SlideShare a Scribd company logo
December 6, 2014 
Andrii Dzynia 
Road Manager to Continuous Delivery 
@adzynia 
Quality Built In
2 
Spotify brings you the right music for 
every moment! 
Started in 2006 (in Sweden) 
Now 1500+ employees, 500+ engineers 
Over 30 million songs available 
Over 20,000 songs added every day 
5 development centres across the globe
What is Quality? 
3
4 
Quality is a “state” …. 
when expectations matches the reality. 
Not just software defects, but stakeholders expectations as well
Why Quality is Important? 
5
Cost of Bug/Change 
6
What is a way to build high 
quality product? 
7
8 Do not release until it’s perfect
9 
No one is perfect.
Example 
Implementing internal back office system
11 
Step 1 - Prototyping phase 
Technical spike investigation 
Hacking different Back-End solutions 
Ad-hoc design discussions 
Stub implementation of Front End 
Getting early user’s feedback on Front End
12 WebApp 
with 
stubs 
Sketches 
on 
whiteboard 
Whatever works 
to get end-user 
understanding
13 
Step 2 - Setting delivery pipeline 
Add more unit tests 
Add more integration tests 
Add more functional tests 
Automate deployment configuration 
Add dashboards and system monitoring 
Clarifying End User Needs
14 
http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment
15 
Step 3 - Connecting the dots 
Connect back-end and front-end parts 
into 
End-2-End system 
Get user’s feedback and iterate
16 
Keep releasing small, usable, incremental pieces
17 
What is under the hood ?
18
Organic Structure 
19 
@joakimsunden Scaled Agile @ Spotify
20 
Test Automation Pyramid
21 
How to avoid Ice-Cream effect?
22 
Be strict with this rule.
Testable micro-services architecture 23 
Unit Tests 
Component Tests 
Functional 
Tests 
Integration Tests 
System/End-2-End Tests 
Unit Tests 
Component Tests 
Functional 
Tests 
JS Unit Tests 
JS Component Tests 
UI 
Functional 
Tests
24 
Unit Tests 
Check logic of minimal code snippet 
with no or mocked dependencies 
https://github.com/mockito/mockito 
https://github.com/junit-team/junit
25 
Component Tests 
Very much like unit tests but emulating 
external components 
e.g. cassandra database 
https://github.com/jsevellec/cassandra-unit
26 
Fake back-end for client apps 
Client App 
Fake Back-End Server 
Spotify Back End 
https://github.com/azagniotov/stubby4j 
https://github.com/dreamhead/moco 
http://jsonstub.com
27 
Integration Tests 
Check an integration between components 
You do not need whole system to check the integration 
https://github.com/spotify/helios/blob/master/docs/testing_framework.md
28 
Create Test Environment ‘on the fly’ 
@docker containers 
https://www.docker.com
29 
Functional Tests 
Check services API behaviour 
or 
End user use cases 
http://hc.apache.org 
http://seleniumhq.org
30 
Model Based Testing 
http://graphwalker.org
32 
Code Review 
& 
Human Testing 
on 
Pull Request
Delivery Pipeline 
33
34 
Was not covered during this talk … 
Load Testing 
Gradual Rollouts 
A/B Testing 
Feature Toggles 
Monitoring/Alerting 
Information Radiators 
… but stay tuned
35 
Test Engineering Roles and Responsibilities 
Technical Test Engineer Test Engineer 
! = Test Automator ! = Manual Tester 
~ Software Engineer in Test 
~ Context Driven Tester
36 
Technical Test Engineer 
Work as a software developer 
Advocate testability of the product 
Argue on software design with software engineers 
Help with building new/injecting existed development tools
37 
Development Productivity Tools 
e.g. cli control over amazon cloud http://dashing.io 
Docker orchestration 
https://github.com/spotify/helios 
Cassandra Unit 
https://github.com/jsevellec/cassandra-unit 
Spoticloud 
Jenkins job-dsl-plugin 
https://github.com/mikaellanger/job-dsl-plugin 
Dashboards
38 
Test Engineer 
Knows business domain 
Focused on exploring the product 
Free to use any programming language to test specific use case 
Free to use any programming language to automate his work
39 
Mind Map as a Tool 
https://www.mindmup.com 
Product tree 
Scenarios 
Playbooks 
Checklists 
Session notes
Quality Engineers 
40 
@jamesmarcusbach calls us Test Jumpers 
TEST JUMPERS: ONE VISION OF AGILE TESTING 
http://www.satisfice.com/blog/archives/1372 
“RESPONSIBLE TESTER” 
http://www.satisfice.com/blog/archives/1364 
OMEGA TESTER 
http://www.satisfice.com/articles/omega_tester.pdf
41 
Any other ways 
to improve 
quality?
42 
Break Uncertainty 
test ideas during healthy 
discussions 
test requirements via end 
users collaboration 
discuss system design and 
conner cases earlier in the 
planing/design meetings 
define definition of done
43 
Use Google Docs collaboration for finding group consensus 
But have responsible person to figure out what consensus means in each case
Social Programming 
44 
write “checks” during implementation 
shape your thoughts via pair discussions 
peer review before merging to master
45 
Not mentioned Engineering Practices 
User Stories 
Planning meeting 
Standup meeting 
Open workspace 
TDD 
Refactoring 
etc…… 
http://www.extremeprogramming.org/rules.html
Employee Early Builds Testing 
46
47 
Hackathons 
One week to do whatever 
you want 
Do it alone or with a team 
Crazy ideas comes out
48 
QA engineer at Spotify 
http://continuousdelivery.com/2014/02/visualizations-of-continuous-delivery/
How to check you are on 
a right path ? 
49
That is easy to spot 50
51 Ready for a journey? 
Make Quality explicit 
Find Quality promoters 
Define your way of Quality 
improvements 
Set right Quality constraints 
Share results as early as possible 
Keep looking further quality improvements 
Probably you will never end :)
52 
@adzynia 
adzynia@spotify.com 
Spotify Engineering Culture 
part 1 
part 2
and … 
Stop saying QA 
when you mean Testing 
Want to join the band? 
Check out http://www.spotify.com/jobs or @Spotifyjobs for more information

More Related Content

What's hot

Introduction to CI/CD
Introduction to CI/CDIntroduction to CI/CD
Introduction to CI/CD
Steve Mactaggart
 
The story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps EngineerThe story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps Engineer
Manu Pk
 
Build CICD Pipeline for Container Presentation Slides
Build CICD Pipeline for Container Presentation SlidesBuild CICD Pipeline for Container Presentation Slides
Build CICD Pipeline for Container Presentation Slides
Amazon Web Services
 
Developing a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps SuccessDeveloping a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps Success
DevOps.com
 
Guide to Agile testing
Guide to Agile testingGuide to Agile testing
Guide to Agile testing
Subrahmaniam S.R.V
 
Introduction to CI/CD
Introduction to CI/CDIntroduction to CI/CD
Introduction to CI/CD
Hoang Le
 
Test Automation Framework with BDD and Cucumber
Test Automation Framework with BDD and CucumberTest Automation Framework with BDD and Cucumber
Test Automation Framework with BDD and Cucumber
Rhoynar Software Consulting
 
Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle
Kari Kakkonen
 
Yale Jenkins Show and Tell
Yale Jenkins Show and TellYale Jenkins Show and Tell
Yale Jenkins Show and Tell
E. Camden Fisher
 
Agile testing - Testing From Day 1
Agile testing - Testing From Day 1Agile testing - Testing From Day 1
Agile testing - Testing From Day 1
Kaizenko
 
Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration
Amazon Web Services
 
SonarQube: Continuous Code Inspection
SonarQube: Continuous Code InspectionSonarQube: Continuous Code Inspection
SonarQube: Continuous Code Inspection
Michael Jesse
 
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Edureka!
 
DevOps, Common use cases, Architectures, Best Practices
DevOps, Common use cases, Architectures, Best PracticesDevOps, Common use cases, Architectures, Best Practices
DevOps, Common use cases, Architectures, Best Practices
Shiva Narayanaswamy
 
An Introduction To Jenkins
An Introduction To JenkinsAn Introduction To Jenkins
An Introduction To Jenkins
Knoldus Inc.
 
Testes E2E em Cypress com JS
Testes E2E em Cypress com JSTestes E2E em Cypress com JS
Testes E2E em Cypress com JS
Nàtali Cabral
 
Introduction to DevOps
Introduction to DevOpsIntroduction to DevOps
Introduction to DevOps
Hawkman Academy
 
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
CI/CD with Jenkins and Docker - DevOps Meetup Day ThailandCI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
Troublemaker Khunpech
 
Agile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile TesterAgile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile Tester
Declan Whelan
 
Component testing with cypress
Component testing with cypressComponent testing with cypress
Component testing with cypress
Walmyr Lima e Silva Filho
 

What's hot (20)

Introduction to CI/CD
Introduction to CI/CDIntroduction to CI/CD
Introduction to CI/CD
 
The story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps EngineerThe story of SonarQube told to a DevOps Engineer
The story of SonarQube told to a DevOps Engineer
 
Build CICD Pipeline for Container Presentation Slides
Build CICD Pipeline for Container Presentation SlidesBuild CICD Pipeline for Container Presentation Slides
Build CICD Pipeline for Container Presentation Slides
 
Developing a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps SuccessDeveloping a Testing Strategy for DevOps Success
Developing a Testing Strategy for DevOps Success
 
Guide to Agile testing
Guide to Agile testingGuide to Agile testing
Guide to Agile testing
 
Introduction to CI/CD
Introduction to CI/CDIntroduction to CI/CD
Introduction to CI/CD
 
Test Automation Framework with BDD and Cucumber
Test Automation Framework with BDD and CucumberTest Automation Framework with BDD and Cucumber
Test Automation Framework with BDD and Cucumber
 
Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle
 
Yale Jenkins Show and Tell
Yale Jenkins Show and TellYale Jenkins Show and Tell
Yale Jenkins Show and Tell
 
Agile testing - Testing From Day 1
Agile testing - Testing From Day 1Agile testing - Testing From Day 1
Agile testing - Testing From Day 1
 
Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration
 
SonarQube: Continuous Code Inspection
SonarQube: Continuous Code InspectionSonarQube: Continuous Code Inspection
SonarQube: Continuous Code Inspection
 
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
 
DevOps, Common use cases, Architectures, Best Practices
DevOps, Common use cases, Architectures, Best PracticesDevOps, Common use cases, Architectures, Best Practices
DevOps, Common use cases, Architectures, Best Practices
 
An Introduction To Jenkins
An Introduction To JenkinsAn Introduction To Jenkins
An Introduction To Jenkins
 
Testes E2E em Cypress com JS
Testes E2E em Cypress com JSTestes E2E em Cypress com JS
Testes E2E em Cypress com JS
 
Introduction to DevOps
Introduction to DevOpsIntroduction to DevOps
Introduction to DevOps
 
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
CI/CD with Jenkins and Docker - DevOps Meetup Day ThailandCI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
 
Agile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile TesterAgile Testing: The Role Of The Agile Tester
Agile Testing: The Role Of The Agile Tester
 
Component testing with cypress
Component testing with cypressComponent testing with cypress
Component testing with cypress
 

Similar to Quality Built In @ Spotify

Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...
Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...
Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...
GoIT
 
Testing practicies not only in scala
Testing practicies not only in scalaTesting practicies not only in scala
Testing practicies not only in scala
Paweł Panasewicz
 
Agile & DevOps - It's all about project success
Agile & DevOps - It's all about project successAgile & DevOps - It's all about project success
Agile & DevOps - It's all about project success
Adam Stephensen
 
Continous integration and delivery for single page applications
Continous integration and delivery for single page applicationsContinous integration and delivery for single page applications
Continous integration and delivery for single page applications
Sunil Dalal
 
Leveraging Open Source Automation: A Selenium WebDriver Example
Leveraging Open Source Automation: A Selenium WebDriver ExampleLeveraging Open Source Automation: A Selenium WebDriver Example
Leveraging Open Source Automation: A Selenium WebDriver Example
TechWell
 
Part 2 improving your software development v1.0
Part 2   improving your software development v1.0Part 2   improving your software development v1.0
Part 2 improving your software development v1.0
Jasmine Conseil
 
Agile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery WorkshopAgile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery Workshop
Michael Palotas
 
Large scale agile development practices
Large scale agile development practicesLarge scale agile development practices
Large scale agile development practices
Skills Matter
 
Agile a to z chapter 4 feedback loop
Agile a to z chapter 4 feedback loopAgile a to z chapter 4 feedback loop
Agile a to z chapter 4 feedback loop
Wajih Aslam
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
Sauce Labs
 
DevOps and Build Automation
DevOps and Build AutomationDevOps and Build Automation
DevOps and Build Automation
Heiswayi Nrird
 
Agile A to Z Chapter 4 Feedback Loop Part 2 DevOps
Agile A to Z Chapter 4 Feedback Loop Part 2 DevOpsAgile A to Z Chapter 4 Feedback Loop Part 2 DevOps
Agile A to Z Chapter 4 Feedback Loop Part 2 DevOps
Agile ME
 
CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton
Araf Karsh Hamid
 
CookpadTechConf2018-(Mobile)TestAutomation
CookpadTechConf2018-(Mobile)TestAutomationCookpadTechConf2018-(Mobile)TestAutomation
CookpadTechConf2018-(Mobile)TestAutomation
Kazuaki Matsuo
 
Continuous integration and delivery for java based web applications
Continuous integration and delivery for java based web applicationsContinuous integration and delivery for java based web applications
Continuous integration and delivery for java based web applications
Sunil Dalal
 
STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
Anna Russo
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for you
Ambientia
 
[ENGLISH] TDC 2015 - PHP Trail - Tests and PHP Continuous Integration Enviro...
[ENGLISH] TDC 2015 - PHP  Trail - Tests and PHP Continuous Integration Enviro...[ENGLISH] TDC 2015 - PHP  Trail - Tests and PHP Continuous Integration Enviro...
[ENGLISH] TDC 2015 - PHP Trail - Tests and PHP Continuous Integration Enviro...
Bruno Tanoue
 
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
Anna Russo
 
Gherkin /BDD intro
Gherkin /BDD introGherkin /BDD intro

Similar to Quality Built In @ Spotify (20)

Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...
Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...
Встреча Quality built in @Spotify от Андрея Дзыни : как стать лучшим тестиров...
 
Testing practicies not only in scala
Testing practicies not only in scalaTesting practicies not only in scala
Testing practicies not only in scala
 
Agile & DevOps - It's all about project success
Agile & DevOps - It's all about project successAgile & DevOps - It's all about project success
Agile & DevOps - It's all about project success
 
Continous integration and delivery for single page applications
Continous integration and delivery for single page applicationsContinous integration and delivery for single page applications
Continous integration and delivery for single page applications
 
Leveraging Open Source Automation: A Selenium WebDriver Example
Leveraging Open Source Automation: A Selenium WebDriver ExampleLeveraging Open Source Automation: A Selenium WebDriver Example
Leveraging Open Source Automation: A Selenium WebDriver Example
 
Part 2 improving your software development v1.0
Part 2   improving your software development v1.0Part 2   improving your software development v1.0
Part 2 improving your software development v1.0
 
Agile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery WorkshopAgile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery Workshop
 
Large scale agile development practices
Large scale agile development practicesLarge scale agile development practices
Large scale agile development practices
 
Agile a to z chapter 4 feedback loop
Agile a to z chapter 4 feedback loopAgile a to z chapter 4 feedback loop
Agile a to z chapter 4 feedback loop
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
 
DevOps and Build Automation
DevOps and Build AutomationDevOps and Build Automation
DevOps and Build Automation
 
Agile A to Z Chapter 4 Feedback Loop Part 2 DevOps
Agile A to Z Chapter 4 Feedback Loop Part 2 DevOpsAgile A to Z Chapter 4 Feedback Loop Part 2 DevOps
Agile A to Z Chapter 4 Feedback Loop Part 2 DevOps
 
CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton
 
CookpadTechConf2018-(Mobile)TestAutomation
CookpadTechConf2018-(Mobile)TestAutomationCookpadTechConf2018-(Mobile)TestAutomation
CookpadTechConf2018-(Mobile)TestAutomation
 
Continuous integration and delivery for java based web applications
Continuous integration and delivery for java based web applicationsContinuous integration and delivery for java based web applications
Continuous integration and delivery for java based web applications
 
STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STARWEST 2010 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
 
Making software development processes to work for you
Making software development processes to work for youMaking software development processes to work for you
Making software development processes to work for you
 
[ENGLISH] TDC 2015 - PHP Trail - Tests and PHP Continuous Integration Enviro...
[ENGLISH] TDC 2015 - PHP  Trail - Tests and PHP Continuous Integration Enviro...[ENGLISH] TDC 2015 - PHP  Trail - Tests and PHP Continuous Integration Enviro...
[ENGLISH] TDC 2015 - PHP Trail - Tests and PHP Continuous Integration Enviro...
 
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
STAREAST 2011 - 7 Steps To Improving Software Quality using Microsoft Test Ma...
 
Gherkin /BDD intro
Gherkin /BDD introGherkin /BDD intro
Gherkin /BDD intro
 

More from Andrii Dzynia

Continuous Delivery as you want it
Continuous Delivery as you want itContinuous Delivery as you want it
Continuous Delivery as you want it
Andrii Dzynia
 
Test coaching your agile team
Test coaching your agile teamTest coaching your agile team
Test coaching your agile team
Andrii Dzynia
 
Testing at Spotify
Testing at SpotifyTesting at Spotify
Testing at Spotify
Andrii Dzynia
 
What does it mean to be a test engineer?
What does it mean to be a test engineer?What does it mean to be a test engineer?
What does it mean to be a test engineer?
Andrii Dzynia
 
Exploring your unit tests
Exploring your unit testsExploring your unit tests
Exploring your unit tests
Andrii Dzynia
 
Hermetic environment for your functional tests
Hermetic environment for your functional testsHermetic environment for your functional tests
Hermetic environment for your functional tests
Andrii Dzynia
 
Applying testing mindset to software development
Applying testing mindset to software developmentApplying testing mindset to software development
Applying testing mindset to software development
Andrii Dzynia
 
Appium Mobile Test Automation like WebDriver
Appium Mobile Test Automation like WebDriverAppium Mobile Test Automation like WebDriver
Appium Mobile Test Automation like WebDriver
Andrii Dzynia
 
How to Manage Testing in Dynamic World
How to Manage Testing in Dynamic WorldHow to Manage Testing in Dynamic World
How to Manage Testing in Dynamic World
Andrii Dzynia
 
ExtJS WebDriver
ExtJS WebDriverExtJS WebDriver
ExtJS WebDriver
Andrii Dzynia
 
10 правил agile тестировщика IT-Brunch
10 правил agile тестировщика IT-Brunch10 правил agile тестировщика IT-Brunch
10 правил agile тестировщика IT-BrunchAndrii Dzynia
 
Working Software Over Comprehensive Documentation
Working Software Over Comprehensive DocumentationWorking Software Over Comprehensive Documentation
Working Software Over Comprehensive Documentation
Andrii Dzynia
 
«Самоорганизуй» себя, пока не «самоорганизовали» тебя
«Самоорганизуй» себя, пока не «самоорганизовали» тебя«Самоорганизуй» себя, пока не «самоорганизовали» тебя
«Самоорганизуй» себя, пока не «самоорганизовали» тебя
Andrii Dzynia
 
Как не нужно писать Gherkin сценарии
Как не нужно писать Gherkin сценарииКак не нужно писать Gherkin сценарии
Как не нужно писать Gherkin сценарии
Andrii Dzynia
 
iOS and Android Mobile Test Automation
iOS and Android Mobile Test AutomationiOS and Android Mobile Test Automation
iOS and Android Mobile Test Automation
Andrii Dzynia
 
10 правил Agile тестировщика
10 правил Agile тестировщика10 правил Agile тестировщика
10 правил Agile тестировщика
Andrii Dzynia
 
Что такое Exploratory Testing?
Что такое Exploratory Testing?Что такое Exploratory Testing?
Что такое Exploratory Testing?Andrii Dzynia
 
Software testing 2.0
Software testing 2.0Software testing 2.0
Software testing 2.0
Andrii Dzynia
 
Тестировщики Vs Программисты
Тестировщики Vs ПрограммистыТестировщики Vs Программисты
Тестировщики Vs Программисты
Andrii Dzynia
 
Testing Dojo Kyiv - Testing Heuristics
Testing Dojo Kyiv - Testing HeuristicsTesting Dojo Kyiv - Testing Heuristics
Testing Dojo Kyiv - Testing Heuristics
Andrii Dzynia
 

More from Andrii Dzynia (20)

Continuous Delivery as you want it
Continuous Delivery as you want itContinuous Delivery as you want it
Continuous Delivery as you want it
 
Test coaching your agile team
Test coaching your agile teamTest coaching your agile team
Test coaching your agile team
 
Testing at Spotify
Testing at SpotifyTesting at Spotify
Testing at Spotify
 
What does it mean to be a test engineer?
What does it mean to be a test engineer?What does it mean to be a test engineer?
What does it mean to be a test engineer?
 
Exploring your unit tests
Exploring your unit testsExploring your unit tests
Exploring your unit tests
 
Hermetic environment for your functional tests
Hermetic environment for your functional testsHermetic environment for your functional tests
Hermetic environment for your functional tests
 
Applying testing mindset to software development
Applying testing mindset to software developmentApplying testing mindset to software development
Applying testing mindset to software development
 
Appium Mobile Test Automation like WebDriver
Appium Mobile Test Automation like WebDriverAppium Mobile Test Automation like WebDriver
Appium Mobile Test Automation like WebDriver
 
How to Manage Testing in Dynamic World
How to Manage Testing in Dynamic WorldHow to Manage Testing in Dynamic World
How to Manage Testing in Dynamic World
 
ExtJS WebDriver
ExtJS WebDriverExtJS WebDriver
ExtJS WebDriver
 
10 правил agile тестировщика IT-Brunch
10 правил agile тестировщика IT-Brunch10 правил agile тестировщика IT-Brunch
10 правил agile тестировщика IT-Brunch
 
Working Software Over Comprehensive Documentation
Working Software Over Comprehensive DocumentationWorking Software Over Comprehensive Documentation
Working Software Over Comprehensive Documentation
 
«Самоорганизуй» себя, пока не «самоорганизовали» тебя
«Самоорганизуй» себя, пока не «самоорганизовали» тебя«Самоорганизуй» себя, пока не «самоорганизовали» тебя
«Самоорганизуй» себя, пока не «самоорганизовали» тебя
 
Как не нужно писать Gherkin сценарии
Как не нужно писать Gherkin сценарииКак не нужно писать Gherkin сценарии
Как не нужно писать Gherkin сценарии
 
iOS and Android Mobile Test Automation
iOS and Android Mobile Test AutomationiOS and Android Mobile Test Automation
iOS and Android Mobile Test Automation
 
10 правил Agile тестировщика
10 правил Agile тестировщика10 правил Agile тестировщика
10 правил Agile тестировщика
 
Что такое Exploratory Testing?
Что такое Exploratory Testing?Что такое Exploratory Testing?
Что такое Exploratory Testing?
 
Software testing 2.0
Software testing 2.0Software testing 2.0
Software testing 2.0
 
Тестировщики Vs Программисты
Тестировщики Vs ПрограммистыТестировщики Vs Программисты
Тестировщики Vs Программисты
 
Testing Dojo Kyiv - Testing Heuristics
Testing Dojo Kyiv - Testing HeuristicsTesting Dojo Kyiv - Testing Heuristics
Testing Dojo Kyiv - Testing Heuristics
 

Recently uploaded

Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...
Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...
Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...
IJECEIAES
 
Computational Engineering IITH Presentation
Computational Engineering IITH PresentationComputational Engineering IITH Presentation
Computational Engineering IITH Presentation
co23btech11018
 
CEC 352 - SATELLITE COMMUNICATION UNIT 1
CEC 352 - SATELLITE COMMUNICATION UNIT 1CEC 352 - SATELLITE COMMUNICATION UNIT 1
CEC 352 - SATELLITE COMMUNICATION UNIT 1
PKavitha10
 
Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...
bijceesjournal
 
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
Gino153088
 
Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...
IJECEIAES
 
Software Quality Assurance-se412-v11.ppt
Software Quality Assurance-se412-v11.pptSoftware Quality Assurance-se412-v11.ppt
Software Quality Assurance-se412-v11.ppt
TaghreedAltamimi
 
cnn.pptx Convolutional neural network used for image classication
cnn.pptx Convolutional neural network used for image classicationcnn.pptx Convolutional neural network used for image classication
cnn.pptx Convolutional neural network used for image classication
SakkaravarthiShanmug
 
Data Driven Maintenance | UReason Webinar
Data Driven Maintenance | UReason WebinarData Driven Maintenance | UReason Webinar
Data Driven Maintenance | UReason Webinar
UReason
 
AI assisted telemedicine KIOSK for Rural India.pptx
AI assisted telemedicine KIOSK for Rural India.pptxAI assisted telemedicine KIOSK for Rural India.pptx
AI assisted telemedicine KIOSK for Rural India.pptx
architagupta876
 
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
IJECEIAES
 
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
ecqow
 
An improved modulation technique suitable for a three level flying capacitor ...
An improved modulation technique suitable for a three level flying capacitor ...An improved modulation technique suitable for a three level flying capacitor ...
An improved modulation technique suitable for a three level flying capacitor ...
IJECEIAES
 
Properties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptxProperties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptx
MDSABBIROJJAMANPAYEL
 
Welding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdfWelding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdf
AjmalKhan50578
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Sinan KOZAK
 
Curve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods RegressionCurve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods Regression
Nada Hikmah
 
Manufacturing Process of molasses based distillery ppt.pptx
Manufacturing Process of molasses based distillery ppt.pptxManufacturing Process of molasses based distillery ppt.pptx
Manufacturing Process of molasses based distillery ppt.pptx
Madan Karki
 
People as resource Grade IX.pdf minimala
People as resource Grade IX.pdf minimalaPeople as resource Grade IX.pdf minimala
People as resource Grade IX.pdf minimala
riddhimaagrawal986
 
Null Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAMNull Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAM
Divyanshu
 

Recently uploaded (20)

Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...
Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...
Redefining brain tumor segmentation: a cutting-edge convolutional neural netw...
 
Computational Engineering IITH Presentation
Computational Engineering IITH PresentationComputational Engineering IITH Presentation
Computational Engineering IITH Presentation
 
CEC 352 - SATELLITE COMMUNICATION UNIT 1
CEC 352 - SATELLITE COMMUNICATION UNIT 1CEC 352 - SATELLITE COMMUNICATION UNIT 1
CEC 352 - SATELLITE COMMUNICATION UNIT 1
 
Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...Comparative analysis between traditional aquaponics and reconstructed aquapon...
Comparative analysis between traditional aquaponics and reconstructed aquapon...
 
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
 
Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...
 
Software Quality Assurance-se412-v11.ppt
Software Quality Assurance-se412-v11.pptSoftware Quality Assurance-se412-v11.ppt
Software Quality Assurance-se412-v11.ppt
 
cnn.pptx Convolutional neural network used for image classication
cnn.pptx Convolutional neural network used for image classicationcnn.pptx Convolutional neural network used for image classication
cnn.pptx Convolutional neural network used for image classication
 
Data Driven Maintenance | UReason Webinar
Data Driven Maintenance | UReason WebinarData Driven Maintenance | UReason Webinar
Data Driven Maintenance | UReason Webinar
 
AI assisted telemedicine KIOSK for Rural India.pptx
AI assisted telemedicine KIOSK for Rural India.pptxAI assisted telemedicine KIOSK for Rural India.pptx
AI assisted telemedicine KIOSK for Rural India.pptx
 
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
 
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
一比一原版(CalArts毕业证)加利福尼亚艺术学院毕业证如何办理
 
An improved modulation technique suitable for a three level flying capacitor ...
An improved modulation technique suitable for a three level flying capacitor ...An improved modulation technique suitable for a three level flying capacitor ...
An improved modulation technique suitable for a three level flying capacitor ...
 
Properties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptxProperties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptx
 
Welding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdfWelding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdf
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
 
Curve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods RegressionCurve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods Regression
 
Manufacturing Process of molasses based distillery ppt.pptx
Manufacturing Process of molasses based distillery ppt.pptxManufacturing Process of molasses based distillery ppt.pptx
Manufacturing Process of molasses based distillery ppt.pptx
 
People as resource Grade IX.pdf minimala
People as resource Grade IX.pdf minimalaPeople as resource Grade IX.pdf minimala
People as resource Grade IX.pdf minimala
 
Null Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAMNull Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAM
 

Quality Built In @ Spotify

  • 1. December 6, 2014 Andrii Dzynia Road Manager to Continuous Delivery @adzynia Quality Built In
  • 2. 2 Spotify brings you the right music for every moment! Started in 2006 (in Sweden) Now 1500+ employees, 500+ engineers Over 30 million songs available Over 20,000 songs added every day 5 development centres across the globe
  • 4. 4 Quality is a “state” …. when expectations matches the reality. Not just software defects, but stakeholders expectations as well
  • 5. Why Quality is Important? 5
  • 7. What is a way to build high quality product? 7
  • 8. 8 Do not release until it’s perfect
  • 9. 9 No one is perfect.
  • 10. Example Implementing internal back office system
  • 11. 11 Step 1 - Prototyping phase Technical spike investigation Hacking different Back-End solutions Ad-hoc design discussions Stub implementation of Front End Getting early user’s feedback on Front End
  • 12. 12 WebApp with stubs Sketches on whiteboard Whatever works to get end-user understanding
  • 13. 13 Step 2 - Setting delivery pipeline Add more unit tests Add more integration tests Add more functional tests Automate deployment configuration Add dashboards and system monitoring Clarifying End User Needs
  • 15. 15 Step 3 - Connecting the dots Connect back-end and front-end parts into End-2-End system Get user’s feedback and iterate
  • 16. 16 Keep releasing small, usable, incremental pieces
  • 17. 17 What is under the hood ?
  • 18. 18
  • 19. Organic Structure 19 @joakimsunden Scaled Agile @ Spotify
  • 21. 21 How to avoid Ice-Cream effect?
  • 22. 22 Be strict with this rule.
  • 23. Testable micro-services architecture 23 Unit Tests Component Tests Functional Tests Integration Tests System/End-2-End Tests Unit Tests Component Tests Functional Tests JS Unit Tests JS Component Tests UI Functional Tests
  • 24. 24 Unit Tests Check logic of minimal code snippet with no or mocked dependencies https://github.com/mockito/mockito https://github.com/junit-team/junit
  • 25. 25 Component Tests Very much like unit tests but emulating external components e.g. cassandra database https://github.com/jsevellec/cassandra-unit
  • 26. 26 Fake back-end for client apps Client App Fake Back-End Server Spotify Back End https://github.com/azagniotov/stubby4j https://github.com/dreamhead/moco http://jsonstub.com
  • 27. 27 Integration Tests Check an integration between components You do not need whole system to check the integration https://github.com/spotify/helios/blob/master/docs/testing_framework.md
  • 28. 28 Create Test Environment ‘on the fly’ @docker containers https://www.docker.com
  • 29. 29 Functional Tests Check services API behaviour or End user use cases http://hc.apache.org http://seleniumhq.org
  • 30. 30 Model Based Testing http://graphwalker.org
  • 31.
  • 32. 32 Code Review & Human Testing on Pull Request
  • 34. 34 Was not covered during this talk … Load Testing Gradual Rollouts A/B Testing Feature Toggles Monitoring/Alerting Information Radiators … but stay tuned
  • 35. 35 Test Engineering Roles and Responsibilities Technical Test Engineer Test Engineer ! = Test Automator ! = Manual Tester ~ Software Engineer in Test ~ Context Driven Tester
  • 36. 36 Technical Test Engineer Work as a software developer Advocate testability of the product Argue on software design with software engineers Help with building new/injecting existed development tools
  • 37. 37 Development Productivity Tools e.g. cli control over amazon cloud http://dashing.io Docker orchestration https://github.com/spotify/helios Cassandra Unit https://github.com/jsevellec/cassandra-unit Spoticloud Jenkins job-dsl-plugin https://github.com/mikaellanger/job-dsl-plugin Dashboards
  • 38. 38 Test Engineer Knows business domain Focused on exploring the product Free to use any programming language to test specific use case Free to use any programming language to automate his work
  • 39. 39 Mind Map as a Tool https://www.mindmup.com Product tree Scenarios Playbooks Checklists Session notes
  • 40. Quality Engineers 40 @jamesmarcusbach calls us Test Jumpers TEST JUMPERS: ONE VISION OF AGILE TESTING http://www.satisfice.com/blog/archives/1372 “RESPONSIBLE TESTER” http://www.satisfice.com/blog/archives/1364 OMEGA TESTER http://www.satisfice.com/articles/omega_tester.pdf
  • 41. 41 Any other ways to improve quality?
  • 42. 42 Break Uncertainty test ideas during healthy discussions test requirements via end users collaboration discuss system design and conner cases earlier in the planing/design meetings define definition of done
  • 43. 43 Use Google Docs collaboration for finding group consensus But have responsible person to figure out what consensus means in each case
  • 44. Social Programming 44 write “checks” during implementation shape your thoughts via pair discussions peer review before merging to master
  • 45. 45 Not mentioned Engineering Practices User Stories Planning meeting Standup meeting Open workspace TDD Refactoring etc…… http://www.extremeprogramming.org/rules.html
  • 46. Employee Early Builds Testing 46
  • 47. 47 Hackathons One week to do whatever you want Do it alone or with a team Crazy ideas comes out
  • 48. 48 QA engineer at Spotify http://continuousdelivery.com/2014/02/visualizations-of-continuous-delivery/
  • 49. How to check you are on a right path ? 49
  • 50. That is easy to spot 50
  • 51. 51 Ready for a journey? Make Quality explicit Find Quality promoters Define your way of Quality improvements Set right Quality constraints Share results as early as possible Keep looking further quality improvements Probably you will never end :)
  • 52. 52 @adzynia adzynia@spotify.com Spotify Engineering Culture part 1 part 2
  • 53. and … Stop saying QA when you mean Testing Want to join the band? Check out http://www.spotify.com/jobs or @Spotifyjobs for more information