SlideShare a Scribd company logo
1 of 25
Download to read offline
Experiences Of Test Automation at Spotify 
Kristian Karl, Spotify 
www.eurostarconferences.com 
@esconfs 
#esconfs 
Insert speaker 
picture here, no 
more than 
150x150 pixels
Experiences Of Test Automation 
@ Spotify 
Kristian Karl 
October 16, 2013
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
Scaling Agile @ Spotify with Tribes, Squads, Chapters & Guilds, by Henrik Kniberg & Anders Ivarsson
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
Why automate? 
• We want to be faster 
• Shorter release cycles 
• Facilitate the testing in 
agile dev cycle 
From: ”Intelligent Test 
Automation” by Harry Robinson
What to automate? 
Graphical user interface testing 
Usability testing 
Software performance testing 
System testing 
Functional testing 
Load testing 
Volume testing 
Stress testing 
Security testing 
Scalability testing 
Sanity testing 
Unit testing 
Smoke testing 
Component testing 
API testing 
Regression testing 
Installation testing 
Maintenance testing 
Recovery and failover testing. 
Accessibility testing 
Monkey testing 
Integration testing 
Graphical user interface testing
Our challenges 
• Hard-to-test SUT 
(Experiences of test automation: case study 1, An Agile Team’s Test 
Automation Journey: The First Year), Dorothy Graham & Mark 
Fewster) 
• Maintenance of automation 
• Peoples expectations 
• Flaky SUTs 
• Flaky tests 
• Re-prioritizations of non-critical but test hindering bugs 
• Test data, test environments 
• Supporting services
Our goals 
• Create automated end-user regression tests on 4 major 
platforms 
1) Desktop – Windows and OSX 
2) iOS – iPhone and iPad 
3) Android 
4) Webplayer 
• To facilitate the workload for testers 
• To deliver automated regression tests for a feature as a part of 
definition of done 
• To deliver short feedback loops to teams using Dashboards
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
Model-based testing 
• The models are the abstraction layer 
• The testers designs the automation 
using models 
• The developers implements the code 
of the automation
App not 
running 
Start app 
Login view 
displayed 
Valid credentials 
Main view displayed 
Close 
Log out 
Exit app 
Toggle Invalid credentials 
’Remember Me’ 
Start app
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
public interface SimpleLogin { 
public void e_Close(); 
public void e_Exit(); 
public void e_Init(); 
public void e_InvalidCredentials(); 
public void e_Logout(); 
public void e_StartClient(); 
public void e_ToggleRememberMe(); 
public void e_ValidPremiumCredentials(); 
public void v_ClientNotRunning(); 
public void v_LoginPrompted(); 
public void v_WhatsNew(); 
}
Demo
Developers and developers 
• Why not use developers for TA? 
• Why use developers for TA? 
• Using Test API’s 
• Defined by TA 
• Implemented by developers
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
Before 
android.view.View seekBarView = 
solo.getView(com.spotify.mobile.android.ui.view.CancellableSeekBar.class, 0); 
int[] xy = new int[2]; 
seekBarView.getLocationOnScreen(xy); 
solo.clickOnScreen(xy[0] + 9 + (seekBarView.getWidth() - 18) * position, xy[1] + 
seekBarView.getHeight() / 2.0f); 
After 
page().seekTrack(position)
Supporting services 
• Continuous testing 
• QA Lab 
• TDS – Test Data Service 
• TRS – Test Result Service
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013
Open source tools that we use 
• yEd – editing FSM models [www.yworks.com] 
• GraphWalker – generate test sequences from models 
[graphwalker.org github.com/spotify/python-graphwalker] 
• Sikuli – image recognition tool [www.sikuli.org] 
• NuRemote - communicating with iOS client 
[github.com/nevyn/NuRemoting] 
• TestNG – group tests into suites and and run them [testng.org] 
• Java – code implementation 
• Jenkins – [test] job scheduler 24/7 [jenkins-ci.org]
Demo
Q & A

More Related Content

What's hot

Appium Presentation
Appium Presentation Appium Presentation
Appium Presentation OmarUsman6
 
Api testing and steps to do it
Api testing and steps to do itApi testing and steps to do it
Api testing and steps to do itZoe Gilbert
 
[webinar] Best of Breed: Successful Test Automation Practices from Innovative...
[webinar] Best of Breed: Successful Test Automation Practices from Innovative...[webinar] Best of Breed: Successful Test Automation Practices from Innovative...
[webinar] Best of Breed: Successful Test Automation Practices from Innovative...Applitools
 
API Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj RollisonAPI Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj RollisonTEST Huddle
 
UI5 supportability tools explained
UI5 supportability tools explainedUI5 supportability tools explained
UI5 supportability tools explainedMaxim Naidenov
 
API Test Automation Tips and Tricks
API Test Automation Tips and TricksAPI Test Automation Tips and Tricks
API Test Automation Tips and Trickstesthive
 
Test automation_strategy_for_legacysystems
Test automation_strategy_for_legacysystemsTest automation_strategy_for_legacysystems
Test automation_strategy_for_legacysystemsVodqaBLR
 
Diffy : Automatic Testing of Microservices @ Twitter
Diffy : Automatic Testing of Microservices @ TwitterDiffy : Automatic Testing of Microservices @ Twitter
Diffy : Automatic Testing of Microservices @ TwitterPuneet Khanduri
 
Test Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & ToolsTest Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & ToolsAmit Rawat
 
API Testing: Answers to Your Top 3 Questions
API Testing: Answers to Your Top 3 QuestionsAPI Testing: Answers to Your Top 3 Questions
API Testing: Answers to Your Top 3 QuestionsQASource
 
An Introduction To Automated API Testing
An Introduction To Automated API TestingAn Introduction To Automated API Testing
An Introduction To Automated API TestingSauce Labs
 
API Testing With Katalon Studio
API Testing With Katalon StudioAPI Testing With Katalon Studio
API Testing With Katalon StudioKnoldus Inc.
 
Testing Design System Changes Across Your Application -- Intuit Use Case -- w...
Testing Design System Changes Across Your Application -- Intuit Use Case -- w...Testing Design System Changes Across Your Application -- Intuit Use Case -- w...
Testing Design System Changes Across Your Application -- Intuit Use Case -- w...Applitools
 
Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...
Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...
Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...Applitools
 
Introduction to APIs & how to automate APIs testing with selenium web driver?
Introduction to APIs & how to automate APIs testing with selenium web driver?Introduction to APIs & how to automate APIs testing with selenium web driver?
Introduction to APIs & how to automate APIs testing with selenium web driver?BugRaptors
 
How to Leverage XCUITest for Continuous Testing
How to Leverage XCUITest for Continuous TestingHow to Leverage XCUITest for Continuous Testing
How to Leverage XCUITest for Continuous TestingPerfecto by Perforce
 
Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...
Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...
Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...Narayanan Palani
 
4 Major Advantages of API Testing
4 Major Advantages of API Testing4 Major Advantages of API Testing
4 Major Advantages of API TestingQASource
 
Stop Testing (Only) The Functionality of Your Mobile Apps!
Stop Testing (Only) The Functionality of Your Mobile Apps!Stop Testing (Only) The Functionality of Your Mobile Apps!
Stop Testing (Only) The Functionality of Your Mobile Apps!Applitools
 

What's hot (20)

Appium Presentation
Appium Presentation Appium Presentation
Appium Presentation
 
Api testing and steps to do it
Api testing and steps to do itApi testing and steps to do it
Api testing and steps to do it
 
[webinar] Best of Breed: Successful Test Automation Practices from Innovative...
[webinar] Best of Breed: Successful Test Automation Practices from Innovative...[webinar] Best of Breed: Successful Test Automation Practices from Innovative...
[webinar] Best of Breed: Successful Test Automation Practices from Innovative...
 
API Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj RollisonAPI Testing: The heart of functional testing" with Bj Rollison
API Testing: The heart of functional testing" with Bj Rollison
 
UI5 supportability tools explained
UI5 supportability tools explainedUI5 supportability tools explained
UI5 supportability tools explained
 
API Test Automation Tips and Tricks
API Test Automation Tips and TricksAPI Test Automation Tips and Tricks
API Test Automation Tips and Tricks
 
Test automation_strategy_for_legacysystems
Test automation_strategy_for_legacysystemsTest automation_strategy_for_legacysystems
Test automation_strategy_for_legacysystems
 
Diffy : Automatic Testing of Microservices @ Twitter
Diffy : Automatic Testing of Microservices @ TwitterDiffy : Automatic Testing of Microservices @ Twitter
Diffy : Automatic Testing of Microservices @ Twitter
 
Test Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & ToolsTest Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & Tools
 
API Testing: Answers to Your Top 3 Questions
API Testing: Answers to Your Top 3 QuestionsAPI Testing: Answers to Your Top 3 Questions
API Testing: Answers to Your Top 3 Questions
 
An Introduction To Automated API Testing
An Introduction To Automated API TestingAn Introduction To Automated API Testing
An Introduction To Automated API Testing
 
API Testing With Katalon Studio
API Testing With Katalon StudioAPI Testing With Katalon Studio
API Testing With Katalon Studio
 
Testing Design System Changes Across Your Application -- Intuit Use Case -- w...
Testing Design System Changes Across Your Application -- Intuit Use Case -- w...Testing Design System Changes Across Your Application -- Intuit Use Case -- w...
Testing Design System Changes Across Your Application -- Intuit Use Case -- w...
 
Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...
Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...
Testing Hourglass at Jira Frontend - by Alexey Shpakov, Sr. Developer @ Atlas...
 
Introduction to APIs & how to automate APIs testing with selenium web driver?
Introduction to APIs & how to automate APIs testing with selenium web driver?Introduction to APIs & how to automate APIs testing with selenium web driver?
Introduction to APIs & how to automate APIs testing with selenium web driver?
 
Api testing
Api testingApi testing
Api testing
 
How to Leverage XCUITest for Continuous Testing
How to Leverage XCUITest for Continuous TestingHow to Leverage XCUITest for Continuous Testing
How to Leverage XCUITest for Continuous Testing
 
Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...
Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...
Web Accessibility Testing Trends and Shift Left Testing of accessibility usin...
 
4 Major Advantages of API Testing
4 Major Advantages of API Testing4 Major Advantages of API Testing
4 Major Advantages of API Testing
 
Stop Testing (Only) The Functionality of Your Mobile Apps!
Stop Testing (Only) The Functionality of Your Mobile Apps!Stop Testing (Only) The Functionality of Your Mobile Apps!
Stop Testing (Only) The Functionality of Your Mobile Apps!
 

Similar to Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013

Diving into the World of Test Automation The Approach and the Technologies
Diving into the World of Test Automation The Approach and the TechnologiesDiving into the World of Test Automation The Approach and the Technologies
Diving into the World of Test Automation The Approach and the TechnologiesQASymphony
 
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...Turkish Testing Board
 
Suparna - Shift Left Testing in Mobile.pptx
Suparna - Shift Left Testing in Mobile.pptxSuparna - Shift Left Testing in Mobile.pptx
Suparna - Shift Left Testing in Mobile.pptxSUPARNA KHAMARU
 
Test Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_PresentationTest Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_PresentationUiPathCommunity
 
Unit tests & TDD
Unit tests & TDDUnit tests & TDD
Unit tests & TDDDror Helper
 
How Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlHow Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlSmartBear
 
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test AutomationSTARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test AutomationClever Moe
 
Continuous Integration, Deploy, Test From Beginning To End 2014
Continuous Integration, Deploy, Test From Beginning To End 2014Continuous Integration, Deploy, Test From Beginning To End 2014
Continuous Integration, Deploy, Test From Beginning To End 2014Clever Moe
 
Use Jenkins For Continuous Load Testing And Mobile Test Automation
Use Jenkins For Continuous Load Testing And Mobile Test AutomationUse Jenkins For Continuous Load Testing And Mobile Test Automation
Use Jenkins For Continuous Load Testing And Mobile Test AutomationClever Moe
 
Battle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java ProjectBattle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java ProjectGlobalLogic Ukraine
 
Neev Independent Testing Services
Neev Independent Testing ServicesNeev Independent Testing Services
Neev Independent Testing ServicesNeev Technologies
 
Agile testing - Principles and best practices
Agile testing  - Principles and best practicesAgile testing  - Principles and best practices
Agile testing - Principles and best practicesDr Ganesh Iyer
 
Accelerating tests with Cypress for a leaderboard platform
Accelerating tests with Cypress for a leaderboard platformAccelerating tests with Cypress for a leaderboard platform
Accelerating tests with Cypress for a leaderboard platformKnoldus Inc.
 
When & How to Successfully use Test Automation for Mobile Applications
When & How to Successfully use Test Automation for Mobile ApplicationsWhen & How to Successfully use Test Automation for Mobile Applications
When & How to Successfully use Test Automation for Mobile ApplicationsTechnologyAssociationOregon
 
End to End Testing with Quality Enthusiasts: SDET Technologies
End to End Testing with Quality Enthusiasts: SDET TechnologiesEnd to End Testing with Quality Enthusiasts: SDET Technologies
End to End Testing with Quality Enthusiasts: SDET Technologiessdettech
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test AutomationSauce Labs
 
Thomas Haver - Mobile Testing.pdf
Thomas Haver - Mobile Testing.pdfThomas Haver - Mobile Testing.pdf
Thomas Haver - Mobile Testing.pdfQA or the Highway
 
Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...
Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...
Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...Riccardo Coppola
 
Large-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in PracticeLarge-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in PracticeTechWell
 

Similar to Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013 (20)

Diving into the World of Test Automation The Approach and the Technologies
Diving into the World of Test Automation The Approach and the TechnologiesDiving into the World of Test Automation The Approach and the Technologies
Diving into the World of Test Automation The Approach and the Technologies
 
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
 
Suparna - Shift Left Testing in Mobile.pptx
Suparna - Shift Left Testing in Mobile.pptxSuparna - Shift Left Testing in Mobile.pptx
Suparna - Shift Left Testing in Mobile.pptx
 
Test Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_PresentationTest Automation with Gen AI_Final_Presentation
Test Automation with Gen AI_Final_Presentation
 
Unit tests & TDD
Unit tests & TDDUnit tests & TDD
Unit tests & TDD
 
How Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian KarlHow Spotify Does Test Automation - Kristian Karl
How Spotify Does Test Automation - Kristian Karl
 
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test AutomationSTARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
STARWest: Use Jenkins For Continuous 
Load Testing And Mobile Test Automation
 
Continuous Integration, Deploy, Test From Beginning To End 2014
Continuous Integration, Deploy, Test From Beginning To End 2014Continuous Integration, Deploy, Test From Beginning To End 2014
Continuous Integration, Deploy, Test From Beginning To End 2014
 
Use Jenkins For Continuous Load Testing And Mobile Test Automation
Use Jenkins For Continuous Load Testing And Mobile Test AutomationUse Jenkins For Continuous Load Testing And Mobile Test Automation
Use Jenkins For Continuous Load Testing And Mobile Test Automation
 
Battle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java ProjectBattle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java Project
 
Neev Independent Testing Services
Neev Independent Testing ServicesNeev Independent Testing Services
Neev Independent Testing Services
 
Agile testing - Principles and best practices
Agile testing  - Principles and best practicesAgile testing  - Principles and best practices
Agile testing - Principles and best practices
 
Accelerating tests with Cypress for a leaderboard platform
Accelerating tests with Cypress for a leaderboard platformAccelerating tests with Cypress for a leaderboard platform
Accelerating tests with Cypress for a leaderboard platform
 
When & How to Successfully use Test Automation for Mobile Applications
When & How to Successfully use Test Automation for Mobile ApplicationsWhen & How to Successfully use Test Automation for Mobile Applications
When & How to Successfully use Test Automation for Mobile Applications
 
End to End Testing with Quality Enthusiasts: SDET Technologies
End to End Testing with Quality Enthusiasts: SDET TechnologiesEnd to End Testing with Quality Enthusiasts: SDET Technologies
End to End Testing with Quality Enthusiasts: SDET Technologies
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
 
Thomas Haver - Mobile Testing.pdf
Thomas Haver - Mobile Testing.pdfThomas Haver - Mobile Testing.pdf
Thomas Haver - Mobile Testing.pdf
 
Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...
Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...
Automated Generation, Evolution and Maintenance: a perspective for mobile GUI...
 
Suparna - XCUITest
Suparna - XCUITestSuparna - XCUITest
Suparna - XCUITest
 
Large-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in PracticeLarge-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in Practice
 

More from TEST Huddle

Why We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- AccentureWhy We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- AccentureTEST Huddle
 
Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar TEST Huddle
 
Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway TEST Huddle
 
Being a Tester in Scrum
Being a Tester in ScrumBeing a Tester in Scrum
Being a Tester in ScrumTEST Huddle
 
Leveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional TestsLeveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional TestsTEST Huddle
 
Using Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test WorkUsing Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test WorkTEST Huddle
 
Big Data: The Magic to Attain New Heights
Big Data:  The Magic to Attain New HeightsBig Data:  The Magic to Attain New Heights
Big Data: The Magic to Attain New HeightsTEST Huddle
 
Will Robots Replace Testers?
Will Robots Replace Testers?Will Robots Replace Testers?
Will Robots Replace Testers?TEST Huddle
 
TDD For The Rest Of Us
TDD For The Rest Of UsTDD For The Rest Of Us
TDD For The Rest Of UsTEST Huddle
 
Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)TEST Huddle
 
Creating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger EnterprisesCreating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger EnterprisesTEST Huddle
 
Is There A Risk?
Is There A Risk?Is There A Risk?
Is There A Risk?TEST Huddle
 
Are Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test CoverageAre Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test CoverageTEST Huddle
 
Growing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for TestersGrowing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for TestersTEST Huddle
 
Do we need testers on agile teams?
Do we need testers on agile teams?Do we need testers on agile teams?
Do we need testers on agile teams?TEST Huddle
 
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfullyTEST Huddle
 
Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey TEST Huddle
 
Practical Test Strategy Using Heuristics
Practical Test Strategy Using HeuristicsPractical Test Strategy Using Heuristics
Practical Test Strategy Using HeuristicsTEST Huddle
 
Thinking Through Your Role
Thinking Through Your RoleThinking Through Your Role
Thinking Through Your RoleTEST Huddle
 
Using Selenium 3 0
Using Selenium 3 0Using Selenium 3 0
Using Selenium 3 0TEST Huddle
 

More from TEST Huddle (20)

Why We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- AccentureWhy We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- Accenture
 
Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar
 
Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway
 
Being a Tester in Scrum
Being a Tester in ScrumBeing a Tester in Scrum
Being a Tester in Scrum
 
Leveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional TestsLeveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional Tests
 
Using Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test WorkUsing Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test Work
 
Big Data: The Magic to Attain New Heights
Big Data:  The Magic to Attain New HeightsBig Data:  The Magic to Attain New Heights
Big Data: The Magic to Attain New Heights
 
Will Robots Replace Testers?
Will Robots Replace Testers?Will Robots Replace Testers?
Will Robots Replace Testers?
 
TDD For The Rest Of Us
TDD For The Rest Of UsTDD For The Rest Of Us
TDD For The Rest Of Us
 
Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)
 
Creating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger EnterprisesCreating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger Enterprises
 
Is There A Risk?
Is There A Risk?Is There A Risk?
Is There A Risk?
 
Are Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test CoverageAre Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test Coverage
 
Growing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for TestersGrowing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for Testers
 
Do we need testers on agile teams?
Do we need testers on agile teams?Do we need testers on agile teams?
Do we need testers on agile teams?
 
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfully
 
Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey
 
Practical Test Strategy Using Heuristics
Practical Test Strategy Using HeuristicsPractical Test Strategy Using Heuristics
Practical Test Strategy Using Heuristics
 
Thinking Through Your Role
Thinking Through Your RoleThinking Through Your Role
Thinking Through Your Role
 
Using Selenium 3 0
Using Selenium 3 0Using Selenium 3 0
Using Selenium 3 0
 

Kristian Karl - Experiences of Test Automation at Spotify - EuroSTAR 2013

  • 1. Experiences Of Test Automation at Spotify Kristian Karl, Spotify www.eurostarconferences.com @esconfs #esconfs Insert speaker picture here, no more than 150x150 pixels
  • 2. Experiences Of Test Automation @ Spotify Kristian Karl October 16, 2013
  • 4. Scaling Agile @ Spotify with Tribes, Squads, Chapters & Guilds, by Henrik Kniberg & Anders Ivarsson
  • 6. Why automate? • We want to be faster • Shorter release cycles • Facilitate the testing in agile dev cycle From: ”Intelligent Test Automation” by Harry Robinson
  • 7. What to automate? Graphical user interface testing Usability testing Software performance testing System testing Functional testing Load testing Volume testing Stress testing Security testing Scalability testing Sanity testing Unit testing Smoke testing Component testing API testing Regression testing Installation testing Maintenance testing Recovery and failover testing. Accessibility testing Monkey testing Integration testing Graphical user interface testing
  • 8. Our challenges • Hard-to-test SUT (Experiences of test automation: case study 1, An Agile Team’s Test Automation Journey: The First Year), Dorothy Graham & Mark Fewster) • Maintenance of automation • Peoples expectations • Flaky SUTs • Flaky tests • Re-prioritizations of non-critical but test hindering bugs • Test data, test environments • Supporting services
  • 9. Our goals • Create automated end-user regression tests on 4 major platforms 1) Desktop – Windows and OSX 2) iOS – iPhone and iPad 3) Android 4) Webplayer • To facilitate the workload for testers • To deliver automated regression tests for a feature as a part of definition of done • To deliver short feedback loops to teams using Dashboards
  • 11. Model-based testing • The models are the abstraction layer • The testers designs the automation using models • The developers implements the code of the automation
  • 12. App not running Start app Login view displayed Valid credentials Main view displayed Close Log out Exit app Toggle Invalid credentials ’Remember Me’ Start app
  • 14. public interface SimpleLogin { public void e_Close(); public void e_Exit(); public void e_Init(); public void e_InvalidCredentials(); public void e_Logout(); public void e_StartClient(); public void e_ToggleRememberMe(); public void e_ValidPremiumCredentials(); public void v_ClientNotRunning(); public void v_LoginPrompted(); public void v_WhatsNew(); }
  • 15. Demo
  • 16. Developers and developers • Why not use developers for TA? • Why use developers for TA? • Using Test API’s • Defined by TA • Implemented by developers
  • 18. Before android.view.View seekBarView = solo.getView(com.spotify.mobile.android.ui.view.CancellableSeekBar.class, 0); int[] xy = new int[2]; seekBarView.getLocationOnScreen(xy); solo.clickOnScreen(xy[0] + 9 + (seekBarView.getWidth() - 18) * position, xy[1] + seekBarView.getHeight() / 2.0f); After page().seekTrack(position)
  • 19. Supporting services • Continuous testing • QA Lab • TDS – Test Data Service • TRS – Test Result Service
  • 23. Open source tools that we use • yEd – editing FSM models [www.yworks.com] • GraphWalker – generate test sequences from models [graphwalker.org github.com/spotify/python-graphwalker] • Sikuli – image recognition tool [www.sikuli.org] • NuRemote - communicating with iOS client [github.com/nevyn/NuRemoting] • TestNG – group tests into suites and and run them [testng.org] • Java – code implementation • Jenkins – [test] job scheduler 24/7 [jenkins-ci.org]
  • 24. Demo
  • 25. Q & A