SlideShare a Scribd company logo
1 of 55
Download to read offline
Get Your Testing 
Process in Place 
Niels Frydenholm 
Senior iOS developer 
nfrydenholm@ebay.com
Get Your Testing 
Process in Place 
Karl Krukow 
Technical Lead,, Xamarin Test Cloud 
@karlkrukow
Getting your mobile test automation process in place - using Cucumber and Calabash
“Continuous Delivery is a software 
development discipline where you build 
software in such a way that the software 
can be released to production at any time.” 
Martin Fowler 
ThoughtWorks
Continuous Delivery
Mobile Continuous Delivery Challenges 
• App Store Distribution Model 
•Maturity - No Industry Standards (yet) 
• Complicated Tool Setup 
• Realistic tests are challenging 
• Tested on your users’ devices 
• Vary Device Conditions - Device settings, Network…
Towards Continuous Delivery for Mobile 
Aspects 
Tooling 
People 
Process
Towards Continuous Delivery for Mobile 
Aspects Examples 
Tooling 
People 
Process 
• Xamarin.UITest, Calabash 
• Jenkins, TeamCity, TFS, … 
• Xamarin Test Cloud
Towards Continuous Delivery for Mobile 
Aspects Examples 
Tooling 
People 
Process 
• Xamarin.UITest, Calabash 
• Jenkins, TeamCity, TFS, … 
• Xamarin Test Cloud 
• Cross-functional Teams 
• Behaviour Driven Development 
• Shared responsibility
Xamarin.UITest & Calabash
Xamarin.UITest & Calabash 
Xamarin.UITest 
• Write Tests in C# 
• Run with NUnit 
• Xamrin/Visual Studio or CLI 
Calabash 
• Write Tests in Ruby 
• Supports BDD, run with Cucumber 
• RubyMine or CLI
Xamarin.UITest - Architecture 
Your App 
Test Cloud 
Agent
Calabash - Architecture 
Your App 
Test Cloud 
Agent 
! 
Scenario: Login 
Given I am on the Login screen 
When I login as "Nat" 
Then I should go to the Assignments
Calabash - Architecture 
Your App 
Test Cloud 
Agent 
Cucumber 
! 
Scenario: Login 
Given I am on the Login screen 
When I login as "Nat" 
Then I should go to the Assignments
Calabash - Architecture 
Your App 
Test Cloud 
Agent 
Cucumber 
! 
Scenario: Login 
Given I am on the Login screen 
When I login as "Nat" 
Then I should go to the Assignments
Get Your Testing 
Process in Place 
Niels Frydenholm 
Senior iOS developer 
nfrydenholm@ebay.com
Agenda 
■ From story to running tests 
■ Improvements in development and release cycle 
■ Demo 
■ Best practices (based on our learnings) 
■ Final words 
■ Questions?
eBay Classifieds, Denmark 
Two popular consumer apps and sites 
■ DBA - DK’s #1 Classifieds site 
■ BilBasen - DK’s #1 Online Vehicle Marketplace 
A few numbers 
■ Approx. 1/4 of the danish population is using the 
DBA app 
■ Mobile traffic share is 57% for DBA 
■ Approx 1/8 of the danish population is using the 
BilBasen app 
■ Mobile traffic share is 54% for BilBasen
How we are organised 
The team 
■ 4 iOS developers / (backend) 
■ 1 backend developer 
■ 1 QA expert 
■ 1 Scrum master
From Story 
to Running Tests
Implementing a story 
Before sprint 
■ Product owner (PO) introduces story 
■ Team and PO agrees on accept criteria’s 
New feature idea 
Lots of questions
Implementing a story 
Before sprint 
■ Product owner (PO) introduces story 
■ Team and PO agrees on accept criteria’s 
New feature idea 
Lots of questions 
Implement feature 
Write tests 
Review 
Test 
In Sprint 
■ Developer(s) implements story and tests 
■ Unit, Integration and UI tests 
■ Done with code and tests are green 
■ Review code (and tests) 
■ Manuel test from other team member 
■ Inspect code coverage to ensure test quality
Continuous Integration 
■ Jenkins on a couple of Mac Mini’s 
- Unit -> Integration -> UI tests 
- UI tests only in Simulator 
■ A subset after commit 
■ A subset in Xamarin Cloud 
■ All UI tests nightly 
! 
■ Keep status visible and Green!
Focus on Automated Tests 
Has Improved ‘Everything’
Releases 2013 
DBA versions 
JAN 
2.0 2.0.1 2.1 2.2 2.3 2.3.1 2.4 3.0 
FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 
BilBasen versions 
4.2 5.0 5.0.1 5.0.2 
Refactored tests 
to Page objects 
4.1.2 
4.1.3
3.3.1 3.5 
3.2 3.3 3.4.1 
3.0.1 3.1 3.2.1 3.4 3.4.2 
FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 
5.0.4 5.0.5 5.0.6 5.1 5.2 5.2.2 
5.2.1 
3.2.3 
Releases 2014 
DBA versions 
3.1.1 3.2.2 
JAN 
5.0.3 
BilBasen versions
Development improvements 
In early 2013 
• Limited refactoring - risk too high 
• All changes = huge test effort 
•Many bugs were not found until later… 
• Lots of “legacy code” 
• See first bullet
Development improvements 
In early 2013 Now 
• Limited refactoring - risk too high 
• All changes = huge test effort 
•Many bugs were not found until later… 
• Lots of “legacy code” 
• See first bullet 
• Planned refactoring of larger areas 
• Boy scout clean up 
• Adding tests to missing areas 
• (Most) bugs are found immediately after 
commit 
• QA can spend time on better things 
■ Identify critical business areas 
■ Find the crazy bugs
Project example 
BilBasen - new search 
■ Search criteria pages and search logic 
was totally re-written 
■ From hardcoded options in the app 
■ To meta driven from the server 
■ Server side meta data was build as 
part of a project 
■ Saved searches now in sync with web
BilBasen - new search 
Test results 
■ Most scenarios were already covered 
with UI tests 
■ Manuel test effort 
■ About one day of testing 
■ Focused on network timings 
■ Compare results with website 
■ No serious bugs were found
Release improvements 
In early 2013 
• Entire team spent 2-5 days on test runs 
and bug fixes 
• Serious bugs could be found in last 
minute 
• Big releases to save the overhead 
• Releases were often delayed
Release improvements 
In early 2013 Now 
• Entire team spent 2-5 days on test runs 
and bug fixes 
• Serious bugs could be found in last 
minute 
• Big releases to save the overhead 
• Releases were often delayed 
• Upload when all tests are green (all the 
time!) 
• PO flexibility to change scope of 
release 
• 1 hour test run for some releases 
■ Bugs are very rarely found
Demo
Best Practices 
(based on our learnings)
Best Practices 
• Structure your test code well 
• Page objects 
• Replace “sleeps” with “wait_for”
Page objects and wait_for 
Before page objects (Ruby example) 
Scenario: Check that change classification clears 
matrixdata with warning 
Given I am logged in as "Buyer" 
And I am on the SYI hub 
And I select classification "Hovedtelefoner" 
And I set "Type" to “In-ear” 
And I set price to "250" 
!
Page objects and wait_for 
Before page objects (Ruby example) 
Scenario: Check that change classification clears 
matrixdata with warning 
Given I am logged in as "Buyer" 
And I am on the SYI hub 
And I select classification "Hovedtelefoner" 
And I set "Type" to “In-ear” 
And I set price to "250" 
! 
And /^I set price to "(.*?)"$/ do |price| 
macro 'I swipe up' 
sleep(1) 
touch("view marked:'Price'") 
sleep(0.5) 
set_text("view marked:'Price'", price) 
sleep(0.5) 
touch("view marked:'OK'") 
sleep(1) 
end
Page object and wait_for 
With page objects (Ruby example) 
And /^I set price to "(.*?)"$/ do |price| 
@page.write_price(price) 
end
Page object and wait_for 
With page objects (Ruby example) 
And /^I set price to "(.*?)"$/ do |price| 
@page.write_price(price) 
end 
SellYourItemPage class (page object) 
def write_price(price) 
scroll_and_wait_for_row_with_mark("priceCell") 
touch("view marked:'Price'") 
keyboard_enter_text price 
close_keyboard 
end 
def select_bundle(bundle_name) 
..logic here.. 
end 
! 
def write_headline(headline) 
..logic here.. 
end
Best Practices 
• Structure your test code well 
• Page objects 
• replace “sleeps” with “wait_for” 
• Ensure stable test suite execution 
• Give failed tests an extra run
Stability in your CI setup 
• UIAutomation + Simulator can often fail when running 
a big test suite 
• Re-run failed tests an extra time 
• Cucumber rerun formatter 
• NUnit-retry 
• Red builds should be trusted, investigated and fixed 
• Only use this if test suite starts to be unstable
Stability in your CI setup 
• UIAutomation + Simulator can often fail when running 
a big test suite 
• Re-run failed tests an extra time 
• Cucumber rerun formatter 
• NUnit-retry 
• Red builds should be trusted, investigated and fixed 
• Only use this if test suite starts to be unstable
Best Practices 
• Structure your test code well 
• Page objects 
• replace “sleeps” with “wait_for” 
• Ensure stable test suite execution 
• Give failed tests an extra 
• Run tests on real devices 
• Xamarin Test Cloud 
• Screen sizes, OS versions
Getting your mobile test automation process in place - using Cucumber and Calabash
Best Practices 
• Structure your test code well 
• Page objects 
• replace “sleeps” with “wait_for” 
• Ensure stable test suite execution 
• Give failed tests an extra run 
• Run tests on real devices 
• Xamarin Test Cloud 
• Screen sizes, OS versions 
• Strive for fast test execution 
• Use different test suites 
• Avoid too much UI interaction for 
setup/teardown operations
Fast test execution 
• Use the “Backdoor” to the app 
• Xamarin.UITest.IApp.Invoke 
• Invoke(“method”, param) 
• Invoke logic - without using the UI 
• Great for Setup / Teardown behaviour 
• Backdoor logic should be tested with UI 
interactions in other tests
Fast test execution 
• Use the “Backdoor” to the app 
• Xamarin.UITest.IApp.Invoke 
• Invoke(“method”, param) 
• Invoke logic - without using the UI 
• Great for Setup / Teardown behaviour 
• Backdoor logic should be tested with UI 
interactions in other tests 
•We use it for things like 
• Create a user / Login 
• Create listings (things for sale) 
• Cleaning up 
• Delete favorites 
• Delete users listings
Fast test execution - backdoor optimised 
Scenario: I can manage my listing and see the changes 
Given I am logged in as "UniqueSeller" using quick login 
And I have created a listing for "Hovedtelefoner" and is on 
the SYI VIP 
Then I see the VIP for "Her kommer en rimelig lang tekst" 
! 
When I go back to my listings page 
And the created listing is in the list 
…more steps omitted…
Fast test execution - backdoor optimised 
Scenario: I can manage my listing and see the changes 
Given I am logged in as "UniqueSeller" using quick login 
And I have created a listing for "Hovedtelefoner" and is on 
the SYI VIP 
Then I see the VIP for "Her kommer en rimelig lang tekst" 
! 
When I go back to my listings page 
And the created listing is in the list 
…more steps omitted… 
1 scenario (1 passed) 
19 steps (19 passed) 
0m34.054s 
1 scenario (1 passed) 
19 steps (19 passed) 
1m16.610s
Best Practices 
• Structure your test code well 
• Page objects 
• replace “sleeps” with “wait_for” 
• Ensure stable test suite execution 
• Give failed tests an extra run 
• Run tests on real devices 
• Xamarin Test Cloud 
• Screen sizes, OS versions 
• Strive for fast test execution 
• Use different test suites 
• Avoid too much UI interaction for 
setup/teardown operations 
• Run tests for all active branches in CI 
• Automate job creation
Feature builds 
■ All jobs required for a branch 
■ Red builds are accepted for a period 
■ Only merge to master with green builds
Final words
Start today… 
• QA is a Team effort / responsibility 
• Bugs/Crashes will still live in the app 
• Do not expect it to be effortless to get 
started 
• It will quickly pay off
Start today… 
• QA is a Team effort / responsibility 
• Bugs/Crashes will still live in the app 
• Do not expect it to be effortless to get 
started 
• It will quickly pay off 
See it as an investment that gives you 
• Higher app quality (and code quality) 
• Time to focus on new important features 
• Faster time to market - with less bugs 
• Happy customers - they like frequent 
updates of high quality 
• Happy and calm QA 
• Happy and brave developers
Questions? 
Niels Frydenholm 
@nfrydenholm

More Related Content

What's hot

Story Testing Approach for Enterprise Applications using Selenium Framework
Story Testing Approach for Enterprise Applications using Selenium FrameworkStory Testing Approach for Enterprise Applications using Selenium Framework
Story Testing Approach for Enterprise Applications using Selenium FrameworkOleksiy Rezchykov
 
Getting Predictable - Pragmatic Approach for Mobile Development - Devday.lk ...
Getting Predictable - Pragmatic Approach for Mobile Development  - Devday.lk ...Getting Predictable - Pragmatic Approach for Mobile Development  - Devday.lk ...
Getting Predictable - Pragmatic Approach for Mobile Development - Devday.lk ...Anjana Somathilake
 
Automated Acceptance Tests in .NET
Automated Acceptance Tests in .NETAutomated Acceptance Tests in .NET
Automated Acceptance Tests in .NETWyn B. Van Devanter
 
Building High Quality Android Applications
Building High Quality Android ApplicationsBuilding High Quality Android Applications
Building High Quality Android ApplicationsLeif Janzik
 
Calabash my understanding
Calabash my understandingCalabash my understanding
Calabash my understandingFARHAN SUMBUL
 
Fashionable XCUITest for iOS Apps by Shashikant Jagtap
Fashionable XCUITest for iOS Apps by Shashikant JagtapFashionable XCUITest for iOS Apps by Shashikant Jagtap
Fashionable XCUITest for iOS Apps by Shashikant JagtapSauce Labs
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1Qualitest
 
Calabash Andoird + Calabash iOS
Calabash Andoird + Calabash iOSCalabash Andoird + Calabash iOS
Calabash Andoird + Calabash iOSAnadea
 
API Design - developing for developers
API Design - developing for developersAPI Design - developing for developers
API Design - developing for developersJoy George
 
Integration Testing with Selenium
Integration Testing with SeleniumIntegration Testing with Selenium
Integration Testing with SeleniumAll Things Open
 
Practical Patterns for Developing a Cross-product Cross-version App
Practical Patterns for Developing a Cross-product Cross-version AppPractical Patterns for Developing a Cross-product Cross-version App
Practical Patterns for Developing a Cross-product Cross-version AppAtlassian
 
Chris Mathias Presents Advanced API Design Considerations at LA CTO Forum
Chris Mathias Presents Advanced API Design Considerations at LA CTO ForumChris Mathias Presents Advanced API Design Considerations at LA CTO Forum
Chris Mathias Presents Advanced API Design Considerations at LA CTO ForumChris Mathias
 
Using The Page Object Pattern
Using The Page Object PatternUsing The Page Object Pattern
Using The Page Object PatternDante Briones
 
Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...
Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...
Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...Mozaic Works
 
SauceCon 2017: Testing @ the Speed of Concurrency
SauceCon 2017: Testing @ the Speed of ConcurrencySauceCon 2017: Testing @ the Speed of Concurrency
SauceCon 2017: Testing @ the Speed of ConcurrencySauce Labs
 
Getting started with Appium 2.0
Getting started with Appium 2.0Getting started with Appium 2.0
Getting started with Appium 2.0Anand Bagmar
 
Effective Testing with Ruby
Effective Testing with RubyEffective Testing with Ruby
Effective Testing with RubyAkira Sosa
 
E-TAIL QA: Approach to E-commerce testing in an Agile environment
E-TAIL QA: Approach to E-commerce testing in an Agile environmentE-TAIL QA: Approach to E-commerce testing in an Agile environment
E-TAIL QA: Approach to E-commerce testing in an Agile environmentNurun
 

What's hot (20)

Story Testing Approach for Enterprise Applications using Selenium Framework
Story Testing Approach for Enterprise Applications using Selenium FrameworkStory Testing Approach for Enterprise Applications using Selenium Framework
Story Testing Approach for Enterprise Applications using Selenium Framework
 
Getting Predictable - Pragmatic Approach for Mobile Development - Devday.lk ...
Getting Predictable - Pragmatic Approach for Mobile Development  - Devday.lk ...Getting Predictable - Pragmatic Approach for Mobile Development  - Devday.lk ...
Getting Predictable - Pragmatic Approach for Mobile Development - Devday.lk ...
 
Automated Acceptance Tests in .NET
Automated Acceptance Tests in .NETAutomated Acceptance Tests in .NET
Automated Acceptance Tests in .NET
 
Building High Quality Android Applications
Building High Quality Android ApplicationsBuilding High Quality Android Applications
Building High Quality Android Applications
 
Calabash my understanding
Calabash my understandingCalabash my understanding
Calabash my understanding
 
Fashionable XCUITest for iOS Apps by Shashikant Jagtap
Fashionable XCUITest for iOS Apps by Shashikant JagtapFashionable XCUITest for iOS Apps by Shashikant Jagtap
Fashionable XCUITest for iOS Apps by Shashikant Jagtap
 
Introduction to SoapUI day 1
Introduction to SoapUI day 1Introduction to SoapUI day 1
Introduction to SoapUI day 1
 
Calabash Andoird + Calabash iOS
Calabash Andoird + Calabash iOSCalabash Andoird + Calabash iOS
Calabash Andoird + Calabash iOS
 
API Design - developing for developers
API Design - developing for developersAPI Design - developing for developers
API Design - developing for developers
 
Angular2 Upgrade
Angular2 UpgradeAngular2 Upgrade
Angular2 Upgrade
 
Integration Testing with Selenium
Integration Testing with SeleniumIntegration Testing with Selenium
Integration Testing with Selenium
 
Practical Patterns for Developing a Cross-product Cross-version App
Practical Patterns for Developing a Cross-product Cross-version AppPractical Patterns for Developing a Cross-product Cross-version App
Practical Patterns for Developing a Cross-product Cross-version App
 
How to define an api
How to define an apiHow to define an api
How to define an api
 
Chris Mathias Presents Advanced API Design Considerations at LA CTO Forum
Chris Mathias Presents Advanced API Design Considerations at LA CTO ForumChris Mathias Presents Advanced API Design Considerations at LA CTO Forum
Chris Mathias Presents Advanced API Design Considerations at LA CTO Forum
 
Using The Page Object Pattern
Using The Page Object PatternUsing The Page Object Pattern
Using The Page Object Pattern
 
Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...
Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...
Aki Salmi - Refactoring legacy code: a true story @ I T.A.K.E. Unconference 2...
 
SauceCon 2017: Testing @ the Speed of Concurrency
SauceCon 2017: Testing @ the Speed of ConcurrencySauceCon 2017: Testing @ the Speed of Concurrency
SauceCon 2017: Testing @ the Speed of Concurrency
 
Getting started with Appium 2.0
Getting started with Appium 2.0Getting started with Appium 2.0
Getting started with Appium 2.0
 
Effective Testing with Ruby
Effective Testing with RubyEffective Testing with Ruby
Effective Testing with Ruby
 
E-TAIL QA: Approach to E-commerce testing in an Agile environment
E-TAIL QA: Approach to E-commerce testing in an Agile environmentE-TAIL QA: Approach to E-commerce testing in an Agile environment
E-TAIL QA: Approach to E-commerce testing in an Agile environment
 

Viewers also liked

Kvalitetssikring i et highperformance team
Kvalitetssikring i et highperformance teamKvalitetssikring i et highperformance team
Kvalitetssikring i et highperformance teamNiels Frydenholm
 
Behavior driven development - cucumber, Junit and java
Behavior driven development - cucumber, Junit and javaBehavior driven development - cucumber, Junit and java
Behavior driven development - cucumber, Junit and javaNaveen Kumar Singh
 
Generate Leads With Your Resource Center
Generate Leads With Your Resource CenterGenerate Leads With Your Resource Center
Generate Leads With Your Resource CenterUberflip
 
If the green-eyed giant were more open-minded...
If the green-eyed giant were more open-minded...If the green-eyed giant were more open-minded...
If the green-eyed giant were more open-minded...Sarinee Achavanuntakul
 
『クチコミの技術』におけるクチコミの実践
『クチコミの技術』におけるクチコミの実践『クチコミの技術』におけるクチコミの実践
『クチコミの技術』におけるクチコミの実践masaki ishitani
 
Creating Cultural Agility -The Impact study
Creating Cultural Agility -The Impact study Creating Cultural Agility -The Impact study
Creating Cultural Agility -The Impact study Diversity Best Practices
 
Lyndon Galea - Eat Up - SSE Australia
Lyndon Galea - Eat Up - SSE AustraliaLyndon Galea - Eat Up - SSE Australia
Lyndon Galea - Eat Up - SSE AustraliaLyndon Galea
 
INTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOS
INTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOSINTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOS
INTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOSFEDERICO ALMENARA CHECA
 
中国软实力与新外交
中国软实力与新外交中国软实力与新外交
中国软实力与新外交dario.b
 
μείωση τιμών νεοεισερχομένων φβ
μείωση τιμών νεοεισερχομένων φβμείωση τιμών νεοεισερχομένων φβ
μείωση τιμών νεοεισερχομένων φβLycourgos Papayannopoulos
 
Planificacion de naturales para el viernes
Planificacion de naturales para el viernesPlanificacion de naturales para el viernes
Planificacion de naturales para el viernesmarcialuciazaracho
 
Fail Conference 2012: Nick Decrock
Fail Conference 2012: Nick DecrockFail Conference 2012: Nick Decrock
Fail Conference 2012: Nick Decrockimec
 
How to Crank Up Your Content Marketing Career
How to Crank Up Your Content Marketing CareerHow to Crank Up Your Content Marketing Career
How to Crank Up Your Content Marketing CareerLinkedIn
 
Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión
Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión
Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión John Molina
 
Lady be good chords nate lee
Lady be good chords   nate leeLady be good chords   nate lee
Lady be good chords nate leefiddlestar
 
Diversity and Inclusion in Community Building by Erica McGillivray
Diversity and Inclusion in Community Building by Erica McGillivrayDiversity and Inclusion in Community Building by Erica McGillivray
Diversity and Inclusion in Community Building by Erica McGillivrayErica McGillivray
 

Viewers also liked (20)

Kvalitetssikring i et highperformance team
Kvalitetssikring i et highperformance teamKvalitetssikring i et highperformance team
Kvalitetssikring i et highperformance team
 
Behavior driven development - cucumber, Junit and java
Behavior driven development - cucumber, Junit and javaBehavior driven development - cucumber, Junit and java
Behavior driven development - cucumber, Junit and java
 
Generate Leads With Your Resource Center
Generate Leads With Your Resource CenterGenerate Leads With Your Resource Center
Generate Leads With Your Resource Center
 
If the green-eyed giant were more open-minded...
If the green-eyed giant were more open-minded...If the green-eyed giant were more open-minded...
If the green-eyed giant were more open-minded...
 
『クチコミの技術』におけるクチコミの実践
『クチコミの技術』におけるクチコミの実践『クチコミの技術』におけるクチコミの実践
『クチコミの技術』におけるクチコミの実践
 
Creating Cultural Agility -The Impact study
Creating Cultural Agility -The Impact study Creating Cultural Agility -The Impact study
Creating Cultural Agility -The Impact study
 
Lyndon Galea - Eat Up - SSE Australia
Lyndon Galea - Eat Up - SSE AustraliaLyndon Galea - Eat Up - SSE Australia
Lyndon Galea - Eat Up - SSE Australia
 
INTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOS
INTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOSINTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOS
INTRODUCCIÓN A LA BIBLIA. LA BIBLIA PALABRA DE DIOS
 
中国软实力与新外交
中国软实力与新外交中国软实力与新外交
中国软实力与新外交
 
μείωση τιμών νεοεισερχομένων φβ
μείωση τιμών νεοεισερχομένων φβμείωση τιμών νεοεισερχομένων φβ
μείωση τιμών νεοεισερχομένων φβ
 
Planificacion de naturales para el viernes
Planificacion de naturales para el viernesPlanificacion de naturales para el viernes
Planificacion de naturales para el viernes
 
イントロ
イントロイントロ
イントロ
 
Fail Conference 2012: Nick Decrock
Fail Conference 2012: Nick DecrockFail Conference 2012: Nick Decrock
Fail Conference 2012: Nick Decrock
 
How to Crank Up Your Content Marketing Career
How to Crank Up Your Content Marketing CareerHow to Crank Up Your Content Marketing Career
How to Crank Up Your Content Marketing Career
 
Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión
Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión
Microbiologia - Mecanismos de Evasión de las defensas del organismo anfitrión
 
ETIQUETA SECRETARIAL
ETIQUETA SECRETARIALETIQUETA SECRETARIAL
ETIQUETA SECRETARIAL
 
TUe 2012
TUe 2012TUe 2012
TUe 2012
 
Lady be good chords nate lee
Lady be good chords   nate leeLady be good chords   nate lee
Lady be good chords nate lee
 
Diversity and Inclusion in Community Building by Erica McGillivray
Diversity and Inclusion in Community Building by Erica McGillivrayDiversity and Inclusion in Community Building by Erica McGillivray
Diversity and Inclusion in Community Building by Erica McGillivray
 
EdCrunch
EdCrunchEdCrunch
EdCrunch
 

Similar to Getting your mobile test automation process in place - using Cucumber and Calabash

Continuous delivery is more than dev ops
Continuous delivery is more than dev opsContinuous delivery is more than dev ops
Continuous delivery is more than dev opsAgile Montréal
 
Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...
Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...
Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...Daniel Gallego Vico
 
Badoo: Cross platform Mobile Test Automation and Continuos Delivery
Badoo: Cross platform Mobile Test Automation and Continuos DeliveryBadoo: Cross platform Mobile Test Automation and Continuos Delivery
Badoo: Cross platform Mobile Test Automation and Continuos DeliveryCodeFest
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsAmazon Web Services
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsMahesh Arali
 
Dev ops ci-ap-is-oh-my_security-gone-agile_ut-austin
Dev ops ci-ap-is-oh-my_security-gone-agile_ut-austinDev ops ci-ap-is-oh-my_security-gone-agile_ut-austin
Dev ops ci-ap-is-oh-my_security-gone-agile_ut-austinMatt Tesauro
 
Behaviour driven infrastructure
Behaviour driven infrastructureBehaviour driven infrastructure
Behaviour driven infrastructureLindsay Holmwood
 
Proactive Security AppSec Case Study
Proactive Security AppSec Case StudyProactive Security AppSec Case Study
Proactive Security AppSec Case StudyAndy Hoernecke
 
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...Sauce Labs
 
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source ToolsTYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source ToolsMichael Lihs
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsAmazon Web Services
 
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfullyTEST Huddle
 
Raj Subramanian - Mobile Web Testing
Raj Subramanian - Mobile Web TestingRaj Subramanian - Mobile Web Testing
Raj Subramanian - Mobile Web TestingQA or the Highway
 
Continuous delivery @wcap 5-09-2013
Continuous delivery   @wcap 5-09-2013Continuous delivery   @wcap 5-09-2013
Continuous delivery @wcap 5-09-2013David Funaro
 
Testing for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration MondayTesting for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration MondayBizTalk360
 
Continuous Integration, the minimum viable product
Continuous Integration, the minimum viable productContinuous Integration, the minimum viable product
Continuous Integration, the minimum viable productJulian Simpson
 
Dev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDavide Benvegnù
 
Driving application development through behavior driven development
Driving application development through behavior driven developmentDriving application development through behavior driven development
Driving application development through behavior driven developmentEinar Ingebrigtsen
 
Making security-agile matt-tesauro
Making security-agile matt-tesauroMaking security-agile matt-tesauro
Making security-agile matt-tesauroMatt Tesauro
 

Similar to Getting your mobile test automation process in place - using Cucumber and Calabash (20)

Continuous delivery is more than dev ops
Continuous delivery is more than dev opsContinuous delivery is more than dev ops
Continuous delivery is more than dev ops
 
Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...
Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...
Droidcon Spain 2105 - One app to rule them all: Methodologies, Tools & Tricks...
 
Badoo: Cross platform Mobile Test Automation and Continuos Delivery
Badoo: Cross platform Mobile Test Automation and Continuos DeliveryBadoo: Cross platform Mobile Test Automation and Continuos Delivery
Badoo: Cross platform Mobile Test Automation and Continuos Delivery
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
 
Dev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teamsDev ops for mobile apps at microsoft teams
Dev ops for mobile apps at microsoft teams
 
Dev ops ci-ap-is-oh-my_security-gone-agile_ut-austin
Dev ops ci-ap-is-oh-my_security-gone-agile_ut-austinDev ops ci-ap-is-oh-my_security-gone-agile_ut-austin
Dev ops ci-ap-is-oh-my_security-gone-agile_ut-austin
 
Behaviour driven infrastructure
Behaviour driven infrastructureBehaviour driven infrastructure
Behaviour driven infrastructure
 
Proactive Security AppSec Case Study
Proactive Security AppSec Case StudyProactive Security AppSec Case Study
Proactive Security AppSec Case Study
 
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
 
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source ToolsTYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
 
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfully
 
Raj Subramanian - Mobile Web Testing
Raj Subramanian - Mobile Web TestingRaj Subramanian - Mobile Web Testing
Raj Subramanian - Mobile Web Testing
 
DevOps on AWS
DevOps on AWSDevOps on AWS
DevOps on AWS
 
Continuous delivery @wcap 5-09-2013
Continuous delivery   @wcap 5-09-2013Continuous delivery   @wcap 5-09-2013
Continuous delivery @wcap 5-09-2013
 
Testing for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration MondayTesting for Logic App Solutions | Integration Monday
Testing for Logic App Solutions | Integration Monday
 
Continuous Integration, the minimum viable product
Continuous Integration, the minimum viable productContinuous Integration, the minimum viable product
Continuous Integration, the minimum viable product
 
Dev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps worldDev/Test scenarios in DevOps world
Dev/Test scenarios in DevOps world
 
Driving application development through behavior driven development
Driving application development through behavior driven developmentDriving application development through behavior driven development
Driving application development through behavior driven development
 
Making security-agile matt-tesauro
Making security-agile matt-tesauroMaking security-agile matt-tesauro
Making security-agile matt-tesauro
 

Recently uploaded

Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Incrobinwilliams8624
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 

Recently uploaded (20)

Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Inc
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 

Getting your mobile test automation process in place - using Cucumber and Calabash

  • 1. Get Your Testing Process in Place Niels Frydenholm Senior iOS developer nfrydenholm@ebay.com
  • 2. Get Your Testing Process in Place Karl Krukow Technical Lead,, Xamarin Test Cloud @karlkrukow
  • 4. “Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time.” Martin Fowler ThoughtWorks
  • 6. Mobile Continuous Delivery Challenges • App Store Distribution Model •Maturity - No Industry Standards (yet) • Complicated Tool Setup • Realistic tests are challenging • Tested on your users’ devices • Vary Device Conditions - Device settings, Network…
  • 7. Towards Continuous Delivery for Mobile Aspects Tooling People Process
  • 8. Towards Continuous Delivery for Mobile Aspects Examples Tooling People Process • Xamarin.UITest, Calabash • Jenkins, TeamCity, TFS, … • Xamarin Test Cloud
  • 9. Towards Continuous Delivery for Mobile Aspects Examples Tooling People Process • Xamarin.UITest, Calabash • Jenkins, TeamCity, TFS, … • Xamarin Test Cloud • Cross-functional Teams • Behaviour Driven Development • Shared responsibility
  • 11. Xamarin.UITest & Calabash Xamarin.UITest • Write Tests in C# • Run with NUnit • Xamrin/Visual Studio or CLI Calabash • Write Tests in Ruby • Supports BDD, run with Cucumber • RubyMine or CLI
  • 12. Xamarin.UITest - Architecture Your App Test Cloud Agent
  • 13. Calabash - Architecture Your App Test Cloud Agent ! Scenario: Login Given I am on the Login screen When I login as "Nat" Then I should go to the Assignments
  • 14. Calabash - Architecture Your App Test Cloud Agent Cucumber ! Scenario: Login Given I am on the Login screen When I login as "Nat" Then I should go to the Assignments
  • 15. Calabash - Architecture Your App Test Cloud Agent Cucumber ! Scenario: Login Given I am on the Login screen When I login as "Nat" Then I should go to the Assignments
  • 16. Get Your Testing Process in Place Niels Frydenholm Senior iOS developer nfrydenholm@ebay.com
  • 17. Agenda ■ From story to running tests ■ Improvements in development and release cycle ■ Demo ■ Best practices (based on our learnings) ■ Final words ■ Questions?
  • 18. eBay Classifieds, Denmark Two popular consumer apps and sites ■ DBA - DK’s #1 Classifieds site ■ BilBasen - DK’s #1 Online Vehicle Marketplace A few numbers ■ Approx. 1/4 of the danish population is using the DBA app ■ Mobile traffic share is 57% for DBA ■ Approx 1/8 of the danish population is using the BilBasen app ■ Mobile traffic share is 54% for BilBasen
  • 19. How we are organised The team ■ 4 iOS developers / (backend) ■ 1 backend developer ■ 1 QA expert ■ 1 Scrum master
  • 20. From Story to Running Tests
  • 21. Implementing a story Before sprint ■ Product owner (PO) introduces story ■ Team and PO agrees on accept criteria’s New feature idea Lots of questions
  • 22. Implementing a story Before sprint ■ Product owner (PO) introduces story ■ Team and PO agrees on accept criteria’s New feature idea Lots of questions Implement feature Write tests Review Test In Sprint ■ Developer(s) implements story and tests ■ Unit, Integration and UI tests ■ Done with code and tests are green ■ Review code (and tests) ■ Manuel test from other team member ■ Inspect code coverage to ensure test quality
  • 23. Continuous Integration ■ Jenkins on a couple of Mac Mini’s - Unit -> Integration -> UI tests - UI tests only in Simulator ■ A subset after commit ■ A subset in Xamarin Cloud ■ All UI tests nightly ! ■ Keep status visible and Green!
  • 24. Focus on Automated Tests Has Improved ‘Everything’
  • 25. Releases 2013 DBA versions JAN 2.0 2.0.1 2.1 2.2 2.3 2.3.1 2.4 3.0 FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC BilBasen versions 4.2 5.0 5.0.1 5.0.2 Refactored tests to Page objects 4.1.2 4.1.3
  • 26. 3.3.1 3.5 3.2 3.3 3.4.1 3.0.1 3.1 3.2.1 3.4 3.4.2 FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 5.0.4 5.0.5 5.0.6 5.1 5.2 5.2.2 5.2.1 3.2.3 Releases 2014 DBA versions 3.1.1 3.2.2 JAN 5.0.3 BilBasen versions
  • 27. Development improvements In early 2013 • Limited refactoring - risk too high • All changes = huge test effort •Many bugs were not found until later… • Lots of “legacy code” • See first bullet
  • 28. Development improvements In early 2013 Now • Limited refactoring - risk too high • All changes = huge test effort •Many bugs were not found until later… • Lots of “legacy code” • See first bullet • Planned refactoring of larger areas • Boy scout clean up • Adding tests to missing areas • (Most) bugs are found immediately after commit • QA can spend time on better things ■ Identify critical business areas ■ Find the crazy bugs
  • 29. Project example BilBasen - new search ■ Search criteria pages and search logic was totally re-written ■ From hardcoded options in the app ■ To meta driven from the server ■ Server side meta data was build as part of a project ■ Saved searches now in sync with web
  • 30. BilBasen - new search Test results ■ Most scenarios were already covered with UI tests ■ Manuel test effort ■ About one day of testing ■ Focused on network timings ■ Compare results with website ■ No serious bugs were found
  • 31. Release improvements In early 2013 • Entire team spent 2-5 days on test runs and bug fixes • Serious bugs could be found in last minute • Big releases to save the overhead • Releases were often delayed
  • 32. Release improvements In early 2013 Now • Entire team spent 2-5 days on test runs and bug fixes • Serious bugs could be found in last minute • Big releases to save the overhead • Releases were often delayed • Upload when all tests are green (all the time!) • PO flexibility to change scope of release • 1 hour test run for some releases ■ Bugs are very rarely found
  • 33. Demo
  • 34. Best Practices (based on our learnings)
  • 35. Best Practices • Structure your test code well • Page objects • Replace “sleeps” with “wait_for”
  • 36. Page objects and wait_for Before page objects (Ruby example) Scenario: Check that change classification clears matrixdata with warning Given I am logged in as "Buyer" And I am on the SYI hub And I select classification "Hovedtelefoner" And I set "Type" to “In-ear” And I set price to "250" !
  • 37. Page objects and wait_for Before page objects (Ruby example) Scenario: Check that change classification clears matrixdata with warning Given I am logged in as "Buyer" And I am on the SYI hub And I select classification "Hovedtelefoner" And I set "Type" to “In-ear” And I set price to "250" ! And /^I set price to "(.*?)"$/ do |price| macro 'I swipe up' sleep(1) touch("view marked:'Price'") sleep(0.5) set_text("view marked:'Price'", price) sleep(0.5) touch("view marked:'OK'") sleep(1) end
  • 38. Page object and wait_for With page objects (Ruby example) And /^I set price to "(.*?)"$/ do |price| @page.write_price(price) end
  • 39. Page object and wait_for With page objects (Ruby example) And /^I set price to "(.*?)"$/ do |price| @page.write_price(price) end SellYourItemPage class (page object) def write_price(price) scroll_and_wait_for_row_with_mark("priceCell") touch("view marked:'Price'") keyboard_enter_text price close_keyboard end def select_bundle(bundle_name) ..logic here.. end ! def write_headline(headline) ..logic here.. end
  • 40. Best Practices • Structure your test code well • Page objects • replace “sleeps” with “wait_for” • Ensure stable test suite execution • Give failed tests an extra run
  • 41. Stability in your CI setup • UIAutomation + Simulator can often fail when running a big test suite • Re-run failed tests an extra time • Cucumber rerun formatter • NUnit-retry • Red builds should be trusted, investigated and fixed • Only use this if test suite starts to be unstable
  • 42. Stability in your CI setup • UIAutomation + Simulator can often fail when running a big test suite • Re-run failed tests an extra time • Cucumber rerun formatter • NUnit-retry • Red builds should be trusted, investigated and fixed • Only use this if test suite starts to be unstable
  • 43. Best Practices • Structure your test code well • Page objects • replace “sleeps” with “wait_for” • Ensure stable test suite execution • Give failed tests an extra • Run tests on real devices • Xamarin Test Cloud • Screen sizes, OS versions
  • 45. Best Practices • Structure your test code well • Page objects • replace “sleeps” with “wait_for” • Ensure stable test suite execution • Give failed tests an extra run • Run tests on real devices • Xamarin Test Cloud • Screen sizes, OS versions • Strive for fast test execution • Use different test suites • Avoid too much UI interaction for setup/teardown operations
  • 46. Fast test execution • Use the “Backdoor” to the app • Xamarin.UITest.IApp.Invoke • Invoke(“method”, param) • Invoke logic - without using the UI • Great for Setup / Teardown behaviour • Backdoor logic should be tested with UI interactions in other tests
  • 47. Fast test execution • Use the “Backdoor” to the app • Xamarin.UITest.IApp.Invoke • Invoke(“method”, param) • Invoke logic - without using the UI • Great for Setup / Teardown behaviour • Backdoor logic should be tested with UI interactions in other tests •We use it for things like • Create a user / Login • Create listings (things for sale) • Cleaning up • Delete favorites • Delete users listings
  • 48. Fast test execution - backdoor optimised Scenario: I can manage my listing and see the changes Given I am logged in as "UniqueSeller" using quick login And I have created a listing for "Hovedtelefoner" and is on the SYI VIP Then I see the VIP for "Her kommer en rimelig lang tekst" ! When I go back to my listings page And the created listing is in the list …more steps omitted…
  • 49. Fast test execution - backdoor optimised Scenario: I can manage my listing and see the changes Given I am logged in as "UniqueSeller" using quick login And I have created a listing for "Hovedtelefoner" and is on the SYI VIP Then I see the VIP for "Her kommer en rimelig lang tekst" ! When I go back to my listings page And the created listing is in the list …more steps omitted… 1 scenario (1 passed) 19 steps (19 passed) 0m34.054s 1 scenario (1 passed) 19 steps (19 passed) 1m16.610s
  • 50. Best Practices • Structure your test code well • Page objects • replace “sleeps” with “wait_for” • Ensure stable test suite execution • Give failed tests an extra run • Run tests on real devices • Xamarin Test Cloud • Screen sizes, OS versions • Strive for fast test execution • Use different test suites • Avoid too much UI interaction for setup/teardown operations • Run tests for all active branches in CI • Automate job creation
  • 51. Feature builds ■ All jobs required for a branch ■ Red builds are accepted for a period ■ Only merge to master with green builds
  • 53. Start today… • QA is a Team effort / responsibility • Bugs/Crashes will still live in the app • Do not expect it to be effortless to get started • It will quickly pay off
  • 54. Start today… • QA is a Team effort / responsibility • Bugs/Crashes will still live in the app • Do not expect it to be effortless to get started • It will quickly pay off See it as an investment that gives you • Higher app quality (and code quality) • Time to focus on new important features • Faster time to market - with less bugs • Happy customers - they like frequent updates of high quality • Happy and calm QA • Happy and brave developers