This document discusses how to define a mobile test automation strategy. It covers deciding when to automate tests, challenges of mobile testing like device diversity, and approaches for automation including record and playback, behavior driven development, and keyword driven testing. It also discusses continuous integration, selecting tools, and providing an abstraction layer approach to a test automation framework.

1. How to Define Mobile Test
Automation Strategy
Serhat BOLSU – Selin GUNGOR – Erol SELITEKTAY

2. Selin Güngör
Software Test Engineer
ÇiçekSepeti.com / Bright E-Ventures

3. Serhat Bolsu
Service & QA Manager
tmob
References

4. Erol Selitektay
Quality Assurance Engineer
VNGRS

6. Outline
Decision to Automate
Mobile Challenges
Demo
Mobile Automation Approaches
Continuous Integration
A Fine Approach

7. Decision to Automate
Aim of Automation
Reduce Regression Cost
Device Support
Better Quality
Early Detection
3500
3000
2500
2000
1500
1000
500
0
1 2 3 4 5 6 7 8 9 10 11
Android/iOS
Platform 2x
Phones Manual
Android/iOS
Platform 2x
Phones
Automated
900
800
700
600
500
400
300
200
100
0
1 2 3 4 5 6 7 8 9 10 11
Thousands
Manual 1
Platform
Automated Multi
Platform

8. Decision to Automate
What to Automate
Stable Version
Test Types
• Regression Testing
• Smoke Test
• Performance & Load Testing
Regression Testing
Performance & Load Testing Smoke Testing

9. Decision to Automate
Profiles

10. Mobile Challenges
Device Diversity Multi Platform Hardware
Dependencies
Network
1112. 8d6if3fe Dreisnttin Acnt dArnodidro viedr Dsioevnisc essu pbpyo 2r0te1d3 boyn Gthoeo mglea r ket.

11. Mobile Challenges
App-Behaviour
1 1
2

12. Easy to Debug
Ideal Case
Automation
Requirements
Support Integration
to Third Party Libraries
Easy to Use
Sustainable
Maintainable
Human Readable
Support CI Tools

13. Test automation is a software development process.
Test automation projects often fail because of a lack of
discipline and project management. Many testers don’t realize
that when they automate tests they’re actually developing
software.
Lessons Learned in Software Testing
Authors: Cem Kaner, James Bach and Bret Pettichord

14. Automation is Software Development
login();
searchItem();
addToBasket();
buyWithMoneyOrder();

15. Automation is Software Development
//Login
driver.driver.get("get("http://http://somewebsite.somewebsite.com");
com");
WebElement WebElement signIn signIn = driver.= driver.findElement(findElement(By.linkText("By.SIGN linkText("IN").click();
SIGN IN").click();
WebElement email= driver.findElement(By.id("email")).sendKeys("someemail@address.com");
WebElement WebElement password email= = driver.driver.findElement(findElement(By.id("By.password")).id("email")).sendKeys("sendKeys("somepassword");
someemail@address.com");
WebElement WebElement submitButton password = = driver.driver.findElement(findElement(By.id("By.submit_id("password")).form")).click();
sendKeys("somepassword");
WebElement Wait<WebDriver> submitButton wait = new WebDriverWait(= driver.findElement(driver, 10);
By.id("submit_form")).click();
//WebElement Search resim Item
=wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("img[alt='Dracena Massengena İkili']")));
WebElement city = wait.until(visibilityOfElementLocated(By.className("select2-choice"))).click();
Wait<WebElement WebDriver> dist = wait.wait until(= ExpectedConditions.new WebDriverWait(visibilityOfElementLocated(driver, 10);
By.className("select2-default"))).click();
WebElement WebElement distbox resim = driver.=wait.findElement(until(ExpectedConditions.By.className("select2-visibilityOfElementLocated(focused")).sendKeys("AKSARAY");
By.cssSelector("img[alt='Dracena Massengena İkili']")));
WebElement cldist = driver.findElement(By.className("select2-result-label")).click(),
//WebElement Add to picerd Basket
= driver.findElement(By.id("prdatepicker")).click();
WebElement WebElement selectd city = = driver.wait.until(findElement(visibilityOfElementLocated(By.linkText("30")).click();
By.className("select2-choice"))).click();
WebElement AddBasket=driver.findElement(By.id("addToBasket")).click();
WebElement dist = wait.until(ExpectedConditions.visibilityOfElementLocated(By.className("select2-default"))).click();
WebElement SatinAl = wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[@id='Basket']/div[2]/div[2]/input"))).click();
WebElement WebElement EkUrun distbox = wait.= until(driver.ExpectedConditions.findElement(By.visibilityOfElementLocated(className("select2-focused")).By.xpath("(//button[@sendKeys("type='AKSARAY");
button'])[5]"))).click();
WebElement cldist = driver.findElement(By.className("select2-result-label")).click(),
WebElement picerd = driver.findElement(By.id("prdatepicker")).click();
WebElement selectd = driver.findElement(By.linkText("30")).click();
WebElement AddBasket=driver.findElement(By.id("addToBasket")).click();
//Buy with Money Order
WebElement SatinAl = wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[@id='Basket']/div[2]/div[2]/input"))).click();
WebElement EkUrun = wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("(//button[@type='button'])[5]"))).click();
public LoginPage login(){
driver.get("http://somewebsite.com");
WebElement signIn = driver.findElement(By.linkText("SIGN IN").click();
WebElement email= driver.findElement(By.id("email")).sendKeys("someemail@address.com");
WebElement password = driver.findElement(By.id("password")).sendKeys("somepassword");
WebElement submitButton = driver.findElement(By.id("submit_form")).click();
return new HomePage(driver)
}
login();
public SearchPage searchItem(){
.
.
.
searchItem();
ilerle1=driver.findElement(By.cssSelector("button.goNext.sendAddition")).click();
}
public ProductDetailPage addBasket(){
.
.
.
}
public OrderPage buyWithMoneyOrder(){
.
.
.
}
WebElement UyeOlmaDevam= wait.until(ExpectedConditions.visibilityOfElementLocated(By.linkText("Üye olmadan devam et"))).click();
WebElement AliciName= wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("OrderProducts_1__ReceiverModel_OrderReceiver_NameRec"))).sendKeys("Selenium
Test ");
WebElement WebElement addToBasket();
AliciTel=driver.findElement(By.xpath("//*[@id='OrderProducts_1__ReceiverModel_OrderReceiver_PhoneRec']")).sendKeys("11111111111111111111");
AliciAdres = driver.findElement(By.id("OrderProducts_1__ReceiverModel_OrderReceiver_AddressLineRec")).sendKeys("Selenium Test Adress");
WebElement ilerle2= driver.findElement(By.cssSelector("button.goNext.sendRec")).click();
WebElement ilerle3 = wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("button.goNext.sendCardMessage"))).click();
WebElement FaturaAd = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("InvoiceInfoViewModel_Order_SenderName"))).sendKeys("Selenium Test Fatura Ad");
WebElement FaturaTel ilerle1== driver.driver.findElement(findElement(By.id("By.InvoiceInfoViewModel_cssSelector("button.Order_goNext.SenderPhone")).sendAddition")).sendKeys("click();
111111111111");
WebElement buyWithMoneyOrder();
FaturaEmail=driver.findElement(By.id("InvoiceInfoViewModel_Email")).sendKeys("erol.selitektay@gmail.com");
WebElement OdemeSec= driver.findElement(By.cssSelector("p.billOther")).click();
WebElement Havale = wait.until(ExpectedConditions.visibilityOfElementLocated(By.linkText("Havale / EFT ile Ödeme"))).click();
WebElement BankaSec = wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("span.selectBox-label"))).click();
WebElement UyeOlmaDevam= wait.until(ExpectedConditions.visibilityOfElementLocated(By.linkText("Üye olmadan devam et"))).click();
WebElement AliciName= wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("OrderProducts_1__ReceiverModel_OrderReceiver_NameRec"))).sendKeys("Selenium
Test ");
WebElement garanti AliciTel== driver.driver.findElement(findElement(By.xpath("By.html/xpath("//*[@body/ul/li[2]/id='a")).OrderProducts_click();
1__ReceiverModel_OrderReceiver_PhoneRec']")).sendKeys("11111111111111111111");
WebElement WebElement SiparisVer AliciAdres = driver.= driver.findElement(findElement(By.xpath("//By.input[@id("OrderProducts_value='Sipariş Ver']")).1__click();
ReceiverModel_OrderReceiver_AddressLineRec")).sendKeys("Selenium Test Adress");
WebElement String sipNo=driver.ilerle2= findElement(driver.By.findElement(xpath("//*[@id='By.wrapper']/cssSelector("div[3]/div/button.div/div/div[goNext.1]/p[2]/sendRec")).span")).getText();
click();
System.out.println("Sipariş Numarası: "+ sipNo);
WebElement driver.quit();
ilerle3 = wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("button.goNext.sendCardMessage"))).click();
WebElement FaturaAd = wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("InvoiceInfoViewModel_Order_SenderName"))).sendKeys("Selenium Test Fatura Ad");
WebElement FaturaTel = driver.findElement(By.id("InvoiceInfoViewModel_Order_SenderPhone")).sendKeys("111111111111");
WebElement FaturaEmail=driver.findElement(By.id("InvoiceInfoViewModel_Email")).sendKeys("erol.selitektay@gmail.com");

16. Demo
DEMO TIME
http://youtu.be/fJxgWwAMwbM

17. http://youtu.be/fJxgWwAMwbM

18. Mobile Automation Approaches
Record & Play
Record -> Edit -> Play -> Edit -> Add Verify -> Edit and so on...
Good for getting to know technology
none of these tests will be maintainable
none of these tests will be sustainable

19. Mobile Automation Approaches
Behaviour Driven Development
#1 USER STORIES describe BUSINESS VALUE
#2 SCENARIOS help describe ACCEPTANCE CRITERIA
#3 SCENARIOS act as AUTOMATED TESTS

20. Mobile Automation Approaches
Behaviour Driven Development
BDD is good for :
BDD is not ideal for :

21. Mobile Automation Approaches
Keyword Driven Testing
keyword :
1) A function or method that can be used to test one aspect of the SUT
2) Can be defined by other keywords
Higher-level keywords:
a concrete aspect of the business logic
Lower-level keywords:
to keep the implementation of the higher-level keywords at a decent size
Technical keywords:
provide the technical implementation to access and thus test the system.
By codeCentric

22. Mobile Automation Approaches
BDD vs Keyword-Driven Testing
Requirement: Allow customer to choose a round or square shaped pizza
BDD: Given the customer logged into the pizza build app.
When the customer choose to build a square shaped pizza,
Then customer should be able to get a square shaped pizza,
When the customer choose to build a round shaped pizza,
Then customer should be able to get a round shaped pizza.
KDT: Login
Build pizza | square
Verify pizza is built
Build pizza | round
Verify pizza is built
by Jim Cornelius BDD presentation

23. Tools
Test tool is NOT A Strategy
Using the tool is NOT A Goal

24. Mobile Automation Approaches
Tool Technology
Object Recognition
Highest ROI in long perspective.
Test cases developed are more reusable and flexible.
Scope and coverage for functional testing will be very high compared to Image recognition.
ex: WebElement AddToBasket=driver.findElement(By.id("btn_Basket"));
Image Recognition
Identify and control GUI components.
Useful when there is no easy access to a GUI's internal or source code.
Create Image Library.
Text Recognition (OCR)
Highest ROI in short perspective.
No need object repository.

25. Mobile Automation Approaches
How to Select Tool
 Application Type (native, hybrid, mobile-web)
 Operating System (Platform)
 Application Functionality (Business, communication, gaming etc.)
 Design Methodology (Keyword, DDT, BDD etc.)
 Support Real Device Testing

26. Continuous Integration
Continuous Integration – Builds Execute
Automated Regression Tests
Automated Regression Tests
Continuous Integration – Builds Execute
Automated Unit Tests
Automated Unit Tests
Test Maturity Model
Build Automation (.apk, .ipa)
by Jim Cornelius BDD presentation

27. A Fine Approach
Input: Test Design Output: Test Suite
QA Responsibilities
Test Case Design
Test Case
Maintanence
Debugging
Object Library
Platform Support
iOS
Android
Windows
Source Repository
Commit test cases/suites
Objects

28. Android Phone
iOS
Android Tablet
iPad
Windows
Tablet
A Fine Approach
pull object library
pull test cases/suites
Jenkins
Server
3 Step Execution
Full Suite Execution
Failing case Execution
Report/Mail Results
Input: Test Design Output: Test Suite
Source Repository
Windows
Phone

29. A Fine Approach: Abstraction Layers
Test Suite 2 Test SuiteT nest Case n
Test Case 1 Test Case n Test Case 1
Higher Level Keywords
Lower Level Keywords
Object Library
Windows Agent
iOS Agent Android Agent
iPhone / iPad
Android Phone /
Android Tablet
Third-party Libraries
Test Suite 1
Test Case n Test Case 1
Windows Phone / Tablet
Test Case n
Stable
Variable
Stable
Technical Keywords
Test Framework

31. Q&A
Serhat Bolsu
sbolsu@tmob.com.tr
Selin Güngör
selin.gungor@ciceksepeti.com
Erol Selitektay
erol@vngrs.com