8. Decision to Automate
What to Automate
Stable Version
Test Types
• Regression Testing
• Smoke Test
• Performance & Load Testing
Regression Testing
Performance & Load Testing Smoke Testing
12. Easy to Debug
Ideal Case
Automation
Requirements
Support Integration
to Third Party Libraries
Easy to Use
Sustainable
Maintainable
Human Readable
Support CI Tools
13. Test automation is a software development process.
Test automation projects often fail because of a lack of
discipline and project management. Many testers don’t realize
that when they automate tests they’re actually developing
software.
Lessons Learned in Software Testing
Authors: Cem Kaner, James Bach and Bret Pettichord
14. Automation is Software Development
login();
searchItem();
addToBasket();
buyWithMoneyOrder();
18. Mobile Automation Approaches
Record & Play
Record -> Edit -> Play -> Edit -> Add Verify -> Edit and so on...
Good for getting to know technology
none of these tests will be maintainable
none of these tests will be sustainable
19. Mobile Automation Approaches
Behaviour Driven Development
#1 USER STORIES describe BUSINESS VALUE
#2 SCENARIOS help describe ACCEPTANCE CRITERIA
#3 SCENARIOS act as AUTOMATED TESTS
21. Mobile Automation Approaches
Keyword Driven Testing
keyword :
1) A function or method that can be used to test one aspect of the SUT
2) Can be defined by other keywords
Higher-level keywords:
a concrete aspect of the business logic
Lower-level keywords:
to keep the implementation of the higher-level keywords at a decent size
Technical keywords:
provide the technical implementation to access and thus test the system.
By codeCentric
22. Mobile Automation Approaches
BDD vs Keyword-Driven Testing
Requirement: Allow customer to choose a round or square shaped pizza
BDD: Given the customer logged into the pizza build app.
When the customer choose to build a square shaped pizza,
Then customer should be able to get a square shaped pizza,
When the customer choose to build a round shaped pizza,
Then customer should be able to get a round shaped pizza.
KDT: Login
Build pizza | square
Verify pizza is built
Build pizza | round
Verify pizza is built
by Jim Cornelius BDD presentation
23. Tools
Test tool is NOT A Strategy
Using the tool is NOT A Goal
24. Mobile Automation Approaches
Tool Technology
Object Recognition
Highest ROI in long perspective.
Test cases developed are more reusable and flexible.
Scope and coverage for functional testing will be very high compared to Image recognition.
ex: WebElement AddToBasket=driver.findElement(By.id("btn_Basket"));
Image Recognition
Identify and control GUI components.
Useful when there is no easy access to a GUI's internal or source code.
Create Image Library.
Text Recognition (OCR)
Highest ROI in short perspective.
No need object repository.
25. Mobile Automation Approaches
How to Select Tool
Application Type (native, hybrid, mobile-web)
Operating System (Platform)
Application Functionality (Business, communication, gaming etc.)
Design Methodology (Keyword, DDT, BDD etc.)
Support Real Device Testing
26. Continuous Integration
Continuous Integration – Builds Execute
Automated Regression Tests
Automated Regression Tests
Continuous Integration – Builds Execute
Automated Unit Tests
Automated Unit Tests
Test Maturity Model
Build Automation (.apk, .ipa)
by Jim Cornelius BDD presentation
27. A Fine Approach
Input: Test Design Output: Test Suite
QA Responsibilities
Test Case Design
Test Case
Maintanence
Debugging
Object Library
Platform Support
iOS
Android
Windows
Source Repository
Commit test cases/suites
Objects
28. Android Phone
iOS
Android Tablet
iPad
Windows
Tablet
A Fine Approach
pull object library
pull test cases/suites
Jenkins
Server
3 Step Execution
Full Suite Execution
Failing case Execution
Report/Mail Results
Input: Test Design Output: Test Suite
Source Repository
Windows
Phone
29. A Fine Approach: Abstraction Layers
Test Suite 2 Test SuiteT nest Case n
Test Case 1 Test Case n Test Case 1
Higher Level Keywords
Lower Level Keywords
Object Library
Windows Agent
iOS Agent Android Agent
iPhone / iPad
Android Phone /
Android Tablet
Third-party Libraries
Test Suite 1
Test Case n Test Case 1
Windows Phone / Tablet
Test Case n
Stable
Variable
Stable
Technical Keywords
Test Framework
I’m Selin. I am a Software Test Engineer. I am working for ciceksepeti.com. In my company, we have different kind of e-commerce websites. I responsible for the test activities of ecommerce websites and all mobile applications. Currently, we have two mobile applications. One of them is ciceksepeti mobile application which is hybrid application. People who can order gifts, flowers, cakes and cookies. The other one is fieldterminal project which is native application. Also i responsible for test automation.
So today we are going to tell you about test au. How many have you tried test automation?
But this is different kind of ta presentation as you realize it is not named how to do test automation. We are going to talk about strategy, the strategy that is going to define your mobile test automation.
Actually automation is a hot topic. Everyone wants to automate their application. But they don’t know how or where to start automation. As you know automation can be a big challange but mobile automation is a bigger challange. Mobile test automation presents unique difficulties that a tester would not consider in a web or desktop application. So the goal of our paper is to help you to overcome the challange. We have been passed through this ways and we have gained some experience. So we are here to share these experiences with you.
Our talk is particularly relevant to those of you who wants to automate their mobile application.
First of all we wrote this paper to show the people who wants to automate their mobile application. We would like to guide people…
First we’ll be looking at decision to automate. Next, mobile challanges. After that we have demo to show you. I think it will be fun to watch it. It will be exciting for you. After that we’ll go on to mobile automation approaches. Finally, We’ll move on continuous integration. This presentation will take 45 minutes. There will be time for questions after our presentation like 15 minutes.
----- Meeting Notes (22/05/14 20:03) -----
ikinci resim çok anlatıcı değil - bunu farklılık ile anlatalım
better quality -> Another layer for Quality
early detection - CI ile entegre çalışması
Non deterministic olan durumu otomasyona almayız. Belirsizlik.
----- Meeting Notes (22/05/14 20:03) -----
if you automate a mess you get automated mess
Test type ı kontrol edebilebilr : tekrar kontrol et.
multi profile / multi user test.
Better knowledge of scripting and coding.
Software Developer in Test.
----- Meeting Notes (22/05/14 20:03) -----
iyibir test automated requirement dan kopacağı için aynı kişi olmasa daha iyi olur.
Behaviour does not lie
-Otomasyon sisteminize import edebilirsiniz
Requirement değişebilir.
System implementasyonu değişebilir.
----- Meeting Notes (22/05/14 20:03) -----
robust -
belki konular azaltılabilir - birbininin altına yazılabilir.
-Ana mottomuz
-çıkış noktamız
Yazılıma uyguladığınız herşeyi, otomasyonda bir yazılım olduğu için uygulayabiliriz
https://www.youtube.com/watch?v=zADOv65AmiI
Teknik tarafı tdd den base.
J-bhave, cucumber, calabash
Mobil : frank
----- Meeting Notes (22/05/14 19:31) -----
sadece 3. den yola çıkarak bdd için testi çözebilirsin diyemeyiz.
----- Meeting Notes (22/05/14 20:03) -----
object oriented similar exercies
Optical Character Recognition (OCR)(Text recognation)
Optical Character Recognition essentially uses a smart software engine that converts scanned images of handwritten, typewritten or printed text into machine-encoded text.
----- Meeting Notes (22/05/14 20:03) -----
use source control tools : Github / Bitbucket etc.
----- Meeting Notes (22/05/14 19:31) -----
hizalamayı düzeltlim.
object yeşil olması.
agentlar yazılabilir veya tool kullanılabilir.
----- Meeting Notes (22/05/14 20:03) -----
object library = yeşilve yeri değişiçek