© Utopia Solutions
Extending Test Automation To Mobile Applications
Lee Barnes, CTO
Utopia Solutions
© Utopia Solutions
Agenda
•What is Successful MTA?
•How Do We Achieve It?
•Getting Started
•Visual Based Tools
•Object Bas...
© Utopia Solutions
SECTION 1
What Is Successful Mobile Test Automation?
© Utopia Solutions4
WIFI / Carrier Networks
Mobile Test Automation – Goal
Automation
Framework
Test Cases
Backend System
S...
© Utopia Solutions5
Test Automation Success Criteria
• Issue detection and recovery
• Accurate verification
• Unattended e...
© Utopia Solutions
SECTION 2
How Do We Achieve These Goals?
© Utopia Solutions7
Test Automation Approach
Mobile Test Automation Framework
Automated Test Cases Test Results
Test Defin...
© Utopia Solutions
Mobile Test Automation Framework
Business / Application Testing Components
Login
GoTo Stocks
GoTo News
...
© Utopia Solutions9
Test Automation Challenges
• Defining appropriate scope
• Integrating with SDLC
• Addressing technical...
© Utopia Solutions
SECTION 3
Let’s Get Started
© Utopia Solutions11
Mobile Test Automation Tool Categories
Native Platform
Frameworks
Multi-Platform Tools
Visual Based O...
© Utopia Solutions12
Start With a Plan
Scope
OS/Device
Selection
Test Case
Selection
Non-Testing
Tasks
Technical
Challenge...
© Utopia Solutions13
Automation Scope Selection
• Frequently tested
• Predictable results
• Easy to automate
• Manually te...
© Utopia Solutions
SECTION 4
Visual-based Tool Approach
© Utopia Solutions15
Visual Based Identification
Label
Label + Boundary
Image Recognition
Label Offset
Multi-Label Offset
© Utopia Solutions16
Cross Platform Issues – Inconsistent UI/UX
Button
Image/Position
Screen
Titles
Button
Labels
© Utopia Solutions17
Visual Based Techniques – Complex Interaction
iOS On/Off Switch
Function iosSwitchGetState (Label)
De...
© Utopia Solutions18
Visual Interaction – Reusability?
Maybe…
• Label / object offset
• On / Off reversed
not
• Label offs...
© Utopia Solutions19
Object Identification - Defining Master Object Properties
Environment Object Type Attribute Value
iPh...
© Utopia Solutions20
Object Identification - Define Object Map
ObjectScreenPlatform
iOS
Login
User
Name
Password
Add
Stock...
© Utopia Solutions
SECTION 5
Object-based Tool Approach
© Utopia Solutions© Utopia Solutions
Object Based Interaction
Tool Libraries Included
in Project Build
Object Properties
V...
© Utopia Solutions23
Object Based Interaction – Cross Platform Differences
Simple Interaction – “Tap” the Save/Done Button...
© Utopia Solutions24
Object Identification – Handling Cross Platform
Properties
Class: Button
Name: Save
Create Mapping of...
© Utopia Solutions25
Function TouchButton (Name)
Get execution platform
Retrieve platform specific object info
from Object...
© Utopia Solutions
SECTION 6
Evaluating Your Apps for Potential Challenges
© Utopia Solutions27
Analyze Business Processes for Differences
Differences
• Menu always visible
on iOS
• Different edit ...
© Utopia Solutions28
Analyze Business Processes for Differences
Differences
• Different screen titles
• Different “add”
bu...
© Utopia Solutions29
Analyze Business Processes for Differences
Differences
• Cancel button
embedded in search
field on iO...
© Utopia Solutions
SECTION 7
Handling Workflow Differences
© Utopia Solutions31
Handling Workflow Differences – Go To Stocks
Android
1. Tap “MENU”
2. Tap “Stocks”
3. Sync on “My
Sto...
© Utopia Solutions32
Mobile Test Automation Tool Evaluation
Top Ten Questions to Ask Tool Vendors
• Is your tool visual or...
© Utopia Solutions33
Key Summary Points
• Set appropriate automation goals
• Start with a plan
• Scope
• Technical Challen...
© Utopia Solutions34
Questions…
…. and Answers!
Direct future questions to:
Lee Barnes
Founder and CTO
Utopia Solutions, I...
Upcoming SlideShare
Loading in …5
×

Mobile Test Automation

1,463 views

Published on

The explosion of multi-platform mobile application development has created an extensive testing burden for quality organizations – and a huge opportunity for test automation ROI. Factoring in the idiosyncrasies of individual device models and carriers magnifies both the challenge and opportunity. Attend this session to learn how to extend test automation to the mobile platform and realize significant ROI. You will come away with best practices for implementing test automation effectively and efficiently across multiple mobile platforms and devices utilizing various types of mobile test automation tools. Specifically, tool independent techniques for creating a single test suite capable of executing across multiple platforms and devices will be discussed.

Published in: Mobile
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,463
On SlideShare
0
From Embeds
0
Number of Embeds
108
Actions
Shares
0
Downloads
78
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Mobile Test Automation

  1. 1. © Utopia Solutions Extending Test Automation To Mobile Applications Lee Barnes, CTO Utopia Solutions
  2. 2. © Utopia Solutions Agenda •What is Successful MTA? •How Do We Achieve It? •Getting Started •Visual Based Tools •Object Based Tools •Evaluating Your Apps •Tool Evaluation •Summary & Questions Mobile Test Automation (MTA)
  3. 3. © Utopia Solutions SECTION 1 What Is Successful Mobile Test Automation?
  4. 4. © Utopia Solutions4 WIFI / Carrier Networks Mobile Test Automation – Goal Automation Framework Test Cases Backend System Single set of test cases executing across a diverse set of real devices
  5. 5. © Utopia Solutions5 Test Automation Success Criteria • Issue detection and recovery • Accurate verification • Unattended execution Reliable • Minimum sensitivity to application and test case changes • Test case definition separate from automation code Maintainable • Test coverage expanded efficiently • Automated test cases created by non- automation resources Scalable
  6. 6. © Utopia Solutions SECTION 2 How Do We Achieve These Goals?
  7. 7. © Utopia Solutions7 Test Automation Approach Mobile Test Automation Framework Automated Test Cases Test Results Test Definition Test Execution Non-technical / Test Analysts & Subject Matter Experts Technical / Automation Engineers
  8. 8. © Utopia Solutions Mobile Test Automation Framework Business / Application Testing Components Login GoTo Stocks GoTo News Add Stock Edit Stock Remove Stock Logout Get Position Verify Position … … Object Interaction Methods Object Exists Set Value Get Value Select Item Touch Touch and Hold Get Image Get Text Send Key Test Case Login GoTo Stocks Add Stock Verify Position Logout MobileAutomationFramework
  9. 9. © Utopia Solutions9 Test Automation Challenges • Defining appropriate scope • Integrating with SDLC • Addressing technical issues • Reliability, maintainability, scalability General • Platform diversity • Device diversity • Tool diversity • Tool immaturity • Lack of skilled resources • Rate of technology change • Rapid development cycles Mobile Specific
  10. 10. © Utopia Solutions SECTION 3 Let’s Get Started
  11. 11. © Utopia Solutions11 Mobile Test Automation Tool Categories Native Platform Frameworks Multi-Platform Tools Visual Based Object Based • Provided by mobile platform vendor as part of SDK • Typically interact with the application at the UI object level • Interacts with devices using visual methods (text and image recognition) • Interacts directly with application UI objects using native methods and properties • Object based interaction with app UI controls • Greatest degree of support for native UI objects • Control of device settings • Supports multiple platforms • Ability to execute tests across multiple devices • Interact with entire device • Supports multiple platforms • Ability to execute tests across multiple platforms • Interacts with apps at the object level for greater reliability • Limited to a single platform • Requires code level access to the app • May be limited to simulator (e.g. Blackberry) • Generally less reliable • No access to object properties • May require jail-breaking / rooting • Interaction is limited to app under test • Requires app to be instrumented • Android UI Automator • iOS UI Automation • SeeTest (Experitest) • eggPlant (TestPlant) • Perfecto Mobile • Zap-Fix (Zap Tech.) • Trust (Mobile Labs) • TouchTest (SOASTA) • M-eux Test (Jamo Solutions) DescriptionProsConsExample
  12. 12. © Utopia Solutions12 Start With a Plan Scope OS/Device Selection Test Case Selection Non-Testing Tasks Technical Challenges Platform/Device Diversity Object Identification Process Integration Test Suite Maintenance Continuous Integration
  13. 13. © Utopia Solutions13 Automation Scope Selection • Frequently tested • Predictable results • Easy to automate • Manually tedious “Traditional” rules still apply • Tests executed across all target devices • Avoid apps and tests that face many mobile automation challenges (to be discussed) Mobile specific
  14. 14. © Utopia Solutions SECTION 4 Visual-based Tool Approach
  15. 15. © Utopia Solutions15 Visual Based Identification Label Label + Boundary Image Recognition Label Offset Multi-Label Offset
  16. 16. © Utopia Solutions16 Cross Platform Issues – Inconsistent UI/UX Button Image/Position Screen Titles Button Labels
  17. 17. © Utopia Solutions17 Visual Based Techniques – Complex Interaction iOS On/Off Switch Function iosSwitchGetState (Label) Define OCR parameter ranges Search for Label on device screen If Label not found status = switch not found error Else Loop through OCR parameter range Search for switch state(ON/OFF) If state found status = switch state Else status = state not found error End If End Loop End If Return status End Function Function iosSwitchSetState (Label, State) CurrentState = iosSwitchGetState (Label) If CurrentState = error status = error Else If CurrentState = desired state status = success Else If desired state = ON Touch right side of switch Else Touch left side of switch End if End if status = success End if Return status End Function Hidden Complexities
  18. 18. © Utopia Solutions18 Visual Interaction – Reusability? Maybe… • Label / object offset • On / Off reversed not • Label offset right • No On / Off label
  19. 19. © Utopia Solutions19 Object Identification - Defining Master Object Properties Environment Object Type Attribute Value iPhone Screen Method Unique Text iPhone Screen Label Screen name iPhone Screen Location 20,25:40,200 iPhone Edit Label Object name iPhone Edit Offset 0,-20 … • Defines general rules for identifying objects by platform • Avoids adding all application objects to Object Repository • One master file per application (typically) Master Property File
  20. 20. © Utopia Solutions20 Object Identification - Define Object Map ObjectScreenPlatform iOS Login User Name Password Add Stock Symbol • Contains any screens and objects that don’t meet criteria defined in Master Property File • Objects organized hierarchically by platform and screen UI Object Map
  21. 21. © Utopia Solutions SECTION 5 Object-based Tool Approach
  22. 22. © Utopia Solutions© Utopia Solutions Object Based Interaction Tool Libraries Included in Project Build Object Properties Visible to Tool Class: UIButton Label: Save Class: UITextField Label: Name
  23. 23. © Utopia Solutions23 Object Based Interaction – Cross Platform Differences Simple Interaction – “Tap” the Save/Done Button Class: avwButton Method: Push Implementation: avwWindow(“MyStock”).avwButton(“Save”).Push Class: iosButton Method: Tap Implementation: iosView(“EditStock”).iosButton(“Done”).Tap
  24. 24. © Utopia Solutions24 Object Identification – Handling Cross Platform Properties Class: Button Name: Save Create Mapping of Universal Objects to Platform Specific Objects Properties Class: avwButton Label: Save Method Name Touch: Push Object Path avwWindow(“MyStock”) Properties Class: iosButton Label: Done Method Name Touch: Tap Object Path iosView(“EditStock”) Step 1
  25. 25. © Utopia Solutions25 Function TouchButton (Name) Get execution platform Retrieve platform specific object info from Object Map Construct statement to perform operation Execute operation Return operation status End Function Object Identification – Handling Cross Platform Create Universal Method for All PlatformsStep 2 Properties Class: Button Name: Save Properties Class: iosButton Label: Done Method Name Touch: Tap Object Path iosView(“EditStock”) iosView(“EditStock”).iosButton(“Done”).Tap
  26. 26. © Utopia Solutions SECTION 6 Evaluating Your Apps for Potential Challenges
  27. 27. © Utopia Solutions27 Analyze Business Processes for Differences Differences • Menu always visible on iOS • Different edit buttons • Different column headers
  28. 28. © Utopia Solutions28 Analyze Business Processes for Differences Differences • Different screen titles • Different “add” buttons • Different “save” buttons • Different background color on list
  29. 29. © Utopia Solutions29 Analyze Business Processes for Differences Differences • Cancel button embedded in search field on iOS • Different “expand” icons
  30. 30. © Utopia Solutions SECTION 7 Handling Workflow Differences
  31. 31. © Utopia Solutions31 Handling Workflow Differences – Go To Stocks Android 1. Tap “MENU” 2. Tap “Stocks” 3. Sync on “My Stocks” Screen iOS 1. Tap “My Stocks” 2. Sync on “My Stocks” Screen Implement at the Business Process Component Level Public Function GoToStocks Platform = GetPlatform() Select Case Platform Case “iOS” Tap (“MENU”) Tap (“Stocks”) status = WaitScreen (“My Stocks”) Case “Android” Tap (“My Stocks”) status = WaitScreen (“My Stocks”) Case Else status = INVALID_PLATFORM End Select GoToStocks = status End Function
  32. 32. © Utopia Solutions32 Mobile Test Automation Tool Evaluation Top Ten Questions to Ask Tool Vendors • Is your tool visual or object based? • Does your tool require jailbreaking or rooting? • How quickly do you support new mobile OS releases? • How strong is the company (e.g. sales, customers, years, etc.)? • How do I execute a single test across multiple platforms with your tool? • Can (or must) your tool integrate with other scripting engines? • Do your support our mobile development platform? • Can your tool interact with devices remotely? • Does your tool integrate with our test management solution? • Will you assist me with a proof of concept?
  33. 33. © Utopia Solutions33 Key Summary Points • Set appropriate automation goals • Start with a plan • Scope • Technical Challenges • Process Integration • Understand how visual/object based tools work • Analyze your app for platform differences • Address differences at appropriate level in your framework • Evaluate tools carefully • Start small and expand over time
  34. 34. © Utopia Solutions34 Questions… …. and Answers! Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: lee.barnes@utopiasolutions.com Twitter: twitter.com/USI_LeeBarnes LinkedIn: linkedin.com/in/leebarnes Blog: www.RaiseYourMQ.com

×