Your SlideShare is downloading. ×
Mobile Test Automation
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Mobile Test Automation


Published on

The explosion of multi-platform mobile application development has created an extensive testing burden for quality organizations – and a huge opportunity for test automation ROI. Factoring in the …

The explosion of multi-platform mobile application development has created an extensive testing burden for quality organizations – and a huge opportunity for test automation ROI. Factoring in the idiosyncrasies of individual device models and carriers magnifies both the challenge and opportunity. Attend this session to learn how to extend test automation to the mobile platform and realize significant ROI. You will come away with best practices for implementing test automation effectively and efficiently across multiple mobile platforms and devices utilizing various types of mobile test automation tools. Specifically, tool independent techniques for creating a single test suite capable of executing across multiple platforms and devices will be discussed.

Published in: Mobile

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. © Utopia Solutions Extending Test Automation To Mobile Applications Lee Barnes, CTO Utopia Solutions
  • 2. © Utopia Solutions Agenda •What is Successful MTA? •How Do We Achieve It? •Getting Started •Visual Based Tools •Object Based Tools •Evaluating Your Apps •Tool Evaluation •Summary & Questions Mobile Test Automation (MTA)
  • 3. © Utopia Solutions SECTION 1 What Is Successful Mobile Test Automation?
  • 4. © Utopia Solutions4 WIFI / Carrier Networks Mobile Test Automation – Goal Automation Framework Test Cases Backend System Single set of test cases executing across a diverse set of real devices
  • 5. © Utopia Solutions5 Test Automation Success Criteria • Issue detection and recovery • Accurate verification • Unattended execution Reliable • Minimum sensitivity to application and test case changes • Test case definition separate from automation code Maintainable • Test coverage expanded efficiently • Automated test cases created by non- automation resources Scalable
  • 6. © Utopia Solutions SECTION 2 How Do We Achieve These Goals?
  • 7. © Utopia Solutions7 Test Automation Approach Mobile Test Automation Framework Automated Test Cases Test Results Test Definition Test Execution Non-technical / Test Analysts & Subject Matter Experts Technical / Automation Engineers
  • 8. © Utopia Solutions Mobile Test Automation Framework Business / Application Testing Components Login GoTo Stocks GoTo News Add Stock Edit Stock Remove Stock Logout Get Position Verify Position … … Object Interaction Methods Object Exists Set Value Get Value Select Item Touch Touch and Hold Get Image Get Text Send Key Test Case Login GoTo Stocks Add Stock Verify Position Logout MobileAutomationFramework
  • 9. © Utopia Solutions9 Test Automation Challenges • Defining appropriate scope • Integrating with SDLC • Addressing technical issues • Reliability, maintainability, scalability General • Platform diversity • Device diversity • Tool diversity • Tool immaturity • Lack of skilled resources • Rate of technology change • Rapid development cycles Mobile Specific
  • 10. © Utopia Solutions SECTION 3 Let’s Get Started
  • 11. © Utopia Solutions11 Mobile Test Automation Tool Categories Native Platform Frameworks Multi-Platform Tools Visual Based Object Based • Provided by mobile platform vendor as part of SDK • Typically interact with the application at the UI object level • Interacts with devices using visual methods (text and image recognition) • Interacts directly with application UI objects using native methods and properties • Object based interaction with app UI controls • Greatest degree of support for native UI objects • Control of device settings • Supports multiple platforms • Ability to execute tests across multiple devices • Interact with entire device • Supports multiple platforms • Ability to execute tests across multiple platforms • Interacts with apps at the object level for greater reliability • Limited to a single platform • Requires code level access to the app • May be limited to simulator (e.g. Blackberry) • Generally less reliable • No access to object properties • May require jail-breaking / rooting • Interaction is limited to app under test • Requires app to be instrumented • Android UI Automator • iOS UI Automation • SeeTest (Experitest) • eggPlant (TestPlant) • Perfecto Mobile • Zap-Fix (Zap Tech.) • Trust (Mobile Labs) • TouchTest (SOASTA) • M-eux Test (Jamo Solutions) DescriptionProsConsExample
  • 12. © Utopia Solutions12 Start With a Plan Scope OS/Device Selection Test Case Selection Non-Testing Tasks Technical Challenges Platform/Device Diversity Object Identification Process Integration Test Suite Maintenance Continuous Integration
  • 13. © Utopia Solutions13 Automation Scope Selection • Frequently tested • Predictable results • Easy to automate • Manually tedious “Traditional” rules still apply • Tests executed across all target devices • Avoid apps and tests that face many mobile automation challenges (to be discussed) Mobile specific
  • 14. © Utopia Solutions SECTION 4 Visual-based Tool Approach
  • 15. © Utopia Solutions15 Visual Based Identification Label Label + Boundary Image Recognition Label Offset Multi-Label Offset
  • 16. © Utopia Solutions16 Cross Platform Issues – Inconsistent UI/UX Button Image/Position Screen Titles Button Labels
  • 17. © Utopia Solutions17 Visual Based Techniques – Complex Interaction iOS On/Off Switch Function iosSwitchGetState (Label) Define OCR parameter ranges Search for Label on device screen If Label not found status = switch not found error Else Loop through OCR parameter range Search for switch state(ON/OFF) If state found status = switch state Else status = state not found error End If End Loop End If Return status End Function Function iosSwitchSetState (Label, State) CurrentState = iosSwitchGetState (Label) If CurrentState = error status = error Else If CurrentState = desired state status = success Else If desired state = ON Touch right side of switch Else Touch left side of switch End if End if status = success End if Return status End Function Hidden Complexities
  • 18. © Utopia Solutions18 Visual Interaction – Reusability? Maybe… • Label / object offset • On / Off reversed not • Label offset right • No On / Off label
  • 19. © Utopia Solutions19 Object Identification - Defining Master Object Properties Environment Object Type Attribute Value iPhone Screen Method Unique Text iPhone Screen Label Screen name iPhone Screen Location 20,25:40,200 iPhone Edit Label Object name iPhone Edit Offset 0,-20 … • Defines general rules for identifying objects by platform • Avoids adding all application objects to Object Repository • One master file per application (typically) Master Property File
  • 20. © Utopia Solutions20 Object Identification - Define Object Map ObjectScreenPlatform iOS Login User Name Password Add Stock Symbol • Contains any screens and objects that don’t meet criteria defined in Master Property File • Objects organized hierarchically by platform and screen UI Object Map
  • 21. © Utopia Solutions SECTION 5 Object-based Tool Approach
  • 22. © Utopia Solutions© Utopia Solutions Object Based Interaction Tool Libraries Included in Project Build Object Properties Visible to Tool Class: UIButton Label: Save Class: UITextField Label: Name
  • 23. © Utopia Solutions23 Object Based Interaction – Cross Platform Differences Simple Interaction – “Tap” the Save/Done Button Class: avwButton Method: Push Implementation: avwWindow(“MyStock”).avwButton(“Save”).Push Class: iosButton Method: Tap Implementation: iosView(“EditStock”).iosButton(“Done”).Tap
  • 24. © Utopia Solutions24 Object Identification – Handling Cross Platform Properties Class: Button Name: Save Create Mapping of Universal Objects to Platform Specific Objects Properties Class: avwButton Label: Save Method Name Touch: Push Object Path avwWindow(“MyStock”) Properties Class: iosButton Label: Done Method Name Touch: Tap Object Path iosView(“EditStock”) Step 1
  • 25. © Utopia Solutions25 Function TouchButton (Name) Get execution platform Retrieve platform specific object info from Object Map Construct statement to perform operation Execute operation Return operation status End Function Object Identification – Handling Cross Platform Create Universal Method for All PlatformsStep 2 Properties Class: Button Name: Save Properties Class: iosButton Label: Done Method Name Touch: Tap Object Path iosView(“EditStock”) iosView(“EditStock”).iosButton(“Done”).Tap
  • 26. © Utopia Solutions SECTION 6 Evaluating Your Apps for Potential Challenges
  • 27. © Utopia Solutions27 Analyze Business Processes for Differences Differences • Menu always visible on iOS • Different edit buttons • Different column headers
  • 28. © Utopia Solutions28 Analyze Business Processes for Differences Differences • Different screen titles • Different “add” buttons • Different “save” buttons • Different background color on list
  • 29. © Utopia Solutions29 Analyze Business Processes for Differences Differences • Cancel button embedded in search field on iOS • Different “expand” icons
  • 30. © Utopia Solutions SECTION 7 Handling Workflow Differences
  • 31. © Utopia Solutions31 Handling Workflow Differences – Go To Stocks Android 1. Tap “MENU” 2. Tap “Stocks” 3. Sync on “My Stocks” Screen iOS 1. Tap “My Stocks” 2. Sync on “My Stocks” Screen Implement at the Business Process Component Level Public Function GoToStocks Platform = GetPlatform() Select Case Platform Case “iOS” Tap (“MENU”) Tap (“Stocks”) status = WaitScreen (“My Stocks”) Case “Android” Tap (“My Stocks”) status = WaitScreen (“My Stocks”) Case Else status = INVALID_PLATFORM End Select GoToStocks = status End Function
  • 32. © Utopia Solutions32 Mobile Test Automation Tool Evaluation Top Ten Questions to Ask Tool Vendors • Is your tool visual or object based? • Does your tool require jailbreaking or rooting? • How quickly do you support new mobile OS releases? • How strong is the company (e.g. sales, customers, years, etc.)? • How do I execute a single test across multiple platforms with your tool? • Can (or must) your tool integrate with other scripting engines? • Do your support our mobile development platform? • Can your tool interact with devices remotely? • Does your tool integrate with our test management solution? • Will you assist me with a proof of concept?
  • 33. © Utopia Solutions33 Key Summary Points • Set appropriate automation goals • Start with a plan • Scope • Technical Challenges • Process Integration • Understand how visual/object based tools work • Analyze your app for platform differences • Address differences at appropriate level in your framework • Evaluate tools carefully • Start small and expand over time
  • 34. © Utopia Solutions34 Questions… …. and Answers! Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: Twitter: LinkedIn: Blog: