Mobile App Automated Testing


Published on

The slides cover the challenges faced in developing an #automated #mobile #app (iOS/Android) project, as well as tips and tricks that can used to quickly achieve it with multiple platform without duplicate code & less efforts using #Calabash.

Published in: Technology
1 Comment
  • The slides cover the challenges faced in developing an #automated #mobile #app (iOS/Android) project, as well as tips and tricks that can used to quickly achieve it with multiple platform without duplicate code & less efforts using #Calabash.
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Functional and Acceptance tests• Actual app, as opposed to an isolated component• Often based on use-cases written in natural (domain) language• Visual appearance of app screens matter! (Design guidelines, etc)• As realistic an environment as practically possibleoften a manual process: repetitive, expensive• Many devices, screens, OS versions, languages
  • Minimal Distance between Use Cases and Test Cases High Level (robust against minor UI changes)Supports Testing in Realistic Environment (Real Devices, Multiple OS Versions….)Supports Continuous IntegrationExecuting a test produces a test report for each step, did it succeed or not exception/error message if presentFlexible Report output formats Machine readable (XML, JSON,...) Human readable, console
  • Cucumber Providesa framework for writing software specificationsa tool for executing those specifications. Specifications are written in a business readable language (DSL) that is close to natural language.Works with Ruby, Java, .NET, Flex or web applications written in any language. It has been translated to over 40 spoken languages.
  • NarrativeIn order to ‘Conditon’As a RoleI want to ‘Verify’
  • The Calabash client library makes HTTP requests to the server when it wants to do something with the current viewFeature files. The feature files describe the app use-cases you want to test. They are simultaneously a specification of the behaviour of your app and an executable test suite.
  • Mobile App Automated Testing

    1. 1. Hello I’m Utsav
    2. 2. SOFTWARE DEVELOPMENT DONE RIGHT Netherlands | USA | India | France | UK; Blog :
    3. 3. Mobile App Automated Acceptance Testing Utsav Sinha
    4. 4. Problems • Development and Maintenance • Adaptable to changes • Accuracy • Completeness
    5. 5. Automated Testing Desiderata • Use Cases are the Test Cases. • Robust • Realistic Environment (Real Devices, Multiple OS Versions….) • Continuous Integration • Comprehensive Test report for each step. • Flexible Report output formats.
    6. 6. • Cucumber Provides o a framework for writing software specifications o a tool for executing those specifications. • Specifications are written DSLs • Works with Ruby, Java, .NET, Flex and many more •
    7. 7. Example – Feature
    8. 8. Step Definition • Make Cucumber Tests come alive. • Written in various Programming Language (mostly Ruby)
    9. 9. Calabash • One Interface Cucumber for iOS and Android o Reuse of Test Cross Platform (iOS and Android) • • • • • Targets Native and Hybrid apps for Android and iOS Runs on Physical Device Support for Hybrid Apps (embedded web-views) Open Source Tool Cross Platform Testing
    10. 10. Calabash iOS architecture • Calabash tests are executed with the Cucumber tool and consist of two parts: • Client Library written in Ruby • Calabash.framework, a server framework written in Objective-C
    11. 11. Calabash iOS • To use Calabash we make a special test target in Xcode that links with calabash.framework • The Server Framework starts HTTP Server inside the App, that listens to the request from Client Library. • The Cucumber tool executes the feature file that represents the intended behaviour of the App Under Test. • The Feature Files contains steps (Predefined or Custom)
    12. 12. Calabash Androidarchitecture • Features • Step-Definitions • Instrumentation Test Server: This is another app which will be installed and executed into device.
    13. 13. Calabash iOS • App is built for Testing by linking a static library (calabash-server) o Uses a combination of UIAutomation and other APIs • Supports Interactive test-development • Based on Frank but changes some core parts. o Setting up Frank Requires modification of the app source, inclusion of static resources and source files manually, where as Calabash has Automated Set Up. 3 commands and we are done • Touch Synthesis supports multi-touch, gestures • Can use device accessibility for identifying views
    14. 14. Queries • • • • Explore the App Interactively Run Command „calabash-ios console‟ It gives us a Calabash console (it is just a Ruby irb with calabash loaded) Example irb(main):001:0> query("button") [0] { }, "class" => "UIButton", "frame" => { "y" => 20, "width" => 101, "x" => 20, "height" => 101 "UIType" => "UIControl", "description" => "<UIButton: 0x4cc1120; frame = (20 20; 101 101); opaque = NO; autoresize = RM+BM; layer = <CALayer: 0x4c89cc0>>" }, Returns an array with objects that are descriptions of the buttons in the current screen in your app
    15. 15. Queries • The query function o takes a string query as an argument. o the query argument is similar to a css selector • The syntax for queries is based on UIScript, o but is a new implementation with additional features added (and some removed). • UIScript gives a nice "CSS-selector" like approach to finding view objects in your app screens. irb(main):001:0> query("tabBarButton").count => 4 irb(main):002:0> query("tabBarButton")[0] => {"class"=>"UITabBarButton", "frame"=>{"y"=>1, "width"=>76, "x"=>2, "height"=>48}, "UIType"=>"UIControl", "description"=>"<UITabBarButton: 0x856a820; frame = (2 1; 76 48); opaque = NO; layer = <CALayer: 0x856d210>>" }
    16. 16. Let’s test…
    17. 17. Prototype gestures • Calabash iOS has a number of built-in event sequences that can be played back. o (touch, swipe, pinch, etc) • Events can be relocated (translated) to different views, and optionally offsets. • Extensible: you can record your own gestures if none of the built in suits you.
    18. 18. Summary • Calabash iOS tradeoffs and mitigations DSLs: Cucumber, and Query language (decl, high level) • Full power of Ruby in tests • Advanced touch synthesis o o recordings give some brittleness, which is mitigated by supporting relocating and manipulating of recorded event sequences (which are just lists of dictionaries). • Interactive development experience • Requires linking of a framework • Not good for some games (randomness, “gameplay” & “feel”, timing critical) • Not good for Phone-to-Phone coordination (call, text, bluetooth)
    19. 19. Thank You!! Utsav Sinha
    20. 20. References • • • •
    21. 21. Agile Testing Current Competencies Mobile Testing: Appium, Calabash Performance Testing Tools: JMeter, LoadUI Automation Frameworks in place - Selenium/Webdriver keyword driven - SoapUI ATDD Tools: Cucumber, Fitnesse, JBehave, Geb Language Proficiencies: Java, Ruby, Groovy, Python Functional automation Tools: Selenium/Webdriver, AUTO IT, SoapUI, QTP Knowledge Sharing: Speakers in national and international conferences
    22. 22. Contact us @ Websites Xebia India Thought Leadership Htto://