Tips and Tricks with Calabash for Mobile App Testing

3,098 views

Published on

Watch a live presentation at http://offer.bitbar.com/tips-and-tricks-with-calabash-for-mobile-app-testing

Calabash is a cross-platform test automation framework for Android and iOS native and hybrid applications. Calabash's easy-to-understand language syntax enables - not only developers, testers and QA folks - but also people with no programming background or experience can write and execute automated acceptance tests for their mobile apps.

Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,098
On SlideShare
0
From Embeds
0
Number of Embeds
261
Actions
Shares
0
Downloads
59
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Tips and Tricks with Calabash for Mobile App Testing

  1. 1. Tips & Tricks with Calabash for Mobile App Testing 13 March 2014 Ville-Veikko Helppi Technical Product Manager ville-veikko.helppi@bitbar.com
  2. 2. Agenda • Prerequisites for Mobile App Testing • Calabash Introduction and How It Compares to Other Test Automation Frameworks • Inside the Calabash – Building Those Scripts • Tips & Tricks with Calabash • Demonstration • Q&A © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 2
  3. 3. Agenda • Prerequisites for Mobile App Testing • Calabash Introduction and How It Compares to Other Test Automation Frameworks • Inside the Calabash – Building Those Scripts • Tips and Tricks with Calabash • Demonstration • Q&A © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 3
  4. 4. Why Apps Fail? © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 4
  5. 5. Prerequisites for Mobile App Testing • Testing environment as authentic as possible – Real devices <-> emulators/simulators • Test coverage needs to be adequate – Software is not identical in all Android/iOS devices – Hardware is even more fragmented landscape • Surrounding testing conditions (e.g. network) • Try not to build in repetitive, manual processes • Test automation framework that works for you © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 5
  6. 6. What Framework Works You The Best? • High-level & declarative test automation frameworks work well when app is under dev • Easy to integrate with continuous integration • Distance between use case and actual test • What do you build: Native, Hybrid or Web app? • The technical competence in your organization • Desired outcome from test automation © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 6
  7. 7. What Improves the App Quality? • Automate generic things as much as you can • During the development your app changes – the same must apply for testing! • Carefully select (testing) technology & partner • Use all 24 hours per day with test automation • Cloud-based platform is the only solution to get you quickly covered in the global markets © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 7
  8. 8. Why Real Devices are Must-to-Have? • Emulators/simulators won’t help you to test… – User Experience and Usability – Hardware – Software – Infrastructure 0 % = the percentage of your app users that use emulator to run your app! © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 8
  9. 9. Basics: Different Ways of Doing Test Automation Automatic test exercisersRecord and PlaybackHand written test scripts Benefits: Accurate, specific to your testing needs, plenty of options with frameworks, tools Fast to create, accurate, not as sensitive to human-errors as hand-written tests, tools avail’ty Fastest & extremely automated, excellent for smoke testing/quick testing, availability Tradeoffs: Takes a lot of time, ties resources to write test cases/scripts, error- prone (humans) Compelling Recorder+Playback tools available for only few test automation frameworks Not accurate as real test cases © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 9
  10. 10. Agenda • Prerequisites for Mobile App Testing • Calabash Introduction and How It Compares to Other Test Automation Frameworks • Inside the Calabash – Building Those Scripts • Tips and Tricks with Calabash • Demonstration • Q&A © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 10
  11. 11. Introduction to Calabash • Behavior driven test framework for native Android & iOS, and for hybrid web apps (embedded webviews) • Tests are described in Cucumber and then converted to Robotium or “similar-to-Frank“ in run time • Supports currently about 80 different natural language commands (controllers). New controllers can be implemented in Ruby or Java • If your app uses custom UI Widgets you have to create controllers for those in order to test your app • Calabash tools include a command line inspector for finding right UI element names/ids © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 11
  12. 12. Android Example: The Family Tree of Test Automation Frameworks JUnit Android Instrumentation Framework Robotium Espresso UI Automator Calabash Appium © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 12
  13. 13. Why Calabash is A Good Choice? • Cross-platform test automation framework • Human-readable syntax/tests • Not much changes needed for tests when the mobile app changes (e.g. UI components) • Pretty easy to integrate with any CI • Great choice for teams that have limited resources for development and/or testing © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 13
  14. 14. Calabash-iOS: Architecture • Calabash installs an HTTP server as an instrumentation package that listens commands from Calabash server • Tests are executed on server side • Each test scenario is described in Cucumber • Ruby Client library converts Cucumber commands to ”Frank” method calls • Webview support is implemented with Javascript injection to the Webview iOS device Your app Calabash HTTP server Test server Step definitions, Ruby client library Features © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 14
  15. 15. Calabash-Android: Architecture • Calabash uses Android instrumentation test server (a separate app) • Based on ActivityInstrumentationTestCase2 • Tests are executed on server side • Predefined and custom steps • Ruby Client library converts Cucumber commands to Robotium method calls • Webview support is implemented with Javascript injection to the Webview Android device Your app Instrumentation Test Server Test server Step definitions, Ruby client library Features © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 15
  16. 16. Calabash: Code Example Feature: Login feature Scenario: As a valid user I can log into my app I wait for text "Hello" Then I press view with id "Sign in" Then I enter text "username" into "login_username" Then I enter text "password" into "login_password" Then I wait for activity "HomeTabActivity" Then I press view with id "menu_compose_tweet" Then I enter text "Testdroid" into field with id "edit" Then I press view with id "composer_post" © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 16
  17. 17. How Calabash Compares to Others? Calabash Robotium uiautomator Espresso Appium Android Yes Yes Yes Yes Yes iOS Yes No No No Yes Mobile web Hybrid (webviews) Yes (Android) Limited to x.y clicks No Yes (Android & iOS) Scripting Language Ruby Java Java Java Almost any Test creation tools CLI Testdroid Recorder UI Automator viewer Hierarchy Viewer Appium.app Supported API levels All All 16 => 8, 10, 15-19 All Community Active Contributors Google Google Active © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 17
  18. 18. Example #1: Calabash with Cloud Developers Source code Repository Build failed Successful build QA & Testers Error reporting Test Cases Application Users / Consumers Commit © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 18
  19. 19. Example #2: Calabash with CI Developers Source code Repository Build failed Successful build QA & Testers Error reporting Test Cases Application Users / Consumers Commit © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 19
  20. 20. Agenda • Prerequisites for Mobile App Testing • Calabash Introduction and How It Compares to Other Test Automation Frameworks • Inside the Calabash – Building Those Scripts • Tips and Tricks with Calabash • Demonstration • Q&A © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 20
  21. 21. Step Definitions • Fast and easy to modify definitions and features • Predefined and Custom steps – Predefined steps offer the way to start instantly – Custom steps provide more options & can be easily tailored for your app • Works for both, Android and iOS, when added: – calabash.framework – ActivityInstrumentationTestCase2 • Custom steps extremely easy to modify & create © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 21
  22. 22. Test Script – called Features © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 22 Feature: Login feature Scenario: As a valid user I can log into my app I wait for text "Hello" Then I press view with id "Sign in" Then I enter text "username" into "login_username" Then I enter text "password" into "login_password" Then I wait for activity "HomeTabActivity" Then I press view with id "menu_compose_tweet" Then I enter text "Testdroid" into field with id "edit" Then I press view with id "composer_post"
  23. 23. Example: Predefined Steps © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 23 Given /^I press the "([^"]*)" button$/ do |buttonText| performAction('press_button_with_text', buttonText) end Then /^I press button number (d+)$/ do |buttonNumber| performAction('press_button_number', buttonNumber) end Then /^I press image button number (d+)$/ do |buttonNumber| performAction('press_image_button_number', buttonNumber) end Then /^I press view with id "([^"]*)"$/ do |view_id| performAction('click_on_view_by_id',view_id) end
  24. 24. From Definition to Feature © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 24 Then /^I press view with id "([^"]*)"$/ do |view_id| performAction('click_on_view_by_id',view_id) end Then I press view with id "menu_compose_tweet" Definition: Feature:
  25. 25. Agenda • Prerequisites for Mobile App Testing • Calabash Introduction and How It Compares to Other Test Automation Frameworks • Building the Calabash Scripts • Tips and Tricks with Calabash • Demonstration • Q&A © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 25
  26. 26. #1: Use Only Real Devices © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 26
  27. 27. #2: Calabash is Perfect for “Test Early, Test Often” Approach © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 27
  28. 28. #3: Integrate Dev & Testing -> CI © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 28 Commit Developers Source code Repository Build failed Successful build QA & Testers Error reporting Test Cases Application Users / Consumers
  29. 29. Agenda • Prerequisites for Mobile App Testing • Calabash Introduction and How It Compares to Other Test Automation Frameworks • Inside the Calabash – Building Those Scripts • Tips and Tricks with Calabash • Demonstration • Q&A © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 29
  30. 30. Agenda • Prerequisites for Mobile App Testing • Calabash Introduction and How It Compares to Other Test Automation Frameworks • Inside the Calabash – Building Those Scripts • Tips and Tricks with Calabash • Demonstration • Q&A © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 30
  31. 31. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 31

×