TiCalabash: Fully automated Acceptance Testing @ TiConf EU 2014


Published on

Acceptance testing has become a crucial step of software development,
but it has been a noticeably missing piece of the Titanium mobile
application development puzzle. TiCalabash is a new tool, developed
specifically for Titanium to bring mobile development a significant step
forward in reducing time and resource risk and increasing user acceptance
and overall quality of development. Fully automated, using human-readable
language, and offering full end-to-end acceptance testing, TiCalabash should
be a major part of your mobile development toolbox.

Published in: Technology

TiCalabash: Fully automated Acceptance Testing @ TiConf EU 2014

  1. 1. Ti Calabash: TiCalabash for Titanium: Fully Automated Testing. !
  2. 2. Platino Talk?
  3. 3. Game Development with Titanium https://github.com/gouldjw/tiConf-Mobile-Dev-Wars-Demo https://vimeo.com/96590732 https://www.slideshare.net/gouldjw13/ticonf2014-gdtph-10may14ext http://www.avatar-soft.com/#tiConf2014
  4. 4. Acceptance Testing
  5. 5. Acceptance Testing AKA User Acceptance Testing What is it?
  6. 6. Acceptance Testing Test the entire App from the UI down rather than Units (aka Unit Testing) Unit Testing + Acceptance Testing === Outside-in BDD <sidenote> http://www.sarahmei.com/blog/2010/05/29/outside-in-bdd/ credit: http://lostechies.com </sidenote>
  7. 7. Before Acceptance Testing Some Pain Points Many devices, OS versions, Languages, Form Factors, Screen Density/Size… Time consuming process of manually clicking through screens. Catch and stop regressions
  8. 8. The Original Vision A Natural Language Like DSL Extensible via ‘step definitions’ Supports Continuous Integration No Titanium Modules No touching tiapp.xml Use only a Titanium CLI hook No Ruby Dependencies
  9. 9. Calabash Overview “Calabash supports Cucumber. Cucumber lets us express the behavior of our app using natural language that can be understood by business experts and non- technical QA staff.” — Calaba.sh “…Calabash is cross-platform, supporting Android and iOS native apps…”
  10. 10. About that no Ruby thing… TiCucumber was too much work for too little reward. Matt Apperson had a better idea
  11. 11. TiCalabash A Titanium CLI hook for Calabash Calabash is Cucumber for Mobile npm install -g ticalabash https://github.com/appersonlabs/ticalabash
  12. 12. A Feature This is executable code: https://github.com/CodexLabs/alloy_fugitive/blob/master/features/fugitive.feature
  13. 13. Step Definitions https://github.com/calabash/calabash-ios/blob/master/calabash-cucumber/features/step_definitions/calabash_steps.rb https://github.com/calabash/calabash-android/tree/master/ruby-gem/lib/calabash-android/steps An Example
  14. 14. Custom Step Definitions Add your own steps:
  15. 15. Demo
  16. 16. Architecture of Calabash FYI (for the curious) features Cucumber (On Dev computer) iOS/ Android App Calabash library (contains a web server iOS: Frank Android: Robotium) *On iOS it uses -cal scheme Cucumber test reporter REST API
  17. 17. Calabash Queries You can use calabash to query your UI http://blog.lesspainful.com/2012/12/18/Android-Query/ http://roadtoautomation.blogspot.nl/2013/12/road-to-identifying-elements-using.html https://github.com/calabash/calabash-ios/wiki/05-Query-syntax query("all button") query("all view marked:'something'")
  18. 18. Recording Touches We haven’t exposed it yet “for reasons” https://github.com/calabash/calabash-ios/wiki/04-Touch-recording-and-playback Only works for iOS < 7 There appears to be hope for iOS 7 http://stackoverflow.com/questions/18205130/calabash-how-to-record-touches-on-iphone ! Calabash Android doesn’t implement it. There is a Robotium Recorder however. If you need touch recording today, just use Appcelerator Platform.
  19. 19. Recording Touches $ calabash-ios console ! Now use the command record_begin ! > record_begin => "" This begins recording touch events. Now in the simulator perform the touch events you want to record. This should be a short sequence of events corresponding to a gesture on a view. Then use record_end "mytouches", for example ! > record_end "drag_one_up" => "drag_one_up_ios5_iphone.base64" This saves the touch events under the name "mytouches". In the example, you see the string "drag_one_up_ios5_iphone.base64". This is actually a file being saved in your directory. ! You can test that the recording does what it should by running ! > playback "drag_one_up" => ["<UIView: 0x7dc70b0; frame = (-48.7356 26.2644; 417.471 417.471); ... This will playback the events you just recorded at the same coordinates you recorded them. If you're unhappy with the results just try again. ! If you're happy, exit the console and move the generated file to the "features" folder.
  20. 20. Jenkins The Configuration cd ~/Desktop/ticonf2014testappdemo/alloytest # or actually pull from git. titanium clean titanium build --platform=iphone --test & sleep 30 killall "iPhone Simulator" export PATH=“/Users/andrewmcelroy/.rbenv/shims:/Users/andrewmcelroy/bin:/usr/local/bin: /Users/andrewmcelroy/.rbenv/bin:/Users/andrewmcelroy/android/tools:/Users/andrewmcelroy/android/: /Users/andrewmcelroy/android//tools:/Users/andrewmcelroy/android/platform-tools/:/usr/bin:/bin: /usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:" ! echo | titanium calabash —platform=iphone ! cd ~/Library/Application Support/iPhone Simulator/7.1/Applications/ AppGUID = $(find . -name "alloytest" | cut -d "/" -f 2) ! cp "~/Library/Application Support/iPhone Simulator/7.1/Applications/$AppGUID/Documents/jenkins.xml" $WORKSPACE/jenkins.xml
  21. 21. Questions? !
  22. 22. Thank You Andrew McElroy https://CodexLabs.com github.com/CodexLabs@CodexLabs Matt Apperson @mattapperson Special thanks to Matt for the Android implementation