Calabash, an open-source automated testing technology for native mobile, by Karl Krukow

6,107 views

Published on

Mobile is now facing problems that have been existed in web for many years: fragmentation. In mobile, it is supporting the many phone and tablet models, operating system and versions, and languages. Calabash is an open source technology for functional testing of Android and iOS apps. It combines advanced automation technology with the Cucumber tool. Tests are written in a natural language of the business domain, and the cucumber tool can execute tests on Android and iOS devices or simulators.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,107
On SlideShare
0
From Embeds
0
Number of Embeds
543
Actions
Shares
0
Downloads
170
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Calabash, an open-source automated testing technology for native mobile, by Karl Krukow

  1. 1. Introducing Calabash automated functional testing for mobile native apps Karl Krukow, karl@lesspainful.com, LessPainful CodeMotion 2012, Rome 1fredag den 23. marts 12
  2. 2. About me 2fredag den 23. marts 12
  3. 3. About me • PhD, Computer Science, University of Aarhus, 2006. 2fredag den 23. marts 12
  4. 4. About me • PhD, Computer Science, University of Aarhus, 2006. • Developer at Trifork for about 6 years mostly Java enterprise, last two years on iOS. 2fredag den 23. marts 12
  5. 5. About me • PhD, Computer Science, University of Aarhus, 2006. • Developer at Trifork for about 6 years mostly Java enterprise, last two years on iOS. • Co-owner, iOS responsible at 2fredag den 23. marts 12
  6. 6. About me • PhD, Computer Science, University of Aarhus, 2006. • Developer at Trifork for about 6 years mostly Java enterprise, last two years on iOS. • Co-owner, iOS responsible at • Spare-time Hickey & Clojure fan-boy! 2fredag den 23. marts 12
  7. 7. Agenda 3fredag den 23. marts 12
  8. 8. Agenda • Automated functional testing for native mobile • Some desirable properties for a functional testing tool 3fredag den 23. marts 12
  9. 9. Agenda • Automated functional testing for native mobile • Some desirable properties for a functional testing tool • Introduce Calabash • Focus on iOS only 3fredag den 23. marts 12
  10. 10. Agenda • Automated functional testing for native mobile • Some desirable properties for a functional testing tool • Introduce Calabash • Focus on iOS only • Live Demo: • Calabash iOS • LessPainful: test service and device cloud 3fredag den 23. marts 12
  11. 11. Functional testing 4fredag den 23. marts 12
  12. 12. Functional testing • Functional and Acceptance tests • Actual app, as opposed to an isolated component • Often based on use-cases written in natural language (for domain) • Visual appearance of screens matter! (Design guidelines, branding,..) • As realistic an environment as practically possible 4fredag den 23. marts 12
  13. 13. Functional testing • Functional and Acceptance tests • Actual app, as opposed to an isolated component • Often based on use-cases written in natural language (for domain) • Visual appearance of screens matter! (Design guidelines, branding,..) • As realistic an environment as practically possible • For mobile apps, in particular • Many devices, screens, OS versions, languages • often a manual process: repetitive, expensive • regressions: why should I test that again? 4fredag den 23. marts 12
  14. 14. Functional testing desiderata 5fredag den 23. marts 12
  15. 15. Functional testing desiderata • Minimize distance between use cases and actual test code (DSLs?). • Expressive and efficient to write. • Extensible • High-level, declarative (robustness against “minor” UI changes). • Support testing in realistic environments (multiple real devices, on multiple OS versions, languages). • Support Continuous integration. 5fredag den 23. marts 12
  16. 16. Cucumber and Calabashfredag den 23. marts 12
  17. 17. • Cucumber provides • a framework for writing software specifications • a tool for executing those specifications • Specifications are written in a business readable language that is close to natural language. • Extremely popular tool for test and specs of web applications. • http://cukes.info/fredag den 23. marts 12
  18. 18. Cucumber Example Feature: As an administrator. I want to be able to add and remove users, so I can control access to the application Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr" Scenario: ...fredag den 23. marts 12
  19. 19. Cucumber Example Feature: As an administrator. I want to be able to add and remove users, so I can control access to the application Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr" Scenario: ...fredag den 23. marts 12
  20. 20. Cucumber Example Feature: As an administrator. I want to be able to add and remove users, so I can control access to the application Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr" Scenario: ...fredag den 23. marts 12
  21. 21. Cucumber Example Feature: As an administrator. I want to be able to add and remove users, so I can control access to the application Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr" Scenario: ...fredag den 23. marts 12
  22. 22. Cucumber Example Feature: As an administrator. I want to be able to add and remove users, so I can control access to the application Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr" Scenario: ...fredag den 23. marts 12
  23. 23. Cucumber Example Feature: As an administrator. I want to be able to add and remove users, so I can control access to the application Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr" Scenario: ...fredag den 23. marts 12
  24. 24. Step Definitions • Make the cucumber tests “come alive” • Written in ordinary programming languages • Mostly Ruby (but cucumber-jvm: Java, Clojure,...) Feature Step definitionsfredag den 23. marts 12
  25. 25. Step Definitions • Make the cucumber tests “come alive” • Written in ordinary programming languages • Mostly Ruby (but cucumber-jvm: Java, Clojure,...) Feature Step definitions Scenario: Add test user When I touch the Add User button ...fredag den 23. marts 12
  26. 26. Step Definitions • Make the cucumber tests “come alive” • Written in ordinary programming languages • Mostly Ruby (but cucumber-jvm: Java, Clojure,...) Feature Step definitions Scenario: Add test user When /^I touch the Add User button$/ do When I touch the Add User button btn_txt = Add user ... touch("button text:#{btn_txt}") endfredag den 23. marts 12
  27. 27. Calabash • One interface: Cucumber, for Android and iOS tests. • Predefined and custom steps (Ruby + soon: JVM). • Reuse of tests across platform possible. • Run on physical devices and simulators. • Support for hybrid apps (embedded webviews) • Free, open source with commercial add-ons: • Run tests in a device cloud using the LessPainful service, • we can write and run your tests, • setting up continuous integration, consulting, training, support. 10fredag den 23. marts 12
  28. 28. Architecture iOS iOS App -------------------- Calabash libraryfredag den 23. marts 12
  29. 29. Architecture iOS features iOS App -------------------- Cucumber Calabash library (ruby api) Test report (cucumber)fredag den 23. marts 12
  30. 30. LessPainful Test Execution in the Cloud • Execute Calabash tests concurrently on many devices, OS’es, languages. • Authentic: Not jailbroken, iOS and Android devices, rotation. • Visual test reports. • Comparison across models and operating systems. • Continuous integration.fredag den 23. marts 12
  31. 31. fredag den 23. marts 12
  32. 32. fredag den 23. marts 12
  33. 33. fredag den 23. marts 12
  34. 34. Calabash iOS: more detail 15fredag den 23. marts 12
  35. 35. Calabash: more detail • Very easy to get started for iOS developers/QAs. • Declarative query language for finding views. • Based on UISpec, but simplified and extended. (New Implementation, EPL licensed). • Touch synthesis, supports (multitouch) gestures. Extensible. (pan, swipe, pinch, your own) • Full power of Ruby programming language for test logic • Interactive, exploratory development experience. • Can use device accessibility for identifying views. 16fredag den 23. marts 12
  36. 36. Queries 17fredag den 23. marts 12
  37. 37. Queries • Queries are like CSS selectors or XPath • label text:‘Hello‘ • label index:2 • view marked:‘thepane’ label • view:‘MyClassName’ • label {text LIKE ‘Hel*’} • webView css:‘#header a.cssclass‘ • webView xpath:’//node()’ 17fredag den 23. marts 12
  38. 38. Demo: - Calabash iOS - LessPainful Device Cloud 18fredag den 23. marts 12
  39. 39. iOS Comparisons • Several options available. To my knowledge: • Calabash • UIAutomation, Apple • Zucchini, iOS Testing Framework • Frank, Pete Hodgson, ThoughtWorks • UISpec, http://code.google.com/p/uispec/ • FoneMonkey => MonkeyTalk, GorillaLogic • KIF, Square • NativeDriver, http://code.google.com/p/nativedriver/ 19fredag den 23. marts 12
  40. 40. References • https://github.com/calabash • https://github.com/calabash/calabash-ios • https://github.com/calabash/calabash-ios/wiki • https://github.com/calabash/calabash-ios-server • https://github.com/calabash/calabash-android • http://blog.lesspainful.com/ • https://www.lesspainful.com/ 20fredag den 23. marts 12
  41. 41. Questions? Making app testing less painful... Please contact us with any questions: contact@lesspainful.com karl@lesspainful.com - iOS jonas@lesspainful.com - Android http://www.lesspainful.comfredag den 23. marts 12

×