System Testing with Robotium


Published on

While Unit and Integration testing in mobile software is a great way to help maintain quality and produce stable software, testing user interaction until more recently has been done solely by humans. System testing for Mobile isn't a new concept and yet is very hard to accomplish. System Testing with Robotium can help Android Developers shorten the feedback loop between QA and will help developers to start thinking the way users will actually use their software. My presentation discusses the why, what, when, where and how of System Testing with Robotium.

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

System Testing with Robotium

  1. 1. © mutualmobile SYSTEMTESTING WITH ROBOTIUM Why,What,When,Where and How
  2. 2. © mutualmobile WHO AM I? Elliott Chenger
 Android Engineer @ Mutual Mobile @echenger erchenger • Likes • Long walks on the beach • Everything Android • Ice Hockey • Auburn • Cars • Dislikes • the F word (fragmentation) • when carriers delay software updates • King Joffery from GOT • I-35 traffic
  3. 3. © mutualmobile AGENDA • Brief overview of different types ofTesting • What value does system testing add? • Different tools for system testing • What we chose and why. • Do’s and Don’ts to system testing • How to write system tests • Live Demo • Q&A
  4. 4. © mutualmobile DIFFERENTTYPES OFTESTING White Box • Tests code for precision and correctness • Tester understands and is testing internal structure and design of software • Generally Unit and Integration testing Black Box • Software internal structure is not known to the tester • Mainly done through acceptance and system testing • Tests are based on requirements
  5. 5. © mutualmobile WHAT IS SYSTEMTESTING?
  6. 6. © mutualmobile SYSTEMTESTING • Subset of Black BoxTesting • Tests software as a whole to make sure it meets specified requirements • Defined by acceptance criteria • Mimics the human
  8. 8. © mutualmobile
  9. 9. © mutualmobile SYSTEMTESTINGTOOLS • Robotium • Espresso • MonkeyTalk • UIAutomator
  10. 10. © mutualmobile WHY WE CHOSE ROBOTIUM? • Written in Java and structured like JUnit tests • Bootstrapping was easy • Allows us to write tests that mimic what a human does • Strong community support
  11. 11. © mutualmobile PROCESS/ACTIVITIES • The project kicks off and a full audit of all current system testing tools is performed. • After a month of reviewing various tools Robotium becomes the clear selection among all of the stakeholders. • A machine was purchased to be used as our testing server. • The script and boiler plate for testing was created. • Through trial and error our script and the server became more stable. • We switched from emulators to physical devices and we decided that Flow testing will be more beneficial instead of screen testing.
  12. 12. © mutualmobile • Worked with QA to identify how we are going to capture tests and communicate what should and shouldn’t be tested. • We started a pilot program for Robotium on two major projects. • Much was learned about how we actually write these tests. • Now we are going through the process of educating and advocating our department to add system testing into our normal development process
  13. 13. © mutualmobile LESSONS LEARNED
  14. 14. © mutualmobile AUTOMATE Let Continuous Integration systems do the heavy lifting for you.This will help you catch issues before it gets into QA’s hands.
  15. 15. © mutualmobile USE PHYSICAL DEVICES We have found through testing that Android emulators are simply slow and unreliable.Another benefit is being able to test on different device subsets.
  16. 16. © mutualmobile WRITETESTSTHAT MIMIC A HUMAN Put yourself in the user’s shoes, don’t get hung up testing pieces of the UI directly. Thinking this way makes your tests more stable and more valuable to QA and your customer.
  17. 17. © mutualmobile DON’T STRIVE FOR 100% COVERAGE Strive for good tests, not a certain quantity of test coverage. Beware of writing tests just to have tests. Like most things SystemTesting has a time and a place.
  18. 18. © mutualmobile SCREENTESTING VS FLOWTESTING
  19. 19. © mutualmobile SCREENTESTING
  20. 20. © mutualmobile FLOWTESTING
  21. 21. © mutualmobile CODE The fun part
  22. 22. © mutualmobile OUR EXAMPLE APP
  23. 23. © mutualmobile AGENDA • Setting up Robotium in a project • Organizing your tests • Boilerplate code • Writing tests • RunningTests • Live Demo
  24. 24. © mutualmobile SETTING UP ROBOTIUM WITH ECLIPSE • Download jar
  25. 25. © mutualmobile SETTING UP ROBOTIUM GENERIC • Download jar • Add the jar to your classpath
  26. 26. © mutualmobile SETTING UPTHE MANIFEST
  27. 27. © mutualmobile BOILER PLATE
  28. 28. © mutualmobile WRITINGYOU FIRSTTEST
  29. 29. © mutualmobile BEFORE WE START DO’S AND DON’TS • DO write tests that mimic a human. • DO write your tests to encompass the overall acceptance criteria • Assert DO NOT assume (also follow assertions with detailed error messages) • DO NOT depend on time.
 Example: Don’t assume a REST call will always take 2 seconds.
  30. 30. © mutualmobile LET’STEST A STORY • User Story:As a user I would like to be able to login to the app, so that I can see the next screen • Acceptance Criteria • Button for logging in • A place for the user to enter their name for logging in. • Once they enter their name and click the log in button they go to the next screen. • If they don’t enter a name they get an error dialog
  31. 31. © mutualmobile SO FAR • We have written tests for the happy path for testing login. • We have an Activity that has visuals but nothing else in it.
  32. 32. © mutualmobile LET’S NOW RUNTHETESTS
  34. 34. © mutualmobile NOW WE CAN WRITE SOME SAD PATH TESTS
  35. 35. © mutualmobile NOW • We have written happy path tests and they are successful. • We just finished writing our sad path tests and they should not pass. • We have an Activity that has visuals and some of our interactions but we still need to code the rest of the sad path test.
  37. 37. © mutualmobile LIVE DEMO
  38. 38. © mutualmobile WHATS NEXT? • Go explore: • Download the example app code for this presentation: • Have questions or feedback? or @echenger
  39. 39. © mutualmobile Q&A