Android testing


Published on

This presentation was presented in Android Only! 2011 conference on June 14th.

With more than 300 different Android devices out on 6 different platform versions, application developers are facing a real nightmare when trying validate that their applications really work on their customers' devices. While fragmentation is a new thing in Android platform, it is not new in software industry and there are several ways to deal with device fragmentation from testing point of view.

This presentation discusses most common approaches to tackle fragmentation from application developer's point of view and explains why testing for device compatibility is a must for any serious Android application developer.

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Android testing

  1. 1. Strategies for coping with device fragmentation<br />
  2. 2. About Bitbar<br /> A technology company specialized in creating tools and environments for Android testing<br /> Experience ranging from hosting the build and test process of the whole Android stack to creating custom application testing device clusters<br /> Products: Testdroid Recorder and Testdroid Server<br />
  3. 3. What problem are we solving?<br />
  4. 4. Never able to log in! Uninstalling… Galaxy S<br />Hung phone, drains power (Inspire)<br />Real junk, although installed would not run<br />Doesn’t work half of the time – HTC Evo<br />Why is it important?<br />Does not work on Motorola Zoom<br />I cannot get this program to uninstall<br />There seems to be more bugs than before update<br />Xperia X10 - Widget doesn’t update and crashes all the time<br />Could not connect using my Incredible<br />
  5. 5. Approaches for managing this problem<br />1. Do nothing<br />2. Test manually every time you publish a new version<br />3. Outsource your manual testing to low cost countries<br />4. Automate your tests and run those on real hardware every time the code is changed<br />
  6. 6. Frameworks for automated Android testing<br />Monkey – a random UI exerciser tool<br />Monkeyrunner – Jython based test scripting tool <br />Robotium – It’s like Selenium but for Android <br />Roboelectric – Unit testing for Android applications <br />Cucumber – scenario based testing implemented in Ruby <br />
  7. 7. Monkey<br /> A UI/Application exerciser monkey for creating stream of random user events such as clicks, touches, gestures and some system level events <br /> Useful way to stress test your application to find non-obvious usage patterns and corner cases <br />$ adb shell monkey -v 500<br />
  8. 8. Monkeyrunner<br /> An API toolkit for writing programs that control an Android device or emulator from outside of Android code<br /># Runs the component<br />device.startActivity(component=runComponent)<br /># Presses the Menu button<br />'KEYCODE_MENU','DOWN_AND_UP')<br /># Takes a screenshot<br />result = device.takeSnapshot()<br /># Writes the screenshot to a file<br />result.writeToFile('myproject/shot1.png','png')<br />
  9. 9. Robotium<br /> Powerful and robust automatic black-box test cases for Android applications <br /> Function, system and acceptance test scenarios spanning multiple Android activities<br />@Smoke<br />public void AddNewHudson() throws Exception {<br />solo.clickOnText("Configuration");<br /> solo.enterText(0, "hudson:8080");<br /> solo.enterText(1, "hudson");<br /> solo.clickOnImageButton(0);<br />boolean expected = true;<br />boolean actual = solo.searchText("(http://hudson:8080)"); <br />assertEquals("Can't add new Hudson Instance", expected, actual);<br />}<br />
  10. 10. Roboelectric<br />public class MyActivityTest {<br /> private Activity activity;<br /> private Button pressMeButton;<br /> private TextView results;<br /> @Before<br /> public void setUp() throws Exception {<br /> activity = new MyActivity();<br />activity.onCreate(null);<br />pressMeButton = (Button) activity.findViewById(;<br /> results = (TextView) activity.findViewById(;<br /> }<br /> @Test<br /> public void shouldUpdateResultsWhenButtonIsClicked() throws Exception {<br />pressMeButton.performClick();<br /> String resultsText = results.getText().toString();<br />assertThat(resultsText, equalTo("Testing Android Rocks!"));<br /> }<br />}<br /> Running unit tests on virtual machine – instead of running the tests on emulator or real device<br />
  11. 11. Cucumber<br /> Scenario based testing implemented in Ruby<br /> Use cases are described in natural language where keywords are mapped to Android activities<br />Given My ”TestExample" app is running<br /> And I wait for "Do Androids Dream of Electric Sheep?" to appear<br /> Then take picture<br /> Then I press the menu key<br /> Then take picture<br /> And I select ”Options" from the menu<br /> Then take picture<br /> And I wait for "Enter text into input field" to appear<br /> And I select "Enter text into input field"<br /> And I wait for "Save new configuration" to appear<br /> Then take picture<br />
  12. 12. Running tests on Android<br />InstrumentationTestRunner – the primary plumbing for running tests on Android<br /> Android Debug Bridge (ADB) – connects the devices and manages all debug information flow<br /> Asserts – rich validation capabilities enabled by JUnit. Helps validating test results, system outputs or practically anything.<br /> Mock objects – methods for creating mock system objects such as content, service and intent.<br />
  13. 13. Testing for device compliancy<br />1. Test only on real devices<br />2. Select at least one phone with each major Android release<br />3. Choose different form factors: Side sliders, tablets, small resolutions , large resolutions<br />4. Integrate test execution to your development process – catch failures early in development and notice regressions immediately<br />
  14. 14. Some best practices <br />1. Application developers should create the test cases at the same time when they are writing the code<br />2. All test cases should be stored in version control – together with the source code <br />3. Use Continuous Integration and run the tests every time the code is changed <br />4. Avoid using emulators and rooted devices<br />5. Publish the test results to whole organization – real time<br />
  15. 15. Typical failures <br />1. Application installation fails<br />2. Application crashes during execution<br />3. Scaling/Layout problems<br />4. Application hangs if some resource is not available<br />5. Problems in landscape/portrait mode<br />
  16. 16. Our solution for multi device testing<br />Testdroid Recorder<br />An Eclipse plugin that automates the manual tasks of writing tests for Android applications<br />Testdroid Server<br />A server software that manages all aspects of automatically running tests on multiple Android devices simultaneously.<br />
  17. 17. Demo (Video)<br />
  18. 18. Questions?<br />
  19. 19. Thank you!<br />