Using GUI Ripping for Automated Testing of Android Apps

3,086 views

Published on

As mobile applications become more complex and business-critical, use of well-defined software engineering techniques becomes essential to assure the necessary software quality. In particular, testing and its automation play a strategic part for assuring the quality of applications that are often developed by small teams, working on strict timelines and under the pressure of short time-to-market. This paper presents an automated GUI based testing technique for Android apps. The technique is based on a ripper that automatically explores the GUI with the aim of exercising the application and revealing run-time crashes. At the same time, the ripper builds a GUI model and an executable test suite based on the JUnit test framework. The technique has been evaluated by an experiment where the ripper has been used to test a real Android application. The experiment has shown the fault-detection capability of the technique and its cost-effectiveness in smoke testing processes.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,086
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
49
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Using GUI Ripping for Automated Testing of Android Apps

  1. 1. Using GUI Ripping for Automated Testing of Android Apps Domenico Amalfitano Salvatore De Carmine Anna Rita Fasolino Porfirio Tramontana Dipartimento di Informatica e Sistemistica University of Naples Federico II, Italy Atif Memon Department of Computer Science University of Maryland
  2. 2. Motivation Android is a very popular operative system and framework for mobile devices  Its diffusion is increasing: recently it overtook ioS and Symbian in terms of diffusion  Over than 500,000 apps are now on the Google Play market  The most part of the apps are realized by single programmers or little teams and lacks in quality Android apps are often developed according to an evolutionary life cycle, with very frequent releases via Google Play  Bugs causing failures and, in particular, crashes are very common There is a strong need for testing automation processes and tools  In this talk, our techniques and tool supporting a process of Android GUI Testing Automation will be presentedPorfirio Tramontana – ASE 2012 – Essen – September 6, 2012 2
  3. 3. Android apps An Android application is composed of several types of Java components instantiated at run-time:  Activities, Services, Intents, Broadcast Receivers and Content Providers. An Activity component:  is responsible for presenting a visual user interface;  provides a screen with which the users can interact in order to do something. An Android application usually includes one or more Activity classes that extend the base Activity class provided by the Android development framework. Android Applications are Event-Driven Systems  Three types of events can be triggered:  user events (such as Click, MouseOver, etc.);  events related to the lifecycle of the Activity classes  onStart, on Pause, onResume, onStop, onDestroy, …  events that are triggered by other input sources  such as GPS receiver, phone, network, etc.Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 3
  4. 4. Existing approaches to Android Testing Automation JUnit Testing  Android Testing Framework  Supports the coding of executable test classes  Classes supporting the instrumentation and monitoring of Android apps are provided in the Android framework itself  Libraries such as Robotium and Monkeyrunner extends the framework Random Testing  Monkey (included in the Android framework) supports the randomly automatic exploration of Android apps Model Based Testing  TEMA Tools (Takala, Katara, Harty, 2011) represent a first tentative of approach for the generation of test code from a design model of Android app Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 4
  5. 5. Android GUI Ripping We propose a technique and a tool for:  Automatic exploration of the GUI of an Android application via a systematic Ripping approach  Similar to approaches adopted for Web crawling  Crash testing  GUI model reverse engineering  JUnit test case generationPorfirio Tramontana – ASE 2012 – Essen – September 6, 2012 5
  6. 6. Android GUI Ripping Algorithm Customizable  Selection of types of events to be fired  Adoption of a GUI traversing strategy (e.g. depth first, breadth first, etc.)  Termination criteria used to stop the GUI traversal  Setting of specific preconditions Efficient  Provides heuristic solutions to the well known problem of GUI interface states and events explosion  GUI exploration criteria based on a Interface Equivalence CriterionPorfirio Tramontana – ASE 2012 – Essen – September 6, 2012 6
  7. 7. Ripping ≡ Algorithm Rotate ≡ Click Pages Click Click Refresh Press Menu 1. Task List Initialization;1. New Post Task List Initialization; 2. while (Task List Is Not Empty) {2. while (Task List Is Not Empty) { Extract a Task From The 3. Task List;3. Extract a Task From The Task List; Execute the Task; 4. 5. Abstract the Current GUI Abstract State;4. Execute the Task; … Update the GUI Tree 6. Model;5. Abstract the Current GUI Abstract 7. if (GUI Exploration Criterion) then { State; 8. Define New Tasks ; 9. Add New Tasks To6. Update the GUI Tree Model; The Task List; 10. }7. if (GUI Exploration Criterion) then Click Add 11. } { Click About Account8. Click Save Define New Tasks ;9. Add New Tasks To The Task List; …10. } Crash11. } Click Edit 7 … … Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
  8. 8. Crash Testing Process Preconconditions are set by generating a Snapshot Image of the device for the emulator In the task execution step, the GUI Ripping algorithm is executedPorfirio Tramontana – ASE 2012 – Essen – September 6, 2012 8
  9. 9. A Demonstrational example Android for Wordpress  Measured metrics  A Wordpress client for  Effectiveness Android  Number of crashes found  334 classes, ~10,000 LOCs  Number of different  More than 250,000 downloads bugs causing crashes  A set of three preconditions  LOCs coverage  No Login  Single Blog  Efficiency  Multiple Blogs  Number of test cases  Comparison between  Number of events Ripping and Random  Ripping time Testing with MonkeyPorfirio Tramontana – ASE 2012 – Essen – September 6, 2012 9
  10. 10. Results Preconditions No Login Single Blog Multiple Blog Random (Single Blog) #Crashes 0 6 8 3 #Bugs 0 3 4 1 LOC Cov % 2.65 39.32 37.83 25.27 Time (h) 0.2 4.88 4.58 4.46 #TC 18 166 166 - #Events 36 517 502 14400Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 10
  11. 11. Bugs and Crashes The 14 crashes Id Crash Description Java Exception Ticket and Changeset found by the Ripping B1 The app crashes trying StringIndex https://android.tra process belongs to to opening the default OutOfBoun c.wordpress.org/tic post “Hello World” dException ket/206 https://android.tra four different bugs c.wordpress.org/ch angeset/398  Bugs have been B2 The app crashes when BadTokenE https://android.tra the Stats activity is xception c.wordpress.org/tic submitted to the bug rapidly opened closed (via the Back and ket/208 https://android.tra tracking system and key). c.wordpress.org/ch angeset/420 accepted/fixed by the B3 The app crashes when NullPointerE https://android.tra the Stats activity is open xception c.wordpress.org/tic programmer and the Refresh button is clicked while the ket/212 https://android.tra Monkey was able to progress bar widget is still loading. c.wordpress.org/ch angeset/423 find only one of B4 The app crashes when NullPointerE https://android.tra the user opens a post xception c.wordpress.org/tic these bugs and tries to share it ket/218 within his blog. The https://android.tra crash occurs when there c.wordpress.org/ch is a single blog in the angeset/446 app.Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 11
  12. 12. Conclusions and Future Works Our technique supports the execution of completely automatic generation and execution of test cases for Android applications. In our preliminary demonstration example:  Real bugs and crashes have been detected;  Good level of code coverage has been reached;  Testing time is compatible with smoke testing processes;  Reusable test cases has been automatically generated;  Our technique demonstrates its effectiveness with respect to standard Random testing tools such as MonkeyPorfirio Tramontana – ASE 2012 – Essen – September 6, 2012 12
  13. 13. Future Works This presentation covers the first part of a larger project on testing automation of Android applications Some future works  Publication of the tool  As a standalone tool  As a service  A preliminary demo is available at http://wpage.unina.it/ptramont/GUIRipperWiki.htm  Further experiments with more applications and ripping parameters  Delay Time, ripping strategy (depth first/breadth first, activity equivalence criteria, maximum depth), input generation strategies, …  Sensor event testingPorfirio Tramontana – ASE 2012 – Essen – September 6, 2012 13
  14. 14. Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 14

×