Using GUI Ripping for Automated Testing of Android Apps

Using GUI Ripping for Automated Testing of Android Apps

• 1.
  Using GUI Rippingfor Automated Testing of Android Apps Domenico Amalfitano Salvatore De Carmine Anna Rita Fasolino Porfirio Tramontana Dipartimento di Informatica e Sistemistica University of Naples Federico II, Italy Atif Memon Department of Computer Science University of Maryland
• 2.
  Motivation  Android is a very popular operative system and framework for mobile devices  Its diffusion is increasing: recently it overtook ioS and Symbian in terms of diffusion  Over than 500,000 apps are now on the Google Play market  The most part of the apps are realized by single programmers or little teams and lacks in quality  Android apps are often developed according to an evolutionary life cycle, with very frequent releases via Google Play  Bugs causing failures and, in particular, crashes are very common  There is a strong need for testing automation processes and tools  In this talk, our techniques and tool supporting a process of Android GUI Testing Automation will be presented Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 2
• 3.
  Android apps  An Android application is composed of several types of Java components instantiated at run-time:  Activities, Services, Intents, Broadcast Receivers and Content Providers.  An Activity component:  is responsible for presenting a visual user interface;  provides a screen with which the users can interact in order to do something.  An Android application usually includes one or more Activity classes that extend the base Activity class provided by the Android development framework.  Android Applications are Event-Driven Systems  Three types of events can be triggered:  user events (such as Click, MouseOver, etc.);  events related to the lifecycle of the Activity classes  onStart, on Pause, onResume, onStop, onDestroy, …  events that are triggered by other input sources  such as GPS receiver, phone, network, etc. Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 3
• 4.
  Existing approaches toAndroid Testing Automation  JUnit Testing  Android Testing Framework  Supports the coding of executable test classes  Classes supporting the instrumentation and monitoring of Android apps are provided in the Android framework itself  Libraries such as Robotium and Monkeyrunner extends the framework  Random Testing  Monkey (included in the Android framework) supports the randomly automatic exploration of Android apps  Model Based Testing  TEMA Tools (Takala, Katara, Harty, 2011) represent a first tentative of approach for the generation of test code from a design model of Android app Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 4
• 5.
  Android GUI Ripping  We propose a technique and a tool for:  Automatic exploration of the GUI of an Android application via a systematic Ripping approach  Similar to approaches adopted for Web crawling  Crash testing  GUI model reverse engineering  JUnit test case generation Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 5
• 6.
  Android GUI RippingAlgorithm  Customizable  Selection of types of events to be fired  Adoption of a GUI traversing strategy (e.g. depth first, breadth first, etc.)  Termination criteria used to stop the GUI traversal  Setting of specific preconditions  Efficient  Provides heuristic solutions to the well known problem of GUI interface states and events explosion  GUI exploration criteria based on a Interface Equivalence Criterion Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 6
• 7.
  Ripping ≡ Algorithm Rotate ≡ Click Pages Click Click Refresh Press Menu 1. Task List Initialization; 1. New Post Task List Initialization; 2. while (Task List Is Not Empty) { 2. while (Task List Is Not Empty) { Extract a Task From The 3. Task List; 3. Extract a Task From The Task List; Execute the Task; 4. 5. Abstract the Current GUI Abstract State; 4. Execute the Task; … Update the GUI Tree 6. Model; 5. Abstract the Current GUI Abstract 7. if (GUI Exploration Criterion) then { State; 8. Define New Tasks ; 9. Add New Tasks To 6. Update the GUI Tree Model; The Task List; 10. } 7. if (GUI Exploration Criterion) then Click Add 11. } { Click About Account 8. Click Save Define New Tasks ; 9. Add New Tasks To The Task List; … 10. } Crash 11. } Click Edit 7 … … Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
• 8.
  Crash Testing Process  Preconconditions are set by generating a Snapshot Image of the device for the emulator  In the task execution step, the GUI Ripping algorithm is executed Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 8
• 9.
  A Demonstrational example  Android for Wordpress  Measured metrics  A Wordpress client for  Effectiveness Android  Number of crashes found  334 classes, ~10,000 LOCs  Number of different  More than 250,000 downloads bugs causing crashes  A set of three preconditions  LOCs coverage  No Login  Single Blog  Efficiency  Multiple Blogs  Number of test cases  Comparison between  Number of events Ripping and Random  Ripping time Testing with Monkey Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 9
• 10.
  Results Preconditions No Login Single Blog Multiple Blog Random (Single Blog) #Crashes 0 6 8 3 #Bugs 0 3 4 1 LOC Cov % 2.65 39.32 37.83 25.27 Time (h) 0.2 4.88 4.58 4.46 #TC 18 166 166 - #Events 36 517 502 14400 Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 10
• 11.
  Bugs and Crashes  The 14 crashes Id Crash Description Java Exception Ticket and Changeset found by the Ripping B1 The app crashes trying StringIndex https://android.tra process belongs to to opening the default OutOfBoun c.wordpress.org/tic post “Hello World” dException ket/206 https://android.tra four different bugs c.wordpress.org/ch angeset/398  Bugs have been B2 The app crashes when BadTokenE https://android.tra the Stats activity is xception c.wordpress.org/tic submitted to the bug rapidly opened closed (via the Back and ket/208 https://android.tra tracking system and key). c.wordpress.org/ch angeset/420 accepted/fixed by the B3 The app crashes when NullPointerE https://android.tra the Stats activity is open xception c.wordpress.org/tic programmer and the Refresh button is clicked while the ket/212 https://android.tra  Monkey was able to progress bar widget is still loading. c.wordpress.org/ch angeset/423 find only one of B4 The app crashes when NullPointerE https://android.tra the user opens a post xception c.wordpress.org/tic these bugs and tries to share it ket/218 within his blog. The https://android.tra crash occurs when there c.wordpress.org/ch is a single blog in the angeset/446 app. Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 11
• 12.
  Conclusions and FutureWorks  Our technique supports the execution of completely automatic generation and execution of test cases for Android applications. In our preliminary demonstration example:  Real bugs and crashes have been detected;  Good level of code coverage has been reached;  Testing time is compatible with smoke testing processes;  Reusable test cases has been automatically generated;  Our technique demonstrates its effectiveness with respect to standard Random testing tools such as Monkey Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 12
• 13.
  Future Works  This presentation covers the first part of a larger project on testing automation of Android applications  Some future works  Publication of the tool  As a standalone tool  As a service  A preliminary demo is available at http://wpage.unina.it/ptramont/GUIRipperWiki.htm  Further experiments with more applications and ripping parameters  Delay Time, ripping strategy (depth first/breadth first, activity equivalence criteria, maximum depth), input generation strategies, …  Sensor event testing Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 13
• 14.
  Porfirio Tramontana –ASE 2012 – Essen – September 6, 2012 14