The document presents a technique called GUI ripping for automatically testing Android apps. GUI ripping systematically interacts with an app's user interface to generate test cases. When applied to an Android wordpress app, GUI ripping found 14 crashes from 4 bugs, whereas random testing found only 1 of the bugs. Future work will include publishing the GUI ripping tool and experimenting with more apps and test parameters.

1. Using GUI Ripping for Automated
Testing of Android Apps
Domenico Amalfitano
Salvatore De Carmine
Anna Rita Fasolino
Porfirio Tramontana
Dipartimento di Informatica e Sistemistica
University of Naples Federico II, Italy
Atif Memon
Department of Computer Science
University of Maryland

2. Motivation
 Android is a very popular operative system and framework for
mobile devices
 Its diffusion is increasing: recently it overtook ioS and Symbian in terms of
diffusion
 Over than 500,000 apps are now on the Google Play market
 The most part of the apps are realized by single programmers or little teams and
lacks in quality
 Android apps are often developed according to an evolutionary life
cycle, with very frequent releases via Google Play
 Bugs causing failures and, in particular, crashes are very common
 There is a strong need for testing automation processes
and tools
 In this talk, our techniques and tool supporting a process of Android
GUI Testing Automation will be presented
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 2

3. Android apps
 An Android application is composed of several types of Java
components instantiated at run-time:
 Activities, Services, Intents, Broadcast Receivers and Content Providers.
 An Activity component:
 is responsible for presenting a visual user interface;
 provides a screen with which the users can interact in order to do
something.
 An Android application usually includes one or more Activity
classes that extend the base Activity class provided by the
Android development framework.
 Android Applications are Event-Driven Systems
 Three types of events can be triggered:
 user events (such as Click, MouseOver, etc.);
 events related to the lifecycle of the Activity classes
 onStart, on Pause, onResume, onStop, onDestroy, …
 events that are triggered by other input sources
 such as GPS receiver, phone, network, etc.
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 3

4. Existing approaches to Android Testing Automation
 JUnit Testing
 Android Testing Framework
 Supports the coding of executable test classes
 Classes supporting the instrumentation and monitoring of Android apps are provided in the
Android framework itself
 Libraries such as Robotium and Monkeyrunner extends the framework
 Random Testing
 Monkey (included in the Android framework) supports the randomly
automatic exploration of Android apps
 Model Based Testing
 TEMA Tools (Takala, Katara, Harty, 2011) represent a first tentative of
approach for the generation of test code from a design model of
Android app
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 4

5. Android GUI Ripping
 We propose a technique and a tool for:
 Automatic exploration of the GUI of an
Android application via a systematic Ripping
approach
 Similar to approaches adopted for Web crawling
 Crash testing
 GUI model reverse engineering
 JUnit test case generation
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 5

6. Android GUI Ripping Algorithm
 Customizable
 Selection of types of events to be fired
 Adoption of a GUI traversing strategy (e.g. depth first, breadth first, etc.)
 Termination criteria used to stop the GUI traversal
 Setting of specific preconditions
 Efficient
 Provides heuristic solutions to the well known problem of GUI interface
states and events explosion
 GUI exploration criteria based on a Interface Equivalence Criterion
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 6

7. Ripping
≡ Algorithm
Rotate ≡ Click Pages
Click
Click Refresh Press Menu 1. Task List Initialization;
1. New Post
Task List Initialization; 2.
while (Task List Is Not
Empty) {
2.
while (Task List Is Not Empty) { Extract a Task From The
3.
Task List;
3.
Extract a Task From The Task List; Execute the Task;
4.
5. Abstract the Current GUI
Abstract State;
4.
Execute the Task;
… Update the GUI Tree
6.
Model;
5.
Abstract the Current GUI Abstract 7.
if (GUI Exploration
Criterion) then {
State; 8.
Define New Tasks ;
9.
Add New Tasks To
6.
Update the GUI Tree Model; The Task List;
10.
}
7.
if (GUI Exploration Criterion) then Click Add 11.
}
{ Click About
Account
8. Click Save
Define New Tasks ;
9.
Add New Tasks To The Task
List;
…
10.
}
Crash
11.
}
Click
Edit
7
… …
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012

8. Crash Testing Process
 Preconconditions are set
by generating a Snapshot
Image of the device for
the emulator
 In the task execution step,
the GUI Ripping algorithm
is executed
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 8

9. A Demonstrational example
 Android for Wordpress  Measured metrics
 A Wordpress client for  Effectiveness
Android  Number of crashes
found
 334 classes, ~10,000 LOCs
 Number of different
 More than 250,000 downloads
bugs causing crashes
 A set of three preconditions  LOCs coverage
 No Login
 Single Blog  Efficiency
 Multiple Blogs  Number of test cases
 Comparison between  Number of events
Ripping and Random  Ripping time
Testing with Monkey
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 9

10. Results
Preconditions
No Login Single Blog Multiple Blog Random
(Single Blog)
#Crashes 0 6 8 3
#Bugs 0 3 4 1
LOC Cov % 2.65 39.32 37.83 25.27
Time (h) 0.2 4.88 4.58 4.46
#TC 18 166 166 -
#Events 36 517 502 14400
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 10

11. Bugs and Crashes
 The 14 crashes Id Crash Description Java
Exception
Ticket and
Changeset
found by the Ripping B1 The app crashes trying StringIndex https://android.tra
process belongs to
to opening the default OutOfBoun c.wordpress.org/tic
post “Hello World” dException ket/206
https://android.tra
four different bugs c.wordpress.org/ch
angeset/398
 Bugs have been B2 The app crashes when BadTokenE https://android.tra
the Stats activity is xception c.wordpress.org/tic
submitted to the bug rapidly opened
closed (via the Back
and ket/208
https://android.tra
tracking system and key). c.wordpress.org/ch
angeset/420
accepted/fixed by the B3 The app crashes when NullPointerE https://android.tra
the Stats activity is open xception c.wordpress.org/tic
programmer and the Refresh button
is clicked while the
ket/212
https://android.tra
 Monkey was able to progress bar widget is
still loading.
c.wordpress.org/ch
angeset/423
find only one of B4 The app crashes when NullPointerE https://android.tra
the user opens a post xception c.wordpress.org/tic
these bugs
and tries to share it ket/218
within his blog. The https://android.tra
crash occurs when there c.wordpress.org/ch
is a single blog in the angeset/446
app.
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 11

12. Conclusions and Future Works
 Our technique supports the execution of
completely automatic generation and execution
of test cases for Android applications.
In our preliminary demonstration example:
 Real bugs and crashes have been detected;
 Good level of code coverage has been reached;
 Testing time is compatible with smoke testing
processes;
 Reusable test cases has been automatically generated;
 Our technique demonstrates its effectiveness with
respect to standard Random testing tools such as
Monkey
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 12

13. Future Works
 This presentation covers the first part of a larger
project on testing automation of Android
applications
 Some future works
 Publication of the tool
 As a standalone tool
 As a service
 A preliminary demo is available at
http://wpage.unina.it/ptramont/GUIRipperWiki.htm
 Further experiments with more applications and
ripping parameters
 Delay Time, ripping strategy (depth first/breadth first, activity
equivalence criteria, maximum depth), input generation
strategies, …
 Sensor event testing
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 13

14. Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012 14