Android has become the most popular mobile OS, as it enables device manufacturers to introduce customizations to compete with value-added services. However, customizations make the OS less dependable and secure, since they can introduce software flaws. Such flaws can be found by using fuzzing, a popular testing technique among security researchers. We present Chizpurfle, a novel “gray-box” fuzzing tool for vendor-specific Android services. Testing these services is challenging for existing tools, since vendors do not provide source code and the services cannot be run on a device emulator. Chizpurfle has been designed to run on an unmodified Android OS on an actual device. The tool automatically discovers, fuzzes, and profiles proprietary services. We evaluate the applicability and performance of Chizpurfle on the Samsung Galaxy S6 Edge, and discuss software bugs found in privileged vendor services.

1. CHIZPURFLE:
A GRAY-BOX ANDROID FUZZER
FOR VENDOR SERVICE CUSTOMIZATIONS
Antonio Ken Iannillo, Roberto Natella, Domenico Cotroneo, Cristina Nita-Rotaru
CHIZPURFLE is a reference to the wizarding world of J. K. Rowling
The 28th IEEE International Symposium on Software Reliability Engineering (ISSRE)
October 23-26,2017,Toulouse,France
Best Research Paper Award

2. ANDROID
• Android is the major OS for smartphones
• More than 20 Original Equipment Manufacturers (OEMs)
base their devices on theAndroid Open-Source Project
(AOSP)
• adding hardware and software customizations
• Vendor customizations often introduce new vendor-
specific software defects
• do not benefit from the feedback loop of the Android ecosystem
• Vendors’ customizations are often code runningwith
special privileges
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 1

3. ANDROID VENDOR CUSTOMIZATIONS
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 2
device drivers,stock applications,and system services
69%
31%
Samsung Galaxy S6 Edge
45%
55%
Huawei P8 Lite
28%
72%
HTC One M9

4. Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 3
FUZZING
WELL-ESTABLISHED AND EFFECTIVE
TESTING TECHNIQUE
TO IDENTIFY WEAKNESSES IN FRAGILE
SOFTWARE INTERFACES
BY INJECTING
INVALID AND UNEXPECTED INPUTS

5. BLACK-BOX
easy to
implement
• random
inputs, or
• grammar-
based inputs
low
coverage
• may
repeatedly
execute the
same code
paths over
and over
high
coverage
• exploits the
visibility of
the paths
covered by
the tests
hard to
implement
• needs source
code, or
• needs the
ability to run
in virtualized
environments
WHITE-BOX
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 4
FUZZING
Vendor customizations do not provide source
code, and cannot run on a device emulator!

6. CHIZPURFLE: A GRAY-BOX SOLUTION
• Chizpurfle is a fuzzing tool designed
to run on the actual device from
the vendor
• no need for recompiling the target code
• no execution in a special environment
• It automatically identifies and fuzzes
vendor customizations on the
deviceChizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 5

7. ARCHITECTURE OF CHIZPURFLE
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 6
ANDROID DEVICE
METHOD
EXTRACTOR
INSTRUMENTATION
MODULE
SEED
MANAGER
FUZZ INPUT
GENERATOR
TEST
EXECUTOR
OUTPUT
ANALYZER
STORAGE
ORCHESTRATOR
SYSTEM SERVICE
Identifies the Android
services that have been
added or modified by the
vendor

8. ARCHITECTURE OF CHIZPURFLE
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 7
ANDROID DEVICE
METHOD
EXTRACTOR
INSTRUMENTATION
MODULE
SEED
MANAGER
FUZZ INPUT
GENERATOR
TEST
EXECUTOR
OUTPUT
ANALYZER
STORAGE
ORCHESTRATOR
SYSTEM SERVICE
This loop iterates over service inputs
to maximize the test coverage
Computes the test score for
every tested input (the more
new blocks are covered, the
higher the score)
Keeps track of test coverage
(using dynamic binary
instrumentation)

9. CHIZPURFLE TARGET PROCESS
INSTRUM.
MODULE
STALKER
SERVER
PROCESS
THREAD
for each test
for each block
DYN LIB INJECT
FOLLOW
INJECT
START
STOP
ADDRESSES
REWRITE
BLOCK
ADDRESS
CHIZPURFLE:
INSTRUMENTATION
MODULE
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 8
The instrumentation module injects a
small server (“stalker”) in the target
process using ptrace()
Before a code block is executed,the
stalker rewrites the branch
instructions in the block,replacing
them with a branch to the stalker’s
code (in order to instrument the next
code block, and to track coverage)
mov x29, sp
mov x3, x30
ldp x29,x30,[sp],16
stp x29,x30,[sp-16]!
add x30,x0,#4
save address
stalk (bl f_label)
save address
stalk(ret)
mov x29, sp
mov x3, x30
ldp x29,x30,[sp],16
stp x29,x30,[sp-16]!
add x30,x0,#4
ret
bl f_label
Block
(before rewriting)
Block
(after rewriting)

10. CHIZPURFLE:
FUZZ INPUT GENERATOR
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 9
The seed manager generates
new test inputs, by mutating
previous test inputs that
increased the test coverage (the
test score π)
Previous test inputs are randomly
selected: the higher the test score
π, the higher the probability of
selecting that test input
com.samsung.android.cocktailbar.ICocktailBarService
{
"name":"updateCocktail",
"parameters":[
"java.lang.String",
"com.samsung.android.cocktailbar.CocktailInfo",
"int”
]
}
“string” a CocktailInfo object 2
• Random
• Substring
• Truncate
• Substitute Char
• Very long string
• Null
• New object
with random
values
• Fuzz field
values
• Random
• Zero
• One
• Add/Sub
Random
Delta
• Max value
• Min value

11. EVALUATION CAMPAIGN
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 10
Samsung Galaxy S6 Edge – Android 7.0 Nougat

12. FUZZ TESTING CAMPAIGN
• Chizpurfle detected 2,272 service methods
from Samsung customizations
• Chizpurfle performed 34,645 tests on these
methods
• Found failures were critical
• 9 tests failed, due to 2 distinct bugs
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 11

13. BUGS IN SAMSUNG S6 EDGE
(CASE 1)
System Server process
reboots à Smartphone
reboots
FATAL EXCEPTION
(NullPointerException)
Not trivial input,detected by
the gray-box approach
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 12
SPENGESTURE SERVICE
injectInput(…, android.view.InputEvent [ ], ...)
android.view.InputEvent array is
non-null and non-empty, and at
least one of its elements is null
REBOOT

14. BUGS IN SAMSUNG S6 EDGE
(CASE 2)
Phone process crashes à
Call interrupted
SQLite-Exception
input strings that include
specific SQL control
expressions (such as single
quotes),triggering an SQL
injection
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 13
VOIP SERVICE
callInVoIP(String SIPAddress)
SELECT reject_number FROM
reject_num WHERE reject
number=’001?u0 [...a random string
with a single quote...]
CRASH

15. MEASUREMENTS
• throughput
• code coverage
ANDROID DEVICE
METHOD
EXTRACTOR
SEED
MANAGER
FUZZ INPUT
GENERATOR
TEST
EXECUTOR
STORAGE
ORCHESTRATOR
SYSTEM SERVICE
PERFORMANCE EVALUATION METHODOLOGY
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 14
INSTRUMENTATION
MODULE
OUTPUT
ANALYZER

16. MEASUREMENTS
PERFORMANCE
OVERHEAD
• throughput
• code coverage
• How much more
time does Chizpurfle
take compared to a
black-box approach?
ANDROID DEVICE
METHOD
EXTRACTOR
SEED
MANAGER
FUZZ INPUT
GENERATOR
TEST
EXECUTOR
STORAGE
ORCHESTRATOR
SYSTEM SERVICE
PERFORMANCE EVALUATION METHODOLOGY
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 15
INSTRUMENTATION
MODULE
OUTPUT
ANALYZER
we simulate a black-box fuzzer, by
disabling the instrumentation module
and the coverage analysis

17. MEASUREMENTS
PERFORMANCE
OVERHEAD
COVERAGE
GAIN
• throughput
• code coverage
• How much more
time does Chizpurfle
take compared to a
black-box approach?
• How much more
code does Chizpurfle
cover compared to a
black-box approach?
ANDROID DEVICE
METHOD
EXTRACTOR
SEED
MANAGER
FUZZ INPUT
GENERATOR
TEST
EXECUTOR
STORAGE
ORCHESTRATOR
SYSTEM SERVICE
PERFORMANCE EVALUATION METHODOLOGY
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 16
INSTRUMENTATION
MODULE
OUTPUT
ANALYZER
we again simulate a black-box fuzzer; we still generate
black-box inputs (no feedback from the test score), but
we enable the instrumentation module to measure the
coverage of black-box fuzzing

18. PERFORMANCE EVALUATION
METHODOLOGY
Throughput
Chizpurfle ChizpurfleBB
In ChizpurfleBB,the inputs are
generated randomly
we used ChizpurfleBB by
applyingthe same number
of inputs that were also
generated by the gray-box
Chizpurfle for the same
methods,and compared the
execution times (T andT’)
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 17
T
T’

19. PERFORMANCE EVALUATION
METHODOLOGY
In ChizpurfleBB+COV,
instrumentation is active
we compensate for the
slowdown due to
instrumentation by granting
ChizpurfleBB+COV a higher
time budget than gray-box
Chizpurfle fuzzing.
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 18
Code Coverage
Chizpurfle ChizpurfleBB+COV
C
C’
T T(1+T/T’)

20. CODE
COVERAGE GAIN
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 19
On average, Chizpurfle covers 2.3x more code

21. PERFORMANCE
OVERHEAD
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 20
On average, Chizpurfle has an execution slow-down of 11.97x

22. CONCLUSION
• Chizpurfle,a novel gray-box fuzzer designed to test custom system services
fromAndroid vendors
• The gray-box approach can discover relevant vulnerabilities,it has a reasonable
overhead,and it can increase the test coverage compared to the black-box
approach
• Opportunity for research on fuzzing in mobile devices,by allowing to
experiment with different heuristics for evolutionary fuzzing (e.g.,for
determining when to stop fuzzing,for prioritizing seeds,and for selecting fuzz
operators)
Chizpurfle: a Gray-Box Android Fuzzer for Vendor Service Customizations 21