AFLGo: Directed Greybox Fuzzing

PhD Thesis Defense
Presented by Marcel Boehme
Directed 
Greybox Fuzzing
Marcel Böhme Thuan Pham Abhik RoychoudhuryM.-D. Nguyen

Presented by Abhik Roychoudhury
Directed Greybox Fuzzing
Motivation
• Automated vulnerability detection techniques

1. Blackbox Fuzzing (no program analysis, no feedback)

2. Whitebox Fuzzing (mostly program analysis)

3. Greybox Fuzzing (no program analysis, but coverage feedback)

Motivation

📄 Model-Based
Blackbox
Fuzzing
Peach, Spike …
Seed Input

Motivation

📄 Model-Based
Blackbox
Fuzzing
Peach, Spike …
Seed Input
📄
📄
📄
Valid input
Semi-valid input
Semi-valid input
Mutated Inputs

Motivation

📄 Model-Based
Blackbox
Fuzzing
Input model
Peach, Spike …
Seed Input
📄
📄
📄
Valid input
Semi-valid input
Semi-valid input
Mutated Inputs

Motivation



Motivation



📄 📄📄 📄Greybox
Fuzzing
…
Seed Input

Motivation



Fuzzing
…
📄📄 Enqueue
Seed Input Mutated Inputs
Queue of  
“interesting” seeds
Retain inputs 
that increase 
coverage!

Motivation



Fuzzing
…
📄📄 EnqueueDequeue
Queue of  
Retain inputs 
that increase 
coverage!

Motivation
• Greybox Fuzzing is frequently used
• State-of-the-art in automated vulnerability detection

• Extremely eﬃcient coverage-based input generation

• All program analysis before/at instrumentation time.

• Start with a seed corpus, choose a seed ﬁle, fuzz it.

• Add to corpus only if new input increases coverage.

Motivation
• Greybox Fuzzing is frequently used
• State-of-the-art in automated vulnerability detection

• Extremely eﬃcient coverage-based input generation

• All program analysis before/at instrumentation time.

• Start with a seed corpus, choose a seed ﬁle, fuzz it.

• Add to corpus only if new input increases coverage.

• Cannot be directed!

Motivation
• Directed Fuzzing has many applications
• Patch Testing: reach changed statements
• Crash Reproduction: exercise stack trace

• SA Report Veriﬁcation: reach “dangerous” location

• Information Flow Detection: exercise source-sink pairs

Motivation
• Directed Fuzzing: classical constraint satisfaction prob.

• Program analysis to identify program paths  
that reach given program locations.

• Symbolic Execution to derive path conditions  
for any of the identified paths.

• Constraint Solving to find an input that

• satisfies the path condition and thus

• reaches a program location that was given.
φ1 = (x>y)∧(x+y>10) 
φ2 = ¬(x>y)∧(x+y>10)
x > y
a = x a = y
x+y>10
b = a
return b

Motivation

• Program analysis to identify program paths  
that reach given program locations.

• Symbolic Execution to derive path conditions  
for any of the identiﬁed paths.



φ1 = (x>y)∧(x+y>10) 
φ2 = ¬(x>y)∧(x+y>10)
x > y
a = x a = y
x+y>10
b = a
return b
Requires 
heavy-weight 
machinery!

Overview
• Directed Fuzzing as optimisation problem!

1. Instrumentation Time:

1. Extract call graph (CG) and control-ﬂow graphs (CFGs).

2. For each BB, compute distance to target locations.

3. Instrument program to aggregate distance values.

Overview





2. Runtime, for each input

1. collect coverage and distance information, and

2. decide how long to be fuzzed based on distance.

• If input is closer to the targets, it is fuzzed for longer.

• If input is further away from the targets, it is fuzzed for shorter.

Instrumentation
• Function-level target distance using call graph (CG)
main
a b
cd e

Instrumentation

1. Identify target functions in CG
main
a b
cd e

main
a b
cd e
Instrumentation

1. Identify target functions in CG

2. For each function, compute the 
harmonic mean of the length  
of the shortest path to targets
1
2
0
2
3N/A

Instrumentation

• BB-level target distance using CFG
CFG for function b
main
a b
cd e
1
2
0
2
3N/A

Instrumentation

• BB-level target distance using control-ﬂow graph (CFG)

1. Identify target BBs and 
assign distance 0 
(none in function b)
CFG for function b
main
a b
cd e
1
2
0
2
3N/A

Instrumentation


assign distance 0

2. Identify BBs that 
call functions
CFG for function b
main
a b
cd e
1
2
0
2
3N/A
c
a

Instrumentation


assign distance 0

call functions and 
assign 10*FLTD
CFG for function b
main
a b
cd e
1
2
0
2
3N/A
c
a 10
30

Instrumentation


assign distance 0

assign 10*FLTD

3. For each BB, compute harmonic 
mean of (length of shortest path to 
any function-calling BB + 10*FLTD).
CFG for function b
c
a 10
30
[(1+30)-1+(2+10)-1]-1

Instrumentation


assign distance 0

assign 10*FLTD

3. For each BB, compute harmonic 
mean of (length of shortest path to 
any function-calling BB + 10*FLTD).
CFG for function b
8.7
11
10
30
13
12
N/A

Runtime


• Seed distance from instrumented binary
CFG for function b
8.7
11
10
30
13
12
N/A

Runtime



• Two 64-bit shared memory entries

• Aggregated BB-level distance values

• Number of executed BBs
Seed Distance: 19.4  
= (8.7+30)/2
8.7
11
10
30
13
12
N/A

Runtime





8.7
11
10
30
13
12
N/A
= (8.7+11+10+12)/4

Runtime





8.7
11
10
30
13
12
N/A
= (8.7+11+10+12)/4
Now that we know how to 
compute seed distance,
— let’s minimise it! —

• Background: Coverage-based Greybox Fuzzing
Directed Fuzzing as  
Optimisation Problem
Seed File
📄

•
Mutation Operators:
• Bitﬂips
• BoundaryValues  
(0,1,-1,INT_MAX,INT_MIN)
• Simple arithmetics 
(add/subtract 1)
• Block deletion
• Block insertion
Seed File
📄

Fuzzing
…
Seed Input

Fuzzing
…
📄📄 Enqueue
Queue of  
Retain inputs 
that increase 
coverage!

Fuzzing
…
📄📄 EnqueueDequeue
Queue of  
Retain inputs 
that increase 
coverage!

📄📄
Queue of  
📄📄 📄
energy
low energy 
- generated less 
test inputs!

📄📄
Queue of  
📄📄 📄
high energy 
- generated more 
test inputs!📄
energy


• Seed’s energy:

• Number of inputs generated when chosen for fuzzing

• Local property: each seed has its own energy

• Power schedule:

• Assigns energy to seeds according to a pre-deﬁned formula

★ Boosted Greybox Fuzzing (AFLFast CCS’16)

• Assign more energy to seeds exercising low-frequency paths.
★ Directed Greybox Fuzzing (AFLGo CCS’17)
• Assign more energy to seeds that a closer to the given targets!

• Directed Greybox Fuzzing

• Assign more energy to seeds that a closer to the given targets!

• Problem (Stochastic Gradient Descent)
• If we always assign more energy to closer seeds,  
we typically reach only a local minimum, 
but never a global minimum distance!

• Solution (Simulated Annealing)

• Sometimes assign more energy to further-away seeds!

• Approaches global minimum distance.

• Simulated Annealing (SA)

• Exploration phase:

• Energy of closer seeds similar to energy 
of further-away seeds

• Exploitation phase:

• Energy of closer seeds is assigned  
to be higher and higher

• Energy of further-away seeds is  
assigned to be lower and lower

• We are increasing the “importance”  
of seed distance over time.

• Simulated Annealing (SA)

• Annealing from metallurgy: control the cooling of material 
to reduce defects (e.g., cracks or bubbles) in the material.

• Temperature T ∈ [0,1] speciﬁes “importance” of distance.

• At T=1, exploration (normal AFL)

• At T=0, exploitation (gradient descent)

• Cooling schedule controls (global) temperature

• Classically, exponential cooling.

• Integrating Simulated Annealing as power schedule

• In the beginning (t = 0min),  
assign the same energy 
to all seeds.
dene the normalized seed
set of target locations Tb .
s. This trace contains the
ed distance d(s,Tb ) as
(m,Tb )
|
(3)
set S of seeds to fuzz. We
,Tb ) as the dierence be-
he minimum seed distance
by the dierence between
0.00
0.25
0.50
0.75
1.00
0.00 0.25 0.50 0.75 1.00
Distance d(s,Tb)
Energyp(s,Tb)
t = 0min t =10min t = 80min


to all seeds.

• Later (t=10min), assign 
a bit more energy to 
seeds that are closer. 
(m,Tb )
|
(3)
0.00
0.25
0.50
0.75
1.00
0.00 0.25 0.50 0.75 1.00
Distance d(s,Tb)
Energyp(s,Tb)


to all seeds.

seeds that are closer.

• At exploitation (t=80min), 
assign maximal energy to 
seeds that are closest.
(m,Tb )
|
(3)
0.00
0.25
0.50
0.75
1.00
0.00 0.25 0.50 0.75 1.00
Distance d(s,Tb)
Energyp(s,Tb)


to all seeds.


0.00
0.25
0.50
0.75
1.00
0.00 0.25 0.50 0.75 1.00
Distance d(s,Tb)
Energyp(s,Tb)
0.00
0.25
0.50
0.75
1.00
0 20 40 60 80
Current time t (in min)
Energyp(s,Tb)
d = 1 d = 0.5 d = 0

Results
• Patch Testing: Reach changed statements

• State-of-the-art in patch testing

• KATCH (based on Klee symbolic exec. tool)

• Experimental Setup

• Reuse original KATCH-benchmark

• Measure patch coverage (#changed BBs reached)

• Measure vuln. detection (#errors discovered)
KATCH: High-Coverage Testing of Software Patches
Paul Dan Marinescu
Department of Computing
Imperial College London, UK
p.marinescu@imperial.ac.uk
Cristian Cadar
Department of Computing
Imperial College London, UK
c.cadar@imperial.ac.uk
ABSTRACT
One of the distinguishing characteristics of software systems
is that they evolve: new patches are committed to software
repositories and new versions are released to users on a
continuous basis. Unfortunately, many of these changes
bring unexpected bugs that break the stability of the system
or a↵ect its security. In this paper, we address this problem
using a technique for automatically testing code patches.
Our technique combines symbolic execution with several
novel heuristics based on static and dynamic program anal-
!#$%'$!(
Figure 1: KATCH is integrated in the software

Results



• Patch Coverage (#changed BBs reached)
is unrelated to the concept [32]. We make a conscious eort to
explain the observed phenomena and distinguish conceptual from
technical origins. Moreover, we encourage the reader to consider
the perspective of a security researcher who is actually handling
these tools to establish whether there exists a vulnerability.
5.1 Patch Coverage
We begin by analyzing the patch coverage achieved by both K
and AFLG as measured by the number of previously uncovered
basic blocks that were changed in the respective patch.
Table 1: Patch coverage results showing the number of previ-
ously uncovered targets that K and AFLG could cover
in the stipulated time budget, respectively.
#Changed #Uncovered
Basic Blocks Changed BBs K AFLG
Binutils 852 702 135 159
Diutils 166 108 63 64
Sum 1018 810 198 223
complex inp
executed onl
for certain a
dened stru
higher-quali
approach [2
To unders
we investiga
we can see
cannot cove
cover. We at
vidual streng
dicult con
be dicult t
quickly exp
stuck in a pa
AFLG and
282 targets
cover indi

Results




• While we would expect Klee to take a substantial lead,  
AFLGo outperforms KATCH in terms of patch coverage.
is unrelated to the concept [32]. We make a conscious eort to
explain the observed phenomena and distinguish conceptual from
technical origins. Moreover, we encourage the reader to consider
the perspective of a security researcher who is actually handling
these tools to establish whether there exists a vulnerability.
5.1 Patch Coverage
We begin by analyzing the patch coverage achieved by both K
and AFLG as measured by the number of previously uncovered
basic blocks that were changed in the respective patch.
Table 1: Patch coverage results showing the number of previ-
ously uncovered targets that K and AFLG could cover
in the stipulated time budget, respectively.
#Changed #Uncovered
Basic Blocks Changed BBs K AFLG
Binutils 852 702 135 159
Diutils 166 108 63 64
Sum 1018 810 198 223
complex inp
executed onl
for certain a
dened stru
higher-quali
approach [2
To unders
we investiga
we can see
cannot cove
cover. We at
vidual streng
dicult con
be dicult t
quickly exp
stuck in a pa
AFLG and
282 targets
cover indi

Results




• While we would expect Klee to take a substantial lead,  
AFLGo outperforms KATCH in terms of patch coverage.

• BUT: Together they cover 42% and 26%  
more than AFLGo and KATCH individually.  
They complement each other! 
oject Tools diff, sdiff, diff3, cmp
ogram Size 42,930 LoC
n Commits 175 commits from Nov’09–May’12
GNU Diutils
oject Tools
addr2line, ar, cxxfilt, elfedit, nm,
objcopy, objdump, ranlib, readelf
size, strings, strip
ogram Size 68,830 LoC + 800kLoC from libraries
n Commits 181 commits from Apr’11–Aug’12
GNU Binutils
K — 59 139 84 — AFLG

Results




• Vulnerability Detection (#errors discovered)
As future work, we are planning to integrate symbolic-execution-
based directed whitebox fuzzing and directed greybox fuzzing to
achieve a directed fuzzing technique that is both very eective and
very ecient in terms of reaching pre-specied target locations.
We believe that such an integration would be superior to each
technique individually. An integrated directed fuzzing technique
that leverages both symbolic execution and search as optmization
problem would be able to draw on their combined strengths to
mitigate their indiviual weaknesses. Driller [38] is an example of
an (undirected) fuzzer that integrates the AFL greybox fuzzer and
the K whitebox fuzzer.
5.2 Vulnerability Detection
Table 2: Showing the number of previously unreported bugs
found by AFLG (reported Apr’2017) in addition to the num-
ber of bug reports for K (reported Feb’2013).
K AFLG
#Reports #Reports14 #New Reports #CVEs
Binutils 7 4 12 7
Diutils 0 N/A 1 0
Sum 7 4 13 7
AFLG found 13 previously unreported bugs in addition to 4 of
Table 3: Bug rep
Report-ID
Binutils
21408
21409
21412
21414
21415
21417
Diutils http://lists.g
We attribute m
to the eciency of
program analysis a
magnitute more in
is the runtime che
error detection mec
the program when
we instrumented o
Sanitizer (ASAN). I
e.g., by reading be
writing memory th
a SEGFAULT even
fuzzer uses this sig
for a generated in
(i.e., interprets) th
and uses constrain

Results




• Vulnerability Detection (#errors discovered)

• AFLGo found 13 previously unreported bugs (7 CVEs)  
in addition to 4 of the 7 bugs that were found by KATCH.
As future work, we are planning to integrate symbolic-execution-
based directed whitebox fuzzing and directed greybox fuzzing to
achieve a directed fuzzing technique that is both very eective and
very ecient in terms of reaching pre-specied target locations.
We believe that such an integration would be superior to each
technique individually. An integrated directed fuzzing technique
that leverages both symbolic execution and search as optmization
problem would be able to draw on their combined strengths to
mitigate their indiviual weaknesses. Driller [38] is an example of
an (undirected) fuzzer that integrates the AFL greybox fuzzer and
the K whitebox fuzzer.
5.2 Vulnerability Detection
Table 2: Showing the number of previously unreported bugs
found by AFLG (reported Apr’2017) in addition to the num-
ber of bug reports for K (reported Feb’2013).
K AFLG
#Reports #Reports14 #New Reports #CVEs
Binutils 7 4 12 7
Diutils 0 N/A 1 0
Sum 7 4 13 7
AFLG found 13 previously unreported bugs in addition to 4 of
Table 3: Bug rep
Report-ID
Binutils
21408
21409
21412
21414
21415
21417
Diutils http://lists.g
We attribute m
to the eciency of
program analysis a
magnitute more in
is the runtime che
error detection mec
the program when
we instrumented o
Sanitizer (ASAN). I
e.g., by reading be
writing memory th
a SEGFAULT even
fuzzer uses this sig
for a generated in
(i.e., interprets) th
and uses constrain

Results
• Crash Reproduction: Exercise stack trace

• State-of-the-art in crash reproduction

• BugRedux (based on Klee symbolic exec. tool)


• Reuse original BugRedux-benchmark

• Determine whether or not crash can be reproduced
Software
developer
Instrumenter
Application Instrumented
application
Crash report
(execution data)
Analyzer
BugRedux
Test input
Debugging
tool
Software
tester
Figure 4. Intuitive high-level view of BUGREDUX.
Test input
Oracle

Results
• Crash Reproduction: Exercise stack trace

• State-of-the-art in crash reproduction

• BugRedux (based on Klee symbolic exec. tool)


• Reuse original BugRedux-benchmark

• Determine whether or not crash can be reproduced
experimental setup, starting conguration, and timeouts. BugRedux
[18] is a directed whitebox fuzzer based on K, takes as input a
sequence of program statements, and generates as output a test case
that exercises that sequence and crashes the program. It was shown
that BugRedux works best of the complete method-call sequence is
provided that lead to the crash. However, as discussed earlier often
only the stack-trace is available, which does not contain methods
that have already “returned”, i.e., nished execution. Hence, for our
comparison, we set the method-calls in the stack trace as targets.
Despite our request for all subjects from the original dataset, only
a subset of nine subjects could be located for us. For two subjects
(exim, xmail), we could not obtain the stack-trace that would specify
the target locations. Specically, the crash in exim can only be
reproduced on 32bit architectures while the crash in xmail overows
the stack such that the stack-trace is overridden. The results for the
remaining seven subjects are shown in Table 6.
Table 6: Bugs reproduced for the original BugRedux subjects
Subjects BugRedux AFLG Comments
sed.fault1 7 7 Takes two les as input
sed.fault2 7 3
grep 7 3
gzip.fault1 7 3
gzip.fault2 7 3
ncompress 3 3
polymorph 3 3
same benchmarks that th
in the original papers [18
The second concern is
a study minimizes system
nal validity for fuzzer exp
However, for our experi
was readily available, suc
Binutils and the K ex
OSS-Fuzz experiments, a
classically provides for th
when comparing two fuzz
seed corpus such that bot
Second, like implementati
faithfully implement the
shown in the comparison
The third concern is co
a test measures what it c
note that results of tool co
grain of salt. An empirical
implementations of two c
selves. Improving the ec
fuzzer may only be a que
lated to the concept [32].
explain the observed phen
technical origins. Moreov
the perspective of a secur
these tools to establish w
AFLGo reproduces  
3 times more crashes!

Result Summary
• Our directed greybox fuzzer (AFLGo) outperforms  
symbolic execution-based directed fuzzers (KATCH BugRedux)

• in terms of reaching more target locations and

• in terms of detecting more vulnerabilities,
• on their own, original benchmark sets.

Conclusion
Motivation

• Program analysis to identify program paths that reach given
program locations.

• Symbolic Execution to derive path conditions for any of the
identiﬁed paths.



Requires 
heavy-weight 
machinery!
Overview










to all seeds.


0.00
0.25
0.50
0.75
1.00
0.00 0.25 0.50 0.75 1.00
Distance d(s,Tb)
Energyp(s,Tb)
0.00
0.25
0.50
0.75
1.00
0 20 40 60 80
Energyp(s,Tb)
d = 1 d = 0.5 d = 0
Result Summary


*Tool comparisons should always be taken with a grain of salt.
*

Conclusion
Motivation

• Program analysis to identify program paths that reach given
program locations.

• Symbolic Execution to derive path conditions for any of the
identiﬁed paths.



Requires 
heavy-weight 
machinery!
Overview










to all seeds.


0.00
0.25
0.50
0.75
1.00
0.00 0.25 0.50 0.75 1.00
Distance d(s,Tb)
Energyp(s,Tb)
0.00
0.25
0.50
0.75
1.00
0 20 40 60 80
Energyp(s,Tb)
d = 1 d = 0.5 d = 0
Result Summary


*Tool comparisons should always be taken with a grain of salt.
*
Questions?

AFLGo: Directed Greybox Fuzzing

More Related Content

What's hot

Similar to AFLGo: Directed Greybox Fuzzing

More from mboehme

Recently uploaded

AFLGo: Directed Greybox Fuzzing