American Fuzzy Lop (AFL) is a security-oriented fuzz testing tool. In this talk, I demonstrate how dead-simple AFL is to use. I show how I used it to fuzz a Python library, discovering a subtle bug in the process.

1. Black Box Fuzzing
2016-10-24

2. • American Fuzzy Lop (AFL) is a fuzzing tool
• Also, a breed of rabbit
• Written by Michał Zalewski (`lcamtuf`)
• Used as part of a bunch of DEFCON presentations (including CGC)
• Big in the computer security field
• Finds crashes in programs
• Works on (instrumented) binaries
• Has no understanding of problem domain
• Has found A LOT of tricky, impressive bugs:
• http://lcamtuf.coredump.cx/afl/
• Firefox, OpenSSL, clang, glibc, perl, screen, Redis…
American Fuzzy Lop

3. • Fuzz testing is the generation of random, hopefully invalid inputs
• Meant to catch the edge cases that you didn’t anticipate / test for
Aside: Fuzz Testing

4. 1. Create a binary that reads from stdin, returns non-zero on exception
2. Give AFL the binary and a few sample inputs
3. Profit! (analyze crash logs)
Usage

5. • We want to check for crashes in the encode/decode functions of pyhocon
• HOCON (Human-Optimized Config Object Notation)
• https://github.com/typesafehub/config/blob/master/HOCON.md
• pyhocon is a python library for HOCON SerDes
• https://github.com/chimpler/pyhocon
Example

6. • We want to check for crashes in the encode/decode functions of pyhocon
1. Write a simple wrapper program
2. Since we’re in Python, we also use the python-afl library
3. Add the expected Exceptions
4. Run!
5. Wait…
Example

7. The code

8. It’s running!

9. • Bit + Byte flips
• Arithmetic offsets
• Troublesome values (0, 1, INT_MAX, etc)
• Random overwrites + appends
• Inserts/Deletes/Splices of inputs at random offsets
Checks

10. • Bit + Byte flips
• Arithmetic offsets
• Troublesome values (0, 1, INT_MAX, etc)
• Random overwrites + appends
• Inserts/Deletes/Splices of inputs at random offsets
Checks

11. • https://github.com/chimpler/pyhocon/issues/103
• It happens when you try to append to an list that is nested in a dictionary
• Internally, a boolean was being passed in when it should have been a string
• While type checking would have also found this, a person manually testing likely
would not (and did not) find it
Bug found!

12. • American Fuzzy Lop is a very good tool for black box fuzz testing of software.
• Very easy to use (nothing to learn, no domain knowledge)
• Especially useful for code:
• That is complex
• That you didn’t write
• That you don’t have the source code for
• ie. Code you don’t understand
• Further reading:
• https://github.com/mirrorer/afl/blob/master/docs/technical_details.txt
• “10/10; would crash again”
Conclusions