Yury Chemerkin, profile picture
Uploaded byYury Chemerkin
PDF, PPTX1,342 views

Fuzzing sucks!

This document introduces the Sulley fuzzing framework. It begins with background information on past fuzzing tools and their limitations. It then discusses Sulley's architecture, including its component breakdown and advanced features. Next, it covers usage and demos of Sulley through audits of Hewlett-Packard and Trend Micro software. Finally, it briefly mentions future development plans for Sulley.

Embed presentation

Download as PDF, PPTX
Background Architecture Usage and Demos Future Development Fuzzing Sucks! Introducing Sulley Fuzzing Framework Pedram Amini 1 Aaron Portnoy 2 1 pamini@tippingpoint.com 2 aportnoy@tippingpoint.com Black Hat US 2007 Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development About Us Work at TippingPoint’s Digital Vaccine Labs Responsible for vuln-dev, patch analysis, pen-testing Keep tabs on us at http://dvlabs.tippingpoint.com Launched OpenRCE.org over two years ago How many here are members? Some interesting updates on the horizon after BlackHat Creators of PaiMei RE framework How many here have heard of it? Lot of exciting developments coming up after BlackHat Co-authored ”Fuzzing: Brute Force Vulnerability Discovery” Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Talk Outline Background Why does fuzzing suck? How can we make it better? Sulley’s Architecture Component Breakdown Advanced Features Usage and Walkthrough Hewlett-Packard Data Protector Audit Trend Micro Server Protect Audit Future Development What’s still on the drawing board Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Is Fuzzing a ”Dead Horse”? Negative Entire BlackHat track, 3 dedicated books, more commercial vendors and still highly effective. Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Old School antiparser David McKinney, Python, x-platform, API driven DFUZ Diego Bauche, custom language, Unix SPIKE Dave Aitel, C, Unix, block based The list goes on ... Angel Fuzzer Framework Fuzzled Fuzzy Packet The Art of Fuzzing SPIKEfile ... Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development DFUZ FTP Example Notes The custom language is easy to understand but very limiting. port=21/tcp peer write: @ftp:user("user") peer read peer write: @ftp:pass("pass") peer read peer write: "CWD /", %random:data(1024,alphanum), 0x0a peer read peer write: @ftp:quit() peer read repeat=1024 wait=1 # No Options Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development SPIKE FTP Example Notes SPIKE data representation syntax is very simple, perhaps why it’s the most commonly used fuzzer? s_string("HOST "); s_string_variable("10.20.30.40"); s_string("rn"); s_string_variable("USER"); s_string(" "); s_string_variable("bob"); s_string("rn"); s_string("PASS "); s_string_variable("bob"); s_string("rn"); s_string("SITE "); s_string_variable("SEDV"); s_string("rn"); s_string("CWD "); s_string_variable("."); s_string("rn"); Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development New School Peach Michael Eddington, Python, x-platform, highly modularized Codenomicon Commercial vendor, Java, x-platform, pre-recorded test cases GPF Jared Demott, mixed, x-platform, varying fuzz modes Autodafe Martin Vuagnoux, C, Unix, next-gen SPIKE First fuzzer to bundle debugger functionality Evolutionary Fuzzers SideWinder, Sherri Sparks et al. EFS, Jared Demott Protocol Informatics Framework Marshall Beddoe, Python, x-platform, automated protocol field identification tool Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Peach FTP Example Notes There is a non trivial learning curve to writing Peach fuzzers. from Peach import * from Peach.Transformers import * from Peach.Generators import * from Peach.Protocols import * from Peach.Publishers import * loginGroup = group.Group() loginBlock = block.Block() loginBlock.setGenerators(( static.Static("USER usernamernPASS "), dictionary.Dictionary(loginGroup, "dict.txt"), static.Static("rnQUITrn") )) loginProt = null.NullStdout(ftp.BasicFtp(’127.0.0.1’, 21), loginBlock) script.Script(loginProt, loginGroup, 0.25).go() Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development GPF FTP Example Notes Data representation format is very different from other examples. Source:S Size:20 Data:220 (vsFTPd 1.1.3) Source:C Size:12 Data:USER jared Source:S Size:34 Data:331 Please specify the password. Source:C Size:12 Data:PASS jared Source:S Size:33 Data:230 Login successful. Have fun. Source:C Size:6 Data:QUIT Source:S Size:14 Data:221 Goodbye. ... The command line can be a bit unwieldy: GPF ftp.gpf client localhost 21 ? TCP 8973987234 100000 0 + 6 6 100 100 5000 43 finsih 0 3 auto none -G b Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development So Why Does Fuzzing Hurt So Bad? The existing tools contribute solid ideas but are limited in usage Basically all of them are focused solely on data generation Let’s jump through some fuzzer requirements to get a feel for what’s missing Essentially Chapter 5 from the fuzzing book At each juncture we’ll briefly cover Sulley’s solution We’ll drill down into the specifics when we cover architecture Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Easy to Use and Powerfully Flexible Pain Powerful frameworks have a huge learning curve Simple frameworks quickly reach limitations Remedy Sulley utilizes block based data representation Sulley fuzzers start simple and don’t have messy syntax Optional elements and keyword arguments Fuzzers are written in pure Python and can benefit from the languages features and ease of use Development efforts can be easily shared Can handle challenge-response and prev-packet-length situations Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Reproducibility and Documentation Pain Individual test cases must be reproducible Progress and interim results should be recorded Remedy Sulley can replay individual test cases Sulley keeps a bi-directional PCAP of every transaction A built in web interface provides interactive feedback Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Reusability Pain Non-generic fuzzers can never be used again Widely used protocol components are re-developed all the time Remedy Sulley supports the creation and reuse of complex types and helper functions The more Sulley is used, the smarter it gets Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Process State and Process Depth Pain None of the existing fuzzers consider state paths Fuzzing A-B-D vs. A-C-D Many fuzzers can only scratch a protocol surface Fuzzing A only Remedy In Sulley you build fuzzers in manageable chunks called requests These requests are tied together in a graph The graph is automatically walked and each state path and depth is individually fuzzed Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Tracking, Code Coverage and Metrics Pain How much of the target code was exercised? What code was executed to handle a specific test case? Remedy Sulley supports an extensible agent model Utilizes PaiMei/PyDbg for breakpoint-based and MSR-based code coverage tracking Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Fault Detection and Recovery Pain Most fuzzers rely solely on a lack of response to determine when something bad happens ahem ahem Codenomicon Once a fault is discovered, most fuzzers simply stop! Mu Security and BreakingPoint take the interesting approach of power cycling Remedy Sulley bundles a debugger monitor agent Sulley can restore target health and continue testing by: Restarting the target service Restoring a VMware snapshot Amini, Portnoy Fuzzing Sucks!
Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Resource Constraints Pain Non-technical constraints such as time and man power often get in the way Remedy Sulley bundles utilities such as a PDML parser to save time Sulley is designed to allow multiple people to work together easily The monitoring and self recording features of the framework save a great deal of time Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Sulley Architecture Diagram User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Four Major Components Data Generation You build requests out of primitives and legos Legos are complex types that extend the framework Session Management / Driver Requests are chained together in a graph to form a session The session class exposes a standalone web interface for monitoring and control The driver ties targets, agents and requests together Agents Interface with the target for instrumentation and logging purposes Utilities Standalone command line utilities that perform a variety of tasks Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities General Usage Sniff some traffic or reverse some protocol parsing binary Break the target protocol into individual requests Represent each request with a series of primitives Individual requests can be tasked out to different people Setup some targets in conjunction with various agents Write a driver script which instantiates a session and ties requests, agents and the targets together Fuzz! Review results Drill Down Let’s take a look at each individual component in detail... Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Data Generation Legos Blocks Primitives Utils Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Overview Aitel really had it right The block based approach to protocol representation is simple, flexible and powerful Protocols are represented in Sulley as a collection of primitives, blocks and block helpers These elements have many optional arguments The name optional argument gives you direct access to an element without having to walk the stack Refer to the Epydoc generation API documentation for a complete reference of optional arguments Begin the definition of a request with: s initialize("request name") Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Static and Random Primitives s static() is the simplest primitive adding a constant to the stack Aliases include s dunno(), s raw() and s unknown() s binary() is related and should be familiar to SPIKE users: s binary("0xde 0xad be ef xca fe 00 01 02 0xba0xdd") Sulley primitives are driven by heuristics, with the exception of s random() s random() used to generate random data of varying lengths min length and max length are mandatory arguments num mutations defaults to 25 and specifies how many values to cycle through prior to returning to default Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Integer Primitives Simple types for dealing with integer fields s char(), s short(), s long(), s double() Convenience aliases exist like byte, word, dword, int, etc... You can fuzz through the entire valid range Defaults to a subset of potentially interesting values To increase throughput Supports ASCII (signed or unsigned) and binary output rendering Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Strings and Delimiters s string() supports static sizes, variable padding and custom Over 1,000 test cases in string fuzz library Strings are frequently parsed into sub-fields with delimiters Sulley has a special primitive for delimiters, s delim(), here is an example: # fuzzes the string: <BODY bgcolor="black"> s_delim("<") s_string("BODY") s_delim(" ") s_string("bgcolor") s_delim("=") s_delim(""") s_string("black") s_delim(""") s_delim(">") Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Blocks Primitives (and blocks) can be organized and nested within blocks Blocks are opened and closed with s block start() and s block end() respectively Blocks can be associated with a group, encoder or dependency Grouping, encoding and dependencies are powerful features we examine individually s block start() returns True so you can tab out for readability: # Blocks must be given a name. if s block start("my block"): s string("fuzzy") s block end() Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Groups Groups tie a block to a defined set of values The block is cycled through for each value in the group Useful for representing a valid list of opcodes or verbs: # define a group primitive listing the various HTTP verbs we wish to fuzz. s group("verbs", values=["GET", "HEAD", "POST", "TRACE"]) # define a new block named "body" and associate with the above group. if s block start("body", group="verbs"): s delim(" ") s delim("/") s string("index.html") s delim(" ") s string("HTTP") s delim("/") s string("1") s delim(".") s string("1") s static("rnrn") s block end("body") Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Encoders Simple yet powerful block modifier Connect a function with a block to modify contents post render Implement compression, custom obfuscation, etc: def trend xor encode (str): key = 0xA8534344 pad = 4 - (len(str) % 4) if pad == 4: pad = 0 str += "x00" * pad while str: dword = struct.unpack("<L", str[:4])[0] str = str[4:] dword ^= key ret += struct.pack("<L", dword) key = dword return ret Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Dependencies Allow you to apply a conditional to the rendering of a block Done by specifying dep, dep value(s) and dep compare Block dependencies can be chained together arbitrarily s_short("opcode", full_range=True) if s_block_start("auth", dep="opcode", dep_value=10): s_string("aaron") s_block_end() if s_block_start("hostname", dep="opcode", dep_values=[15, 16]): s_string("aportnoy.openrce.org") s_block_end() # the rest of the opcodes take a string prefixed with two underscores. if s_block_start("something", dep="opcode", dep_values=[10, 15, 16], dep_compare="!="): s_static("__") s_int(10) s_block_end() Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Sizers Use s size() to dynamically measure and render a blocks size Many optional arguments for flexibility length of size field, default is 4 endianess of size field, default is little Output format, ”binary” (default) or ”ascii” inclusive, whether the sizer should count itself With ASCII output control signed vs. unsigned Sizers can also be fuzzed Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Checksums Similar to a sizer, s checksum() calculates and render the checksum for a block Keyword argument algorithm can be one of: ”crc32” ”adler32” ”md5” ”sha1” or any arbitrary function pointer endianness can be toggled Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Repeaters Use s repeat() to handle block repetitions Useful for fuzzing multi-entry table parsers Can variable step from min reps to max reps Alternatively the repeat factor can be tied to another variable # table entry: [type][len][string] if s block start("table entry"): s random("x00x00", 2, 2) s size("string field", length=2) if s block start("string field"): s string("C" * 10) s block end() s block end() # repeat the table entry from 100 to 1,000 reps stepping 50 elements on each iteration. s repeat("table entry", min reps=100, max reps=1000, step=50) Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Legos Sulley supports the creation of complex types called Legos Example Legos include: E-mail addresses, IP addresses, DCERPC, XDR and ASN.1 / BER primitives, XML tags, etc... The more Legos you define, the easier fuzzing is in the future class tag (blocks.block): def init (self, name, request, value, options=): blocks.block. init (self, name, request, None, None, None, None) self.value = value self.options = options # [delim][string][delim] self.push(primitives.delim("<")) self.push(primitives.string(self.value)) self.push(primitives.delim(">")) # example instantiation. s_lego("tag", "center") Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Session Management pGraph Session Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities pGraph Python graph abstraction library Developed originally for PaiMei Allows for simple graph construction, manipulation and rendering Rendering formats supported are GML, GraphViz and uDraw The session class extends from this... Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Session Class Connect multiple requests in a graph Register pre and post send callbacks Assign a callback to each edge Add multiple network targets Exposes a custom web interface Automatically communicates with registered agents Responsible for walking the graph and fuzzing at each level Tightly used and related to the creation of drivers Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Session Illustrated ehlo callback_one() pre_send() mail from rcpt to data post_send() helo callback_two() This example demonstrates multiple paths and depths Callbacks aren’t really needed here More applicable in cases of RPC, challenge-response, prev-packet specifies length, etc... Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Interactive Web Interface View fuzzer progress View detected faults Retrieve per-fault crash dump and packet capture Pause and resume fuzzing Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agents VMControl Netmon Procmon ... Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agents A flexible sub-system allows you to create custom agents Client-server communication is extremely simple over ”PedRPC” Create a class that extend from pedrpc.server Instantiate pedrpc.client Call class members as if they are local Some agents have already been developed... Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agent: Netmon Monitors network traffic and saves PCAPs to disk Per test case bi-directional packet capture ERR> USAGE: network monitor.py <-d|--device DEVICE #> device to sniff on (see list below) [-f|--filter PCAP FILTER] BPF filter string [-p|--log path PATH] log directory to store pcaps to [-l|--log level LEVEL] log level (default 1), increase for more verbosity [--port PORT] TCP port to bind this agent to Network Device List: [0] DeviceNPF GenericDialupAdapter [1] {2D938150-427D-445F-93D6-A913B4EA20C0} 192.168.181.1 [2] {9AF9AAEC-C362-4642-9A3F-0768CDA60942} 0.0.0.0 [3] {9ADCDA98-A452-4956-9408-0968ACC1F482} 192.168.81.193 ... Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agent: Procmon PyDbg based fault monitoring engine Used to monitor target health Detected faults are catalogued in a ”crash bin” Allows for simplistic (backtrace driven) fault clustering More on this later ERR> USAGE: process monitor.py <-c|--crash bin FILENAME> filename to serialize crash bin class to [-p|--proc name NAME] process name to search for and attach to [-i|--ignore pid PID] ignore this PID when searching for the target process [-l|--log level LEVEL] log level (default 1), increase for more verbosity [--port PORT] TCP port to bind this agent to Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agent: VMControl Exposes network API for VMWare instrumentation Simple PedRPC wrapper around vmrun.exe Start, stop, suspend, snapshot, revert, etc... Used to restore target health after a fault is induced ERR> USAGE: vmcontrol.py <-x|--vmx FILENAME> path to VMX to control <-r|--vmrun FILENAME> path to vmrun.exe [-s|--snapshot NAME> set the snapshot name [-l|--log level LEVEL] log level (default 1), increase for more verbosity [-i|--interactive] Interactive mode, prompts for input values [--port PORT] TCP port to bind this agent to Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
R2 R Background 3 Architecture Overview Data Representation Usage and Demos Future Development Fuzzing Session and Agents Utilities seq Driver R1 R3 R2 Fil cr R4 R5 Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Driver The driver is where it all comes together: Import requests from the request library Instantiate a session instance Instantiate and add target instances to the session Interconnect the requests to form a graph Start fuzzing This is where edge and pre/post send callbacks should be defined The driver is entirely free form, though must of them will follow a simple and similar structure Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Example Driver from sulley import * from requests import jabber def init message (sock): init = ’<?xml version="1.0" encoding="UTF-8" ?>n’ init += ’<stream:stream to="10.10.20.16" xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org">’ sock.send(init) sock.recv(1024) sess = sessions.session(session filename="audits/trillian.session") target = sessions.target("10.10.20.16", 5298) target.netmon = pedrpc.client("10.10.20.16", 26001) target.procmon = pedrpc.client("10.10.20.16", 26002) target.vmcontrol = pedrpc.client("127.0.0.1", 26003) # start up the target. target.vmcontrol.restart target() print "virtual machine up and running" sess.add target(target) sess.pre send = init message sess.connect(s get("chat message")) sess.fuzz() Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Utilities crashbin_explorer ida_fuzz_library_extender pcap_cleaner pdml_parser sequence_honer Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities crashbin explorer.py View every test case which caused a fault List every location where a fault occurred Retrieve saved crash dumps Render a graph which clusters faults by stack trace $ ./utils/crashbin explorer.py USAGE: crashbin explorer.py <xxx.crashbin> [-t|--test #] dump the crash synopsis for a specific test case number [-g|--graph name] generate a graph of all crash paths, save to ’name’.udg Pedram will actually demo this in a bit Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities ida fuzz library extender.py IDA Python script Simple concept Enumerate all constant integer comparisons in target binary Enumerate all constant string comparisons in target binary Add to fuzz library heuristics This is a pre-fuzz static analysis script A more advanced run-time implementation is still in the works Fuzz library extensions are handled via .fuzz strings and .fuzz ints Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities pcap cleaner.py Simple utility Iterates through a crashbin and removes any PCAPs not associated with a fault Save on disk space prior to archiving a completed audit $ ./utils/pcap cleaner.py USAGE: pcap cleaner.py <xxx.crashbin> <path to pcaps> Amini, Portnoy Fuzzing Sucks!
Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities pdml parser.py Convenience utility Converts PDML dump from Wireshark to a Sulley request Easier then doing so manually, work is still required of course Amini, Portnoy Fuzzing Sucks!
Background Architecture Hewlett-Packard Data Protector Usage and Demos Trend Micro ServerProtect Future Development Hewlett-Packard Data Protector Simple protocol to reverse and represent Simple bug (currently 0day, fix is just around the corner) Good starting example Amini, Portnoy Fuzzing Sucks!
Background Architecture Hewlett-Packard Data Protector Usage and Demos Trend Micro ServerProtect Future Development Trend Micro ServerProtect Microsoft RPC interface Tons of bugs in this thing Some have been reported and fixed, others are still pending Interesting demo since we can show off the fact that Sulley can fuzz DCE/RPC Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Sequence ”Honing” Say test case #100 triggered a fault but replaying it does not Probably relies on some previous test or sequence of tests We can automatically deduce the exact sequence required for replication This is done, but I’m still playing with the reduction ”algorithm” Here is how the current incarnation of the approach works... Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development The Honing ”Algorithm” Start from the last case and step back to find the window 100. 99, 100. 98, 99, 100. 40, 41 ... 99, 100 Start eliminating sequential bundles (ratio to window-size) and check for fault Once exhausted, increase granularity to eliminate single test cases at a time Sulleys health-restoration methods are used to ensure a ”clean slate” per test Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Parallel Fuzzing Combinatorial explosion is a common fuzzing problem We can increase throughput by replicating the target, ex: Run 2 targets in VMWare with PyDbg monitoring Run a target on real hardware for MSR-based code coverage recording Parallel fuzzing is as simple as instantiating and add multiple targets in your driver code Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Heuristic Feedback Loop Along the same lines as the fuzz extender command-line utility Simple concept that may improve your fuzz: Attach to the target process and trace Look for all comparisons to int and string constants Feed those back to the fuzzer, adding them to the current primitives fuzz library There are far more scientific ways of doing this Hoglund and Halvar both are researching what I call ”path-math” Was talking to a friend at Microsoft, they already have it Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Code Coverage Monitor An agent similar to netmon or procmon, records code coverage per test case There are 2 code coverage monitors completed MSR based Process Stalker based Neither are included as they add a lot more requirements (PaiMei) We’re working on PaiMei 2.0 which will be SQL driven and will include it then Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development PCAP Binning Crash binning is great, we can apply it to packets as well Monitor and parse network responses from target Group them together as best we can ex: 404 responses vs. 202 responses from a web server Most useful when you are fuzzing a target you can’t monitor with other tools Say the Xbox for example Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development File Fuzzing The session class, driver etc. was all designed for network protocol fuzzing The data representation stuff however is generic Peter Silberman has already written a beta file-session for file fuzzing Alternatively, you can... Represent the file in Sulley blocks Write a 3 line .render() loop to generate your test cases Use Cody Pierce’s PaiMei FileFuzz module as a testing harness Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Web Based GUI Coding up blocks is an easy task, but not easy enough for widespread use We’d like to have a drag and drop GUI Create and nest blocks visually Insert, re-arrange, etc. primitives View and modify options for each primitive Save it off to a request library Create a session and connect requests Configure and add targets Generate the driver automatically We have no GUI skills, someone else needs to do this Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Appliance Based Distribution Extending on the previous though... With a nice GUI we can move to an appliance model Distribute Sulley as a virtual machine Simply start it up, configure over web and attack a target We can do auto updates, etc... Maybe one day Amini, Portnoy Fuzzing Sucks!
Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Questions? Comments? About the tool About ... Amini, Portnoy Fuzzing Sucks!
Appendix Slide Count Total Slide Count 65 Amini, Portnoy Fuzzing Sucks!

More Related Content

PDF
Entering the Fourth Dimension of OCR with Tesseract
by🎤 Hanno Embregts 🎸
 
PPTX
Browser Fuzzing with a Twist (and a Shake) -- ZeroNights 2015
byJeremy Brown
 
PPTX
0-knowledge fuzzing
byVincenzo Iozzo
 
PDF
FUZZING & SOFTWARE SECURITY TESTING
byMuH4f1Z
 
PDF
The Non-Advanced Persistent Threat
byImperva
 
PDF
Software Security - Vulnerability&Attack
byEmanuela Boroș
 
PPTX
Web vulnerability seminar3
bySakuya Izayoi
 
PDF
How to find_vulnerability_in_software
bysanghwan ahn
 
Entering the Fourth Dimension of OCR with Tesseract
by🎤 Hanno Embregts 🎸
 
Browser Fuzzing with a Twist (and a Shake) -- ZeroNights 2015
byJeremy Brown
 
0-knowledge fuzzing
byVincenzo Iozzo
 
FUZZING & SOFTWARE SECURITY TESTING
byMuH4f1Z
 
The Non-Advanced Persistent Threat
byImperva
 
Software Security - Vulnerability&Attack
byEmanuela Boroș
 
Web vulnerability seminar3
bySakuya Izayoi
 
How to find_vulnerability_in_software
bysanghwan ahn
 

Viewers also liked

PPTX
Variables and constants
byBSCSComputingIT
 
PPTX
Security Testing: Fuzzing
byAndrei Rubaniuk
 
PPTX
Web vulnerability seminar4
bySakuya Izayoi
 
PDF
Codegate 2013 Junior - Music Player Exploit
bysweetchip
 
PPTX
Fuzzinator - In bug we trust.
byRenata Hodovan
 
PPTX
Fuzzing
byKhalegh Salehi
 
PDF
Web vulnerability seminar1
bySakuya Izayoi
 
PDF
취약점(Vulnerability) db 구조 설명
byeungjin cho
 
PDF
Distributed Fuzzing Framework Design
bybannedit
 
PDF
MBFuzzer : MITM Fuzzing for Mobile Applications
byFatih Ozavci
 
PPTX
Web vulnerability seminar2
bySakuya Izayoi
 
PDF
Inc0gnito fuzzing for_fun_sweetchip
bysweetchip
 
PDF
Fuzzing underestimated method of finding hidden bugs
byPawel Rzepa
 
PPTX
EZ KEY_EZ
byJaesung Lee
 
PDF
[2014 CodeEngn Conference 10] 심준보 - 급전이 필요합니다
byCode Engn
 
PDF
Introduction to Browser Fuzzing
byn|u - The Open Security Community
 
Variables and constants
byBSCSComputingIT
 
Security Testing: Fuzzing
byAndrei Rubaniuk
 
Web vulnerability seminar4
bySakuya Izayoi
 
Codegate 2013 Junior - Music Player Exploit
bysweetchip
 
Fuzzinator - In bug we trust.
byRenata Hodovan
 
Fuzzing
byKhalegh Salehi
 
Web vulnerability seminar1
bySakuya Izayoi
 
취약점(Vulnerability) db 구조 설명
byeungjin cho
 
Distributed Fuzzing Framework Design
bybannedit
 
MBFuzzer : MITM Fuzzing for Mobile Applications
byFatih Ozavci
 
Web vulnerability seminar2
bySakuya Izayoi
 
Inc0gnito fuzzing for_fun_sweetchip
bysweetchip
 
Fuzzing underestimated method of finding hidden bugs
byPawel Rzepa
 
EZ KEY_EZ
byJaesung Lee
 
[2014 CodeEngn Conference 10] 심준보 - 급전이 필요합니다
byCode Engn
 
Introduction to Browser Fuzzing
byn|u - The Open Security Community
 

Similar to Fuzzing sucks!

PDF
Fuzzing and You: Automating Whitebox Testing
byNetSPI
 
PDF
Fuzzing malware for fun & profit. Applying Coverage-Guided Fuzzing to Find Bu...
byMaksim Shudrak
 
PPTX
Blaze Information Security: Slaying bugs and improving software security thro...
byBlaze Information Security
 
PDF
Democratizing Fuzzing at Scale by Abhishek Arya
byabh.arya
 
PDF
Ch 16 & 17 Fault Injection & Fuzzing
bySam Bowne
 
PPTX
OWASP Poland Day 2018 - Andrzej Dyjak - Zero Trust Theorem
byOWASP
 
PDF
Awesome_fuzzing_for _pentester_red-pill_2017
byManich Koomsusi
 
PDF
Reverse engineering – debugging fundamentals
byEran Goldstein
 
PPTX
The Hacking Game - Think Like a Hacker Meetup 12072023.pptx
bylior mazor
 
PDF
Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...
byCodemotion
 
PDF
DotDotPwn v3.0 [GuadalajaraCON 2012]
byWebsec México
 
PDF
Guadalajara con 2012
byJaime Restrepo
 
PDF
Keynote fx try harder 2 be yourself
byDefconRussia
 
PPTX
Vulnerability, exploit to metasploit
byTiago Henriques
 
PDF
Zero bugs found? Hold my beer AFL! how to improve coverage-guided fuzzing and...
byMaksim Shudrak
 
PDF
DEF CON 27 - MAKSIM SHUDRAK - zero bugs found hold my beer afl how to improve...
byFelipe Prado
 
PDF
[Wroclaw #4] Fuzzing - underestimated method of finding hidden bugs
byOWASP
 
PDF
Fuzzing: An introduction to Sulley Framework
byHigh-Tech Bridge SA (HTBridge)
 
PPTX
nullcon 2011 - Fuzzing with Complexities
byn|u - The Open Security Community
 
PDF
Fuzzing: Challenges and Reflections
bymboehme
 
Fuzzing and You: Automating Whitebox Testing
byNetSPI
 
Fuzzing malware for fun & profit. Applying Coverage-Guided Fuzzing to Find Bu...
byMaksim Shudrak
 
Blaze Information Security: Slaying bugs and improving software security thro...
byBlaze Information Security
 
Democratizing Fuzzing at Scale by Abhishek Arya
byabh.arya
 
Ch 16 & 17 Fault Injection & Fuzzing
bySam Bowne
 
OWASP Poland Day 2018 - Andrzej Dyjak - Zero Trust Theorem
byOWASP
 
Awesome_fuzzing_for _pentester_red-pill_2017
byManich Koomsusi
 
Reverse engineering – debugging fundamentals
byEran Goldstein
 
The Hacking Game - Think Like a Hacker Meetup 12072023.pptx
bylior mazor
 
Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...
byCodemotion
 
DotDotPwn v3.0 [GuadalajaraCON 2012]
byWebsec México
 
Guadalajara con 2012
byJaime Restrepo
 
Keynote fx try harder 2 be yourself
byDefconRussia
 
Vulnerability, exploit to metasploit
byTiago Henriques
 
Zero bugs found? Hold my beer AFL! how to improve coverage-guided fuzzing and...
byMaksim Shudrak
 
DEF CON 27 - MAKSIM SHUDRAK - zero bugs found hold my beer afl how to improve...
byFelipe Prado
 
[Wroclaw #4] Fuzzing - underestimated method of finding hidden bugs
byOWASP
 
Fuzzing: An introduction to Sulley Framework
byHigh-Tech Bridge SA (HTBridge)
 
nullcon 2011 - Fuzzing with Complexities
byn|u - The Open Security Community
 
Fuzzing: Challenges and Reflections
bymboehme
 

More from Yury Chemerkin

PDF
YURY_CHEMERKIN__NullCon_2013_Conference.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN__ITA_2013_Proceedings.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN__I-Society-2013_Proceedings.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN__CYBER_CRIME_FORUM_2012.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN__CONFidence_2013_Conference.pdf
byYury Chemerkin
 
PDF
Security_Compliance_Challenges_On_Clouds.pdf
byYury Chemerkin
 
PDF
DefCamp_2017_Conference_Chemerkin_Yury_-_full.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN__ICITST_2012_Conference.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN__AthCon_2013._Conference.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN_InfoSecurityRussia_2012.pdf.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN__ICITST-2012_Proceedings.pdf
byYury Chemerkin
 
PDF
DefCamp_2016_Chemerkin_Yury_--_publish.pdf
byYury Chemerkin
 
PDF
OWASP_Russia_2016_-_Yury_Chemerkin_--_run.pdf
byYury Chemerkin
 
PDF
DefCamp_2015_Conference_Chemerkin_Yury.pdf
byYury Chemerkin
 
PDF
InfoSecurityRussia-_2014_Conferences.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN_NotaCon_2014_Conference.pdf
byYury Chemerkin
 
PDF
YURY_CHEMERKIN_HackMiami_2014_Conference.pdf
byYury Chemerkin
 
PDF
Mobile_Security_Challenges_On_Compliance.pdf
byYury Chemerkin
 
PDF
Mobile_Security_From_The_BYOD_Viewpoint.pdf
byYury Chemerkin
 
PDF
INFOSEC_UAB_2016_Conference_Chemerkin_Yury.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__NullCon_2013_Conference.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__ITA_2013_Proceedings.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__I-Society-2013_Proceedings.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__CYBER_CRIME_FORUM_2012.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__CONFidence_2013_Conference.pdf
byYury Chemerkin
 
Security_Compliance_Challenges_On_Clouds.pdf
byYury Chemerkin
 
DefCamp_2017_Conference_Chemerkin_Yury_-_full.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__ICITST_2012_Conference.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__AthCon_2013._Conference.pdf
byYury Chemerkin
 
YURY_CHEMERKIN_InfoSecurityRussia_2012.pdf.pdf
byYury Chemerkin
 
YURY_CHEMERKIN__ICITST-2012_Proceedings.pdf
byYury Chemerkin
 
DefCamp_2016_Chemerkin_Yury_--_publish.pdf
byYury Chemerkin
 
OWASP_Russia_2016_-_Yury_Chemerkin_--_run.pdf
byYury Chemerkin
 
DefCamp_2015_Conference_Chemerkin_Yury.pdf
byYury Chemerkin
 
InfoSecurityRussia-_2014_Conferences.pdf
byYury Chemerkin
 
YURY_CHEMERKIN_NotaCon_2014_Conference.pdf
byYury Chemerkin
 
YURY_CHEMERKIN_HackMiami_2014_Conference.pdf
byYury Chemerkin
 
Mobile_Security_Challenges_On_Compliance.pdf
byYury Chemerkin
 
Mobile_Security_From_The_BYOD_Viewpoint.pdf
byYury Chemerkin
 
INFOSEC_UAB_2016_Conference_Chemerkin_Yury.pdf
byYury Chemerkin
 

Recently uploaded

PDF
Spec-Driven Development with Kiro: Elevating Software Quality, Traceability, ...
byHaim Michael
 
PDF
UiPath Modern Automation Playbook -Session 2
bysuhanisingh58689
 
PDF
Transcript: Escape from the Forbidden Zone: Smuggling green and inclusive tec...
byBookNet Canada
 
PPTX
Celonis Optimizing your future with process
bydevjeremyappleyard
 
PDF
NTG -Company Profile_Software_Vendor_Company_in_MENA
byMustafa Kuğu
 
PPTX
Introducing VisualSim 2610 The Next Leap in System Level Modeling
byDeepak Shankar
 
PDF
Explaining specific examples of purpose-specific BOM
byakipii ogaoga
 
PDF
GTM-and-Sales-Plan for a cyber security product
byAshish Jangir
 
PDF
UiPath Automation Developer Associate Training Series 2026 - Session 3
byDianaGray10
 
PDF
Oracle Cloud Infrastructure 2025 Architect Professional (1Z0-1127-25) Master ...
byExamCert App
 
PPTX
Microsoft Azure News - February 2026 - BAUG
byDaniel Toomey
 
PDF
TrustArc Webinar - From Trends to Action: Fitting AI Governance into Privacy Ops
byTrustArc
 
PPTX
The Transformative Technology in Contemporary Businesses
bygreyaudrina
 
PPTX
Do You Control the AI, or Does the AI Control You?
bymedhateltoukhy
 
PPTX
Workflow and decision Automation with Flowable
bychakib ch
 
PPTX
apidays Paris 2025 | Building Agentic Workflows
byapidays
 
PDF
Session 1/5: Enhancing Automation with Screenplay & Business Rules
bysuhanisingh58689
 
PPTX
Spacecraft Guidance Quick Research Guide by Arthur Morgan
byArthur Morgan
 
PDF
February 2026 Patch Tuesday hosted by Chris Goettl and Todd Schell
byIvanti
 
PDF
apidays New York 2025 | AI in Application Security
byapidays
 
Spec-Driven Development with Kiro: Elevating Software Quality, Traceability, ...
byHaim Michael
 
UiPath Modern Automation Playbook -Session 2
bysuhanisingh58689
 
Transcript: Escape from the Forbidden Zone: Smuggling green and inclusive tec...
byBookNet Canada
 
Celonis Optimizing your future with process
bydevjeremyappleyard
 
NTG -Company Profile_Software_Vendor_Company_in_MENA
byMustafa Kuğu
 
Introducing VisualSim 2610 The Next Leap in System Level Modeling
byDeepak Shankar
 
Explaining specific examples of purpose-specific BOM
byakipii ogaoga
 
GTM-and-Sales-Plan for a cyber security product
byAshish Jangir
 
UiPath Automation Developer Associate Training Series 2026 - Session 3
byDianaGray10
 
Oracle Cloud Infrastructure 2025 Architect Professional (1Z0-1127-25) Master ...
byExamCert App
 
Microsoft Azure News - February 2026 - BAUG
byDaniel Toomey
 
TrustArc Webinar - From Trends to Action: Fitting AI Governance into Privacy Ops
byTrustArc
 
The Transformative Technology in Contemporary Businesses
bygreyaudrina
 
Do You Control the AI, or Does the AI Control You?
bymedhateltoukhy
 
Workflow and decision Automation with Flowable
bychakib ch
 
apidays Paris 2025 | Building Agentic Workflows
byapidays
 
Session 1/5: Enhancing Automation with Screenplay & Business Rules
bysuhanisingh58689
 
Spacecraft Guidance Quick Research Guide by Arthur Morgan
byArthur Morgan
 
February 2026 Patch Tuesday hosted by Chris Goettl and Todd Schell
byIvanti
 
apidays New York 2025 | AI in Application Security
byapidays
 

Fuzzing sucks!

  • 1.
    Background Architecture Usage and Demos Future Development Fuzzing Sucks! Introducing Sulley Fuzzing Framework Pedram Amini 1 Aaron Portnoy 2 1 pamini@tippingpoint.com 2 aportnoy@tippingpoint.com Black Hat US 2007 Amini, Portnoy Fuzzing Sucks!
  • 2.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development About Us Work at TippingPoint’s Digital Vaccine Labs Responsible for vuln-dev, patch analysis, pen-testing Keep tabs on us at http://dvlabs.tippingpoint.com Launched OpenRCE.org over two years ago How many here are members? Some interesting updates on the horizon after BlackHat Creators of PaiMei RE framework How many here have heard of it? Lot of exciting developments coming up after BlackHat Co-authored ”Fuzzing: Brute Force Vulnerability Discovery” Amini, Portnoy Fuzzing Sucks!
  • 3.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Talk Outline Background Why does fuzzing suck? How can we make it better? Sulley’s Architecture Component Breakdown Advanced Features Usage and Walkthrough Hewlett-Packard Data Protector Audit Trend Micro Server Protect Audit Future Development What’s still on the drawing board Amini, Portnoy Fuzzing Sucks!
  • 4.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Is Fuzzing a ”Dead Horse”? Negative Entire BlackHat track, 3 dedicated books, more commercial vendors and still highly effective. Amini, Portnoy Fuzzing Sucks!
  • 5.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Old School antiparser David McKinney, Python, x-platform, API driven DFUZ Diego Bauche, custom language, Unix SPIKE Dave Aitel, C, Unix, block based The list goes on ... Angel Fuzzer Framework Fuzzled Fuzzy Packet The Art of Fuzzing SPIKEfile ... Amini, Portnoy Fuzzing Sucks!
  • 6.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development DFUZ FTP Example Notes The custom language is easy to understand but very limiting. port=21/tcp peer write: @ftp:user("user") peer read peer write: @ftp:pass("pass") peer read peer write: "CWD /", %random:data(1024,alphanum), 0x0a peer read peer write: @ftp:quit() peer read repeat=1024 wait=1 # No Options Amini, Portnoy Fuzzing Sucks!
  • 7.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development SPIKE FTP Example Notes SPIKE data representation syntax is very simple, perhaps why it’s the most commonly used fuzzer? s_string("HOST "); s_string_variable("10.20.30.40"); s_string("rn"); s_string_variable("USER"); s_string(" "); s_string_variable("bob"); s_string("rn"); s_string("PASS "); s_string_variable("bob"); s_string("rn"); s_string("SITE "); s_string_variable("SEDV"); s_string("rn"); s_string("CWD "); s_string_variable("."); s_string("rn"); Amini, Portnoy Fuzzing Sucks!
  • 8.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development New School Peach Michael Eddington, Python, x-platform, highly modularized Codenomicon Commercial vendor, Java, x-platform, pre-recorded test cases GPF Jared Demott, mixed, x-platform, varying fuzz modes Autodafe Martin Vuagnoux, C, Unix, next-gen SPIKE First fuzzer to bundle debugger functionality Evolutionary Fuzzers SideWinder, Sherri Sparks et al. EFS, Jared Demott Protocol Informatics Framework Marshall Beddoe, Python, x-platform, automated protocol field identification tool Amini, Portnoy Fuzzing Sucks!
  • 9.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Peach FTP Example Notes There is a non trivial learning curve to writing Peach fuzzers. from Peach import * from Peach.Transformers import * from Peach.Generators import * from Peach.Protocols import * from Peach.Publishers import * loginGroup = group.Group() loginBlock = block.Block() loginBlock.setGenerators(( static.Static("USER usernamernPASS "), dictionary.Dictionary(loginGroup, "dict.txt"), static.Static("rnQUITrn") )) loginProt = null.NullStdout(ftp.BasicFtp(’127.0.0.1’, 21), loginBlock) script.Script(loginProt, loginGroup, 0.25).go() Amini, Portnoy Fuzzing Sucks!
  • 10.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development GPF FTP Example Notes Data representation format is very different from other examples. Source:S Size:20 Data:220 (vsFTPd 1.1.3) Source:C Size:12 Data:USER jared Source:S Size:34 Data:331 Please specify the password. Source:C Size:12 Data:PASS jared Source:S Size:33 Data:230 Login successful. Have fun. Source:C Size:6 Data:QUIT Source:S Size:14 Data:221 Goodbye. ... The command line can be a bit unwieldy: GPF ftp.gpf client localhost 21 ? TCP 8973987234 100000 0 + 6 6 100 100 5000 43 finsih 0 3 auto none -G b Amini, Portnoy Fuzzing Sucks!
  • 11.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development So Why Does Fuzzing Hurt So Bad? The existing tools contribute solid ideas but are limited in usage Basically all of them are focused solely on data generation Let’s jump through some fuzzer requirements to get a feel for what’s missing Essentially Chapter 5 from the fuzzing book At each juncture we’ll briefly cover Sulley’s solution We’ll drill down into the specifics when we cover architecture Amini, Portnoy Fuzzing Sucks!
  • 12.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Easy to Use and Powerfully Flexible Pain Powerful frameworks have a huge learning curve Simple frameworks quickly reach limitations Remedy Sulley utilizes block based data representation Sulley fuzzers start simple and don’t have messy syntax Optional elements and keyword arguments Fuzzers are written in pure Python and can benefit from the languages features and ease of use Development efforts can be easily shared Can handle challenge-response and prev-packet-length situations Amini, Portnoy Fuzzing Sucks!
  • 13.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Reproducibility and Documentation Pain Individual test cases must be reproducible Progress and interim results should be recorded Remedy Sulley can replay individual test cases Sulley keeps a bi-directional PCAP of every transaction A built in web interface provides interactive feedback Amini, Portnoy Fuzzing Sucks!
  • 14.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Reusability Pain Non-generic fuzzers can never be used again Widely used protocol components are re-developed all the time Remedy Sulley supports the creation and reuse of complex types and helper functions The more Sulley is used, the smarter it gets Amini, Portnoy Fuzzing Sucks!
  • 15.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Process State and Process Depth Pain None of the existing fuzzers consider state paths Fuzzing A-B-D vs. A-C-D Many fuzzers can only scratch a protocol surface Fuzzing A only Remedy In Sulley you build fuzzers in manageable chunks called requests These requests are tied together in a graph The graph is automatically walked and each state path and depth is individually fuzzed Amini, Portnoy Fuzzing Sucks!
  • 16.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Tracking, Code Coverage and Metrics Pain How much of the target code was exercised? What code was executed to handle a specific test case? Remedy Sulley supports an extensible agent model Utilizes PaiMei/PyDbg for breakpoint-based and MSR-based code coverage tracking Amini, Portnoy Fuzzing Sucks!
  • 17.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Fault Detection and Recovery Pain Most fuzzers rely solely on a lack of response to determine when something bad happens ahem ahem Codenomicon Once a fault is discovered, most fuzzers simply stop! Mu Security and BreakingPoint take the interesting approach of power cycling Remedy Sulley bundles a debugger monitor agent Sulley can restore target health and continue testing by: Restarting the target service Restoring a VMware snapshot Amini, Portnoy Fuzzing Sucks!
  • 18.
    Background Introduction Architecture Past and Present Usage and Demos Pain Points and Solutions Future Development Resource Constraints Pain Non-technical constraints such as time and man power often get in the way Remedy Sulley bundles utilities such as a PDML parser to save time Sulley is designed to allow multiple people to work together easily The monitoring and self recording features of the framework save a great deal of time Amini, Portnoy Fuzzing Sucks!
  • 19.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Sulley Architecture Diagram User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
  • 20.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Four Major Components Data Generation You build requests out of primitives and legos Legos are complex types that extend the framework Session Management / Driver Requests are chained together in a graph to form a session The session class exposes a standalone web interface for monitoring and control The driver ties targets, agents and requests together Agents Interface with the target for instrumentation and logging purposes Utilities Standalone command line utilities that perform a variety of tasks Amini, Portnoy Fuzzing Sucks!
  • 21.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities General Usage Sniff some traffic or reverse some protocol parsing binary Break the target protocol into individual requests Represent each request with a series of primitives Individual requests can be tasked out to different people Setup some targets in conjunction with various agents Write a driver script which instantiates a session and ties requests, agents and the targets together Fuzz! Review results Drill Down Let’s take a look at each individual component in detail... Amini, Portnoy Fuzzing Sucks!
  • 22.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
  • 23.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Data Generation Legos Blocks Primitives Utils Amini, Portnoy Fuzzing Sucks!
  • 24.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Overview Aitel really had it right The block based approach to protocol representation is simple, flexible and powerful Protocols are represented in Sulley as a collection of primitives, blocks and block helpers These elements have many optional arguments The name optional argument gives you direct access to an element without having to walk the stack Refer to the Epydoc generation API documentation for a complete reference of optional arguments Begin the definition of a request with: s initialize("request name") Amini, Portnoy Fuzzing Sucks!
  • 25.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Static and Random Primitives s static() is the simplest primitive adding a constant to the stack Aliases include s dunno(), s raw() and s unknown() s binary() is related and should be familiar to SPIKE users: s binary("0xde 0xad be ef xca fe 00 01 02 0xba0xdd") Sulley primitives are driven by heuristics, with the exception of s random() s random() used to generate random data of varying lengths min length and max length are mandatory arguments num mutations defaults to 25 and specifies how many values to cycle through prior to returning to default Amini, Portnoy Fuzzing Sucks!
  • 26.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Integer Primitives Simple types for dealing with integer fields s char(), s short(), s long(), s double() Convenience aliases exist like byte, word, dword, int, etc... You can fuzz through the entire valid range Defaults to a subset of potentially interesting values To increase throughput Supports ASCII (signed or unsigned) and binary output rendering Amini, Portnoy Fuzzing Sucks!
  • 27.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Strings and Delimiters s string() supports static sizes, variable padding and custom Over 1,000 test cases in string fuzz library Strings are frequently parsed into sub-fields with delimiters Sulley has a special primitive for delimiters, s delim(), here is an example: # fuzzes the string: <BODY bgcolor="black"> s_delim("<") s_string("BODY") s_delim(" ") s_string("bgcolor") s_delim("=") s_delim(""") s_string("black") s_delim(""") s_delim(">") Amini, Portnoy Fuzzing Sucks!
  • 28.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Blocks Primitives (and blocks) can be organized and nested within blocks Blocks are opened and closed with s block start() and s block end() respectively Blocks can be associated with a group, encoder or dependency Grouping, encoding and dependencies are powerful features we examine individually s block start() returns True so you can tab out for readability: # Blocks must be given a name. if s block start("my block"): s string("fuzzy") s block end() Amini, Portnoy Fuzzing Sucks!
  • 29.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Groups Groups tie a block to a defined set of values The block is cycled through for each value in the group Useful for representing a valid list of opcodes or verbs: # define a group primitive listing the various HTTP verbs we wish to fuzz. s group("verbs", values=["GET", "HEAD", "POST", "TRACE"]) # define a new block named "body" and associate with the above group. if s block start("body", group="verbs"): s delim(" ") s delim("/") s string("index.html") s delim(" ") s string("HTTP") s delim("/") s string("1") s delim(".") s string("1") s static("rnrn") s block end("body") Amini, Portnoy Fuzzing Sucks!
  • 30.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Encoders Simple yet powerful block modifier Connect a function with a block to modify contents post render Implement compression, custom obfuscation, etc: def trend xor encode (str): key = 0xA8534344 pad = 4 - (len(str) % 4) if pad == 4: pad = 0 str += "x00" * pad while str: dword = struct.unpack("<L", str[:4])[0] str = str[4:] dword ^= key ret += struct.pack("<L", dword) key = dword return ret Amini, Portnoy Fuzzing Sucks!
  • 31.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Dependencies Allow you to apply a conditional to the rendering of a block Done by specifying dep, dep value(s) and dep compare Block dependencies can be chained together arbitrarily s_short("opcode", full_range=True) if s_block_start("auth", dep="opcode", dep_value=10): s_string("aaron") s_block_end() if s_block_start("hostname", dep="opcode", dep_values=[15, 16]): s_string("aportnoy.openrce.org") s_block_end() # the rest of the opcodes take a string prefixed with two underscores. if s_block_start("something", dep="opcode", dep_values=[10, 15, 16], dep_compare="!="): s_static("__") s_int(10) s_block_end() Amini, Portnoy Fuzzing Sucks!
  • 32.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Sizers Use s size() to dynamically measure and render a blocks size Many optional arguments for flexibility length of size field, default is 4 endianess of size field, default is little Output format, ”binary” (default) or ”ascii” inclusive, whether the sizer should count itself With ASCII output control signed vs. unsigned Sizers can also be fuzzed Amini, Portnoy Fuzzing Sucks!
  • 33.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Checksums Similar to a sizer, s checksum() calculates and render the checksum for a block Keyword argument algorithm can be one of: ”crc32” ”adler32” ”md5” ”sha1” or any arbitrary function pointer endianness can be toggled Amini, Portnoy Fuzzing Sucks!
  • 34.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Repeaters Use s repeat() to handle block repetitions Useful for fuzzing multi-entry table parsers Can variable step from min reps to max reps Alternatively the repeat factor can be tied to another variable # table entry: [type][len][string] if s block start("table entry"): s random("x00x00", 2, 2) s size("string field", length=2) if s block start("string field"): s string("C" * 10) s block end() s block end() # repeat the table entry from 100 to 1,000 reps stepping 50 elements on each iteration. s repeat("table entry", min reps=100, max reps=1000, step=50) Amini, Portnoy Fuzzing Sucks!
  • 35.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Legos Sulley supports the creation of complex types called Legos Example Legos include: E-mail addresses, IP addresses, DCERPC, XDR and ASN.1 / BER primitives, XML tags, etc... The more Legos you define, the easier fuzzing is in the future class tag (blocks.block): def init (self, name, request, value, options=): blocks.block. init (self, name, request, None, None, None, None) self.value = value self.options = options # [delim][string][delim] self.push(primitives.delim("<")) self.push(primitives.string(self.value)) self.push(primitives.delim(">")) # example instantiation. s_lego("tag", "center") Amini, Portnoy Fuzzing Sucks!
  • 36.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
  • 37.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Session Management pGraph Session Amini, Portnoy Fuzzing Sucks!
  • 38.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities pGraph Python graph abstraction library Developed originally for PaiMei Allows for simple graph construction, manipulation and rendering Rendering formats supported are GML, GraphViz and uDraw The session class extends from this... Amini, Portnoy Fuzzing Sucks!
  • 39.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Session Class Connect multiple requests in a graph Register pre and post send callbacks Assign a callback to each edge Add multiple network targets Exposes a custom web interface Automatically communicates with registered agents Responsible for walking the graph and fuzzing at each level Tightly used and related to the creation of drivers Amini, Portnoy Fuzzing Sucks!
  • 40.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Session Illustrated ehlo callback_one() pre_send() mail from rcpt to data post_send() helo callback_two() This example demonstrates multiple paths and depths Callbacks aren’t really needed here More applicable in cases of RPC, challenge-response, prev-packet specifies length, etc... Amini, Portnoy Fuzzing Sucks!
  • 41.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Interactive Web Interface View fuzzer progress View detected faults Retrieve per-fault crash dump and packet capture Pause and resume fuzzing Amini, Portnoy Fuzzing Sucks!
  • 42.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
  • 43.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agents VMControl Netmon Procmon ... Amini, Portnoy Fuzzing Sucks!
  • 44.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agents A flexible sub-system allows you to create custom agents Client-server communication is extremely simple over ”PedRPC” Create a class that extend from pedrpc.server Instantiate pedrpc.client Call class members as if they are local Some agents have already been developed... Amini, Portnoy Fuzzing Sucks!
  • 45.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agent: Netmon Monitors network traffic and saves PCAPs to disk Per test case bi-directional packet capture ERR> USAGE: network monitor.py <-d|--device DEVICE #> device to sniff on (see list below) [-f|--filter PCAP FILTER] BPF filter string [-p|--log path PATH] log directory to store pcaps to [-l|--log level LEVEL] log level (default 1), increase for more verbosity [--port PORT] TCP port to bind this agent to Network Device List: [0] DeviceNPF GenericDialupAdapter [1] {2D938150-427D-445F-93D6-A913B4EA20C0} 192.168.181.1 [2] {9AF9AAEC-C362-4642-9A3F-0768CDA60942} 0.0.0.0 [3] {9ADCDA98-A452-4956-9408-0968ACC1F482} 192.168.81.193 ... Amini, Portnoy Fuzzing Sucks!
  • 46.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agent: Procmon PyDbg based fault monitoring engine Used to monitor target health Detected faults are catalogued in a ”crash bin” Allows for simplistic (backtrace driven) fault clustering More on this later ERR> USAGE: process monitor.py <-c|--crash bin FILENAME> filename to serialize crash bin class to [-p|--proc name NAME] process name to search for and attach to [-i|--ignore pid PID] ignore this PID when searching for the target process [-l|--log level LEVEL] log level (default 1), increase for more verbosity [--port PORT] TCP port to bind this agent to Amini, Portnoy Fuzzing Sucks!
  • 47.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Agent: VMControl Exposes network API for VMWare instrumentation Simple PedRPC wrapper around vmrun.exe Start, stop, suspend, snapshot, revert, etc... Used to restore target health after a fault is induced ERR> USAGE: vmcontrol.py <-x|--vmx FILENAME> path to VMX to control <-r|--vmrun FILENAME> path to vmrun.exe [-s|--snapshot NAME> set the snapshot name [-l|--log level LEVEL] log level (default 1), increase for more verbosity [-i|--interactive] Interactive mode, prompts for input values [--port PORT] TCP port to bind this agent to Amini, Portnoy Fuzzing Sucks!
  • 48.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
  • 49.
    R2 R Background 3 Architecture Overview Data Representation Usage and Demos Future Development Fuzzing Session and Agents Utilities seq Driver R1 R3 R2 Fil cr R4 R5 Amini, Portnoy Fuzzing Sucks!
  • 50.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Driver The driver is where it all comes together: Import requests from the request library Instantiate a session instance Instantiate and add target instances to the session Interconnect the requests to form a graph Start fuzzing This is where edge and pre/post send callbacks should be defined The driver is entirely free form, though must of them will follow a simple and similar structure Amini, Portnoy Fuzzing Sucks!
  • 51.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Example Driver from sulley import * from requests import jabber def init message (sock): init = ’<?xml version="1.0" encoding="UTF-8" ?>n’ init += ’<stream:stream to="10.10.20.16" xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org">’ sock.send(init) sock.recv(1024) sess = sessions.session(session filename="audits/trillian.session") target = sessions.target("10.10.20.16", 5298) target.netmon = pedrpc.client("10.10.20.16", 26001) target.procmon = pedrpc.client("10.10.20.16", 26002) target.vmcontrol = pedrpc.client("127.0.0.1", 26003) # start up the target. target.vmcontrol.restart target() print "virtual machine up and running" sess.add target(target) sess.pre send = init message sess.connect(s get("chat message")) sess.fuzz() Amini, Portnoy Fuzzing Sucks!
  • 52.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities User Developed Utilities Data Generation crashbin_explorer Legos Blocks ida_fuzz_library_extender Request Library pcap_cleaner Primitives Utils R1 pdml_parser R2 R3 sequence_honer Driver R1 Session Management R3 R2 Filesystem pGraph Session crashbins R4 R5 pcaps ... Agents Targets VMControl Netmon Procmon ... Sulley Architecture Amini, Portnoy Fuzzing Sucks!
  • 53.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities Utilities crashbin_explorer ida_fuzz_library_extender pcap_cleaner pdml_parser sequence_honer Amini, Portnoy Fuzzing Sucks!
  • 54.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities crashbin explorer.py View every test case which caused a fault List every location where a fault occurred Retrieve saved crash dumps Render a graph which clusters faults by stack trace $ ./utils/crashbin explorer.py USAGE: crashbin explorer.py <xxx.crashbin> [-t|--test #] dump the crash synopsis for a specific test case number [-g|--graph name] generate a graph of all crash paths, save to ’name’.udg Pedram will actually demo this in a bit Amini, Portnoy Fuzzing Sucks!
  • 55.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities ida fuzz library extender.py IDA Python script Simple concept Enumerate all constant integer comparisons in target binary Enumerate all constant string comparisons in target binary Add to fuzz library heuristics This is a pre-fuzz static analysis script A more advanced run-time implementation is still in the works Fuzz library extensions are handled via .fuzz strings and .fuzz ints Amini, Portnoy Fuzzing Sucks!
  • 56.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities pcap cleaner.py Simple utility Iterates through a crashbin and removes any PCAPs not associated with a fault Save on disk space prior to archiving a completed audit $ ./utils/pcap cleaner.py USAGE: pcap cleaner.py <xxx.crashbin> <path to pcaps> Amini, Portnoy Fuzzing Sucks!
  • 57.
    Background Overview Architecture Data Representation Usage and Demos Fuzzing Session and Agents Future Development Utilities pdml parser.py Convenience utility Converts PDML dump from Wireshark to a Sulley request Easier then doing so manually, work is still required of course Amini, Portnoy Fuzzing Sucks!
  • 58.
    Background Architecture Hewlett-Packard Data Protector Usage and Demos Trend Micro ServerProtect Future Development Hewlett-Packard Data Protector Simple protocol to reverse and represent Simple bug (currently 0day, fix is just around the corner) Good starting example Amini, Portnoy Fuzzing Sucks!
  • 59.
    Background Architecture Hewlett-Packard Data Protector Usage and Demos Trend Micro ServerProtect Future Development Trend Micro ServerProtect Microsoft RPC interface Tons of bugs in this thing Some have been reported and fixed, others are still pending Interesting demo since we can show off the fact that Sulley can fuzz DCE/RPC Amini, Portnoy Fuzzing Sucks!
  • 60.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Sequence ”Honing” Say test case #100 triggered a fault but replaying it does not Probably relies on some previous test or sequence of tests We can automatically deduce the exact sequence required for replication This is done, but I’m still playing with the reduction ”algorithm” Here is how the current incarnation of the approach works... Amini, Portnoy Fuzzing Sucks!
  • 61.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development The Honing ”Algorithm” Start from the last case and step back to find the window 100. 99, 100. 98, 99, 100. 40, 41 ... 99, 100 Start eliminating sequential bundles (ratio to window-size) and check for fault Once exhausted, increase granularity to eliminate single test cases at a time Sulleys health-restoration methods are used to ensure a ”clean slate” per test Amini, Portnoy Fuzzing Sucks!
  • 62.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Parallel Fuzzing Combinatorial explosion is a common fuzzing problem We can increase throughput by replicating the target, ex: Run 2 targets in VMWare with PyDbg monitoring Run a target on real hardware for MSR-based code coverage recording Parallel fuzzing is as simple as instantiating and add multiple targets in your driver code Amini, Portnoy Fuzzing Sucks!
  • 63.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Heuristic Feedback Loop Along the same lines as the fuzz extender command-line utility Simple concept that may improve your fuzz: Attach to the target process and trace Look for all comparisons to int and string constants Feed those back to the fuzzer, adding them to the current primitives fuzz library There are far more scientific ways of doing this Hoglund and Halvar both are researching what I call ”path-math” Was talking to a friend at Microsoft, they already have it Amini, Portnoy Fuzzing Sucks!
  • 64.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Code Coverage Monitor An agent similar to netmon or procmon, records code coverage per test case There are 2 code coverage monitors completed MSR based Process Stalker based Neither are included as they add a lot more requirements (PaiMei) We’re working on PaiMei 2.0 which will be SQL driven and will include it then Amini, Portnoy Fuzzing Sucks!
  • 65.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development PCAP Binning Crash binning is great, we can apply it to packets as well Monitor and parse network responses from target Group them together as best we can ex: 404 responses vs. 202 responses from a web server Most useful when you are fuzzing a target you can’t monitor with other tools Say the Xbox for example Amini, Portnoy Fuzzing Sucks!
  • 66.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development File Fuzzing The session class, driver etc. was all designed for network protocol fuzzing The data representation stuff however is generic Peter Silberman has already written a beta file-session for file fuzzing Alternatively, you can... Represent the file in Sulley blocks Write a 3 line .render() loop to generate your test cases Use Cody Pierce’s PaiMei FileFuzz module as a testing harness Amini, Portnoy Fuzzing Sucks!
  • 67.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Web Based GUI Coding up blocks is an easy task, but not easy enough for widespread use We’d like to have a drag and drop GUI Create and nest blocks visually Insert, re-arrange, etc. primitives View and modify options for each primitive Save it off to a request library Create a session and connect requests Configure and add targets Generate the driver automatically We have no GUI skills, someone else needs to do this Amini, Portnoy Fuzzing Sucks!
  • 68.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Appliance Based Distribution Extending on the previous though... With a nice GUI we can move to an appliance model Distribute Sulley as a virtual machine Simply start it up, configure over web and attack a target We can do auto updates, etc... Maybe one day Amini, Portnoy Fuzzing Sucks!
  • 69.
    Background Nearly Done Architecture On the Drawing Board Usage and Demos Conclusion Future Development Questions? Comments? About the tool About ... Amini, Portnoy Fuzzing Sucks!
  • 70.
    Appendix Slide Count Total Slide Count 65 Amini, Portnoy Fuzzing Sucks!