Fuzzing with complexitiesIntroductionNow before moving on the specifics of this paper I would introduce a little bit about software testingand its limitations. I would try to describe fuzzing first before moving on with the case study.We all have been a witness to major threats in the past years and I guess no one could forget nameslike ‘Conficker’ (1), ‘Stuxnet’ (2) and ‘Aurora Project’ (3). All these malware had a unique deliverysystem which was based on exploiting the host operating system and further talking control of theOS for quite talked about purposes. These threats are always there and only thing we expect toachieve is that, we find vulnerability before a bad guy do and do something about it.Software companies spend a lot of their time and money in making their product more stable, morereliable and more secure. But what these incidents prove us that whatever they are doing, there isalways a change to find a bug in a process.Microsoft has even eliminated the use of several known functions which has historically been provento cause buggy code. For example since Vista Microsoft has made sure that functions like strcpy,sprintf etc. are eliminated at the Software development lifecycle (SDL). Thus normal static analysistechnique used to find the ‘lowest hanging fruit’ would not work if we are working to find bugs inthese windows applications.In fact all major vendors have realized the importance of having a secure SDL and importance oftesting in their product (which also has cost benefits as compared to releasing and developingpatch). Microsoft and adobe has adopted fuzzing into their SDL cycle thus highlighting theimportance of this practice. Figure 1: Microsoft Simplified SDL (4)While referring back to security conferences in the past, significant work has been done in between2005 – 2006 (5). This period showed that security researchers and hackers are increasing theirefforts to better themselves in the art of fuzzing and finding bugs and producing exploit using thistechnique.
Until many vendors were compelled to release their monopoly on popular binary and networkformats by European Union or in some case voluntary disclosure, this field was majorly focused onBlack box testing, meaning the hacker had very little or in some cases no knowledge about theprotocol they are trying to fuzz.Black-box fuzzing focuses on input format—ignoring the tested software target. While being efficientand allowing reuse of the same test tools across different tested targets that share the same dataformats, this method misses significant code paths. These code paths depend on configurationoptions or specific complex conditions that are governed by application logic. (5)Even as Microsoft and other vendors do talk up fuzzing seriously, developing multiple protocolfuzzers for various line of product. There is no doubt that their fuzzer is uber cool and is able to findmost of the bugs, some bugs remains and the security industry is gladly finding them and makingliving out of it.Google and Firefox have a policy of rewarding any researcher who comes up with a bug or aresulting exploit. Other such rewards on submitting a bug is funded by various IPS companies likeiSight Partners, Tipping point etc. They buy your bug and then work towards a responsible disclosurewhile negotiating with the vendor.In the initial years fuzzing was never looked up by academia as it was considered to be naïve amongthe peers, but soon this process gained momentum and attention of many researchers. This simpletechnique is proven to produce result faster and cheaper than any other process known. Moreinformation about fuzzing would follow in the following discussion on fuzzing.Software TestingSoftware testing is any activity aimed at evaluating an attribute or capability of a program or systemand determining that it meets its required results (6). Although crucial to software quality and widelydeployed by programmers and testers, software testing still remains an art, due to limitedunderstanding of the principles of software. The difficulty in software testing stems from thecomplexity of software: we cannot completely test a program with moderate complexity. Testing ismore than just debugging. The purpose of testing can be quality assurance, verification andvalidation, or reliability estimation. Testing can be used as a generic metric as well. Correctnesstesting and reliability testing are two major areas of testing. Software testing is a trade-off betweenbudget, time and quality. (7)Unlike most physical systems, most of the defects in software are design errors, not manufacturingdefects. Software does not suffer from corrosion, wear-and-tear -- generally it will not change untilupgrades, or until obsolescence. So once the software is shipped, the design defects -- or bugs -- willbe buried in and remain latent until activation. (7)
With time it was realised that as fixing bug is quite expensive and there is a need to optimize thisprocess. A huge interest in this field has led to many innovations both from academia andcompanies. Many companies completely dedicated in producing software testing techniques, andautomation has grown in recent years. Looking at these innovations it would be unfair to describesoftware testing in a few words. My motive here is to introduce this process as a concept if a readerwant more details on this topic I encourage him to read books like “A Practitioners Guide toSoftware Test Design” and others.Code CoverageCode coverage is one of the most important metrics used to decide on the completeness of the testcases. This metrics gives us the relationship between the test conducted and instructions executedwith the application. The more the instructions executed the better the code coverage is. It is thequantitative measure of instructions executed among the maximum possible instructions with aparticular application.Of course this metrics can be further broken down into more detailed metrics. Major among themare: (8) Function coverage - Has each function (or subroutine) in the program been called? Statement coverage - Has each node in the program been executed? Decision coverage - Has every edge in the program been executed? For instance, have the requirements of each branch of each control structure (such as in IF and CASE statements) been met as well as not met? Condition coverage - Has each Boolean sub-expression evaluated both to true and false? Condition coverage - Both decision and condition coverage should be satisfied.Code coverage is also used to discard cases that do have low code coverage or they cannot be usedto increase the code coverage. Figure 2: An example of Code coverageThere is a specific class of software specifically designed to monitor the code coverage of theapplication under observation. Based on type of application we can have either source codecoverage analyser or binary code coverage analyser etc.
It is quite clear that this metric is both intuitive and based on testing properties; give us specificenough information about the test and its completeness. Some advance code coverage analysis toolcan even give us information that what should we do to improve our test case. But remember it willonly give us a metric of completeness of the test not the metrics of quality or reliability or in any kindof significance on the application itself.Also remember that coverage analysis is one of many testing techniques; you should not rely on italone. Test needed to find bugs Tests needed for coverage Figure 3: Shows that even on a good coverage some bugs would still be left aloneBlackbox TestingWhile testing software, one has only one of three situations in his hand. Firstly, I would have noknowledge of the inner working of the software, neither of the protocol or kind of input expected,this situation is rightly named as Black-box Testing. While testing software with no knowledge inhand would not be very fruitful, but this strategy has worked well in the past. Some of the featuresof black box testing are described pictorially below with self-expiatory terms. Expected Input Black box testing Boundary values Illegal values Figure 4: Blackbox Testing
Grey box TestingThe other situation can be that users have knowledge of internal data structure and algorithms forpurpose of designing test cases, but the testing is done at blackbox level. Managing user input andoutput are beyond the scope of this kind of testing, this testing condition falls under graybox testing.White box TestingFinally, information on internal data structure and algorithms is completely shared between theproduct development team and the tester’s team. This kind of information can be used to test API’s,Code Coverage, fault injection, Mutation of testing and many more. A basic testing setup isdescribed below. Statement Testing White box testing Decision Testing Conditional Testing Figure 5: Whitebox testingFuzzingFuzzing is a kind of software testing technique which was until recently categorized inside black boxtesting. To put simply, it is the technique in which invalid or mutated or malformed input is suppliedto application with only intention to find bugs the application.The first person credit of working and formulating this technique is Barton Miller and his studentsfrom University of Wisconsin-Madison in 1989. They described this technique as,“Fuzz testing is a simple technique for feeding random input to applications. While random is a time-honoured technique, our approach has three characteristics that, when taken together, make issomewhat different from other approaches. 1. Input is random: We do not use any model of program behaviour, application type, or system description. This is something called black box testing. In the command line studies, the random input was simply random ASCII characters streams. From our x-window study, Windows NT study and MAC OS study, the random input included cases that had only valid keyboard and mouse events.
2. Our reliability criteria are simple: if the application crashes or hung, it is considered to fail the test; otherwise, it passes. Note that the application does not have to respond in a sensible manner to the input and it can even quietly exit. 3. As a result of the first two characteristics, fuzz testing can be automated to a high degree and results can be compared across applications, operating systems and vendors.” (9)Now as we look back at this the early definition of fuzzing, a few things have changed. Weunderstand this process more clearly, more concrete theory is developed and it is proven to worktime and time again.It is important to remember that any type of interaction of user supplied data either supplied bynetwork, file, registry, API etc. can be fuzzed.Microsoft has described some key terms in fuzzing terms that one should know.Term DefinitionDumb fuzzing Corruption of data packets randomly without awareness of data structure.Smart fuzzing Corruption of data packets with awareness of the data structure, such as encodings (for example, base-64 encoding) and relations (checksums, bits indicating the presence of some fields, fields indicating offsets or lengths of other fields).Black-box Sending of malformed data without actual verification of which code paths were hitfuzzing and which were not.White-box Sending of malformed data with verification that all target code paths were hit—fuzzing modifying software configuration and the fuzzed data to traverse all data validations in the tested code.Generation Generation of fuzzed data automatically—not basing on any previous input.Mutation Corruption of valid data according to defect patterns, to produce fuzzed data.Mutation Well-formed buffer that represents an equivalence class of the input. The fuzzertemplate takes the mutation template as an input—producing a fuzzed buffer to be sent to the tested software.Code Technology (such as that which is bundled in Microsoft Visual Studio 2005) thatcoverage allows inspection of which code paths were executed during testing. This is useful for verification of test effectiveness and improvement of test coverage. Table 1: Fuzzing Taxonomy (5)It is observed that fuzzing is most effective against application developed in C/C++, these languagesmake the programmer responsible for memory management whereas managed code i.e. developedin C#, Java etc. would yield bugs of a very different class. This does not mean that you can’t find bugsin them. (5)
Fuzz metrics have been understood, process is divided into two types of fuzzer and further thesefuzzers have been separated into more focused modules which make them work more efficiently.Hitting the target application with valid looking inputs would defiantly result in few anomalies. Theseanomalies can be introduced into the system any kind of user input as previously discussed.Some applications like Web Browser uses a lot of open source library and if any one of the libraries isbuggy and exploit can is made out of it, we will see more of the situations like that of ‘Aurora’.Finally I would like to say that there is an important distinction between Fuzzing and other testingactivity. This distinction is the intent. A testing team knows a lot about the program and basicallytest that whether a program is behaving as it is supposed to behave where as a security researcheronly care that his fuzzer crashes your tested application. Security researchers do not care about theinternal working of the program, he has one and only one purpose i.e. to crash the program or makeit behave differently while playing with it.FuzzerThere have a plethora of fuzzers available in market, both open source and closed.I would like to make note of two python based fuzzing framework available in the open sourcecommunity that I use most extensively. PeachFuzzer - Peach is a SmartFuzzer that is capable of performing both generation and mutation based fuzzing (10). Sulley - Sulley is a fuzzer development and fuzz testing framework consisting of multiple extensible components. Sulley (IMHO) exceeds the capabilities of most previously published fuzzing technologies, commercial and public domain (11).Peach is been improved day in and day out and it is the only other open source fuzzer that ismaintained apart from Metasploit fuzzer. Peach is written as primary data fuzzer, but as it opensource it can be extended to secondary and even nth-class fuzzer. Peach fuzzer is also used by adobein its testing of Adobe reader (12).Sulley is not maintained but is as good as you can get when it comes to generation based fuzzing.These fuzzers have separated the data generation or mutation process from process monitoringtechnique which is again separated from control and even support distributed fuzzing (where afuzzer run on multiple machine). These are some of the reasons why these fuzzers are worthmentioning.A major commercial Fuzzer is Codenomicon (13). This is the child project of the very first fuzzer everdeveloped called PROTOS.If you want to check out other fuzzers available I recommend you to check thishttp://packetstormsecurity.org/fuzzer/.
The ComplexityJiantao Pan of Carnegie Mellon University has described the complexity revolving around bugs as (7)“Software bugs will almost always exist in any software module with moderate size: not becauseprogrammers are careless or irresponsible, but because the complexity of software is generallyintractable -- and humans have only limited ability to manage complexity. It is also true that for anycomplex systems, design defects can never be completely ruled out.”He also describes about the never ending race among security researchers who are finding bugs andthat of vendors and uses the terms, used by Boris Beizer in his book Software Testing Techniques, 2ndEdition, Pesticide paradox to describe this race.Pesticide Paradox – “Every method you use to prevent or find bugs leaves a residue of subtler bugsagainst which those methods are ineffectual” (14).In simple terms on can say that once you have found a bug and fixed it, you are now sure that theinput that earlier did trigger bug would not be able to crash the program. But you can be never surethat pre-error test cases that the application passed in the first time would not come haunting.In many of the fuzzer it is observed that test cases produced fails to achieve the basic packetsanitation test, which prevents it from further testing. In all general purposes the system rejectsvirtually all the randomly generated packets, thus defeating the purpose of testing the remainingcode effectively.What this means is that fuzzers with improper understanding of the input type and structure wouldfails often then they succeed. Intuitively the one with more knowledge about the input structurewould result in better fuzzers.A study done by Microsoft on a 450 lines of code and then testing it with various fuzz combinationsto see the effective results that was produced is shown below :Technique Effort Code coverage Defects foundBlack box + Dumb 10 min 50% 25%White box + Dumb 30 min 80% 50%Black box + Smart 2 hr 80% 50%White box + Smart 2.5 hr 99% 100% Table 2: Analysis based on Effort in producing fuzzer and defects found correlated with kind of fuzzer (5)Know what I believe, and the test shows that investing your time in right place would result in betterproductivity with respect to kind of bugs we found and also the amount of bugs we found, furtherrestricting on the number of cases where input is out rightly rejected on the basis of basic sanitationtechniques of the programs input validation module.If we want to go deep inside the program semantics and actually test the code of our choice or trackthe data input we have no choice but to work with all the knowledge we have or could acquire about
the protocol. This involves reading and understanding of Data Format documentation of disclosedfile formats, network formats and many more. We can easily right such fuzzer that generatescredible input to the tested application if the stick to either an ASCII based or Binary based.WHO #UnrealIRCdJOIN #SupportPONG irc.fyremoon.net irc2.dynam.acSTATS u Table 3: Example of a pure Ascii Based Data formatZIP local file headerOffset Bytes Description 0 4 Local file header signature = 0x04034b50 4 2 Version needed to extract (minimum) 6 2 General purpose bit flag 8 2 Compression method10 2 File last modification time12 2 File last modification date14 4 CRC-3218 4 Compressed size22 4 Uncompressed size26 2 File name length (n)28 2 Extra field length (m)30 n File name30+n m Extra field Table 4: An example of purely binary based data formatBut what happens when the formats no longer sticks to one data format? What happens when ourdata switches from one set of data format like ASCII to binary and then binary to ASCII again and toadd a cherry on top sections are encoded differently even the ascii portion can be encoded and evenimported from other binary or ASCII based formats. If you want more complication here is one moreto your bag, Can the code coverage be a correct metrics when analysing fuzzing input? The answer isNo, you would be lucky to even get 25% code coverage for any given input, this is because of highmodular structure of these programs, distributed among various binaries and not all binary load at agiven input.
Things are getting pretty complicated. Hmm!%PDF-1.5%µµµµ1 0 obj<</Type/Catalog/Pages 2 0 R/Lang(en-IN) /StructTreeRoot 54 0 R/MarkInfo<</Marked true>>>>endobj2 0 obj<</Type/Pages/Count 7/Kids[ 3 0 R 23 0 R 29 0 R 38 0 R 47 0 R 49 0 R 51 0 R] >>endobj3 0 obj<</Type/Page/Parent 2 0 R/Resources<</Font<</F1 5 0 R/F2 10 0 R/F3 12 0 R/F4 14 0 R/F5 16 0R/F6 21 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] >>/MediaBox[ 0 0 595.32 841.92]/Contents 4 0 R/Group<</Type/Group/S/Transparency/CS/DeviceRGB>>/Tabs/S/StructParents0>>endobj4 0 obj<</Filter/FlateDecode/Length 5463>>streamxœ•[•Û¸’~•ÿ GûˆÝ€A I&9ÈbgÏì&»ó0³jµÜÖÄq÷Hêtz~ýÖ…”HY”ÅÁœ´e²X$ëúUÉÑë_£Ÿ~zýË»•?GÉõuôöçwÑÛÏ/_¼þ "™Å‰Ž>ï^¾QÿQ.ãDê(×i,Šèó×—/’8)%±øÿ*K£înññ•ýóå‹ß•J’¤¼ÎáŸ2¿¾*ñßwþ¿ÅÛë+ú>¹¾*ðsÉŸeÆ•æ¹—ñ¿ÅþÞŽ+Òë+‘à Table 5: Example of one such complex formats ie. PDFWe see these being used in every day applications like office documents, Adobe PDF, SMB protocolsand more. One cannot try to randomly fuzz these files as they have pretty good input validationmodules which prevent any dumb attempt to fuzz them. All these formats are considered as veryattractive formats within the security industry. But understanding even one format completelywould take some beautiful years out of once life. So is there a solution? My answer is yes there“are” few solutions available still. Figure 6: Adobe Fuzzing Technique
What we have gathered until here is summarized here as we move ahead you will find answers tothese problems.ComplicationsCode coverage is cannot be used as a metrics while testing test type of input streamsData format consistency i.e. Either ASCII or Binary is not maintainedMultiple encoded levels exists which can be different within the same fileMultiple data format either binary or ASCII are embedded within these input stream Table 6: Discussed problemsCode Coverage fails for these applicationsIn all these cases we have to work with what we have in hand. We are going to use a combination ofthe technique of taint analysis, protocol awareness and Code coverage. I call this as relative codecoverage metric.Protocol awareness can be used as once we have all the information of a protocol that we couldhave, we can intuitively say that the packet which contains the most number of tags or objectswould require more code to be covered with that module. Now this could be said that we cannotguarantee the code coverage still because if we do not find a packet that contains all the tags orobject that our application which is usually the case, we would be not able to test all cases.Testing all cases in one go was never the idea but multiple tests covering every tag is what will befruitful. It is intuitive that when all the tags are covered we would have near complete codecoverage. To further improve upon, one can use Taint analysis to find all the possible check andbalances and the further incorporate them in code. Read the ‘future’ section for more details.Data format inconsistencyThe other problem that we face is of data format inconsistency. One can easy write a fuzzer of eitherand ASCII based packet or for binary based packet. But when these formats get together in a packet,it becomes unnaturally difficult to write one.The solution lays in visualizing and breaking problem in parts which we most comfortable in. We canuse the separate out the data generation capability from both ASCII and Binary format. Rememberhere I have trying to separate out these capabilities not necessarily for fuzzing.Multiple Encoding levelsAs we have separated ASCII with Binary in the same format one can further add custom encoding ineach packet as one like. They will all fall back together when we combine them later. See the casestudy for more clarification.In a PDF file if we have a multiple font embedded inside the file we can make use of differentencoders for each such font as each is generated separately.Multiple Files Embedded in a single packetsWith separating of types we can further separate to a secondary level data production module.What this means is that if a PDF file if we have a font and image embedded inside the file we canactually write a different fuzzer for font and for an image and combines each of these result with thePDF files in the manner similar to multiple encoding level problem.
Now is the right time to talk about the strategy that I have used when fuzzing one such format, PDF.You will find different definition of these terms, but this is what I understand out of them. Thisprocess is typically described in the terms of system under test and called for directed area with thesystem, where as in my study I have taken it out of box and placed these conditions on Data packetitself.Attack Point SelectionThe attach point selection is a simple process in which I have tried to specify a specific point withinthe packet which needs to be tested. Now selection of these points depends a lot upon somegathered intelligence of the system, including pervious vulnerabilities. As this eliminates a few attackpoint as they have already been attacked before. For example if working on a simple PDF file whichcontain a U3D file which is known to previously cause a vulnerability in Adobe reader one can saythis format is previously been tested primarily (after looking at the vulnerability) so a lot moreefforts would be required in finding a vulnerability next time. One can focus his time and energy infinding other routes into the application which has still not been tested by security researchers.Directed FuzzingWhenever a vulnerability is released it is released with a very few information. One such disclosureexample would be. Adobe Flash Player Multiple Tag JPEG Parsing Remote Code Execution Vulnerability -- Vulnerability Details: This vulnerability allows remote attackers to execute arbitrary code on vulnerable installations of Adobe Flash Player. User interaction is required in that a target must visit a malicious website. The specific flaw exists within the code for parsing embedded image data within SWF files. The DefineBits tag and several of its variations are prone to a parsing issue while handling JPEG data. Specifically, the vulnerability is due to decompression routines that do not validate image dimensions sufficiently before performing operations on heap memory. An attacker can exploit this vulnerability to execute arbitrary code under the context of the user running the browser. Figure 7: An example of Vulnerability disclosureNow you would like to get your hands on this kind of vulnerability. Getting this vulnerability is verydifficult task as we have only a specific area with a binary file to be fuzzed and with extensive inputsmutation / generations. Looking at the above example here, of the kind of disclosure available wewould like to fuzz only a specific region as we have previously been aware that certain code mightcontain some vulnerability somewhere.Case Study: Attach Point selectionI have implemented a fuzzer using the Peach fuzzing framework, where I have decided to test PDFfiles which have Open Type font embedded in them. Now I fuzz only the font part within the PDF byusing a secondary fuzzer which contains the complete file format of TTF.
I have done mutated fuzzing, so I had to choose a file that contains most of the tags (to achieve themaximum relative code coverage) that are there in documentation. This can be easily done if wehave a good base of fonts already in our repository that was easy to get using torrents. I could notfind a single file that contains all the tags so I have to go for the one which contains the maximumnumber of tags. I had few other files that contain tags that were not there in the maximum tag file,thus my test case is complete with all the tags that were allowed in the font file.This TTF font is arranged within PDF using Report Lab python API. Once the PDF is formed it is openup inside Adobe Reader to which debugger is attached. There are few other libraries available foryou to actually merge the components with PDF some notable among them is iText and pyPdf. Evenif you do not have one such library available it is easy to embed on file with simple scripts one suchexample is there on Feliam’s blog (15). Figure 8: Data format analysis by 010 Editors custom template of TTF
After making this custom template and getting a thorough understanding of TTF file format I wrote afuzzer for it. Figure 9: Snippet of fuzzer for TTF in Peach PIT formatAfter running this fuzzer for just one day I had the following result. Microsoft (R) Windows Debugger Version 6.11.0001.404 X86 Copyright (c) Microsoft Corporation. All rights reserved. …… CommandLine: C:PROGRA~1AdobeReader~1.0ReaderAcroRd32.exe C:FuzzFileFileFormatTTFfuzzedfile.pdf Symbol search path is: SRV*http://msdl.microsoft.com/download/symbols;symsrv*symsrv.dll*c:symbols*http://msdl.microsoft.com/download/symbol s Executable search path is: ModLoad: 00400000 00454000 AcroRd32Exe.exe ModLoad: 7c900000 7c9b0000 ntdll.dll ……….. ……….. (2e4.674): Access violation - code c0000005 (first chance) r eax=00000001 ebx=07d52594 ecx=04dd57ec edx=00000002 esi=04dd56ac edi=07dfbfe4 eip=080111e1 esp=0012de48 ebp=0012de5c iopl=0 nv up ei pl nz na po nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010202 *** ERROR: Symbol file could not be found. Defaulted to export symbols for C:PROGRA~1AdobeReader~1.0ReaderCoolType.dll - CoolType+0x111e1: 080111e1 660fb64f23 movzx cx,byte ptr [edi+23h] ds:0023:07dfc007=?? Figure 10: Result from fuzzingAnd the following information were logged
Figure 11: Fuzzing Results were loggedFor office application there are actually two ways in which you can do Fuzzing using this concept.First you can use an Library to create office files embedded with test case of your choice (somelibraries test the cases sanctity before embedding, one can bypass this by changing the sanctitychecking code and recompiling the library. To know which section of code is doing that you canalways fall back to code coverage) and the other way would be using COM API of these Officesoftware. By using the second method you can test 2 things. First, is there a bug in loading resourceprocess and secondly is there a bug in processing the resource file.Case Study: Directed FuzzingOnce we are familiar with the process of creating files we can further brings thing down to tag level.This means that if we want to only fuzz one particular tag in a file which would be further embeddedinto some complicated data format, we just have to model that particular tag of the file in ourmutator and the rest of the procedure remain the same. For example, in the example given earlier ifone wants to fuzz DefineBits tag in JPEG file which would be embedded into a SWF file they have toonly write a mutator for this particular tag i.e. DefineBits Tag. Rest of the procedure remains thesame.In my study I will reuse my above code developed in directed fuzzing and will fuzz only a particulartag which is maxp in case of TTF fonts which would be further embedded into the PDF file.ConclusionWith new functionalities being added into application with every release, new modules being addedand legacy code left few holes unplugged. One has the opportunity to find these bugs given a refinedand detailed process to fuzz. More knowledge of the protocol always yields into better fuzzers andmore bugs.Bibliography1. Conficker. Wikipedia. [Online] http://en.wikipedia.org/wiki/Conficker.2. Stuxnet. Wikipedia. [Online] http://en.wikipedia.org/wiki/Stuxnet.3. Operation Aurora. Wikipedia. [Online] http://en.wikipedia.org/wiki/Operation_Aurora.4. SDL. Microsoft . [Online] http://www.microsoft.com/security/sdl/discover/default.aspx.
5. John Neystadt. Automated Penetration Testing with White-Box Fuzzing. Microsoft. [Online]Microsoft Corporation, February 2008. http://msdn.microsoft.com/en-us/library/cc162782.aspx.6. Hetzel, William C. The Complete Guide to Software Testing, 2nd ed. s.l. : Wellesley, Mass.0894352423.7. Jiantao Pan. Software Testing. Carnegie Mellon University. [Online] 1999.http://www.ece.cmu.edu/~koopman/des_s99/sw_testing/.8. Code coverage. Wikipedia. [Online] http://en.wikipedia.org/wiki/Code_coverage.9. Noam Rathaus, Gadi Evron. Fuzzing - Whats that? Open source fuzzing tools. s.l. : Syngress, 2007.10. Michael Eddington. Peach Fuzzing Platform. Peach Fuzzer. [Online] Deja Vu Security.http://peachfuzzer.com/.11. Pedram Amini. Sulley. Google Code. [Online] http://code.google.com/p/sulley/.12. Kyle Randolph. Fuzzing Reader - Lessons learned. Adobe. [Online] Adobe Systems Incorporated,1 December 2009. http://blogs.adobe.com/asset/2009/12/fuzzing_reader_-_lessons_learned.html.13. Codenomicon. [Online] http://www.codenomicon.com/.14. Kaul, Nihit. Pesticide Paradox. Blogs MSDN. [Online] Microsoft Corporation, Jul 2004.http://blogs.msdn.com/b/nihitk/archive/2004/07/16/185836.aspx.15. Taint-based Directed Whitebox Fuzzing. V. Ganesh, T. Leek, and M. Rinard. s.l. : Proceedings ofthe 2009 IEEE 31st International Conference on Software Engineering, 2009.16. Fuzz Testing. Wikipedia. [Online] http://en.wikipedia.org/wiki/Fuzzing.17. The buzz about fuzzer. Info World. [Online] http://www.infoworld.com/d/developer-world/buzz-about-fuzzers-089.18. Software Testing. Wikipedia. [Online] http://en.wikipedia.org/wiki/Software_testing.19. Quality Assurance. Wikipedia. [Online] http://en.wikipedia.org/wiki/Quality_assurance.20. SDL. Wikipedia. [Online] http://en.wikipedia.org/wiki/Software_development_life_cycle.21. Paradox of the pesticides. Wikipedia. [Online]http://en.wikipedia.org/wiki/Paradox_of_the_pesticides.22. Grammar-based Whitebox Fuzzing. Patrice Godefroid, Adam Kiezun, and Michael Y. Levin. s.l. :Microsoft Research, 2007. MSR-TR-2007-154.