Fuzzing101 uvm-reporting-and-mitigation-2011-02-10


Published on

Fuzzing 101 webinar: unknown vulnerability management - reporting and mitigation

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Fuzzing101 uvm-reporting-and-mitigation-2011-02-10

  1. 1. UVM: Reporting and Mitigation Ari Takanen, CTO of Codenomicon Feb 10th, 2011
  2. 2. <ul><li>Fuzzing 101: </li></ul><ul><ul><li>The webcast series for fuzzing industry </li></ul></ul><ul><ul><li>Vendor neutral presentations on fuzzing technologies and use-cases </li></ul></ul><ul><ul><li>Includes invited speakers from the industry </li></ul></ul><ul><li>Codenomicon: </li></ul><ul><ul><li>Fuzzing research since 1996 </li></ul></ul><ul><ul><li>2001, Spinoff from University of Oulu </li></ul></ul><ul><ul><li>50-100% annual growth in number of customers and revenues in fuzzing industry </li></ul></ul>About Fuzzing 101 and Codenomicon
  3. 3. About Ari Takanen <ul><li>The Past: Researcher and Lecturer </li></ul><ul><ul><li>1998-2002 </li></ul></ul><ul><ul><li>University of Oulu </li></ul></ul><ul><ul><li>OUSPG/PROTOS research group </li></ul></ul><ul><ul><li>Software Quality related lectures </li></ul></ul><ul><li>The Present: Entrepreneur and Evangelist </li></ul><ul><ul><li>2001-today </li></ul></ul><ul><ul><li>CTO of Codenomicon </li></ul></ul><ul><ul><li>Evangelist: 10 conference talks every year </li></ul></ul><ul><ul><li>Author of two books: </li></ul></ul><ul><ul><ul><li>VoIP Security </li></ul></ul></ul><ul><ul><ul><li>Fuzzing </li></ul></ul></ul>
  4. 4. Agenda <ul><li>Introduction to UVM: </li></ul><ul><li>UVM Overview and Phases </li></ul><ul><ul><li>Attack Surface Analysis </li></ul></ul><ul><ul><li>Testing = Fuzzing </li></ul></ul><ul><ul><li>Reporting and Mitigation </li></ul></ul><ul><li>Vulnerability Reporting </li></ul><ul><ul><li>Process and best practises </li></ul></ul><ul><li>Mitigating Unknown Threats </li></ul><ul><ul><li>Fuzzing results with security devices </li></ul></ul><ul><ul><li>Challenges </li></ul></ul>
  5. 5. Unknown Vulnerability Management Overview and Phases The Only Means of Catching Zero-Day Vulnerabilities!
  6. 6. Challenges with Vulnerability Management <ul><li>Detect Vulnerabilities as they are found </li></ul><ul><ul><li>Not as they emerge, they are in the hiding already </li></ul></ul><ul><li>Most costs are in patch deployment </li></ul><ul><ul><li>Crisis management, each update needs immediate attention </li></ul></ul><ul><ul><li>Ad-hoc deployment is prone to errors </li></ul></ul><ul><ul><li>Maintenance downtime can be expensive </li></ul></ul><ul><ul><li>New patches emerge several times a week </li></ul></ul><ul><ul><li>No time to test the patch </li></ul></ul>
  7. 7. Zero-Day Vulnerability <ul><li>The biggest threat to IT systems everywhere is a Zero-Day vulnerability </li></ul><ul><ul><li>No patches or updates available </li></ul></ul><ul><ul><li>No security defenses </li></ul></ul><ul><li>Software is released with vulnerabilities </li></ul><ul><li>In the past, those problems were found and disclosed by the security community = hackers </li></ul>
  8. 8. Some Helpful Definitions <ul><li>Vulnerability – a weakness in software, a bug </li></ul><ul><li>Threat/Attack – exploit/worm/virus against a specific vulnerability </li></ul><ul><li>Protocol Modeling – Technique for explaining interface message sequences and message structures </li></ul><ul><li>Fuzzing – process and technique for security testing </li></ul><ul><li>Anomaly – abnormal or unexpected input </li></ul><ul><li>Failure – crash, busy-loop, memory corruption, or other indication of a bug in software </li></ul>
  9. 9. The Challenge: Unknown Vulnerabilities Are Everywhere
  10. 10. Codenomicon Labs Test Results http://www.codenomicon.com/labs/results
  11. 11. Unknown Vulnerability Management (UVM) <ul><li>Another name: </li></ul><ul><li>Zero-Day Vulnerability Management </li></ul><ul><li>Process of: </li></ul><ul><ul><li>Detecting attack vectors </li></ul></ul><ul><ul><li>Finding zero-day vulnerabilities </li></ul></ul><ul><ul><li>Building defenses </li></ul></ul><ul><ul><li>Performing patch verification </li></ul></ul><ul><ul><li>Deployment in one big security push </li></ul></ul>
  12. 12. UVM Phases
  13. 13. Phase 1: Attack Surface Analysis <ul><li>Tools: </li></ul><ul><ul><li>Port scanners </li></ul></ul><ul><ul><li>Resource scanners </li></ul></ul><ul><ul><li>Network analyzers </li></ul></ul><ul><li>Codenomicon Network Analyzer identifies what needs to be tested within your network </li></ul><ul><ul><li>Record traffic at multiple points in your network </li></ul></ul><ul><ul><li>Automatically visualize the network </li></ul></ul><ul><ul><li>You can drill up and down from looking at high-level visualizations to inspecting the corresponding packet data </li></ul></ul><ul><ul><li>Real time analysis </li></ul></ul><ul><ul><li>Reveal hidden interfaces and possible exploits. </li></ul></ul>
  14. 14. Phase 2: Test <ul><li>Fuzzing means crash-testing </li></ul><ul><li>Discover both known and previously unknown vulnerabilities with unparalleled efficiency. </li></ul><ul><li>Specification-based tools for over 200 protocols </li></ul><ul><ul><li>Tools contain all the possible protocol messages and structures </li></ul></ul><ul><ul><li>Genuinely interoperate with the tested system exposing vulnerabilities even in deeper protocol layers </li></ul></ul><ul><li>General purpose fuzzers </li></ul><ul><ul><li>Defensics XML Fuzzer can test all XML applications. </li></ul></ul><ul><ul><li>The Traffic Capture Fuzzer uses real traffic </li></ul></ul><ul><ul><li>Generic File Format Fuzzer tests all file formats. </li></ul></ul>
  15. 15. Phase 3: Report <ul><li>Codenomicon test suites generate different reports for different audiences </li></ul><ul><li>Management reports provide an high-level overview of the test execution </li></ul><ul><li>Log files and spreadsheets help you to identify troublesome tests and to minimize false negatives </li></ul><ul><li>Individual tests by augmenting the already extensive test case documentation with PCAP traffic recordings </li></ul><ul><li>Remediation Packages can be send to third parties for automated reproduction. </li></ul>
  16. 16. Phase 4: Mitigate <ul><li>Mitigation tools quickly and easily reproduce vulnerabilities, perform regression testing and verify patches </li></ul><ul><li>The tools automatically generate reports, which contain risk assessment and CWE values for the found vulnerabilities and direct links to the test suites that triggered the vulnerabilities </li></ul><ul><li>Identification of the test cases that triggered the vulnerability is critical </li></ul><ul><li>The test case documentation can be used to create tailored IDS rules to block possible zero-day attacks. </li></ul>
  17. 17. Vulnerability Reporting Experiences with Bug Reporting
  18. 18. Security View: Window of Vulnerability SW - under vulnerability analysis SW - after product release SW - after the vulnerability process TIME BUG APPEARS RELEASE BUG FOUND VULN FOUND VULN REPORT VULN FIX AVAIL. PATCH RELEASE ADVISORY RELEASE PATCH INSTALL Zero Exposure Limited Exposure Public Exposure
  19. 19. Vulnerability Reporting Process
  20. 20. Finding and Reporting Problems <ul><li>Motivations in finding bugs in third party code: </li></ul><ul><ul><li>Protecting your own system </li></ul></ul><ul><ul><li>Security research in some other component </li></ul></ul><ul><li>Note that the above is one of the most important things to analyze! </li></ul>
  21. 21. How and To Who Do You Report? <ul><li>To whom do you report the findings </li></ul><ul><ul><li>First ask from your support contact (no details yet) </li></ul></ul><ul><ul><li>Vendor security contact (vendor CERT, PSIRT, …) </li></ul></ul><ul><ul><li>Mailing lists, support forums, … </li></ul></ul><ul><ul><li>Third party such as CERT </li></ul></ul><ul><ul><li>Remember confidentiality, encryption (PGP) </li></ul></ul>
  22. 22. A Bug Report <ul><li>What to write to a bug/incident report? </li></ul><ul><ul><li>Note that first purpose is reproduction </li></ul></ul><ul><li>Selected observations: </li></ul><ul><ul><li>Use bug report templates </li></ul></ul><ul><ul><li>Write complete but compact bug reports </li></ul></ul><ul><ul><li>Be prepared that the report might leak or become public </li></ul></ul><ul><ul><li>Clearly indicate what is confidential information </li></ul></ul><ul><ul><li>Consider carefully if you want to write demonstration exploits (but be prepared for it) </li></ul></ul><ul><ul><li>Never propose a specific fix (in worst case it will just be blindly adopted) </li></ul></ul>
  23. 23. Share Test Environment Details <ul><li>Bug reports should clearly indicate the test environment that was used </li></ul><ul><ul><li>Exact software versions </li></ul></ul><ul><ul><li>Operating system versions </li></ul></ul><ul><ul><li>Network architecture (or virtual setup) </li></ul></ul><ul><li>If possible, give the vendor the exact setup </li></ul><ul><ul><li>This is easiest with virtual setups </li></ul></ul>
  24. 24. Defensics Includes Test Setup Details
  25. 25. Collaboration and Document Sharing <ul><li>If your test tools produce reports, share those with the vendor </li></ul><ul><li>In many cases, it might be best to actually share the fuzzing tool that you used to find the issue </li></ul><ul><li>Collaboration platforms and IM will make communication much easier </li></ul><ul><ul><li>Codenomicon can provide the platform for you </li></ul></ul>
  26. 26. Best Method for Reproducing with Defensics
  27. 27. But Often You See This…
  28. 28. PCAPs as Test Documentation <ul><li>PCAPs are quick method for reproduction </li></ul><ul><li>Most fuzzing tools can save tests as PCAP, and re-execute those PCAPs </li></ul><ul><ul><li>E.g. Defensics Traffic Capture Fuzzer reproduces and tests around most PCAPs, wherever they come from </li></ul></ul><ul><li>The biggest challenge with PCAPs is that they are difficult to maintain </li></ul><ul><li>When specification changes, the efficiency of PCAP-based test will quickly degenerate </li></ul>
  29. 29. Risk Assessment from Bug Identification
  30. 30. Different Types of Reports <ul><li>Fuzzing tools will produce different reports for different audiences </li></ul>
  31. 31. Remediation Package <ul><li>Defensics Remediation Package collects all required data </li></ul><ul><li>Typical difficulties with reproduction: </li></ul><ul><ul><li>If only the reported test is reproduced, the next test case can still crash the system </li></ul></ul>
  32. 32. How to Protect Yourself <ul><li>It is good to involve third party in the process: </li></ul><ul><ul><li>Codenomicon (or another security company) </li></ul></ul><ul><ul><li>Internal CERT </li></ul></ul><ul><ul><li>External CERT </li></ul></ul><ul><li>Remember that legal departments will almost always get involved </li></ul><ul><li>If something becomes difficult, let the professionals in the bug reporting process handle it </li></ul><ul><li>Remember: Almost always the biggest failure is in personal communications </li></ul>
  33. 33. What to Expect <ul><li>Timeline: </li></ul><ul><ul><li>From reporting to first acknowledgement should be really really fast (reporters do not have patience) </li></ul></ul><ul><ul><li>From acknowledgement to first threat analysis: “we do not think this is critical” </li></ul></ul><ul><ul><li>From start of fixing to ready patch, this is often the longest wait, up to months or few years </li></ul></ul><ul><li>Quality of the patch: </li></ul><ul><ul><li>Quick and dirty patches are always suspicious </li></ul></ul><ul><ul><li>When vendor is given more time, the quality is often better, and the patch does not break any valid functionality </li></ul></ul><ul><li>What to do while waiting? </li></ul>
  34. 34. Challenges in adopting Fuzzing results with security devices Mitigating Unknown Threats
  35. 35. Challenges with IDS <ul><li>Although all fuzz test cases are easy to convert to IDS rules, those IDS rules are quite easy to circumvent </li></ul><ul><ul><li>“ Advanced Evasion Techniques” ;-) </li></ul></ul><ul><li>Traditionally IDS rules detect individual attack exploits, with the same problem that variants of the same attack can easily pass through </li></ul>
  36. 36. How to Build Good IDS Rules <ul><li>Be generic in the rules, detect string lengths instead of string contents, etc. </li></ul><ul><li>Extensively test the rules with both legal (valid case) traffic and with systematic fuzzing in the vulnerable protocol element </li></ul><ul><li>Note that not all fuzz tests need to be blocked by IDS -> Too complex, does not work </li></ul>
  37. 37. Patch Verification Using Fuzzers <ul><li>Use the same tool and same setup immediately when you receive the first patch candidate </li></ul><ul><li>Also relevant to vendor, before even providing the patch candidate to the reporter </li></ul><ul><ul><li>Note that it is really annoying to reporters when the patch is bad quality, and they might just give up if they do not have motivation to continue </li></ul></ul><ul><li>Some critical systems might already include the patch you are testing, so bugs in the patch are also critical, and have to be handled in such way </li></ul>
  38. 38. Conclusions <ul><li>Most challenges in bug reporting are not in technology, but communication </li></ul><ul><ul><li>Codenomicon provides collaboration platform to assist </li></ul></ul><ul><li>Main purpose of bug reports in reproduction of the found flaws </li></ul><ul><ul><li>Codenomicon Remediation Package!! </li></ul></ul><ul><li>Easiest reproduction of bugs is always with exactly the same test setup, and exactly the same tool </li></ul><ul><ul><li>Codenomicon provides free access to our tools for reproduction </li></ul></ul><ul><li>Fuzz test cases cannot all be blocked by IDS </li></ul><ul><ul><li>Codenomicon provides extensive documentation of every test case </li></ul></ul>
  39. 39. Summary of UVM <ul><li>Security is not about security mechanisms </li></ul><ul><li>For full security analysis, you should study: </li></ul><ul><ul><li>Threats </li></ul></ul><ul><ul><li>Attacks </li></ul></ul><ul><ul><li>Vulnerabilities </li></ul></ul><ul><ul><li>Architectures </li></ul></ul><ul><ul><li>Countermeasures </li></ul></ul><ul><li>Unknown Vulnerability Management is about identification and elimination of zero-day vulnerabilities </li></ul><ul><li>Security is a process not a product! </li></ul>
  40. 40. Our Book On Fuzzing! <ul><li>http://www.fuzz-test.com/book/ </li></ul><ul><li>Takanen, DeMott and Miller: “Fuzzing for Software Security Testing and Quality Assurance” </li></ul><ul><li>Aimed at the general public, you do not need to be a security specialist to read this book </li></ul><ul><li>Purpose of the book is to teach next-gen testing approaches to: </li></ul><ul><ul><li>Software practitioners </li></ul></ul><ul><ul><li>Security engineers </li></ul></ul><ul><ul><li>Academics </li></ul></ul>
  41. 41. More News from Codenomicon <ul><li>Facebook: </li></ul><ul><ul><li>Become fan of Codenomicon and Fuzzing </li></ul></ul><ul><li>Twitter: </li></ul><ul><ul><li>CodenomiconLTD </li></ul></ul><ul><li>Codenomicon Website: </li></ul><ul><ul><li>Newsletter every second month </li></ul></ul><ul><li>Customer portal </li></ul><ul><ul><li>The Backstage! Contact us for access! </li></ul></ul>
  42. 42. PROACTIVE SECURITY AND ROBUSTNESS SOLUTIONS THANK YOU – QUESTIONS? “ Thrill to the excitement of the chase! Stalk bugs with care, methodology, and reason. Build traps for them. .... Testers! Break that software (as you must) and drive it to the ultimate - but don’t enjoy the programmer’s pain.” [from Boris Beizer]