Using ThreadFix to Manage Application Vulnerabilities

2,542 views

Published on

ThreadFix is an open source software vulnerability aggregation and management system that reduces the time it takes to fix software vulnerabilities. It imports the results from dynamic, static and manual testing to provide a centralized view of software security defects across development teams and applications. The system allows organizations to correlate testing results and streamline software remediation efforts by simplifying feeds to software issue trackers. This presentation will walk through the major functionality in ThreadFix and describe several common use cases such as merging the results of multiple open source and commercial scanning tools and services. It will also demonstrate how ThreadFix can be used to track the results of scanning over time and gauge the effectiveness of different scanning techniques and technologies. Finally it will provide examples of how tracking assurance activities across an organization’s application portfolio can help the organization optimize remediation activities to best address risks associated with vulnerable software.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,542
On SlideShare
0
From Embeds
0
Number of Embeds
423
Actions
Shares
0
Downloads
51
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Using ThreadFix to Manage Application Vulnerabilities

  1. 1. Using ThreadFix to Manage Application Vulnerabilities! ! Dan Cornell! CTO, Denim Group! @danielcornell© Copyright 2013 Denim Group - All Rights Reserved
  2. 2. My Background •  Dan Cornell, founder and CTO of Denim Group •  Software developer by background (Java, .NET, etc) •  OWASP San Antonio, Global Membership Committee© Copyright 2013 Denim Group - All Rights Reserved 2
  3. 3. Denim Group Background •  Secure software services and products company –  Builds secure software –  Helps organizations assess and mitigate risk of in-house developed and third party software –  Provides classroom training and e-Learning so clients can build software securely •  Software-centric view of application security –  Application security experts are practicing developers –  Development pedigree translates to rapport with development managers –  Business impact: shorter time-to-fix application vulnerabilities •  Culture of application security innovation and contribution –  Develops open source tools to help clients mature their software security programs •  Remediation Resource Center, ThreadFix –  OWASP national leaders & regular speakers at RSA, SANS, OWASP, ISSA, CSI –  World class alliance partners accelerate innovation to solve client problems© Copyright 2013 Denim Group - All Rights Reserved 3
  4. 4. Agenda •  Introductions •  Application Vulnerability Management •  ThreadFix Background •  Use Cases / Demonstrations –  Track Scan Results Over Time –  De-Duplicate and Merge Multiple Scanners –  Scanner Benchmarking –  Virtual Patching –  Turning Vulnerabilities into Software Defects –  Program Benchmark Reporting •  Future Directions •  Questions© Copyright 2013 Denim Group - All Rights Reserved 4
  5. 5. Application Vulnerability Management •  Application security teams uses automated static and dynamic test results as well as manual testing results to assess the security of an application •  Each test delivers results in different formats •  Different test platforms describe same flaws differently, creating duplicates •  Security teams end up using spreadsheets to keep track manually •  It is extremely difficult to prioritize the severity of flaws as a result •  Software development teams receive unmanageable reports and only a small portion of the flaws get fixed© Copyright 2013 Denim Group - All Rights Reserved 5
  6. 6. The Result •  Application vulnerabilities persist in applications: **Average serious vulnerabilities found per website per year is 79 **Average days website exposed to one serious vulnerability is 231 days **Overall percentage of serious vulnerabilities that are fixed annually is only 63% •  Part of that problem is there is no easy way for the security team and application development teams to work together on these issues •  Remediation quickly becomes an overwhelming project •  Trending reports that track the number of reduced vulnerabilities are impossible to create **WhiteHat Statistics Report (Summer 2012): https://www.whitehatsec.com/assets/WPstats_summer12_12th.pdf© Copyright 2013 Denim Group - All Rights Reserved 6
  7. 7. Vulnerability Fun Facts: •  Average number of serious vulnerabilities found per website per year is 79 ** •  Serious Vulnerabilities were fixed in ~38 days ** •  Percentage of serious vulnerabilities fixed annually is only 63% ** •  Average number of days a website is exposed, at least one serious vulnerability ~231 days WhiteHat Statistics Report (Summer 2012): https://www.whitehatsec.com/assets/WPstats_summer12_12th.pdf© Copyright 2013 Denim Group - All Rights Reserved 7
  8. 8. Vulnerability Remediation Data Vulnerability  Type   Sample  Count   Average  Fix  (minutes)   Dead  Code  (unused  methods)   465   2.6   Poor  logging:  system  output  stream   83   2.9   Poor  Error  Handling:  Empty  catch  block   180   6.8   Lack  of  AuthorizaKon  check   61   6.9   Unsafe  threading   301   8.5   ASP.NET  non-­‐serializable  object  in  session   42   9.3   XSS  (stored)   1023   9.6   Null  Dereference   157   10.2   Missing  Null  Check   46   15.7   XSS  (reflected)   25   16.2   Redundant  null  check   21   17.1   SQL  injecKon   30   97.5  © Copyright 2013 Denim Group - All Rights Reserved 8
  9. 9. Where Is Time Being Spent? 70% Indicates the weighted average versus the average of 60% individual projects 59% 50% 44% 42% 40% 37% 30% 31% 28% 29% 24% 24% 20% 17% 20% 15% 15% 16% 10% 9% 3% 2% 0% 0% 0% Setup Development Fix Vulnerabilities Confirm Fixes / QA 0% Deploy Overhead Environment© Copyright 2013 Denim Group - All Rights Reserved 9
  10. 10. Enter ThreadFix •  An open source software vulnerability aggregation and management system •  Imports dynamic, static and manual testing results into a centralized platform •  Removes duplicate findings across all testing platforms to provide a prioritized list of security faults •  Eases communication across development, security and QA teams •  Exports the prioritized list into the company’s bug tracker of choice to streamline software remediation efforts •  Auto generates web application firewall rules to protect corporate data while the software vulnerability is being fixed •  Empowers managers with vulnerability trending reports that can pinpoint team issues and illustrate application security progress© Copyright 2013 Denim Group - All Rights Reserved 10
  11. 11. ThreadFix Background •  An open source vulnerability management and aggregation platform that allows software security teams to reduce the time it takes to fix software vulnerabilities •  Freely available under the Mozilla Public License (MPL) •  Download available at: www.denimgroup.com/threadfix© Copyright 2013 Denim Group - All Rights Reserved 11
  12. 12. ThreadFix Consolidates reports so managers can speak intelligently about the status and trends of security within their organization© Copyright 2013 Denim Group - All Rights Reserved 12
  13. 13. Vulnerability Import • Pulls in static and dynamic results • Eliminates duplicate results • Allows for results to be grouped© Copyright 2013 Denim Group - All Rights Reserved 13
  14. 14. © Copyright 2013 Denim Group - All Rights Reserved
  15. 15. Real-Time Protection Virtual patching helps protect organizations during remediation© Copyright 2013 Denim Group - All Rights Reserved 15
  16. 16. © Copyright 2013 Denim Group - All Rights Reserved
  17. 17. Defect Tracking • ThreadFix can connect to common defect trackers • Defects can be created for developers Integration • Work can continue uninterrupted© Copyright 2013 Denim Group - All Rights Reserved 17
  18. 18. © Copyright 2013 Denim Group - All Rights Reserved
  19. 19. Large Range of Tool Compatibility© Copyright 2013 Denim Group - All Rights Reserved 19
  20. 20. Supported Tools: Dynamic Scanners SaaS Testing Platforms Acunetix WhiteHat Arachni Veracode Burp Suite QualysGuard WAS 2.0 HP WebInspect IBM Security AppScan IDS/IPS and WAF Mavituna Security Netsparker DenyAll NTO Spider F5 OWASP Zed Attack Proxy Imperva Tenable Nessus mod_security Skipfish Snort w3aF Defect Trackers Static Scanners Atlassian JIRA FindBugs Microsoft Team Foundation Server IBM Security AppScan Source Mozilla Bugzilla HP Fortify SCA Microsoft CAT.NET Brakeman© Copyright 2013 Denim Group - All Rights Reserved 20
  21. 21. Use Cases / Demonstrations •  Track Scan Results Over Time •  De-Duplicate and Merge Multiple Scanners •  Scanner Benchmarking •  Virtual Patching •  Turning Vulnerabilities into Software Defects •  Program Benchmark Reporting© Copyright 2013 Denim Group - All Rights Reserved 21
  22. 22. Track Scan Results Over Time •  Pretty basic, but many software security programs have problems providing even basic metrics and trending graphs •  Goal: Turn a “dude with a scanner” into a “dude with some data” •  Notes: –  Each new scan is diff-ed against the previous scan –  Vulnerabilities are tracked as new, fixed, reopened –  You can durably mark false positives© Copyright 2013 Denim Group - All Rights Reserved 22
  23. 23. Track Scan Results Over Time •  Demonstration© Copyright 2013 Denim Group - All Rights Reserved 23
  24. 24. De-Duplicate and Merge Multiple Scanners •  Q: What’s worse than handing a developer a 300 page PDF? •  A: Handing a developer two 300 page PDFs! •  Communicating vulnerabilities via PDF is a horrible interaction pattern for security and development teams (more on this later)© Copyright 2013 Denim Group - All Rights Reserved 24
  25. 25. What is a Unique Vulnerability? •  (CWE, Relative URL) –  Predictable resource location –  Directory listing misconfiguration •  (CWE, Relative URL, Injection Point) –  SQL injection –  Cross-site Scripting (XSS) •  Injection points –  Parameters – GET/POST –  Cookies –  Other headers© Copyright 2013 Denim Group - All Rights Reserved 25
  26. 26. What Do The Scanner Results Look Like? •  Usually XML –  Skipfish uses JSON and gets packaged as a ZIP •  Scanners have different concepts of what a “vulnerability” is –  We normalize to the (CWE, location, [injection point]) noted before •  Look at some example files •  Several vendors have been really helpful adding additional data to their APIs and file formats to accommodate requests (thanks!)© Copyright 2013 Denim Group - All Rights Reserved 26
  27. 27. Why Common Weakness Enumeration (CWE)? •  Every tool has their own “spin” on naming vulnerabilities –  OWASP Top 10 / WASC XX are helpful but not comprehensive •  We tried to create our own vulnerability classification scheme –  Proprietary –  Not sustainable –  Stupid •  CWE is pretty exhaustive •  Reasonably well-adopted standard •  Many tools have mappings to CWE for their results •  Main site: http://cwe.mitre.org/© Copyright 2013 Denim Group - All Rights Reserved 27
  28. 28. Challenges Using the CWE •  It is pretty big (909 nodes, 693 actual weaknesses) •  But it kind of has to be to be comprehensive… •  Many tools provide mappings •  And sometimes they’re even kind of accurate! •  Some tools provide more than one CWE category for a vulnerability •  So in ThreadFix we make a best guess •  Some tools provide “junk” results •  So in ThreadFix we collapse those into a single vulnerability •  Some organizations have their own classification schemes© Copyright 2013 Denim Group - All Rights Reserved
  29. 29. De-Duplicate and Merge Multiple Scanners •  Demonstration© Copyright 2013 Denim Group - All Rights Reserved 29
  30. 30. Scanner Benchmarking •  Of the scanning technologies you are using, which is providing the most value?© Copyright 2013 Denim Group - All Rights Reserved 30
  31. 31. Scanner Coverage •  You can’t test what you can’t see •  How effective is the scanner’s crawler? •  How are URLs mapped to functionality? •  RESTful •  Parameters •  Possible issues: •  Login routines •  Multi-step processes •  Anti-CSRF protection© Copyright 2013 Denim Group - All Rights Reserved 31
  32. 32. Are You Getting a Good Scan?•  Large financial firm: “Our 500 page website is secure because the scanner did not find any vulnerabilities!”•  Me: “Did you teach the scanner to log in so that it can see more than just the homepage?”•  Large financial firm: “…”© Copyright 2013 Denim Group - All Rights Reserved 32
  33. 33. Did I Get a Good Scan? •  Scanner training is really important •  Read the Larry Suto reports… •  Must sanity-check the results of your scans •  What URLs were accessed? •  If only two URLs were accessed on a 500 page site, you probably have a bad scan •  If 5000 URLs were accessed on a five page site, you probably have a bad scan •  What vulnerabilities were found and not found? •  Scan with no vulnerabilities – probably not a good scan •  Scan with excessive vulnerabilities – possibly a lot of false positives© Copyright 2013 Denim Group - All Rights Reserved 33
  34. 34. Low False Positives•  Reports of vulnerabilities that do not actually exist•  How “touchy” is the scanner’s testing engine?•  Why are they bad? –  Take time to manually review and filter out –  Can lead to wasted remediation time© Copyright 2013 Denim Group - All Rights Reserved 34
  35. 35. Low False Negatives •  Scanner failing to report vulnerabilities that do exist •  How effective is the scanner’s testing engine? •  Why are they bad? –  You are exposed to risks you do not know about –  You expect that the scanner would have found certain classes of vulnerabilities •  What vulnerability classes do you think scanners will find?© Copyright 2013 Denim Group - All Rights Reserved 35
  36. 36. Other Benchmarking Efforts •  Larry Suto’s 2007 and 2010 reports •  Analyzing the Accuracy and Time Costs of Web Application Security Standards –  http://ha.ckers.org/files/Accuracy_and_Time_Costs_of_Web_App_Scanners.pdf •  Vendor reactions were … varied –  [Ofer Shezaf attended this talk at AppSecEU 2012 and had some great questions and comments. See his reactions to the latest Larry Suto scanner report here : http://www.xiom.com/2010/02/09/wafs-are-not-perfect-any-security-tool-perfect ] •  Shay Chen’s Blog and Site •  http://sectooladdict.blogspot.com/ •  http://www.sectoolmarket.com/ •  http://www.infosecisland.com/blogview/21926-A-Step-by-Step-Guide-for-Choosing-the-Best- Scanner.html •  Web Application Vulnerability Scanner Evaluation Project (wavsep) •  http://code.google.com/p/wavsep/© Copyright 2013 Denim Group - All Rights Reserved 36
  37. 37. Scanner Benchmarking •  Demonstration© Copyright 2013 Denim Group - All Rights Reserved 37
  38. 38. Virtual Patching •  Connect vulnerability scanners to IDS/IPS/ WAF systems •  Map data from sensors back to data about vulnerabilities© Copyright 2013 Denim Group - All Rights Reserved 38
  39. 39. Virtual Patches - Formats •  Two approaches 1.  (vulnerability_type, vulnerability_location) 2.  (vulnerability_signature , vulnerability_location) (1) “There is a reflected XSS vulnerability in login.php for the username parameter” versus (2) “Watch out for HTML-ish characters in login.php for the username parameter” •  The snort and mod_security rules follow approach (2) •  Integration with commercial solutions may use approach (1)© Copyright 2013 Denim Group - All Rights Reserved 39
  40. 40. Trivia and Analysis •  IDS/IPS/WAF has an impact on the scanning process –  Snort breaks w3af scanning –  mod_security CRS introduces some false positives into skipfish scanning •  mod_security CRS is quite good –  And getting better all the time: SQL Injection Challenge –  http://blog.spiderlabs.com/2011/06/announcing-the-modsecurity-sql-injection-challenge.html •  Virtual patching appears to win for injection flaws© Copyright 2013 Denim Group - All Rights Reserved 40
  41. 41. Where Is This Useful? •  Environments where you have little or no control over deployed code –  XaaS – PaaS, IaaS –  99% of all corporate data centers •  Environments where you have a large “application security debt” –  Actual code fixes: take time and can be hard to get on the schedule© Copyright 2013 Denim Group - All Rights Reserved 41
  42. 42. What Are The Problems? •  Current vulnerability data formats only allow for coarse-grained virtual patches –  Can lead to false blocks •  Virtual patches likely will not stop well-informed, determined attackers –  See the results of the mod_security SQL Injection Challenge© Copyright 2013 Denim Group - All Rights Reserved 42
  43. 43. Virtual Patching •  Demonstration© Copyright 2013 Denim Group - All Rights Reserved 43
  44. 44. Turning Vulnerabilities Into Software Defects •  Security teams talk about “vulnerabilities” •  Software developers talk about “defects” •  Developers Don’t Speak PDF –  http://blog.denimgroup.com/denim_group/2012/11/hey-security-teams-developers-dont-speak-pdf.html •  Why should developers manage 90% of their workload in defect trackers –  And the magic, special “security” part of their workload … some other way? •  ThreadFix lets you slice, dice and bundle vulnerabilities into software defects –  And track their remediation status over time to schedule re-scans© Copyright 2013 Denim Group - All Rights Reserved 44
  45. 45. But My Bug Tracker Isn’t Supported! •  We are always working on supporting new technologies –  Check out the current support list: https://code.google.com/p/threadfix/wiki/DefectTrackers –  Submit a bug to the TheadFix defect tracker https://code.google.com/p/threadfix/issues/list •  You can add new defect trackers as plugins –  No changes to the core codebase required –  For instructions and sample code check out the wiki article: https://code.google.com/p/threadfix/wiki/CustomDefectTrackerGuide© Copyright 2013 Denim Group - All Rights Reserved 45
  46. 46. Turning Vulnerabilities Into Software Defects •  Demonstration© Copyright 2013 Denim Group - All Rights Reserved 46
  47. 47. Program Benchmark Reporting •  How does your software security organization stack up? –  Look at publicly-shared data from WhiteHat and Veracode •  Compare your progress –  Percentage of vulnerabilities fixed –  Time to fix different vulnerability types –  Age of remaining vulnerabilities© Copyright 2013 Denim Group - All Rights Reserved 47
  48. 48. Program Benchmark Reporting •  Demonstration© Copyright 2013 Denim Group - All Rights Reserved 48
  49. 49. Current Status •  1.0 released September 17th, 2012 •  1.0.1 released October 19th, 2012 •  1.1 (release candidate) released January 28th, 2013 •  Final 1.1 coming in the next couple of weeks© Copyright 2013 Denim Group - All Rights Reserved 49
  50. 50. Future Directions •  Increase the audience that can find ThreadFix useful –  Add native scanning capability –  Add scan scheduling and coordination capability •  Address “enterprise” concerns –  Expanded security model available in version 1.1 –  Continue to grow this area •  Improve the user experience •  Dashboard and reporting© Copyright 2013 Denim Group - All Rights Reserved 50
  51. 51. Common Usage Scenarios •  Use ThreadFix to provide an “enterprise” console for a standalone desktop scanning tool •  Use ThreadFix to normalize and merge multiple sources of vulnerability data –  Including the results of manual code reviews, threat models, etc •  Use ThreadFix as a base for a custom application vulnerability management solution –  We’ve already written a LOT of code and solved a LOT of problems© Copyright 2013 Denim Group - All Rights Reserved 51
  52. 52. How Can You Help? •  Use it and provide feedback –  Bug reports –  Usability recommendations –  Feature requests •  Scan file examples –  Multiple tools, multiple versions, limited sample set –  Help! •  Contribute© Copyright 2013 Denim Group - All Rights Reserved 52
  53. 53. How To Get ThreadFix •  Denim Group ThreadFix homepage: www.denimgroup.com/threadfix •  Google Code site: https://code.google.com/p/threadfix/ •  Google Group: https://groups.google.com/forum/?fromgroups#!forum/ThreadFix© Copyright 2013 Denim Group - All Rights Reserved 53
  54. 54. Conclusions / Questions Dan Cornell dan@denimgroup.com Twitter: @danielcornell www.denimgroup.com www.denimgroup.com/threadfix code.google.com/p/threadfix (210) 572-4400© Copyright 2013 Denim Group - All Rights Reserved 54

×