Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DefCamp 2013 - Are we there yet?

582 views

Published on

Published in: Technology, News & Politics
  • Be the first to comment

  • Be the first to like this

DefCamp 2013 - Are we there yet?

  1. 1. 10 Years Later: Are We There Yet? Carsten Eiram Risk Based Security @CarstenEiram
  2. 2. Quick Bio – VDB Work Experience Involved with VDBs for 10+ years • Currently, CRO at Risk Based Security – commercial arm of Open Security Foundation (runs OSVDB and DatalossDB) – and responsible for the VulnDB service. • Chief Security Specialist at Secunia, running the Research team. • Security Team Lead at Danish Verisign affiliate, running a customer-only accessible vulnerability database. NOT JUST SECURITY , THE RIGHT SECURITY
  3. 3. Quick Bio – Vulnerability Research Officially been doing vulnerability research since 2003 • Focused on a static analysis / reverse engineering approach • Jokingly refer to myself as a "vulnerability connoisseur" - I enjoy analyzing vulnerabilities and their root causes. • Critical vulnerabilities discovered in products from many major software vendors. NOT JUST SECURITY , THE RIGHT SECURITY
  4. 4. INTRODUCTION What will be discussed?
  5. 5. Reason for Talk After 10+ years of VDB work, I felt it was time to reflect on certain areas related to vulnerabilities NOT JUST SECURITY , THE RIGHT SECURITY
  6. 6. Considerations NOT JUST SECURITY , THE RIGHT SECURITY
  7. 7. Metrics and their Usage NOT JUST SECURITY , THE RIGHT SECURITY
  8. 8. Code Quality NOT JUST SECURITY , THE RIGHT SECURITY
  9. 9. Advisory Quality VENDORS MAKE BAD DECISIONS NOT JUST SECURITY , THE RIGHT SECURITY
  10. 10. Vulnerability Handling / Bug Bounties NOT JUST SECURITY , THE RIGHT SECURITY
  11. 11. Million Dollar (or Leu) Question NOT JUST SECURITY , THE RIGHT SECURITY
  12. 12. Quick Show of Hands NOT JUST SECURITY , THE RIGHT SECURITY
  13. 13. Vulnerability A Quick Overview To Set The Stage Statistics
  14. 14. Currently Oldest Recorded Vulnerabilities Vulnerabilities have been around for a very long time - And will continue to be... • Oldest entries in OSVDB are 79399 and 79400 • Marconi wireless telegraph • Dated November 1902 • Message spoofing and message disclosure NOT JUST SECURITY , THE RIGHT SECURITY
  15. 15. Guglielmo Marconi http://www.newscientist.com/article/mg21228440.700-dotdashdiss-the-gentleman-hackers-1903-lulz.html NOT JUST SECURITY , THE RIGHT SECURITY
  16. 16. First Ever Unbreakable Claim! http://www.newscientist.com/article/mg21228440.700-dotdashdiss-the-gentleman-hackers-1903-lulz.html NOT JUST SECURITY , THE RIGHT SECURITY
  17. 17. Nevil Maskelyne Ruins Demo http://www.newscientist.com/article/mg21228440.700-dotdashdiss-the-gentleman-hackers-1903-lulz.html NOT JUST SECURITY , THE RIGHT SECURITY
  18. 18. No Wire-Cutting Please While not providing the privacy and security as promised, the wireless telegraph still had one significant advantage over the wired telegraph: Not possible to cut the wires! NOT JUST SECURITY , THE RIGHT SECURITY
  19. 19. Have We Improved? Obviously, we have progressed a fair bit technically since then, but have we gotten significantly better? NOT JUST SECURITY , THE RIGHT SECURITY
  20. 20. Bringing The Internet Down – Old Lady Style Article: http://news.softpedia.com/news/Old-Lady-Cuts-Off-Internet-in-Armenia-193640.shtml NOT JUST SECURITY , THE RIGHT SECURITY
  21. 21. 10 Year Vulnerability Trend 12000 10000 8000 6000 # Vulns 4000 2000 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 NOT JUST SECURITY , THE RIGHT SECURITY
  22. 22. All Datasets Are Incomplete! All datasets are incomplete - some just more than others Many love taking CVE content that’s free and do random conclusions based on it, but since the dataset is severely lacking, the conclusions are as well NOT JUST SECURITY , THE RIGHT SECURITY
  23. 23. 2006 – 2013 Vulnerability Type Trend NOT JUST SECURITY , THE RIGHT SECURITY
  24. 24. 2012 Data Breaches due to SQL Injection NOT JUST SECURITY , THE RIGHT SECURITY
  25. 25. Companies affected by XSS in 2012 Source: CWN - http://www.cyberwarnews.info/2012/07/04/300000-personal-details-leaked-38-sites-hacked-for-projectdragonfly/ NOT JUST SECURITY , THE RIGHT SECURITY
  26. 26. Companies Impacted By Hacking In 2012 NOT JUST SECURITY , THE RIGHT SECURITY
  27. 27. Vulnerability Metrics Usage
  28. 28. Which is more secure? Product A 10 Vulnerabilities Product B 20 Vulnerabilities NOT JUST SECURITY , THE RIGHT SECURITY
  29. 29. Security State != Number of Vulnerabilities Previously, the security state of a product was considered to be equal to the number of vulnerabilities. Flawed conclusion! Today, people understand that the number of vulnerabilities != security state NOT JUST SECURITY , THE RIGHT SECURITY
  30. 30. Some Apparently Still Don’t Know... “The problem with Java is that a lot of vulnerabilities are constantly being reported in it, and when a lot of vulnerabilities are reported, then there are a lot of hackers using these to access programs built on Java“ - Morten Stengaard, CTO, Secunia http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/ NOT JUST SECURITY , THE RIGHT SECURITY
  31. 31. Dissecting the Statement – Part 1 ”... then there are a lot of hackers using these to access programs built on Java” Most vulnerabilities in Java are not used to target Java applications, but the Java Runtime Environment to compromise the system. http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/ NOT JUST SECURITY , THE RIGHT SECURITY
  32. 32. Dissecting the Statement – Part 2 ”... when a lot of vulnerabilities are reported, then there are a lot of hackers using these…” Just because a lot of vulnerabilities are reported in a product, a lot of hackers may not be exploiting them. http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/ NOT JUST SECURITY , THE RIGHT SECURITY
  33. 33. Dissecting the Statement – Part 3 ”The problem with Java is that a lot of vulnerabilities are constantly being reported in it…” The security state of a product is not defined by the number of vulnerabilities reported in it. http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/ NOT JUST SECURITY , THE RIGHT SECURITY
  34. 34. We Should All Stop Using Popular Software Then 400 350 300 250 Java 200 Chrome Firefox 150 Internet Explorer 100 50 0 Vulnerabilities (2013 - Nov 10th) NOT JUST SECURITY , THE RIGHT SECURITY
  35. 35. Facewall! NOT JUST SECURITY , THE RIGHT SECURITY
  36. 36. Microsoft Argument For SDL (Windows) NOT JUST SECURITY , THE RIGHT SECURITY
  37. 37. Microsoft Argument For SDL (SQL Server) NOT JUST SECURITY , THE RIGHT SECURITY
  38. 38. Microsoft Office Vulnerability Trend Vulnerabilities in Office versions one year after product release (based on Microsoft security bulletins) 14 12 10 8 6 4 2 0 Office 2000 Office 2007 NOT JUST SECURITY , THE RIGHT SECURITY Office 2010
  39. 39. Microsoft Security Bulletin Trend 350 300 250 200 Bulletins 150 CVEs 100 50 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 NOT JUST SECURITY , THE RIGHT SECURITY
  40. 40. Researcher Focus and SCADA NOT JUST SECURITY , THE RIGHT SECURITY
  41. 41. Stop Drawing Conclusions on Vulnerability Counts... NOT JUST SECURITY , THE RIGHT SECURITY
  42. 42. There are so many other aspects to consider! More things to consider incl. Patched vs. Unpatched Vulnerability Type Impact Time-To-Patch Time-To-Vendor-Response Security Mechanisms ... NOT JUST SECURITY , THE RIGHT SECURITY
  43. 43. Vulnerability Metrics Severity
  44. 44. Severity Metrics Many different severity metrics – both public and internal Most popular and hated is CVSS, which currently has problems reflecting real risk Many concerns raised about CVSSv2 by many people e.g. myself and Brian Martin of OSVDB in our open letter: "The CVSSv2 Shortcomings, Faults, and Failures Formulation" http://www.riskbasedsecurity.com/reports/CVSS-ShortcomingsFaultsandFailures.pdf NOT JUST SECURITY , THE RIGHT SECURITY
  45. 45. Limitations of Severity Metrics Reflecting the threat of vulnerability-dependent issues (e.g. sandbox bypass, ASLR bypass related to memory disclosure etc.) By themselves and from a scoring point-of-view, these issues are pretty minor, but when combined with code execution... Jackpot! Ability to disclose a few memory addresses was in the past pretty much a non-issue – today it’s very useful. NOT JUST SECURITY , THE RIGHT SECURITY
  46. 46. Pick A Vuln... Any Vuln... If I’d offer you one vulnerability in e.g. Google Chrome, which would you pick? 1) Code execution within sandbox CVSSv2: 6.8 2) Sandbox bypass CVSSv2: 2.6 NOT JUST SECURITY , THE RIGHT SECURITY
  47. 47. Severity Metrics and Sandbox Bypasses If we conclude that exploiters are more interested in the sandbox bypass and system administrators should focus on fixing such a vulnerability over a code execution vulnerability within the sandbox, why are we not rating them higher? Case of reality not being reflected well by severity metrics NOT JUST SECURITY , THE RIGHT SECURITY
  48. 48. Severity Metrics and Vulnerability Chains And once these issues start occuring in chains, which is becoming more and more common, then it really gets complex... You can have a lot of independent minor issues that when combined suddenly are very serious NOT JUST SECURITY , THE RIGHT SECURITY
  49. 49. Google Chrome Pwn2Own Example CVSSv2: 6.8 CVSSv2: 5.1 OSVDB 89734 IPC channel missing listener process validation OSVDB 80007 Plugin blocking logic not run for NaCl in pre-rendering CVSSv2: 7.6 OSVDB 80293 Unpacked NPAPI extension installation without confirmation CVSSv2: OSVDB 81645 5.1 GPU command decoding integer underflow CVSSv2: 2.6 OSVDB 80741 CVSSv2: 2.6 OSVDB 89736 Too permissive LoadExtension bindings for extension manager Unprivileged renderer can navigate to privileged URLs http://blog.chromium.org/2012/05/tale-of-two-pwnies-part-1.html NOT JUST SECURITY , THE RIGHT SECURITY
  50. 50. When Severity Metrics Met Reality NOT JUST SECURITY , THE RIGHT SECURITY
  51. 51. Vulnerability Metrics Exploitability
  52. 52. Microsoft Severity Ratings Source: http://technet.microsoft.com/en-us/security/gg309177.aspx NOT JUST SECURITY , THE RIGHT SECURITY
  53. 53. Exploitability Index Ratings NOT JUST SECURITY , THE RIGHT SECURITY
  54. 54. Microsoft Approach: Pros and Cons Pros Cons Gives an realistic evaluation of the technical requirements to exploit a given vulnerability and how feasible it is Requires significant technical skills and resources to get right Makes it clear which are theoretical and which are plausible Still requires a bit of guesstimation NOT JUST SECURITY , THE RIGHT SECURITY
  55. 55. No Granularity Really Added... NOT JUST SECURITY , THE RIGHT SECURITY
  56. 56. How Does Adobe Do It? NOT JUST SECURITY , THE RIGHT SECURITY
  57. 57. How Does Adobe Do It? NOT JUST SECURITY , THE RIGHT SECURITY
  58. 58. Adobe Approach: Pros and Cons Pros ...Cons Allows understanding which Does not factor in technical products, versions, and requirements and the nature of architectures are most critical to the vulnerability i.e. does not prioritize differentiate between theoretical issues and straight-forward issues to exploit Dynamic approach that can be easily tweaked Requires very little resources – just an understanding of historical exploitation NOT JUST SECURITY , THE RIGHT SECURITY
  59. 59. How Does CVSSv2 Do It? NOT JUST SECURITY , THE RIGHT SECURITY
  60. 60. CVSSv2 Approach: Pros and Cons Pros Most reliable of all the approaches: If an exploit is available, a vulnerability is clearly exploitable. Requires very little resources – just knowledge of availability of PoCs and exploits Cons Purely reactive, requiring very fast response times Only takes into account when the availability of an exploit is publicly known i.e. may be exploited long before being flagged as such NOT JUST SECURITY , THE RIGHT SECURITY
  61. 61. No information about code quality NOT JUST SECURITY , THE RIGHT SECURITY
  62. 62. Code Quality ... And How To Measure It
  63. 63. Code Quality – Why Measure It? NOT JUST SECURITY , THE RIGHT SECURITY
  64. 64. Code Maturity Metric – The Idea The idea of code maturity is that by evaluating the prevalence of the different vulnerability classes being discovered in a product, we can conclude the maturity of that product. We, naturally, focus on it from a security perspective. NOT JUST SECURITY , THE RIGHT SECURITY
  65. 65. Code Maturity Metric – Scoring • Each vulnerability can be scored based on type, and how easy it is to discover. • Researchers find simple vulnerabilities first - as simple vulnerabilities are eliminated, researchers move on to finding more complex vulnerabilities. • When a vendor secures the code, basic vulnerabilities are easier to spot and remedy or never introduce compared to more complex vulnerabilities. NOT JUST SECURITY , THE RIGHT SECURITY
  66. 66. Code Maturity Metric – Scoring Example Level Vulnerability Classes 0 Classic buffer overflows due to e.g. strcpy, sprintf, sscanf and format string issues. 1 Buffer overflows due to incorrect size being used e.g. strncpy, memcpy and array-indexing issues 2 Arithmetic errors i.e. Integer overflows/underflows, type conversion, signedness. 3 Uninitialized variable, use-afterfree, bad cast, complex logic errors. NOT JUST SECURITY , THE RIGHT SECURITY
  67. 67. Schneider Modbus Serial Driver Buffer Overflow Source: http://www.riskbasedsecurity.com/research/RBS-2013-003.pdf NOT JUST SECURITY , THE RIGHT SECURITY
  68. 68. Schneider Modbus Serial Driver Buffer Overflow NOT JUST SECURITY , THE RIGHT SECURITY
  69. 69. Schneider Modbus Serial Driver Buffer Overflow Code Maturity Level: 1 NOT JUST SECURITY , THE RIGHT SECURITY
  70. 70. Schneider Modbus Serial Driver Buffer Overflow NOT JUST SECURITY , THE RIGHT SECURITY
  71. 71. Schneider Modbus Serial Driver Buffer Overflow NOT JUST SECURITY , THE RIGHT SECURITY
  72. 72. ActiveX Control Vulnerability Code Maturity Level: 3 NOT JUST SECURITY , THE RIGHT SECURITY
  73. 73. Office Vulnerabilities Analysed Office 2000: Office XP: 62 103 Office 2003: 90 Office 2007: 47 Office 2010: 14 NOT JUST SECURITY , THE RIGHT SECURITY
  74. 74. Office Product Code Maturity Scores Office 2010 Office 2007 Office 2003 Code Maturity Office XP Office 2000 0 0.5 1 1.5 2 2.5 NOT JUST SECURITY , THE RIGHT SECURITY 3
  75. 75. Office Vulnerability Type Prevalence Office 2010 Office 2007 Uninitialised Variable Object Type Confusion Use-after-free Office 2003 Arithmetic Array Indexing Incorrect Size Copy Office XP Classic Buffer Overflow Office 2000 0% 5% 10% 15% 20% 25% 30% 35% NOT JUST SECURITY , THE RIGHT SECURITY
  76. 76. Measuring the Efforts Taken By Vendors With this we can put more focus on the code security improvement efforts taken by vendors by being able to measure them. Allows system administrators to know which software to steer clear from... and researcher to understand which types of vulnerabilities they can expect to find in a given product. NOT JUST SECURITY , THE RIGHT SECURITY
  77. 77. Advisory Quality Or Lack Thereof...
  78. 78. Information Needs To Be Publicly Available Most vendors have also acknowledged that publishing vulnerability information is beneficial Juniper recently joined the party Still some black sheep like SAP, trying to keep it a secret… NOT JUST SECURITY , THE RIGHT SECURITY
  79. 79. Needs To Include Vulnerability Type Either clearly descripting the vulnerability type in the advisory description or alternatively including CWEs NOT JUST SECURITY , THE RIGHT SECURITY
  80. 80. Everything Is Memory Corruption These Days NOT JUST SECURITY , THE RIGHT SECURITY
  81. 81. Microsoft MS12-037 vs MS13-080 ---- NOT JUST SECURITY , THE RIGHT SECURITY
  82. 82. Rise In Usage Of Memory Corruption Term NOT JUST SECURITY , THE RIGHT SECURITY
  83. 83. No requirements to include proper info Various standards and formats e.g. CVRF are being proposed, but these deal with required fields – not the content of these. Primary focus is to ensure a structure that is easy to parse in an automated manner. Completely up to the vendors how much information they feel like sharing. Up to customers to raise their voice, if they want/need more. NOT JUST SECURITY , THE RIGHT SECURITY
  84. 84. Vulnerability ... And Bug Bounties Handling
  85. 85. Bug Bounties When I started reporting vulnerabilities to vendors, I was stoked each time I actually got a response - and it wasn't a threat from a lawyer. Had any of you told me back then that vendors today would be offering bug bounties, I'd have smiled and shook my head. NOT JUST SECURITY , THE RIGHT SECURITY
  86. 86. Bug Bounties A few interesting ones are of course Google's bounty, which is one of the more serious vendor bounties, and especially their latest twist: Bounties for other software! Microsoft's bounty for vulnerabilities, but specifically bypassing security mechanisms is very interesting NOT JUST SECURITY , THE RIGHT SECURITY
  87. 87. Shockwave Player Vulnerability Trend 90 80 70 60 50 40 30 20 10 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 NOT JUST SECURITY , THE RIGHT SECURITY 2012 2013
  88. 88. Bug Bounties There has definitely been a shift in how vendors perceive bug bounties. It’s clear to me that if a vendor wants to encourage researchers to look at their code and report it in a coordinated manner, then bug bounties are very effective when done right. NOT JUST SECURITY , THE RIGHT SECURITY
  89. 89. Conclusion Are We There Yet?
  90. 90. Security Software and Shiny Appliances NOT JUST SECURITY , THE RIGHT SECURITY
  91. 91. Everything Is Vulnerable – Even Security Software! About 2.2% of all entries in OSVDB cover vulnerabilities in security software NOT JUST SECURITY , THE RIGHT SECURITY
  92. 92. The Security Software Paradox Reducing attack surface by adding an even greater attack surface is a paradox NOT JUST SECURITY , THE RIGHT SECURITY
  93. 93. Code Quality Improvements(?) Microsoft, Google, and Adobe are examples of vendors noticeably improving their security efforts. Oracle may be on their way after everyone finally realized that Java is a mess... NOT JUST SECURITY , THE RIGHT SECURITY
  94. 94. How Do We Force Vendors To Improve? NOT JUST SECURITY , THE RIGHT SECURITY
  95. 95. Grand Demonstrations! We need that ordinary people can relate to! NOT JUST SECURITY , THE RIGHT SECURITY
  96. 96. FTC vs. TRENDnet After demonstrating how network cameras were easily publicly accessible and e.g. allowing spying on people in their homes, the FTC (Federal Trade Commision) in USA went after TRENDnet. Eventually agreed that TRENDnet was ”prohibited from misrepresenting the security of its cameras”, will establish a comprehensive IS program, and hire outside consulting to review security every two years for 20 years... http://www.ftc.gov/opa/2013/09/trendnet.shtm NOT JUST SECURITY , THE RIGHT SECURITY
  97. 97. Is TRENDnet worse than the rest? This is really something every single software vendor should do – but definitely don’t! Is TRENDnet really that much worse than other embedded device vendors? NOT JUST SECURITY , THE RIGHT SECURITY
  98. 98. TRENDnet Product Vulnerabilities NOT JUST SECURITY , THE RIGHT SECURITY
  99. 99. D-Link Product Vulnerabilities NOT JUST SECURITY , THE RIGHT SECURITY
  100. 100. D-Link User-Agent Backdoor Source: http://www.devttys0.com/2013/10/reverse-engineering-a-d-link-backdoor/ NOT JUST SECURITY , THE RIGHT SECURITY
  101. 101. Is Legislation The Answer? NOT JUST SECURITY , THE RIGHT SECURITY
  102. 102. Software Will Always Have Vulns? Vendors claim that they provide software ”as-is” and have long EULAs to exempt them from liability We seemingly accept that software will always have vulns... ... but the types of vulnerabilities matter as well as how the vendor proactively reduces risk and reactively deals with them. NOT JUST SECURITY , THE RIGHT SECURITY
  103. 103. Conclusion Of all the areas, vulnerability coordination/handling is the biggest improvement and continuing in the right direction. Advisory quality overall seems static with some vendors improving and others devolving. Only a few major vendors really seem to have solid SDLs and can show an improvement in code quality. People are beginning to understand metrics better, and we’re seeing attempts at providing more granularity. NOT JUST SECURITY , THE RIGHT SECURITY
  104. 104. The Good News: There is Room for Improvement NOT JUST SECURITY , THE RIGHT SECURITY
  105. 105. Discussion! NOT JUST SECURITY , THE RIGHT SECURITY

×