State of Software Security - European Update
•Download as PPTX, PDF•
0 likes•68 views
At Black Hat EU (2018) we presented on the latest SoSS report (vol9). This year we've used survival analysis of flaws to understand their lifecycle within organisations. The work confirms that certain industry sectors have much to do make faster progress. Specific countries are leading the way in resolving security defects. Those teams that are implementing automation of scanning do the best in achieving outcomes.
Recommended
Recommended
More Related Content
Similar to State of Software Security - European Update
Similar to State of Software Security - European Update (20)
Recently uploaded
Recently uploaded (20)
State of Software Security - European Update
- 1. © 2018 VERACODE INC..© 2018 VERACODE INC.. State of Software Security: European Update Paul Farrington Veracode, Director Solutions Architecture, EMEA & APJ
- 2. © 2018 VERACODE INC..
- 4. © 2018 VERACODE INC..
- 5. © 2018 VERACODE INC..
- 6. © 2018 VERACODE INC.. Language Flaw Persistence Analysis
- 7. © 2018 VERACODE INC.. 7
- 8. © 2018 VERACODE INC.. Fix Velocity
- 9. © 2018 VERACODE INC.. Flaw Persistence Analysis
- 10. © 2018 VERACODE INC.. Severity Flaw Persistence Analysis
- 11. 1 1 Geography
- 12. © 2018 VERACODE INC.. Flaw Persistence Interval by Region
- 13. © 2018 VERACODE INC.. Flaw Persistence Interval by Country
- 14. 1 4 Superheroes
- 16. © 2018 VERACODE INC.. Flaw Persistence Intervals by Industry
- 17. © 2018 VERACODE INC.. Flaw Persistence Intervals by Industry
- 18. © 2018 VERACODE INC.. OWASP Pass Rates Comparison by Industry
- 19. © 2018 VERACODE INC. A BUSINESS UNIT OF CA TECHNOLOGIES.© 2018 VERACODE INC.. 1 Observer Effect
- 20. © 2018 VERACODE INC.. Scan Frequency & Fix Rates
- 21. © 2018 VERACODE INC.. Flaw Persistence
- 22. © 2018 VERACODE INC.. A Big Difference 50% - 20 Days 50% - 175 Days
- 23. © 2018 VERACODE INC.. Wrap-Up •Initial scans passing OWASP Top 10 down •Developers are closed 12% more flaws •Modern language flaw persistence is less than in older languages •Retail and Healthcare are leading the way in clearing-up flaws •There's evidence that a growing minority of developers have mature DevSecOps practices •Regular scan feedback loops correlate with good behaviour / outcomes
- 24. 2 4 Go 365 (or at least 52)
Editor's Notes
- In the realm of Cybersecurity, on a daily basis there’s an abundance of bad news. For example we see open source javascript libraries being poisoned and then included at dependencies in other Node.js projects – so that a user’s crypto currency (Bitpay’s Copay) wallet can be compromised. https://www.zdnet.com/article/hacker-backdoors-popular-javascript-library-to-steal-bitcoin-funds/ (Flatmap-Stream library) But whilst the environment is always changing a new threats appear, this domain is realty no different to say medicine. If you were considering joining the medical profession - you might ask do I like the sight of blood? Do I like people? What's my bedside manner like ? - it's arguably unlikely that you would think that being a doctor was not for you as there are too many diseases in the world, or some people choose to be unhealthy. Penicillin was discovered in 1928 by Fleming. Within 12 years antibiotic resistance was detected. Now for newer antibiotics it can take just months for resistance to be observed. https://www.cdc.gov/drugresistance/about.html I think the analogy holds true too for those considering a career in secure development - this work will never be finished, just like there will always be disease, but the need for the work that we do in helping development teams – will exist for a long time into the future. [Click]
- Veracode has been producing a State of Software Security Report since 2010. Recently we published vol 9 of this report. You are going to learn about areas that remain challenges. BlackHat is nothing, if not a place where people are realistic about the world. One of the things I'd like you hear from this talk is that there are reasons why we can be optimistic about progress and trends in application security.
- Developers are closing doors on more vulnerabilities than ever before. Across a 12 month period just shy of 70% of flaws end-up being closed, I.e. fixed by developers. That's up 12% compared to last year's vol 8 of the report. This year we've spent a lot of time analysing the lifespan of flaws - why because that speaks to outcomes, which ultimately is something we all care about.
- Against the OWASP Top 10 applications are passing policy 22.5% of the time in the first scan (blue here). Applications fare better against the SANS Top 25. In both cases we see improvement in later scans.
- In previous years, a higher percentage of applications passed policy on the first scan. We have to go back to 2013 to observe a lower pass rate.
- Python seems to show the most marked decrease in open flaws from first discovery. Other languages such as Android and JavaScript do reasonably well too. All sorts of inferences you can take from that, maybe there is a linkage with the language and the mindset of the developer - the data doesn't confirm that hypothesis here.
- In analyzing the data, we found that the most common types of vulnerabilities cropped up in largely the same proportions as last year. The top four vulnerability categories presented themselves in more than half of all tested applications. [click] We discovered highly exploitable cross-site scripting flaws in nearly 49% of applications, and SQL injection appeared nearly as much as ever, showing up in almost 28% of tested software. These rates are stubbornly consistent compared to previous years.
- Flaws tend to be fixed fastest in the first few days of discovery. Velocity slows dramatically after the first month. We see that teams start to run out of steam for some reason - yet we still see higher severity flaws being left open for long periods of time.
- Let's flip the graph and focus now on the open flaws. By doing so, we can consider the radioactivity of those flaws. At 21 days, 75% of flaws remain. The average half life is 121 days 25 % of flaws remain after 472 days. Let's look at this expressed again the other way round - focusing on closures. We'll use this graphic again to look at other populations of flaws.
- As one would expect High and Very High Severity Flaws are addressed at a faster rate compared to Low to Informational Level Flaws - but look here at the half life of the red and blue lines - only around 28 days difference, and in both cases pretty high. So of course developers are more motivated to close higher severity flaws faster, but the decay curves are really very similar. If I showed you a graph of application criticality, we'd see a similar picture. Put another, just because something is viewed as important by some, doesn't mean it will get fixed quickly.
- Lets look at how where companies are based affects speed to fix
- Unsurprisingly, vulnerabilities addressed by organizations in the Americas mostly tracked to the overall average. This was inevitable due to the fact that the large volume of these vulnerabilities weighted the average. However, one thing to note is that companies in the Americas did outperform the average on the tail-end of the vulnerability burndown process. This indicates how badly companies in APAC and EMEA trailed when it came to getting to their last quartile of open vulnerabilities. In examining the APAC companies’ speed of closure, it is interesting to find that these firms jumped on their first chunk of flaws very quickly. It only took APAC companies about a week to close out 25% of their flaws. However, the spread between reaching that first milestone and eventually resolving 75% of flaws was enormous. It took APAC companies well over two years to start working on their last quartile of open vulnerabilities. Meanwhile, EMEA companies lagged behind the average significantly at every milepost of the flaw persistence intervals. It took more than double the average time for EMEA organizations to close out three-quarters of their open vulnerabilities. 25% of vulnerabilities persisted more than two- and-a-half years after discovery.
- That sense of urgency shown by companies in Australia, Germany and Switzerland is in stark contrast with the average. More than three years to reach their final quartile of open vulnerabilities, and it took Australian organizations nearly four years to reach the same milepost.
- In particular, the rapid rate of remediation evidenced by Dutch companies remain a promising bright spot amid the worrying time it took their EMEA counterparts to fix the same percentage of flaws. Dutch firms managed to start working on their last quartile of open flaws within five months of discovery — that is the fastest rate worldwide and three times as fast as the average application. The UK actually reach their first quartile of flaws fixed the fastest but lose momentum after reaching the half way point. Still overall they out pace India which comes in 3rd place in the league of Superhero nations. Again India reached closure in the first quartile very quickly, perhaps at a cost of delayed closure down the line.
- Healthcare organizations are remediating at the most rapid rate at every interval compared to their peers. It takes just a little over seven months for healthcare organizations to reach the final quartile of open vulnerabilities, about eight months sooner than it takes the average organization to reach the same landmark. Similarly, retail and technology firms outpace the average speed of fix at every interval. Retail beats all sectors in getting the to final quartile of open flaws. While infrastructure firms address the first half of their open flaws more rapidly than average, it takes them significantly more time to get to the second half. At least one in four vulnerabilities are left open almost three years after first discovery within infrastructure industry apps. This likely reflects the great difficulty that these firms face in fixing many applications within critical systems that have extremely tight thresholds for uptime and availability. In a mirror to infrastructure situations, government and education firms have a reverse situation. They’re right about on par with the average time to address the first half of their open flaws, but they start to pick up speed once they get over that hump. This could be an indication of bureaucratic inertia that may impede initial progress, but which is likely overcome once security teams and developers cut through the red tape.
- And here we can the decay curves for these sectors Healthcare, Retail and Infrastructure all get out of the gate quickly - but infrastructure tends to run out of steam before the 50% point and the velocity of closure slows markedly.
- The passing rates of applications across industry is pretty heterogenous. Look how poorly Technology performs on the 1st scan. Whilst Healthcare doesn't have the best 1st scan pass rate, we see the greatest gap between 1st scan and latest scan across applications. This is what you want to see. Ultimately being the most secure on 1st scan is not really terribly interesting metric. Get scanning and get fixing as quickly as you is really the take home here.
- If you are a Quantum Physicist - you are of course aware that if you try to observe the position and momentum of a particle - let's say an electron then you change the way it behaves. If you're an engineer at CERN, that's a really big deal, because you have to devise experiments that limit the Observer Effect to prove theories. In AppSec - of course scanning software doesn't necessarily change the underlying nature of a security flaw - the risk remains unchanged. We don't change the behaviour of that flaw just by observing it. But there is an interesting phenomenon that we can report on , if you keep observing a flaw in the presence of a developer - things do change. The more you observe, the more it changes.
- This is the first time that we've created this view of application scanning across a 12 month period. The techniques we developed in our analysis for this SoSS report - that allow us to track the life cycle of a flaw in the same way we might say a disease, mean that we observe a potential catalytic effect of scanning This is a beautiful slide because it opens our minds to the idea that automating scans may well facilitate dramatically better outcomes for development teams. If you put security into your pipelines - and force telemetry to be recorded and to be observed we see much faster progress Too many discussions centre around whether a security defect should be allowed to 'break the build'. That will undoubtedly be a question on some people's minds in this room right now. As a pragmatist, I say right now who cares? As a minimum we should open a ticket when a policy violating flaw is found (and that is then prioritised with parity to a P1/P2 defect.
- This highlights the art of the possible. We are looking now at how long flaws hang around for. Look at the clean-up rate of upto 12 scans per year compared to unto 3 times per year. A 3.5x difference. Look at the Unicorns at upto 300 scans per annum - in comparison to the unto 3 scan distribution. There is a monumental delta here. This data may not be causational. We can't say that an app that infrequently changes would benefit from a higher scan incidence. Equally we couldn't say that an app that is changing frequently, would not benefit from frequent scanning. The correctional is strong - and that's message. You have to discern whether higher intensity scanning would propel your organisation's flaw closure velocity
- Let's look not at the extremes, instead those completing upto 12 scans per year, and those performing 50+ scans per year. The difference in the flaw half life is striking. 20 days to 50+ scan per year. 175 days half life for unto 12 scans per year. 8.75x difference.
- Consider going 365 daily scanning, or at least 52 weekly scans per year+ Of course, you get to choose. In many cases this is likely to have a positive effect over the medium term outlook. Select a vendor that doesn't penalise higher intensity scanning or automation. We look forward to tracking this in Volume 10 of the SoSS. I hope you'll agree, like we implied at the start, we do indeed have reasons to be optimistic. Like our doctor analogy earlier, there will be new challenges and not all therapies will work. But unless you get unlucky, life chances appear to be better when we make the Observer Effect work for us. I hope this applies to your work too.