2. Intelligence on the Intractable Problem of Insecure SoftwareKey Insights from Analyzing Thousands of Applications and Billions of Lines of Code in the Cloud 2 State of Software Security Report, Volume 2 Key Findings
3. No Mature Discipline Can Escape The Need for A System of Measurement…. 3 .…It should be no different for Application Security!
4. Cloud-based Platform: Application Intelligence Enabler 4 Vulnerability Data Application Data 010011011101001011000111101001011001001001100100010101110101010100101100111 Binary / Bytecode SecurityReview® Application Risk Management Services Platform Upload or Specify URL Scan Data Single repository of ever increasing and improving application data Combined with BI engine (Endeca) to harness meaningful metrics and trends Delivers powerful data analytics and intelligence capability with power to: Elevate application security program from managing scans (operating controls) to managing risk (outcomes) Juxtapose your state of application security to that of peers and industry benchmarks Inform the market on emerging trends across the software supply chain
26. Executive Summary Findings More than half of all software failed to meet an acceptable level of security and 8 out of 10 web applications failed to comply with OWASP Top 10. Cross-site Scripting remains the most prevalent of all vulnerabilities. Third-party applications found to have lowest security quality. Developers repaired security vulnerabilities quickly. Suppliers of Cloud/Web applications were the most requested third-party assessments. No single method of application security testing is adequate by itself. Security quality of applications from Banks, Insurance, and Financial Services industries not commensurate with business criticality. 8
27. Question One What percentage of applications tested failed on the first assessment? What percentage of web applications do not comply with the OWASP Top 10? 9
28. 10 More than Half of Software Failed, 8 out of 10 Web Apps Do Not Comply with OWASP Top 10
29. Question Two What is the most prevalent vulnerability category across all tested applications (raw flaw count)? What is the most prevalent vulnerability category when across all applications (percentage of applications effected)? 11
30. Question Two What is the most prevalent vulnerability category across all tested applications (raw flaw count)? What is the most prevalent vulnerability category when across all applications (percentage of applications effected)? 12
32. Question Three Which of the following has the highest first assessment failure rate: Outsourced, internally developed, open source, commercial software What percentage of “Internally Developed” code submitted for analysis was actually identified as “third party” code? 14
34. Question Four Which of the following groups remediates flaws the fastest: Open source, internally developed, or commercial software How many days do you think it takes? Do they get it right the first time or does it take multiple tries? 16
36. Question Five Third party applications being analyzed at the request of a purchasing organization failed ____ percent of the time. Which of the following types of applications were subjected to third party analysis by a purchasing organization the most frequently: Cloud , ISV, Deployed (Cots), Integration, Consulting 18
37. Suppliers of Cloud/Web Apps Most Frequently Subjected to Third-party Risk Assessments 19
38. Question Six What is the most effective method of testing the security of an application: Static assessment, dynamic assessment, manual assessment? 20
40. No single method of application security testing is Adequate by Itself 22
41. Question Seven Financial sub-segments: Insurance, Financial Services, Banks Which of the above has the best raw scores (unadjusted for business criticality) When adjusted for business criticality, does that same segment maintain its strength? Why? 23
42. Security Quality Not Commensurate with Business Criticality for Financial Industry Applications 24
43. Executive Summary Findings More than half of all software failed to meet an acceptable level of security and 8 out of 10 web applications failed to comply with OWASP Top 10. Cross-site Scripting remains the most prevalent of all vulnerabilities. Third-party applications found to have lowest security quality. Developers repaired security vulnerabilities quickly. Suppliers of Cloud/Web applications were the most requested third-party assessments. No single method of application security testing is adequate by itself. Security quality of applications from Banks, Insurance, and Financial Services industries not commensurate with business criticality. 25
44.
45. Static analysis is being performed in addition to dynamic analysis on web applications.
46. First mobile app risks appearing in the wild. Both vulnerabilities such as the PDF iOS 4 vulnerability used by jailbreakme.com and mobile apps with trojan functionality.
50. Future Direction for Application Intelligence Service How would you like to see this service/report evolve? What other metrics and trends should we be reporting on? Are there other sources of data (public or private) that we should be integrating with? Who else would be interested in this data i.e. BD/partnership opportunities for application intelligence? http://www.veracode.com/reports/index.html 28 Veracode Confidential
IntroNamePrevious employers and workCurrent responsibilitiesWhat we will be discussing today
Economics = mature ; measured by moneyMedicine = mature ; measured by numerous stats on health and life longevityApp sec not matureInfo sec as a whole also not yet matureWe don’t measure the right things at the right times in order to make the best possible decisionsMetrics presented here are just a start
Background on VeracodeCloud serviceBinary or URL uploadedStatic Analysis, Dynamic Analysis, Manual Assessment occursPresentation of findings in management services platformResults in list of application flawsInteresting becauseCentral location with LOTS of appsContinuous updating of attack models and vulnerable code structuresData analyzed by business intel engine (Endeca) to give metrics and trendsBased on flaw dataBased on IndustryBased on code languageBased on location in supply chainKeeps the market informedHelps to make informed decisions based on real riskNo longer scan and forget model
The data set we’re working with2,922 Applications Scanned in our datasetThree major sectionsVulnerability Data--The specific results of the analysis-Scan Data--Trending data--Pass fail rates-Application Data--Contains meta data about the application itselfAnalytics on this data produces the SoSS report
How was the data broken outBulk of the data was internally developed (71%)Commercial code second (22%)If we slice the data on language we see that about half of the tested code was Java and nearly 80% was Java+.Net20% was C/C++ code. Cold Fusion and PHP made up the difference.We also see that 56% of the applications analyzed were web based applications.Also of interest is that 2/3 of the dataset came from either the software industry or from finance
Veracode’s risk adjusted score – Based on standards like CWE, CVSS and NIST, it has a sliding scale requiring higher criticality applications to have a higher quality of security. This presents a more pragmatic approach to resource allocation.When an application is submitted to our engine the submitter designates an assurance level (AL 1-5)Based on the severity and number of flaws discovered, and some fancy logarithmic math, we determine a pass fail level.Passing might be 90% at AL1 and only 50% at AL5. The AL is set based on the risk level of the application.Is the application highly sensitive, containing private user data, or is it a basic system that simply crunches public data?OWASP is an industry standard list of critical web application errors. Does not take into account the risk level of the application.If you have one flaw in an OWASP flaw class, you do not comply.
57% of all applications were found to have unacceptable application security quality on first submission, evenwhen standards were adjusted for applications considered less business criticalMore than 80% of internally developed and commercial web applications failed to comply with the OWASP Top 10The level of risk in terms of repair costs, business continuity, and brand from so many business critical applicationsfailing to meet an acceptable level of security on first submission is staggering. The potential exposure to brand reputation and loss of revenue from interruptions to business operations is significant.Recommendation: Utilize industry standards such as OWASP Top 10 and CWE/SANS Top 25 list of most dangeroussoftware errors as minimum thresholds and compliance policies to which applications need to adhere.
This one is harder since I’m not giving you the categories to pick from…
OK Fine.. I’ll give you a hint
Cross-site Scripting (XSS) remains the most prevalent vulnerability category, accounting for 51% of all vulnerabilitiesuncovered by Veracode’s combined static binary, dynamic, and manual security testing methods.NET applications, in particular, exhibited an abnormally high rate of Cross-site Scripting vulnerabilities, resulting fromthe use of .NET controls that do not automatically encode output (Table 4). While not as numerous, Cryptographic Issues—a category that includes unencrypted or inadequate encryption of data—appeared in the most applications, with 41% of all applications containing one or more vulnerabilities in this category (Figure 14). These statistics underscore the need for developers to become better educated and better equipped to avoid common vulnerabilities.Recommendation: These flaws are easy to fix once found. Focusing on developer education andawareness is a cost-effective way to avoid introducing them.
Third party code getting lots of attentionBetween 30% and 70% of software submitted as internally developed contained identifiable third-party components. Safecode.org and research firm Secunia2 have recently released data on the elevated risks associated with third-party software in the supply chain. Veracode shows that applications from all types of third-party suppliers less secure than Internally Developed applications on first submission. Third-party suppliers failed to achieve acceptable levels of security 81% of the time. Third-party code is an essential part of every organization’s portfolio, comprising 29% of all applicationssubmitted to Veracode. Furthermore, between 20% and 37% of very high or high criticality applications aresourced from third-parties.Recommendation: Internal and 3rd party components must be subjected to the same levelof security verificationEnsures consistent security quality across the application portfolio. Procurement contracts for outsourced or commercial software vendors should insist upon the authority to perform independent securitytesting and specify minimum security acceptance criteria.
A common misperception is that it is easy to find defects and difficult to fix them. Often true to functional, not for security flawsNoticing a bug can be easyFinding the cause of a security flaw can be long and difficult (although sometimes easy to fix)Encouraging data in this reportAverage of 16 days and 1.1 resubmissions for Dev teams using Veracode platformStrong reason to equip development teams with effective security testing and trainingWhen properly informed and tooled devs can and will improveRecommendation: Equip development teams with the appropriate application security resources and knowledgeand plan for security verification and remediation in the project timeline from the outset.
3rd party scans sourced by purchasing groups grown linearly over 6qtrsShows increased concern over the security of software in the supply chain Also shows uptick of cloud based non source code required static analysisTypes of applications reviewed by request (3rd party)Suppliers of cloud and web applications made up nearly 60% of all 3rd party assessments requestedIntegrators and commercial software providers made up most of the rest in equal parts. Reasonable security concerns cloud services raise and the criticality of the work they perform.Like other 3rd party software, these assessments resulted in low levels of acceptable security and rapid remediation.Recommendation: Require Third-party Cloud/Web application and service providers to demonstrate verificationof application security quality.
This year web scanning has been demonstrated as inadequateCode-level analysis of vulnerabilities confirms that dynamic web application scanning tools are not sufficient as the sole testing method. Manual penetration testing, lacks consistency of coverage and will rarely detect all instances of commonly occurring vulnerabilities. Evidence shows that static binary analysis provides the most consistent breadth and depth of coverageIt is also true that not all design and business logic vulnerabilities are discoverable with static methods alone.Recommendation: Different testing techniques should be viewed as operating controls where each playsan important role in a comprehensive program. Multiple testing techniques should be adopted based on application business criticality and type of application. The use of multiple techniques is the only way to comply with industry standard security polices such as the OWASP Top 10 and the CWE/SANS Top 25 Most Dangerous Software Errors.
Financial Industry applications - best raw code-level security scores of any industry Financial Industry applications - average levels of acceptability when business criticality was consideredDemonstrates a high awareness of code-level threatsDemonstrations inadequate application risk management practicesFinancial Services applications in particular demonstrated an low prevalence of the most common vulnerabilities- less than half the rate of Cross-site Scripting errors as compared to Banks and InsuranceTraining, testing, and a high degree of focus on specific types of errors can make a significant difference. Improvement is possibleThe most critical of applications remain too insecure.Recommendation: Inventory and classify the application inventory based on business criticality. In the absenceof this business context, an understanding of the code-level security quality is insufficient. What seems to begood code-level security quality may still not render the application fit for purpose when business criticality istaken into account.
Answer sheet time!In general software is still largely insecure.Depending on how measured – 50% to 80% failure rate!XSS is still out there and is still a painRegardless of libraries available to alleviate the problem3rd party code is generally low scoringDon’t trust libraries, outsourced code, etcGood news! Developers can fix their own security flaws, and can do it quickly!People are embracing cloudAnd seriously considering the security implications of moving to the cloudManual, dynamic, static assessment are ineffective in a vaccuumDefense in depth for application means do them all (if risk level requires)Conduct proper risk assessmentsDetermine the level of security required for your app and meet it
A few interesting trends and conclusions that we didn’t mentionSQLi and XSS decreasing indicates that the dev teams gets sec codingPeople are starting to realize that one assessment does not indicate securityLayer the assessment approach to match application risk levelIn the past year we saw significant uptick in mobile application riskBoth at the operating system as well as at the application levelEvidenced by PDF attack to iOS4 and significant increase in mobile spywareBackdoors were discovered at an alarming rate in the last yearSpecifically the ones in Seimens SCADA product being actively exploitedCloud is finally getting reviewedBig uptick in assessments of vendors cloud servicesOlder platforms are cleaning up their actLeaving new emerging areas to find fresh flaws Mobile/Cloud
So what does this mean to you guys.. As CIOs, CISOs, VPs, etc of your organizations…As you look through these questions things what changes you would have made in the last year if you knew these statistics
We plan to continue to release this report semi annuallyFeel free to comment now or email us with recommendations on how you would like to see the report improve
