• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
VeraCode State of software security report volume5 2013
 

VeraCode State of software security report volume5 2013

on

  • 587 views

VeraCode State of software security report volume5 2013

VeraCode State of software security report volume5 2013

Statistics

Views

Total Views
587
Views on SlideShare
587
Embed Views
0

Actions

Likes
0
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    VeraCode State of software security report volume5 2013 VeraCode State of software security report volume5 2013 Document Transcript

    • VOLUME 5 State of Software Security Report The Intractable Problem of Insecure Software APRIL 2013Read Our Predictionsfor 2013 and Beyond
    • Dear SoSS Report Reader,As some of you may know I have spent most of my 25 year career in the IT Security industry, more specifically, I’ve beenfocused on application security as the use of web and mobile applications has flourished. For the past five years I have beenan active participant in the preparation of the report before you today—our annual State of Software Security Report, or aswe fondly refer to it at Veracode, the SoSS Report.Throughout my career I have been evangelizing the need for more secure application development practices, and with therelease of each new SoSS report I find myself of two minds. The optimist in me is proud of the vast improvement in generalawareness of the importance of securing the application layer. But the pessimist remains very concerned that we are notseeing the dramatic decreases in exploitable coding flaws that I expect to see with each passing year. It’s as if for eachcustomer, development team, or application that has become more secure, there are an equal number or more that do not.While the benefits of web applications are clear to organizations, the risks to their brands, infrastructure, and their dataare seemingly not as clear, despite being more apparent than ever. It’s at this point of my letter that I could mention that acyber-Vesuvius is about to bubble over and create a cyber-Pompeii as there are so many breaches reported; but I’ll resist thattemptation. Instead, here are a few links to recently released reports that do a shockingly good job of telling the scary story:• 2013 Trustwave Global Security Report 1• 2012 Verizon Data Breach Investigations Report 2I only cite these examples because the reports illustrate the “after” scenario, evaluating what has happened when vulnera-ble systems are exposed to the threat space. We at Veracode see the SoSS report as different, using data to shine light onwhat is to come by understanding the latent vulnerabilities in software organizations are deploying. The “before” scenariomeans our SoSS reports have become great predictors about future data breaches. For example, this report shows 32%of applications analyzed by Veracode contain SQL injection flaws. Knowing that, you should not be surprised that Trustwavereported that SQL injection was the attack method for 26% of all reported breaches in 2012. I can tell you with confidencethat malicious actors target the flaws that are easy to find and exploit—like SQL injection—therefore the instances of SQLinjection attacks will surely increase in 2013. Put more bluntly, we must figure out a way to code more securely simply tokeep up with attacks from the most basic attacker.As you read this report I urge you to consider your organization’s application portfolio and how you currently make decisionsabout the risks your organization is willing to take. The amount of risk an organization takes should be a strategic businessdecision—not the aftermath of a particular development project. If you’re learning about risks after a breach—be it yours or anindustry counterpart—then the time to act is now. Use this SoSS report to estimate your current application risk landscape—particularly on applications that you have never tested or only tested manually. Consider how you can act now to improvethe security posture of your organization, by addressing the applications that you currently have in development and/or inproduction. Hopefully by the time we release SoSS V6 in 2014, we’ll see that dramatic improvement I’ve been waiting for!I hope you enjoy the report.Chris WysopalCo-Founder, CISO and CTO, Veracode1 www2.trustwave.com/rs/trustwave/images/2013-Global-Security-Report.pdf2 www.verizonenterprise.com/resources/reports/rp_data-breach-investigations-report-2012_en_xg.pdf
    • Veracode State of Software Security Report: Volume 5Table of ContentsIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Key Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Security of Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Compliance with Standard Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Remediation Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Security Quality Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Language Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.NET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16PHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18ColdFusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Applications Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Mobile Threat Landscape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22State of Mobile Application Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Web Application Threat Landscape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26State of Web Application Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Non-Web Applications Threat Landscape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31State of Non-Web Application Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Appendix A: About the Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Appendix B: Understanding How the Veracode Platform Determines Policy Compliance . . . . . . . . . . . . . . . . 37Whisker Plot Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38P-Value Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Generalized Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 1
    • Veracode State of Software Security Report: Volume 5IntroductionFor the past five years, the Veracode State of Software Security (SoSS) report hasexamined trends associated with vulnerabilities in applications. Our initial goal wasto provide key insights to those charged with managing enterprise application securityrisk, to give them a series of benchmarks from which they could measure their ownapplication security posture. After five years and five versions of SoSS our goal nowis to highlight the slow progress in securing the application layer. Since insecureapplications are a leading cause of security breaches and data loss for organizationsof all types and sizes, we can’t continue to whistle past the graveyard. We want thereaders of this report to leverage the data to build a business case for an applicationsecurity program at their organization.As with past SoSS reports, this analysis draws on continuously updated information in Veracode’s cloud-based appli-cation security platform. Unlike a survey, the data comes from actual application security assessments conducted toidentify vulnerabilities and validate remediations. SoSS Volume 5 examines data collected over an 18 month periodfrom January 2011 through June 2012 from 22,430 application builds uploaded and assessed by our platform(compared to 9,910 application builds analyzed in Volume 4, which was published in December 2011).This report examines application security quality, remediation, and policy compliance statistics and trends. The dataanalyzed represents multiple security testing methodologies (including static binary, dynamic and manual) on a widerange of application types (web, mobile and non-web) and programming languages (including Java, C/C++, .NET,PHP and ColdFusion). We also expanded our analysis of the mobile vulnerability landscape with sections on Android,iOS and Blackberry applications. The resulting intelligence is unique in the breadth and depth it offers.Readers of Volume 5 will notice an increased focus on vulnerability distribution trends for each language. Also, newto this Volume is an analysis of the percentage improvement in the vulnerability distribution between the first andsecond application builds. This metric should provide some perspective on which vulnerabilities organizations choseto fix upon receiving the results from their first submission. These new visualizations and analysis are in responseto customer questions about demonstrating the impact of security programs on enterprise risk profiles.Veracode’s data analytics team is always looking for new perspectives on security metrics. We welcome readerquestions, comments and ideas so that we can continually improve and enrich the coverage, quality and detailof our analysis.2
    • Veracode State of Software Security Report: Volume 5Executive SummaryThe following are some significant findings in the Veracode State of Software SecurityReport Volume 5. Each finding is accompanied by a prediction for the next 12 to 18months, where we sketch out the possible futures if the status quo continues. Wealso provide recommendations for altering our predicted trajectory, because we canchange the future.Key Findings70% of applications failed to comply with enterprise security policies on first submission.This represents a significant increase in the failure rate of 60% reported in Volume 4. While the applications mayeventually become compliant, the high initial failure rate validates the concerns CISOs have regarding applicationsecurity risks since insecure applications are a leading cause of security breaches and data loss for organizationsof all types and sizes.Prediction: Average CISO Tenure Continues to Decline. The average tenure of a CISO is 18 months, and more CISOjobs will be at risk given the current state of software security. The expansive threat profile associated with softwaremeans the likelihood of CISOs being negatively affected by a high-impact security event has never been greater.Recommendation: Driving up compliance with enterprise application security policies lowers the risk of high-impactevents. To accomplish this, CISOs and security professionals must work closely with their counterparts in Develop-ment and Procurement to set security policies and enable internal and external developers to consistently complywith those policies.SQL injection prevalence has plateaued, affecting approximately 32% of web applications.The downward trend in SQL injection that we reported in Volumes 3 and 4 has flattened. For six consecutive quarters,from the first quarter of 2011 to the second quarter of 2012, the percentage of applications affected by SQL injectionhas hovered around 32%. This should be a concern, as three of the biggest SQL injection attacks in 2012 resulted inmillions of email addresses, user names, and passwords being exposed and damaged the respective brands.Prediction: The Rise of the Everyday Hacker. Once the sole domain of technical experts, now a simple searchfor “SQL Injection Tutorial” enables anyone to exploit a serious vulnerability and wreak havoc. The data showsthat everyday hackers are on the rise, as Trustwave reported SQL injection to be the attack method for 26% of allreported breaches in 2012. We predict that number to exceed 30% in 2013. SQL injection vulnerabilities are justtoo easy to find and exploit.Recommendation: Organizations should institute zero-tolerance policies for SQL injection vulnerabilities and employroutine monitoring to detect vulnerabilities as new applications are deployed. 3
    • Veracode State of Software Security Report: Volume 5Eradicating SQL injection in web applications remains a challenge as organizations make tradeoffs aroundwhat to remediate first.The percentage of applications affected by SQL injection has hovered around 32% and cross-site scripting around67% for the last six quarters. For the first time we are reporting improvement percentages by language to illustratewhich flaws organizations are choosing to fix after receiving results from their first submission. Java, representing56% of web applications, showed 16% improvement in SQL injection and 14% improvement in cross-site scriptingbetween the first and second submission. .NET, representing 28% of web applications, showed a 25% improvementin SQL injection and 15% improvement in cross-site scripting.Prediction: Decreased Job Satisfaction/Higher Turn-over for Security Professionals. The challenge is daunting.Companies face a seemingly ever-expanding threat profile brought on by new applications and application updatescontaining easy to exploit flaws such as SQL injection (26% of all 2012 reported breaches according to Trustwave).This can create a very frustrating work environment for security pros. The desire to find roles where their effortswill bear more fruit and where success is apparent will drive increased turnover among security pros. There is somegood news, however. According to the Bureau of Labor Statistics,3 the employment segment that includes informa-tion security analysts is projected to grow 22% between 2010 and 2020, faster than the average for all occupations.Recommendation: Making a difference as a security professional often means building relationships with developmentexecutives. Instead of taking a “scan and scold” approach, the program goal should be improving overall developer pro-ductivity by efficiently integrating security remediation into existing development methodologies. Getting developmentexecutives focused on process integration, knowledge transfer, remediation support and incentives for secure codecreation as key success criteria would represent a significant breakthrough in the relationship.Cryptographic issues affect a sizeable portion of Android (64%) and iOS (58%) applications.Using cryptographic mechanisms incorrectly can make it easier for attackers to compromise the application. Forexample, cryptographic keys can be used to protect transmitted or stored data. However, practices such as hard-cod-ing a cryptographic key directly into a mobile application can be problematic. Should these keys be compromised, anysecurity mechanisms that depend on the privacy of the keys are rendered ineffective.Prediction: Default Encryption, Not “Opt-in,” Will Become the Norm. Eavesdropping on mobile communicationscan make it easier for attackers to design successful social engineering attacks against key employees. There is astaggering amount of transmitted data at risk, considering the growth of open (i.e. easy to eavesdrop) Wi-Fi networksin combination with the number of social network users (Facebook 1.2B; Twitter 190M tweets/day) and the numberof mobile devices (Cisco4 predicts that by the end of 2013, the number of mobile devices will exceed the numberof people on earth—7.1B). These concerns have prompted companies like Twitter and Facebook to encrypt all trafficby default, despite the additional computing power required to encrypt every connection. As more business isconducted through applications resident on personal mobile devices, we expect enterprises to insist on mobileapplications that force encryption to protect data in motion.Recommendation: Developers and security professionals should expect data encryption to be involved in all aspectsof designing the business user’s experience with mobile applications. From a data in motion perspective, this wouldinclude understanding the performance impact and incremental infrastructure costs of encrypting traffic between themobile application and the server side application. From a data at rest perspective, additional attention should be paidto the cryptographic techniques used to protect the application itself from unintended data disclosure.3 www.bls.gov/ooh/computer-and-information-technology/information-security-analysts-web-developers-and-computer-network-architects.htm4 www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.html4
    • Veracode State of Software Security Report: Volume 5Security of ApplicationsThe evidence linking organizational intrusions and data breach events to applicationsecurity issues continues to grow. Web-based intrusions and hacking in general accountfor 52% of the breaches in 2011 and 2012 tracked by Open Security Foundation’sDataLossDB (Figure 1).While these categories are extremely broad, hacking and web-based intrusions often involve exploiting softwarevulnerabilities. Reports published by companies that conduct actual breach investigations provide additional insight.The Verizon Data Breach Report, released in March 2012, indicated that 81% of attacks utilized some sort of hacking.This section explores application compliance with standard policies, remediation submission rates and security qualityscores to shed some light on why this connection exists.Data Loss Breaches Categorized by Root Cause 45% Hack 15% Stolen Item 12% Fraud-SE 7% Web 5% Lost or Missing Item 5% Disposal Item 4% Unknown 2% Snail Mail and Fax 2% Email 2% Virus 1% Skimming and SnoopingFigure 1: Data Loss Breaches Categorized by Root Cause (Source: DataLossDB)Compliance with Standard Policies Upon First SubmissionFigure 2 illustrates the compliance upon initial application submission against two standard policies.5 Web applicationsare assessed against the OWASP Top 10 and only 13% complied on first submission. Non-web applications areassessed against the CWE/SANS Top 25 and 31% complied on first submission. Only 30% of applications compliedwith enterprise defined policies. Compliance with policies upon first submission of an application can be a goodindicator of the success or failure of “building-in” security as part of the software development lifecycle (SDLC).5 More details about how the Veracode platform determines policy compliance can be found in the Appendix. 5
    • Veracode State of Software Security Report: Volume 5Because security flaws that are eliminated before deployment, or never created in the first place, are much less expen-sive to remediate, thus building remediation into the SDLC at an early stage is often a key goal for most organizations.Yet, with more than two thirds of the applications failing to comply, our results show that secure software developmentpractices are still not as widespread as they should be. While applications may eventually become compliant, the highinitial failure rate validates the concerns CISOs have regarding the business risks related to application security.Compliance with Policies Upon First Submission Compliant Out of Compliance Enterprise Policy 30% 70%CWE/SANS Top 25 31% 69% OWASP Top 10 13% 29% 87% 71% 0% 20% 40% 60% 80% 100%Figure 2: Compliance with Policies Upon First SubmissionThe OWASP Top 10 compliance rate did not change significantly from Volume 4 (14%). In contrast, the percentageof applications passing enterprise policies declined significantly from Volume 4 (40%). Similarly, the percentagesof non-web applications that complied with SANS/CWE policy were respectively 42% and 31% in Volumes 4 and 5,which is highly statistically significant decrease. We decided to investigate whether language or supplier types wherepotential drivers of the decrease in SANS/CWE policy compliance. Our analysis6 suggests that the major contributorsto this Vol4 to Vol5 decrease in compliance rate were as follows:• There is evidence that Language (99.9% confidence level) influences CWE/SANS compliance with C/C++ being the most significant factor. This means that C/C++ applications, which represent 29% of non-web applications in our dataset, had a significant impact on driving down the CWE/SANS compliance rate from Volume 4.• There is no compelling evidence that software supplier types (internally developed, commercial, outsourced, and open source) influence CWE/SANS compliance.Another possible factor in the decrease of SANS/CWE policy compliance could be the increase in the number of firstsubmissions in our sample set. The Volume 5 data had 75% more first builds than Volume 4. This increase in firstbuilds suggests more broad use of the service by a wider variety of companies, perhaps with higher variation insecure software development practices.6 The analysis that we performed used a generalized linear model to perform logistic regression on a proportional response variable (SANS Compliance) with categorical explanatory variables (Volume, Flaw Category, Industry, Supplier, and Language). See Appendix for additional detail.6
    • Veracode State of Software Security Report: Volume 5Remediation AnalysisWe frequently get questions from customers and analysts on whether discovered vulnerabilities are actuallyremediated and whether those remediations are validated through additional testing. To shed some light on thisissue, we start by examining how frequently organizations resubmit applications following the initial analysis. Theseresubmitted applications typically contain a combination of security remediations for previously reported vulnerabili-ties. Resubmitted applications may also contain new or altered code components to address non-security issuesand new code components representing new functionality.One might expect that more companies would resubmit higher percentages of their very high criticality applicationsthan they would their medium criticality applications. If this expectation were true then one would anticipate thedistribution pattern for medium and very high criticality applications to look very different (possibly a classic bellcurve for the medium criticality applications and an exponential curve for the very high criticality applications). At thevery least, one might expect variability in resubmission rate to decrease as application criticality increases. The datadoes not support those expectations. Figure 3 shows statistically insignificant differences in the distribution patterns.Roughly 45% of organizations resubmit 91-100% of their applications regardless of the business criticality. In Volume4 we reported that the very high group was slightly different from the high and medium groups, since over 50%of companies resubmitting 91-100% of their very high criticality applications, however that slight difference hasdisappeared in Volume 5.Percentage of Applications Resubmitted by Business Criticality Medium High Very High 50 40PERCENT OF ORGANIZATIONS 30 20 10 0 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 PERCENT OF APPLICATIONS RESUBMITTEDFigure 3: Percentage of Applications Resubmitted by Business Criticality 7
    • Veracode State of Software Security Report: Volume 5Security Quality AnalysisWe continue to track the quarterly mean Veracode Security Quality Score (SQS) as a means of determining whensecurity quality becomes a standard part of developing software. We expect that when most organizations have builtsecurity into their SDLC we will begin to see an upward trend in SQS developing. An upward trend would indicatethat applications from new Veracode customers and newly developed applications from existing customers are a lesssignificant force in dragging down the mean with very low scores. Figure 4 shows we still have a lot of work to do inbuilding in security. The best fit line across our analysis timeline has a p-value 7 of 0.37 indicating that the trend is flat.This flat trend is consistent with the trends reported in Volumes 3 and 4—there has been no increase or decrease inthe quarterly mean SQS since the fourth quarter of 2009.Veracode Security Quality Score Trendp-value = 0.37 100 80 MEAN VERACODE SQS 60 40 20 0 2011-1 2011-2 2011-3 2011-4 2012-1 2012-2 QUARTERSFigure 4: Veracode Security Quality Score TrendNext we examine the progress an application makes build-over-build as the developers respond to findings andattempt to remediate flaws using the median value of the Veracode Security Quality Score (SQS) as a progressindicator. The distribution of the Veracode Security Quality Score by application build is shown as a whisker plot 8in Figure 5. The data shows statistically significant build-over-build improvement from the first to third builds. Buildsfour through six remain statistically flat, followed by a marked improvement in builds seven and eight. The medianscore decreased in build nine, however, it is still above the plateau of builds four through six. This pattern suggeststhe security quality in applications with nine or more builds has been permanently improved even as functionalityin the form of new code is being added in the later builds.7 See Appendix for definition.8 See Appendix for definition.8
    • Veracode State of Software Security Report: Volume 5Veracode Security Quality Score by Build 100VERACODE SECURITY QUALITY SCORE 80 60 40 1 2 3 4 5 6 7 8 9 BUILD NUMBERFigure 5: Veracode Security Quality Score by BuildThe pattern of statistically significant improvement in security quality scores for builds one, two and three seen inFigure 5 is consistent with the figures reported in Volumes 3. However, there are significant differences betweenVolumes 4 and 5 in the patterns reported for later builds. In Volume 4 we saw an oscillating behavior with peaksoccurring at builds four, seven and nine. The Volume 4 pattern suggested that new functionality was introducedin the build after each peak, which resulted in a new set of security flaws found and the consequently lower score.It is not immediately clear what has caused this shift in pattern between Volumes 4 and 5. It could be that ourdataset for later application builds is richer in Volume 5 and therefore more representative of the actual improve-ment pattern. It is also possible that developers are starting to introduce new code that does not suffer from thevulnerabilities in the old code. The developers have learned from the mistakes and do not repeat them. 9
    • Veracode State of Software Security Report: Volume 5Language AnalysisIn this section we dive deeper into each language. For each language we look at thedistribution of each vulnerability category. We measure vulnerability distribution interms of share of vulnerabilities found in each language group.We calculate this by first filtering our data by language. For each language we determine the total number of vulnera-bilities found and the number vulnerabilities that belong to a specific category. These values allow us to calculatethe percentage share for each vulnerability category for that language. These vulnerability distribution calculationsallow us to make statements such as, 3% of vulnerabilities found in Java applications are SQL injection vulnerabilities(Figure 6). The vulnerability distribution metrics also give us a historical perspective, since we have been reportingthem since Volume 3.Another metric we explore is the vulnerability prevalence in terms of the percentage of applications affected by eachvulnerability category. To calculate this metric, we also begin by filtering our data by language. Then we identify howmany applications contain one or more vulnerabilities from each category, which allows us to calculate the percentageaffected. These calculations enable us to make statements such as: SQL injection vulnerabilities affect 31% of Javaapplications (Figure 7).Vulnerability distribution and prevalence information can be useful for planning purposes, particularly when internaland/or industry-specific benchmarks9 are not readily available. Organizations can estimate the resource impact ofimplementing or changing application security policies. Consider the situation of a security team writing a policyaimed at eliminating SQL injection flaws and a development team writing their application in Java. The percentageaffected data tells the teams there is a 31% chance that their application will have SQL injection flaw. The vulnerabil-ity prevalence data means that if the application does have SQL injection, it is likely that only 3% of the vulnerabilitiesfound will be SQL injection.Finally, we investigate the percentage improvement in vulnerability distribution between an application’s first andsecond build. This metric should provide some perspective on which vulnerabilities organizations chose to fix uponreceiving the results from their first submission. For each language, we looked at the subset of applications with theirfirst and second builds occurring within the analysis timeframe for this report. This means we excluded applicationswith their first build occurring before January 2011 and applications with their second build occurring after June 2012.We also excluded applications with components written in more than one language. Then we calculated the change invulnerability distribution from the first build to the second build. The percentage change will be affected by the volumeof flaws. For example, consider the case of a development team that has fixed 10 flaws between the first and secondbuild. If there were 20 flaws in the first build then the calculation would show a 50% improvement. However, if thefirst build contained 100 flaws, then the calculation would show a 10% improvement. To acknowledge this impactwe indicate the top vulnerability categories in the percentage improvement charts. The percentage change may alsobe affected by improvements to the Veracode platform, and we’ll discuss those improvements where applicable.9 The Veracode Analytics capabilities enable organizations to benchmark their internal application security metrics with industry benchmarks.10
    • Veracode State of Software Security Report: Volume 5JavaFigure 6 shows that vulnerability distribution in Java applications has not significantly changed since Volume 3. Thecross-site scripting category consistently represents more than half of all vulnerabilities discovered in Java applications.In the Volume 5 dataset, SQL injection makes its first appearance in the top 5 list at fifth place, replacing cryptographicissues. Figure 7 shows code quality, CRLF injection and information leakage affecting the most applications with 82%,68% and 58% respectively.Vulnerability Distribution Trends for Java Applications (Share of Total Vulnerabilities Found)Rank Volume 3 Volume 4 Volume 5 1 50% 56% 51% Cross-Site Scripting (XSS) 2 17% 16% 21% CRLF Injection 3 14% 10% 12% Information Leakage 4 4% 4% 3% Encapsulation 5 5% 3% 3% SQL Injection 6 3% 3% Directory Traversal 7 2% Cryptographic IssuesFigure 6: Vulnerability Distribution Trends for Java Applications (Share of Total Vulnerabilities Found) 11
    • Veracode State of Software Security Report: Volume 5Vulnerability Prevalence in Java Applications (Percentage of Applications Affected) Code Quality 82% CRLF Injection 68% Information Leakage 58% Cross-Site Scripting (XSS) 57% Cryptographic Issues 55% Directory Traversal 49%Insufficient Input Validation 44% Encapsulation 38% API Abuse 37% Credentials Management 34% Time and State 34% SQL Injection 31% Session Fixation 29% Race Conditions 18% OS Command Injection 9% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 7: Vulnerability Prevalence in Java Applications (Percentage of Applications Affected)Figure 8 indicates the untrusted search path category had thehighest improvement percentage from first to second applicationbuild. Although this vulnerability category does not occur very CRLF injection, which holdsoften (it is absent from Figure 6 and Figure 7) it contains some the second place in bothvery high severity flaws. For example, CWE-114 is defined as Java vulnerability distributionexecuting commands or loading libraries from an untrusted source, (21%) and prevalence (68%),or in an untrusted environment, can cause an application to execute showed an improvementmalicious commands (and payloads) on behalf of an attacker.10 percentage of 45% from firstFigure 8 also shows an improvement percentage of 45% for CRLF to second submission.injection, which holds the second place in both Java vulnerabilitydistribution (21%) and prevalence (68%).10 For the complete description see cwe.mitre.org/data/definitions/114.html12
    • Veracode State of Software Security Report: Volume 5Percent Improvement in Java Vulnerability Distribution from First to Second Submission Indicates categories with the highest vulnerability distribution in Java Untrusted Search Path 90% CRLF Injection 45% Untrusted Initialization 45% Session Fixation 44% Dangerous Functions 36% Code Quality 28% Encapsulation 23% Credentials Management 23% Cryptographic Issues 18% API Abuse 18% SQL Injection 16%Insufficient Input Validation 15% Time and State 15% Cross-Site Scripting (XSS) 14% OS Command Injection 8% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 8: Percent Improvement in Java Vulnerability Distribution from First to Second Submission.NETThe vulnerability distribution for .NET applications has not changed significantly over the last three Volumes(Figure 9). Cross-site scripting (XSS) retains the highest share of vulnerabilities at 49%. However, the percentageshave been changing over time. Cross-site scripting and directory traversal categories have been slowly increasingwhile information leakage and cryptographic issues have been slowly decreasing. Cross-site scripting and SQL injection showed improvement from first to second build in terms of share of vulnerabilities discovered, but still affect 60% and 30% of all .NET applications respectively. 13
    • Veracode State of Software Security Report: Volume 5In addition, 61% of .NET applications contain one or more XSS vulnerabilities (Figure 10). The high percentages in bothmetrics indicate that cross-site scripting is a pervasive vulnerability, i.e. it occurs many times in many applications.Figure 11 appears to indicate a fairly low percentage improvement (15%) between the first and second build for XSS.When taken together, these three data points demonstrate the enormity of the task of removing cross-site scriptingfrom existing applications, because there are so many vulnerabilities to remediate.Significantly, the top five categories that showed the most improvement are comprised of less than 10% of alldiscovered flaws and affect at most 50% of all .NET applications (Figure 11). If you leave out SQL injection, the topfour categories that showed improvement comprise at most 20% of all .NET applications. Cross-site scripting andSQL injection showed improvement in terms of share of vulnerabilities discovered, but still affect 60% and 30%of all .NET applications respectively.Vulnerability Distribution Trends for .NET Applications (Share of Total Vulnerabilities Found)Rank Volume 3 Volume 4 Volume 5 1 44% 47% 49% Cross-Site Scripting (XSS) 2 23% 18% 14% Information Leakage 3 11% 10% 11% Directory Traversal 4 8% 9% 9% Cryptographic Issues 5 6% 6% 6% Insufficient Input Validation 6 <6% 5% SQL InjectionFigure 9: Vulnerability Distribution Trends for .NET Applications (Share of Total Vulnerabilities Found)14
    • Veracode State of Software Security Report: Volume 5Vulnerability Prevalence in .NET Applications (Percentage of Applications Affected) Code Quality 73% Cryptographic Issues 62% Cross-site Scripting (XSS) 61% Information Leakage 60% Directory Traversal 56%Insufficient Input Validation 42% CRLF Injection 40% SQL Injection 30% Time and State 22% Credentials Management 15% Untrusted Search Path 11% OS Command Injection 11% Error Handling 5% Potential Backdoor 3% Buffer Overflow 1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 10: Vulnerability Prevalence in .NET Applications (Percentage of Applications Affected)Percent Improvement in .NET Vulnerability Distribution from First to Second Submission Indicates categories with the highest vulnerability distribution in .NET Buffer Overflow 59% Error Handling 50% Credentials Management 31% Dangerous Functions 30% SQL Injection 25% Potential Backdoor 24% Insufficient Input Validation 23% OS Command Injection 18% Code Quality 17% Cross-Site Scripting (XSS) 15% Cryptographic Issues 13% Time and State 10% Directory Traversal 4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 11: Percent Improvement in .NET Vulnerability Distribution from First to Second Submission 15
    • Veracode State of Software Security Report: Volume 5C/C++Vulnerability distribution for C/C++ applications haschanged significantly over the past three Volumes Vulnerability distribution for C/C++(Figure 12). The buffer management errors category applications has changed significantlyhas risen from fifth place in Volume 3 to second place over the past three Volumes, with errorin Volume 5. Buffer overflow issues have fallen to third handling and buffer management errorsplace and cryptographic issues appear for the first time rising to the top of both the vulnerabilityin fifth place. While error handling issues retain the distribution and prevalence lists.top spot from Volume 4, the percentage share hasincreased from 23% to 38%.The vulnerability categories with the highest distribution also affect the highest percentage of applications (Figure 13),from error handling affecting 77%, to numeric errors affecting 44%. One implication of this data for organizationsscanning C/C++ applications for the first time is that it is likely several of these vulnerabilities will be listed in theassessment report.Although Figure 13 shows API abuse affecting relatively few applications (7%), Figure 14 indicates a significantpercentage improvement from first to second submission in API abuse (85%). This is an example of how targetinga few flaws (such as CWE 243: Creation of chroot Jail Without Changing Working Directory, which may allowunauthorized access to data files) can significantly improve an application’s security posture.Vulnerability Distribution Trends for C/C++ Applications (Share of Total Vulnerabilities Found)Rank Volume 3 Volume 4 Volume 5 1 27% 26% 38% Error Handling 2 23% 20% 16% Buffer Management Errors 3 22% 18% 16% Buffer Overflow 4 11% 14% 14% Numeric Errors 5 9% 14% 4% Cryptographic Issues15 <1% Potential BackdoorFigure 12: Vulnerability Distribution Trends for C/C++ Applications (Share of Total Vulnerabilities Found)16
    • Veracode State of Software Security Report: Volume 5Vulnerability Prevalence in C/C++ Applications (Percentage of Applications Affected) Error Handling 78%Buffer Management Errors 53% Buffer Overflow 48% Cryptographic Issues 46% Numeric Errors 44% Directory Traversal 40% Time and State 32% Dangerous Functions 30% Code Quality 28% OS Command Injection 26% Untrusted Search Path 19% Race Conditions 16% Format String 15% API Abuse 7% Information Leakage 5% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 13: Vulnerability Prevalence in C/C++ Applications (Percentage of Applications Affected)Percent Improvement in C/C++ Vulnerability Distribution from First to Second Submission Indicates categories with the highest vulnerability distribution in C/C++ API Abuse 89%Buffer Management Errors 74% Dangerous Functions 53% Directory Traversal 42% Code Quality 37% Buffer Overflow 34% Numeric Errors 34% Untrusted Search Path 33% Error Handling 7% Cryptographic Issues 4% Time and State 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 14: Percent Improvement in C/C++ Vulnerability Distribution from First to Second Submission 17
    • Veracode State of Software Security Report: Volume 5PHPCross-site scripting has the largest share of total vulnerabilities found in PHP applications (Figure 15). The goodnews is that the percentage continues to drop from 80% in Volume 3, to 75% in Volume 4, and to 60% in Volume 5.Cryptographic issues debut at second place with 12% share. While, the share of SQL injection vulnerabilities remainssteady at 7% it drops to fifth place. 27% of PHP applications have SQL injection issues (Figure 16), with just overhalf of those vulnerabilities remediated by the second submission (Figure 17).Interestingly, with PHP we see a similar disconnect between the rankings of vulnerability categories by percentageof affected applications and percent improvement. While XSS holds first place in both Figure 15 and Figure 16, itsimprovement from build one to build two is in third place (Figure 17). Code injection is ranked second in Improvementbut sixth in percent affected applications. Two interpretations are possible. First, the two rankings are not exactlyapples to apples. In the case of percent affected applications, all flaws in a given vulnerability category much be fixedto register a change in the metric; while fixing just one flaw in the Improvement metric will (other things being equal)affect improvement. The second interpretation is that developers are not necessarily prioritizing eradication of all flawsof a given category.Vulnerability Distribution Trends for PHP Applications (Share of Total Vulnerabilities Found)Rank Volume 3 Volume 4 Volume 5 1 80% 75% 60% Cross-Site Scripting (XSS) 2 8% 10% 12% Cryptographic Issues 3 6% 7% 10% Information Leakage 4 3% 4% 9% Directory Traversal 5 1% 2% 7% SQL Injection 6 1% Code InjectionFigure 15: Vulnerability Distribution Trends for PHP Applications (Share of Total Vulnerabilities Found)18
    • Veracode State of Software Security Report: Volume 5Vulnerability Prevalence in PHP Applications (Percentage of Applications Affected) Cross-site Scripting (XSS) 60% Directory Traversal 47% Information Leakage 29% SQL Injection 27% Cryptographic Issues 23% Code Injection 21% OS Command Injection 17% Untrusted Initialization 3% Code Quality <1%Insufficient Input Validation <1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 16: Vulnerability Prevalence in PHP Applications (Percentage of Applications Affected)Percent Improvement in PHP Vulnerability Distribution from First to Second Submission Indicates categories with the highest vulnerability distribution in PHP SQL Injection 54% Code Injection 29%Cross-Site Scripting (XSS) 24% OS Command Injection 18% Information Leakage 12% Directory Traversal 3% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 17: Percent Improvement in PHP Vulnerability Distribution from First to Second Submission 19
    • Veracode State of Software Security Report: Volume 5ColdFusionCross-site scripting (XSS) has remained the dominant vulnerability category in ColdFusion applications assessed bythe Veracode platform. It accounts for 81% of all vulnerabilities found (Figure 18) and affects 94% of applications(Figure 19). Fortunately, recent versions of the ColdFusion platform have built in several fixes and new features toaddress the high occurrence of XSS. We expect the number of XSS vulnerabilities to decline as we update our scansto reflect platform changes and as customers deploy newer platform versions.We also found a much lower percentage of ColdFusion applications have been resubmitted for testing within ourreporting timeframe, therefore we are not reporting the percentage improvement data for ColdFusion. There are noobvious language-specific reasons for low resubmission rates. However, there may be some organizational issuesat work, for example:• The organizations may be focused on baselining and reporting their current security posture for compliance purposes and have yet to allocate resources for remediation.• The organizations are relying on network mitigations, such as creating WAF rules based on Veracode’s findings, until either the development organization or software supply chain can be influenced to remediate the identified flaws.Neither of these situations is ideal, as the ultimate goal of an application security program is to improve the organiza-tion’s security posture over time. Achieving that goal requires vulnerabilities found during the testing process to beremediated, preferably on a prioritized 11 remediation schedule so that developers can focus on the most impactfulremediations first.Vulnerability Distribution Trends for ColdFusion Applications (Share of Total Vulnerabilities Found) Rank Volume 3 Volume 4 Volume 5 1 89% 87% 81% Cross-Site Scripting (XSS) 2 9% 8% 9% SQL Injection 3 <1% 1% 5% Information Leakage 4 <1% 1% 2% Directory Traversal 5 <1% 1% 2% CRLF Injection 6 1% 1% OS Command InjectionFigure 18: Vulnerability Distribution Trends for ColdFusion Applications (Share of Total Vulnerabilities Found)11 The Veracode platform’s “Fix First” analysis automatically prioritizes an application’s vulnerability results based on severity and effort to fix.20
    • Veracode State of Software Security Report: Volume 5Vulnerability Prevalence in ColdFusion Applications (Percentage of Applications Affected) Cross-Site Scripting (XSS) 95% SQL Injection 72% Information Leakage 62% Directory Traversal 22% OS Command Injection 20% CRLF Injection 20% Code Quality 17% Cryptographic Issues 12%Insufficient Input Validation 12% Encapsulation 12% Time and State 11% API Abuse 11% Credentials Management 10% Race Conditions 9% Session Fixation 4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 19: Vulnerability Prevalence in ColdFusion Applications (Percentage of Applications Affected) 21
    • Veracode State of Software Security Report: Volume 5Application TypesIn this section we take a closer look at three types of applications: mobile, web andnon-web. For each application type we start with a brief discussion of the threatlandscape and then explore the available vulnerability data.Mobile Threat LandscapeThe mobile threat landscape has three distinct areas: mobile malware, application behaviors, and code vulnerabilities.Each of these areas is a risk to the enterprise that is specific to the mobile space and each manifests itself in aslightly different attack model. Since the risk and attack models are slightly different, enterprises should considera multi-pronged approach to applying preventative security controls.The traditional, signature-based malware detection methodology is broken in that modern mobile malware is designedto easily evade signature-based detection. Device vendors have adopted different approaches in attempt to minimizethe malware proliferation. Apple adopted the walled approach, in that it has strict controls over which applications areapproved for listing in its App Store. However, it is not clear what security measures are being checked during thisapproval process. Android is more open with more distribution channels and third party app stores. Google Bouncer 12has some static testing but they are trying to balance turnaround time and security—there is research that shows thatthis project has a low detection rate of 15.32% (www.cs.ncsu.edu/faculty/jiang/appverify). Both iOS and Android plat-forms have built in security features such as kill switches to remove identified malware applications from mobile phones.The behavioral aspects of the threat landscape are particularly important. It is not possible to determine whetherdata exfiltration is part of the intended user experience. For example, the application FourSquare’s transmissionof a phone’s GPS location is a core component of the functionality delivered to the user. On the other hand if theapplication is a single-user solitaire game, then the transmission of the phone’s GPS location may signify maliciousbehavior. Having a control point for determining the behavioral aspects of mobile apps may be one reason thatenterprises are interested in creating their own internal app stores. An enterprise operated app store could potentiallygive the enterprise a control point to implement a zero trust model for adopting mobile applications. In a zero trustmodel, every new application is treated as if it is malware and put into the hands of a malware analyst. The analystwould initially conduct a behavioral analysis, (with static testing tools to understand code and data flows and dynamictesting tools to understand the runtime behavior) and then add human intelligence to determine what the applicationdoes, how it maps to malicious intent and what the risk is to their enterprise.In terms of code vulnerabilities, the Cloud Security Alliance Mobile Working Group released findings 13 in October2012 that data loss from missing mobile devices ranks as the top threat related to mobile devices for enterprises.The number two and three threats were mobile malware and data leakage due to poorly written third party applica-tions. In the next section we examine the vulnerability distribution and prevalence in mobile applications scannedby the Veracode platform.1412 Android and Security, by Hiroshi Lockheimer, VP of Engineering, Android, February 2012. See googlemobile.blogspot.com/2012/02/android-and-security.html13 cloudsecurityalliance.org/csa-news/data-loss-mobile-ranks-top-threat-enterprises14 Note that the analysis in this section does not include vulnerability analysis from our recent Marvin Mobile acquisition.22
    • Veracode State of Software Security Report: Volume 5State of Mobile Application SecurityFirst we examine the vulnerability distribution in terms of share of total vulnerabilities discovered across all applica-tion builds associated with each mobile platform. In Volume 5, we have enough data on all three platforms to providea statistically sound basis for comparison.Figure 20 shows all three mobile platforms that we analyzed share cryptographic issues and information leakagein the Top 5 list of vulnerabilities as measured by percent of total vulnerabilities found. As jailbreaking becomesmore common practice and new features such as surviving reboots are supported, cryptographic issues significantlyweaken data protection. Attackers with physical control of a mobile device for a small amount of time can jailbreakit and install a backdoor with keyloggers or other malware and/or copy the content. Both cryptographic issues andinformation leakage vulnerabilities increase the attack surface for mobile applications, and are two of Cloud SecurityAlliance’s top five identified threats to mobile devices.Vulnerability Distribution for Mobile Platforms (Share of Total Vulnerabilities Found) Android iOS Java ME CRLF Injection 37% Information Leakage 62% Cryptographic Issues 47% Cryptographic Issues 33% Error Handling 20% Information Leakage 47% Information Leakage 10% Cryptographic Issues 7% Directory Traversal 3% SQL Injection 9% Directory Traversal 6% Insufficient Input Validation 2% Time and State 4% Buffer Management Errors 3% Credentials Management <1%Figure 20: Vulnerability Distribution for Mobile Platforms (Share of Total Vulnerabilities Found)In Volume 4 we published our first analysis of Android applications and focused on cryptographic issues and dataexfiltration (i.e. transmission of potentially sensitive information off the device). At the time we reported that 61%of Android applications contained at least one insufficient entropy issue and 39% had potential data exfiltrationissues. Figure 21 shows that these categories are still cause for concern. Cryptographic issues affect 64% ofAndroid applications, while information leakage issues occur in 26% of applications. 23
    • Veracode State of Software Security Report: Volume 5Cryptographic issues also affect 58% of iOS applications(Figure 22). These are common coding mistakes that can be Cryptographic issues affect areadily fixed. A hard-coded key is much simpler to extract from sizeable portion of Android (64%)a mobile application than from a J2EE web application since the and iOS (58%) applications.mobile application can simply be copied from the mobile device.Once these keys are compromised any security mechanismsdependent on the secrecy of the keys are rendered ineffective.Android Vulnerability Prevalence (Percentage of Applications Affected) Code Quality 76% Cryptographic Issues 64% CRLF Injection 62% SQL Injection 31% Information Leakage 26% Directory Traversal 14% Time and State 11% Cross-Site Scripting (XSS) 8% Authorization Issues 7% Credentials Management 7% Encapsulation 4% Session Fixation 4% Dangerous Functions 1% API Abuse 1%Insufficient Input Validation 1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 21: Android Vulnerability Prevalence (Percentage of Applications Affected)The vulnerability landscape for iOS is somewhat different than Android in part because the differences in thelanguages used (Objective C in the case of iOS and Java in the case of Android). For example, Figure 22 shows iOSapplications are more susceptible to error handling and credentials management, than Android applications. Similarly,Android developers must pay more attention to SQL injection and code quality issues than iOS developers.24
    • Veracode State of Software Security Report: Volume 5iOS (ObjectiveC) Vulnerability Prevalence (Percentage of Applications Affected) Error Handling 76% Cryptographic Issues 58% Information Leakage 42% Credentials Management 17% Numeric Errors 13%Buffer Management Errors 11% Code Quality 8% Buffer Overflow 6% Directory Traversal 4% Time and State 2% Format String 1% Untrusted Search Path 1% Race Conditions 1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 22: iOS (ObjectiveC) Vulnerability Prevalence (Percentage of Applications Affected)While the number of iOS and Android applications submitted for testing has grown dramatically over our reportingperiods, the same has not been true for BlackBerry applications. The data we collected from BlackBerry applicationsis relatively small, however, we want to give readers some indication of the vulnerability categories being discovered(Figure 23). Due to the limited amount of data, we expect the statistics to experience some variability and we willassess the value of reporting BlackBerry statistics in future reporting periods.Blackberry Vulnerability Prevalence (Percentage of Applications Affected) Information Leakage 77% Cryptographic Issues 40% Code Quality 28% Directory Traversal 21%Insufficient Input Validation 7% Time and State 2% CRLF Injection 2% Credentials Management 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 23: Blackberry Vulnerability Prevalence (Percentage of Applications Affected) 25
    • Veracode State of Software Security Report: Volume 5Web Application Threat LandscapeExploitation of web application vulnerabilities continues to According to WHID, 10% of thebe a significant threat for organizations. The Web Application incidents occurring in 2012 areSecurity Consortium continually tracks media reported security categorized as SQL injectionincidents that can be associated with web application security attacks, yet millions of recordsvulnerabilities in the Web Hacking Incident Database (WHID).15 are leaked through SQL injectionFigure 24 illustrates the trends in the top attack techniques, attacks during 2012.which enables organizations to take a risk-based approachto application security testing and subsequent remediation.Trends in Attack Methods from 2010 to 2012 Rank 2010 2011 2012 1 20% 34% 45% Unknown 2 17% 23% 28% Denial of Service 3 16% 20% 10% SQL Injection 4 6% 3% 3% Stolen Credentials 5 5% 3% 3% Brute Force 6 5% 2% Banking Trojan 7 2% 2% Cross-Site Scripting (XSS)Figure 24: Trends in Attack Methods from 2010 to 2012 (Source: WHID)45% of the incidents occurring in 2012 are currently categorized as unknown, meaning that there is not yet enoughinformation available to conclusively determine the attack method. The consortium updates the attack methodclassification as those breaches are investigated. Therefore, it is likely that many of those unknown attacks usedone of the other popular techniques listed. The listing shows there is a limited set of vulnerability categories that areexploited at scale; therefore an application security program that covers the top vulnerability categories can reducethe attack surface in a cost-effective manner. This is where most organizations should start when designing an initialapplication security policy to address the current threat space.15 projects.webappsec.org/w/page/13246995/Web-Hacking-Incident-Database26
    • Veracode State of Software Security Report: Volume 5The rise in Denial of Service attacks appears to be evidence of the rising activity of hacktivist groups such asAnonymous. At the time of writing, WHID contained 82 records associated with Anonymous attacks in 2012.10% of the incidents occurring in 2012 are categorized as SQL injection attacks, down from 23% in 2011, howeverit remains in third place in spite of the percentage drop. Figure 25 illustrates how devastating SQL injection attackscan be to organizations. THREE OF THE BIGGEST SQL INJECTION ATTACKS IN 2012 #PROJECTWHITEFOX #LINKEDIN 3 # OF RECORDS AFFECTED 2 # OF RECORDS AFFECTED DEC 2012 1.6 MILLION JUNE 2012 6.5 MILLIONHackers gained “#ProjectWhiteFox will conclude Russian hacker “dwdm” accessed andaccess to 31 targets this year’s series of attacks by leaked millions of passwords promoting hacktivism worldwideincluding NASA, and drawing attention to thethe FBI, Interpol, freedom of information on the net.” “ On a grading scale of A through F, experts say,the Pentagon and Team GhostShell LinkedIn, eHarmony and Lastfm.com would get,numerous other at best, a ‘D’ for password security.”educational and ACCESSED New York Timesgovernment Names, email addresses, home ACCESSEDorganizations. addresses, passwords, the SQL Injection vulnerable links, and more Millions of passwords were posted on the internet. GAMIGO: HAMBURG GERMANY JULY 2012 1 # OF RECORDS AFFECTED 11 MILLION HASHED PASSWORDS 8.2 MILLION EMAIL ADDRESSESHacker “8in4ry_Munch3r” accessed ACCESSED “ It’s the largest leak I’veuser account credentials. ever actually seen.” Email addresses, usernames Steve Thomas, Internet and encrypted passwords Security Expert, PwnedListFigure 25: Three of the Biggest SQL Injection Attacks in 2012 27
    • Veracode State of Software Security Report: Volume 5State of Web Application SecurityFigure 26 shows how the top ten vulnerability categories for web applications have varied over the last three SoSSvolumes. Not much has changed. The top five categories remain the same as Volume 4. Cross-site scripting andinformation leakage are at the top with 67% and 65% respectively. Volume 5 reporting includes two additionalCWE categories associated with the insufficient input validation category which vaulted the category to sixth place.API abuse dropped just out of the top ten.Top Vulnerability Categories (Percentage of Affected Web Application Builds)Rank Volume 3 Volume 4 Volume 5 1 73% 68% 67% Cross-Site Scripting (XSS) 2 70% 66% 65% Information Leakage 3 58% 54% 57% CRLF Injection 4 51% 53% 55% Cryptographic Issues 5 47% 49% 49% Directory Traversal 6 38% 32% 43% Insufficient Input Validation 7 31% 30% 32% SQL Injection 8 30% 27% 28% Time and State 9 23% 25% 26% Credentials Management10 22% 25% 25% Encapsulation11 22% 24% 23% API AbuseFigure 26: Top Vulnerability Categories (Percentage of Affected Web Application Builds)28
    • Veracode State of Software Security Report: Volume 5The quarterly trend for the percentage of applications affected by cross-site scripting (XSS) remains statistically flatwith a p-value of 0.441 (Figure 27), as it has been in both Volume 3 and 4. Although this vulnerability is fairly easy tofix, it is often given a lower remediation priority to other vulnerabilities because attackers are not leveraging them asmuch in profit-driven attack scenarios.16 However there are indications that attacks on XSS vulnerabilities are on therise. According to the Microsoft Security Intelligence Report Vol 1317 there has been a significant increase in reportedXSS cases over the past two years. Both Microsoft and Google have added filters to their browsers to block sometypes of XSS attacks. However, these mitigations should not be viewed as permanent defenses because not allusers will implement the filters and filtering capabilities can be bypassed. The best protection is still eliminatingthe flaws and using secure coding techniques to prevent the creation of new flaws.Quarterly Trend for Cross-Site Scripting (XSS) Prevalence (Percentage of Affected Web Applications)p-value = 0.441 100 PERCENT AFFECTED WEB APP BUILDS 80 60 40 20 0 2011-1 2011-2 2011-3 2011-4 2012-1 2012-2 QUARTERSFigure 27: Quarterly Trend for Cross-Site Scripting (XSS) Prevalence (Percentage of Affected Web Applications)Figure 28 shows the downward trend for SQL injection seen in past Volumes has flattened. For six consecutivequarters, from the first quarter of 2011 to the second quarter of 2012, the percentage of applications affected bySQL injection has hovered around 32% (with a p-value of 0.868). In Volume 3 we reported SQL injection decreasingat a rate of 2.4% per quarter. In Volume 4 the downward trend was still visible, going from 38% of applicationsaffected in the fourth quarter of 2009 to 32% in the third quarter of 2011.16 Trustwave 2012 Global Security Report17 www.microsoft.com/security/sir/default.aspx 29
    • Veracode State of Software Security Report: Volume 5The trend data reported here reveals that eradicating XSSand SQL injection continues to be a significant challenge. Eradicating SQL injection and cross-siteEven with increased attention and media coverage link- scripting remains a challenge as theing high profile data breaches to XSS and SQL injection vulnerability prevalence trends remainvulnerabilities, there has not been a dramatic reduction flat for the last 6 quarters. SQL injectionin the occurrence of these critical vulnerabilities. has hovered around 32% and cross-site scripting around 67%.There may be a few factors keeping these trendsrelatively constant over time, for example:• An influx of web applications. It is likely that a significant portion of our dataset includes web applications that have never been tested before.• Expansion of web assessment focus. Security teams are beginning to expand their focus from analyzing a few critical web applications to analyzing the entire website portfolio. This observation implies that we are seeing assessments of many more web applications that were built before security policies were implemented. As these web applications are tested for the first time, we are logging many more instances of XSS and SQL injection.• Frequent web application changes. Agile development techniques drive more new and updated web applications online with increasing frequency. An enterprise web application portfolio can change from week to week. This means most enterprise portfolios have a built-in influx of new vulnerabilities. Until secure coding techniques go from being recommended best practices to standard web application development practices, CISOs will struggle with maintaining constant vigilance over the website portfolios having a constant stream of new vulnerabilities.Quarterly Trend for SQL Injection Prevalence (Percentage of Affected Web Applications)p-value = 0.868 100PERCENT AFFECTED WEB APP BUILDS 80 60 40 20 0 2011-1 2011-2 2011-3 2011-4 2012-1 2012-2 QUARTERSFigure 28: Quarterly Trend for SQL Injection Prevalence (Percentage of Affected Web Applications)30
    • Veracode State of Software Security Report: Volume 5Non-Web Applications Threat LandscapeWhile most of the media attention focuses on incidents perpetrated by external attackers, organizations must alsoprotect themselves from internal attackers. The threat landscape for applications that are internal to an organization ismuch more difficult to report on in a comprehensive manner. Most organizations are not willing or able to share relevantinformation in a public forum in spite of repeated calls for information sharing by law enforcement agencies.18 Organiza-tions rightfully are concerned about the brand damage of reporting data breaches, particularly those caused by insiderscapable of exploiting vulnerabilities in back-office applications. Indeed the 2011 CyberSecurity Watch Survey19 reportedthat 46% of respondents considered insiders a more critical threat than external hackers.State of Non-Web Application SecurityFigure 29 shows the trends in the top vulnerability categories fornon-web applications over the last three Volumes. Cryptographic Cryptography remains theissues and directory traversal have remained the top two vulnerability top vulnerability category forcategories for the last three Volumes, affecting 47% and 38% of all non-web applications. 47% ofnon-web applications in the current reporting period. Information non-web applications containleakage (26%) takes the third spot from error handling, which drops at least one vulnerability into fourth place. Buffer overflow dropped out of the top ten for the this category.first time in this volume, and is replaced by SQL injection which isnow affecting 16% of non-web applications.The good news is that the percentage of applications with buffer management errors is declining, from 20% inVolume 3 to 13% in Volume 5. However, the rise in the percentage of applications containing information leakageand SQL injection vulnerabilities is disturbing since applications are the conduit through which attackers gain accessto confidential or proprietary information.It is noteworthy that the percentages reported in Figure 29 are generally lower than those reported in our softwaresupply chain feature supplement published in November 2012. For example, cryptographic issues affected 62% ofvendor supplied applications but only 47% of all applications (which include internally developed, outsourced, andopen source applications in addition to vendor supplied applications). The relatively higher percentages reported inthe supplement demonstrate the need for vendors to continue to work towards developing more secure software.In Volume 5 we corrected a mistake in how the past SoSS analysis logic was counting potential backdoors. Thecorrection accounts for the significant decline in the percentage of applications affected.18 www.fbi.gov/news/speeches/combating-threats-in-the-cyber-world-outsmarting-terrorists-hackers-and-spies19 www.cert.org/archive/pdf/CyberSecuritySurvey2011.pdf 31
    • Veracode State of Software Security Report: Volume 5Top Vulnerability Categories (Percentage of Affected Non-Web Application Builds)Rank Volume 3 Volume 4 Volume 5 1 53% 46% 47% Cryptographic Issues 2 36% 34% 38% Directory Traversal 3 28% 24% 26% Information Leakage 4 24% 23% 23% Error Handling 5 22% 19% 18% Time and State 6 20% 17% 17% OS Command Injection 7 19% 15% 16% SQL Injection 8 18% 15% 15% CRLF Injection 9 18% 14% 14% Credentials Management10 15% 13% 13% Buffer Management Errors12 11% 11% Buffer Overflow13 10% Numeric ErrorsFigure 29: Top Vulnerability Categories (Percentage of Affected Non-Web Application Builds)32
    • Veracode State of Software Security Report: Volume 5In Figure 2 we saw that 69% of non-web applications contain at least one flaw in a vulnerability category appearingon the 2011 CWE/SANS Top 25 list of most dangerous software errors. Figure 30 provides more details, byshowing the percentage of applications affected by vulnerability categories which appear on the CWE/SANS list.Vulnerabilities that appear on the CWE/SANS list are relatively easy to find and exploit for malicious actors withaccess to the installed software.Vulnerability Categories on the 2011 CWE/SANS Top 25 List (Percentage of Applications Affected) Cryptographic Issues 49% Directory Traversal 38% OS Command Injection 16% SQL Injection 16% Credentials Management 14%Buffer Management Errors 14% Buffer Overflow 11% Numeric Errors 9%Insufficient Input Validation 9% Dangerous Functions 7% Format String 3% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Figure 30: Vulnerability Categories on the 2011 CWE/SANS Top 25 List (Percentage of Applications Affected) 33
    • Veracode State of Software Security Report: Volume 5Appendix A: About the DatasetFigure 31 illustrates how the language and platform distribution has evolved sinceVolume 3. Overall, only small shifts in percentages occurred between Volume 4 and 5.Java and .NET applications continue to dominate, together representing 77% of theassessments conducted. C/C++ applications are holding steady at 9%. Mobileapplications have increased to 3%.Distribution by Language Apps Java .NET C/C++ ColdFusion PHP Mobile (Android + iOS + JavaME)Volume 3 52% 27% 12% 6% 2% 2%Volume 4 53% 26% 9% 9% 2% 3%Volume 5 50% 27% 9% 8% 3% 0% 20% 40% 60% 80% 100%Figure 31: Distribution by Language AppsAll applications analyzed by Veracode are classified according key characteristics such as whether the applicationis web-facing, whether it is developed for a mobile platform (Android, iOS), the remaining applications are typicallydeveloped to operate behind the firewall (although they can include web-based user interfaces). Figure 32 showsthat 73% of the applications assessed were web applications, which is slightly down from 75% in Volumes 3 and 4.Mobile application assessments are filling the gap, growing from <1% in Volume 4 to 3% in the Volume 5.Distribution of Web, Mobile and Other Applications 73% Web 24% Other 3% MobileFigure 32: Distribution of Web, Mobile and Other Applications34
    • Veracode State of Software Security Report: Volume 5Figure 33 and Figure 34 show that Java is the most popular language for both web (56%) and non-web applications(39%) tested by the Veracode platform. 28% of web applications and 29% of non-web applications were written in.NET. C/C++ accounted for 29% of non-web applications but only 2% of web applications.Distribution of Web Applications by Language Distribution of Non-Web Applications by Language 56% Java 39% Java 28% .NET 29% .NET 10% PHP 29% C/C++ 4% ColdFusion 3% PHP 2% C/C++Figure 33: Distribution of Web Applications by Language Figure 34: Distribution of Non-Web Applications by LanguageFigure 35 shows the platform distribution of mobile applications in our dataset, with iOS representing the largestshare (56%). At first glance, the relatively large percentage of applications developed on iOS may appear surprising,since one would expect that most enterprises and software vendors to create and test Android and iOS versionsof their mobile applications. Veracode’s partnership with Good Technology to test iOS applications for inclusion inenterprise app stores accounts for this imbalance.Distribution of Mobile Applications by Platform 56% iOS 34% Android 10% J2MEFigure 35: Distribution of Applications by Supplier 35
    • Veracode State of Software Security Report: Volume 5Figure 36 shows approximately 22% of the applications analyzed were identified as third-party (commercial, opensource and outsourced). The percentage of outsourced applications remains low at 1%. Oftentimes the outsourcednature of applications labeled “internally developed” is only revealed during remediation when an outsourced partyis assigned the task of fixing vulnerabilities. Hence we believe that the true percentage of “outsourced” code ishigher than represented.Distribution of Applications by Supplier 78% Internally Developed 14% Commercial 7% Open Source 1% OutsourcedFigure 36: Distribution of Applications by SupplierFigure 37 represents the distribution of applications by industry segment. Other and Finance represent the largestsegments at 38% and 27% respectively.Distribution of Applications by Industry 38% Other 27% Financial Services 9% Software 6% Technology 6% Telecommunications 4% Government 3% Business Services 3% Education 2% Healthcare 2% Entertainment and MediaFigure 37: Distribution of Applications by Industry36
    • Veracode State of Software Security Report: Volume 5Appendix B: Understanding How the VeracodePlatform Determines Policy ComplianceThe Veracode Platform automatically determines whether an application is compliantwith OWASP Top 10, CWE/SANS Top 25, the assigned Veracode Level and theassigned enterprise policy. Veracode looks for specific flaws enumerated by the CWElist and uses standardized mappings to determine whether a flaw belongs in the mostrecently published OWASP Top 10 and CWE/SANS Top 25 lists. Flaws discovered inan application are compared to the respective standards. If even a single flaw belongingto the standard is discovered, then the application is deemed out of compliance withthe standard. For our report, web applications are assessed against the OWASP Top 10while non-web applications are assessed against the CWE/SANS Top 25.Veracode Levels are Veracode’s independent standard for evaluating an application’s software quality. Veracode Levelsare assigned based on the business criticality of application. Each Veracode Level provides a predefined security policythat aligns with different levels of risk the organization is willing to accept for applications of varying business criticality.Figure 38 shows the five Veracode Levels aligned with the five business criticality levels as defined by the NationalInstitute of Standards and Technology (NIST). When an application is assigned a business criticality, the Veracodeplatform automatically assigns the appropriate Veracode Level as a predefined policy and determines whether theapplication is compliant with the Veracode Level. Each Veracode Level policy contains a combination of the severityof the flaws found in the application, the types of tests being performed on the application and the application’s overallsecurity quality score. An application is deemed compliant when all three aspects of the Veracode Level policy are met.Predefined Policy Requirements for Veracode Levels Predefined Policy Requirements Business Veracode Flaw Severities Not Required Test Minimum Security Criticality Levels Allowed in This Level Methodologies Quality Score Very High VL5 Very High, High, Medium Static and Manual 90 High VL4 Very High, High, Medium Static 80 Medium VL3 Very High, High Static 70 Low VL2 Very High Static or Dynamic or Manual 60 Very Low VL1 Not Applicable Static or Dynamic or Manual Not ApplicableFigure 38: Predefined Policy Requirements for Veracode Levels 37
    • Veracode State of Software Security Report: Volume 5Veracode provides enterprises several options for defining custom policies, including:• Disallowing specific flaw severities• Disallowing specific types of flaws (specified by CWE number)• Attaining a minimum security quality score• Using predefined policies such as OWASP Top 10, CWE/SANS Top 25 or industry standards such as PCI• Specifying application test methodologies and frequencies (e.g. monthly, quarterly, annually)• Meeting timelines for remediating specified flaw severitiesA custom enterprise policy may include any or all of these options. An application is deemed compliant when allaspects of the enterprise’s custom policy are met.Whisker Plot DefinitionA whisker plot (aka box and whisker plot) is a graphic used for exploratory data analysis. Created by the great dataanalyst John W. Tukey, a whisker plot can show the distribution of a dataset in one quick glance. As an example,let us look at the whisker plot in Figure 5. The figure shows nine distribution plots of Veracode Security QualityScore (SQS), one for the first application build scanned, one for the second application build scanned, and so on.Each application build is indicated by the numbers on the x-axis. Looking at the first application build, we see onewhisker (the dotted vertical line) and one box (the colored shape) occurring along each whisker.Veracode Security Quality Score by Build 100VERACODE SECURITY QUALITY SCORE 80 60 40 1 2 3 4 5 6 7 8 9 BUILD NUMBERFigure 5: Veracode Security Quality Score by BuildThe two horizontal lines at the bottom and top of the whisker reflect the minimum and maximum SQS observed forthe first build. For the first build, the maximum score is 100 and the minimum score is 27. Looking across all ninebuilds, we see that the maximum for each build is a perfect score of 100. The minimum SQS values creep up from27, to 31, to 33 for builds 1, 2, and 3. One might expect this as build 2 represents a resubmission of build 1 withseveral flaws remediated and similarly for builds 2 and 3.38
    • Veracode State of Software Security Report: Volume 5The box indicates values of the first quadrant, median, and third quadrant for all SQS values for the associated buildas we move from bottom to top of the figure. The narrow area in the middle of the box is called the “notch” whichis defined by values called notchLow, notchHi and the median. For example, the first build has a notchLow valueof 80 and a notchHi value of 81. The notch visually depicts values above and below the median that mark roughlya 95% confidence interval. The way to interpret this is to look across multiple whiskers for non-overlappingnotches. When two whiskers have non-overlapping notches one can say with 95% confidence that the two setsof observations have a different median. From the above figure one can conclude that improvement in the SQSvalues for application builds actually does improve from build one to two to three, with 95% confidence.Whisker plots also can visually reflect skewness in a distribution. For example, if the box area below the notch ismuch greater than the box area above the notch, then the distribution is skewed low. Build eight is skewed low—there is a wider range of SQS values in build eight below the median than above the median. There appears to belittle skew for builds one and two. Note also that the width of the box area is proportional to the number of applica-tion builds observed. As expected, there are many more SQS observations for build 1 than for later builds.P-Value DefinitionFor all of the quarterly trend graphs that are presented in this report, we also provide a p-value. In this context, thep-value can be viewed as a metric that is derived via a linear regression model. The p-value is the probability ofobserving a result at least as extreme as what we observed, given an assumption that the trend is flat.For example, in Figure 27, we observed what appears to be a decreasing trend in percent of web application buildswith cross-site scripting flaws for six quarters. A p-value of 0.441 indicates that the probably of seeing this in ourdata is over 44% when, in fact, the trend is flat—neither increasing nor decreasing. This means that we can continueto be hopeful that the trend is down and we can continue to monitor progress, but, speaking statistically, the datadoes not support any conclusion other than that the trend is flat.Quarterly Trend for Cross-Site Scripting (XSS) Prevalence (Percentage of Affected Web Applications)p-value = 0.441 100PERCENT AFFECTED WEB APP BUILDS 80 60 40 20 0 2011-1 2011-2 2011-3 2011-4 2012-1 2012-2 QUARTERSFigure 27: Quarterly Trend for Cross-Site Scripting (XSS) Prevalence (Percentage of Affected Web Applications) 39
    • Veracode State of Software Security Report: Volume 5Generalized Linear ModelWe used a generalized linear model (GLM) to evaluate the interactions between Volume, language, industry, andsupplier on the proportion of first builds that passed the CWE/SANS Top 25 compliance policy. This model leveragesthe following properties of our data:• The response variable is strictly bounded,• The variance is non-constant, and• The errors are non-normal.In addition, due to the large number of industry categories (see Figure 37: Distribution of Applications by Industry)with resulting low sample sizes in the model, we consolidated all categories other than Financial, Software, andGovernment into one category named Other. In particular, the new expanded Other category includes: Media &Entertainment, Health, Education, Business Services, Telecommunications, Technology, and Other.The data is proportional but the GLM for interactions using the binomial distribution for errors has dispersion factorsover 1.8 for all explanatory variables so we used the quasibinomial distribution for errors. Additionally, as is customary,we used the logit “link” function, namely the log-odds (log (p/q)) function as the transformation for raw proportionalobservations. The results of the GLM for analyzing the interaction between the Volume and Supplier factors on SANSCompliance Failure Rate for first builds (without any model simplification) are as follows:Call:glm (formula = y – Volume * Supplier, family = quasibinomial)Deviance Residuals: Min 1Q Median 3Q Max–9.6387771 0.2108186 0.3907429 0.7335999 4.5222136Coefficients: Estimate Std.Error t value Pr(>|t|)(Intercept) 1.98176746 0.19030085 10.41387 < 0.000000000000000222 ***Volume SOSSv5 1.48953329 0.37642676 3.95703 0.00008713 ***SupllierInternally Developed –0.04391537 0.23371916 –0.18790 0.8510345SupplierOpen Source 1.67665279 0.57449701 2.91847 0.0036796 **SupplierOutsourced 13.58430079 1005.61138499 0.01351 0.9892276VolumeSOSSv5:SupplierInternally Developed –0.04479984 0.43773916 –0.10234 0.9185258VolumeSOSSv5:SupplierOpen Source 0.44861486 1.36860963 0.32779 0.7432117VolumeSOSSv5:SupplierOutsourced –14.22238819 1005.61294547 –0.01414 0.9887217–––Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1(Dispersion parameter for quasibinomial family taken to be 2.864496721 Null deviance: 1133.05488 on 496 degree of freedom Residual deviance: 885.25567 on 489 degree of freedomAIC:NANumber of Fisher Scoring iterations: 14Performing model simplification, we conclude that there is no compelling evidence that different Supplier typesinfluence CWE/SANS Compliance Failure Rate. For details on both the implementation of the GLM calculationsas well as interpretation of the above output, we refer you to the Handbook of Statistical Analyses Using R(cran.r-project.org/web/packages/HSAUR/vignettes/Ch_logistic_regression_glm.pdf).40
    • Veracode, Inc. ABOUT VERACODE65 Network Drive Veracode is the only independent provider of cloud-based application intelligence and securityBurlington, MA 01803 verification services. The Veracode platform provides the fastest, most comprehensive solution to improve the security of internally developed, purchased or outsourced software applicationsTel +1.339.674.2500 and third-party components. By combining patented static, dynamic and manual testing, extensiveFax +1.339.674.2502 eLearning capabilities, and advanced application analytics, Veracode enables scalable, policy-drivenwww.veracode.com application risk management programs that help identify and eradicate numerous vulnerabilities by leveraging best-in-class technologies from vulnerability scanning to penetration testing and© 2013 Veracode, Inc. static code analysis. Veracode delivers unbiased proof of application security to stakeholders acrossAll rights reserved. All other the software supply chain while supporting independent audit and compliance requirements for allbrand names, product names, applications no matter how they are deployed, via the web, mobile or in the cloud. Veracode worksor trademarks belong to their with customers in more than 80 countries worldwide representing Global 2000 brands. For morerespective holders. information, visit www.veracode.com, follow on Twitter: @Veracode or read the Veracode Blog.SSSR /VOL5/US/0313