Your SlideShare is downloading. ×

Business cases for software security


Published on

OWASP day 4 presentation in Milan Italy on November 6th 2009

OWASP day 4 presentation in Milan Italy on November 6th 2009

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Because of the current economic recession, security managers have to deal with shrinking budgets and limited resources dedicated to application security and often asked to present the “business cases” for application security as well as software security initiatives. Gartner Survey Shows Security Software and Services Budgets to Increase 4 Per Cent in 2010 Analysts Discuss Key Security Issues During Gartner Information Security Summit, 21-22 September in London Egham, UK, September 8, 2009 — Security software and services spending will outpace other IT spending areas in 2010, according to a new survey by Gartner, Inc. Security software budgets are expected to grow by approximately 4 per cent in 2010, outpacing all other areas of infrastructure software. Security services budgets are projected to grow almost 3 per cent, significantly outperforming other service areas. In April and May of 2009, Gartner surveyed more than 1,000 IT professionals with budget responsibility worldwide to determine their budget-planning expectations for 2010. “ In the current highly uncertain economic environment, with overall IT budgets shrinking, even the modest spending increases indicated by the survey show that security spending accounts for a higher percentage of the IT budget,” said Adam Hils, principal research analyst at Gartner. “Security decision makers should work to allocate limited budgets based on enterprise-specific security needs and risk assessments.” Specific areas of projected security-related software spending growth in 2010 includes security information and event management (SIEM), e-mail security, URL filtering, and user provisioning. The continued, comparatively strong emphasis on security extends beyond software. The survey showed that security services spending will also outpace spending in other services areas, with budgets expected to grow 2.74 per cent in 2010. This anticipated increase is being driven in part by a growing movement towards managed security services, cloud-based e-mail/web security solutions, and third-party compliance-related consulting and vulnerability audits and scans.  “ When evaluating and planning 2010 security budgets, organisations should work to achieve a realistic view of current spending and recognise that it may be impossible to capture all security-related spending because of organisationally diffused security budgets,” said Ruggero Contu, principal research analyst at Gartner. “Businesses should also recognise that new threats or vulnerabilities may require security spending that exceeds the amounts allocated, and should consider setting aside up to 15 per cent of the IT security budget to address the potential risks and impact of such unforeseen issues.” Additional detail is available in the Gartner report “ Security Software and Services Spending Will Outpace Other IT Spending Areas in 2010 ." The report is available on Gartner’s website at . Key issues facing the security industry will be discussed during the Gartner Information Security Summit, taking place 21 -22 September, at the Royal Lancaster hotel in London. The Summit hits the critical spot between strategic planning and tactical advice. Gartner analysts, industry experts and IT security practitioners will deliver unbiased, realistic analysis of the current state of information security, as well as an independent vision of how things will evolve over the long term. For complete event details, please visit the Gartner IT Security Summit website at . Members of the media can register by contacting Holly Stevens at [email_address] .
  • # 1 per creare il caso e’ creare awareness come ad esempio le conference OWASP di questo tipo. E’ di importanza critica definere il problema per esempio spiegare ad executive management cosa si intende per secure software security engineering e come sia diversdo da application security e perimiter security for example. #2 preparazione per le FAQs come e perche e dove devo spendere il mio budget in software security?
  • Maybe compliance is #1, analysts opinions and surverys #2 , incidents #3 and cost savings #4 dal punto di vista della mia esperienza
  • Anche tenendo conto che I piu grandi incidenti di perdita di dati sensibili nella storia degli USA sono dovuti a problem di application security Heartland Payment Systems (HPY) data breach: 130 Credit Card Accounts Exploited SQL Injection vulnerabilities to install malware (Hannaford 9/07, Company A & B on 1/08) Used “wardriving” and installed sniffers Acted ad member of a Cybercrime gang Profited from the sale of ACC#, PINs to fake credit cards and commit ATM fraud Engaged in money laundering
  • Come mostrato in questo caso… In same cases the cyber attack vectors are reported step by step Use "xp_cmdshell“ to download hacker tools to the compromised MSSQL server. Obtain valid Windows credentials by using fgdump Install network "sniffers" to identify card data and systems involved in processing credit card transactions. Install backdoors that "beacon" periodically to their command and control servers Target databases, Hardware Security Modules (HSMs), and processing applications in an effort to obtain credit card data or brute-force ATM PINs. Use WinRAR to compress the information they pilfer from the compromised networks.
  • Dal punto di vista dei costi, quantificazione di quanto cost gestire I dfetti del software e’ essenziale per il business cases
  • L’opinione degli analisti e’ che la maggioranza delle vulnerabilita’ e degli incidenti succede a livello delle applicazioni, in realta dipende dal tipo di incidenti. Dal punto di vista dei cost studi dimostrano che c’e significativamente una riduzione dei cost ed un ROI nel risolvere le issues early in the SDLC da un fattore 100X requirements vs. field test ad un fattore 10 X fra coding e test
  • I modelli di maturity permetttono di usare un metodo standard per ottenere una scorecard o pagella della maturita’ dei processi e delle discipline e a che livello siano riaggiunte. La metrica provvede un livello fo visiabilita’ reale e una misura delle cause, trends e bisogni
  • A parte la maturity un buon caso da usare e’ la metrica Defect management metrics Where the issue is found? (e.g. requirements, design, code, application) When has been fixed? (e.g. found in coding fixed before testing) How has been fixed (e.g. design change, coding, configuration) Vulnerability management metrics Which type of vulnerabilities are found the most during design review, code analysis, pen tests? Which security vulnerabilities we having trouble fixing?
  • E per finire I business cases debbono trovare supporto da punto di vista degli sponsors e di chi usa software security secondo diversi obbiettivi
  • Dal circa un anno o due abbiamo la fortuna grazie a Pravir Chandra e Gary McGraw due modelli per la software security maturity. I modelli si basano su survey di casi reali empirici do come varie organizzazioni implementano software security initiatives (35 prese in esame nel caso di BSIMM e 9 ditte di diversi settori Google, MS, Adobe e telco come qualcomm e anche DTCC , Well Fargo banks etc) I modelli sono organizzati per domini BSMIMM, business functions nel caso di SAMM in tutto molto simili anche in numero 12. I livelli di maturity sono da 1 a 3 e varie attivita per obbiettivi e livelli
  • Un esempio di come siano utili a parte la pagella ci danno la mappa degli obbitettivi e delle attivita da raggiungere.
  • Un fatto importante della maturity e’ la learning curve, lo sviluppo non e’ lineare ma accelerato dall’ apprendimento e in alcune fasi e’ piu difficile di altre come il passaggio dal livello 3 a 4 (defined a managed) questo ha importanza sul planning.
  • CBA costi vs benefici il beneficio ne deve valere la pena come si dice…. La formula e’ semplice ed un fattore < 1 significa I benefici sono maggiori dei costi quindi se i nebefici sono I risparmi in costi per il defect management e incidenti vs I costi per processi, people, ed il fattore e’ <1 si puo dire che ne valga la candeal Il problema pero non e’ semplice da evalutate questi costi
  • Alcuni exempi di costi di assunzione del software sicuro includono il costi per lo svulippo training e implementazione mentre I costi di incidenti includono I costi di sviuppo delle patches, pen testing e I costi degli exploits/incidenti che sono I piu difficili da quantificare
  • Se per esempio prendiamo in considerazione il cost delle contromisure vs il costo dei danni dovuti ad un exploit di una vulnerabilita come anche I costi per le patches etc vediamo che man mano che le spese per la rimediazione dei danni aumentano I costi dovute alle perdite diminuiscono. Un esempio di come questi dati possano essere usati per esempio e che se le mei perdite grossolane dovure a frodi via banking on line channel sono 3 millioni di euro un millione di euro in spese in programmi si software security e’ giustificabile. Most organization's directors of technology and security try to sell technologies and new initiatives to high management with the sales pitch of getting the most "Bang For The Buck" BFTB. But the BFTB business case need to answer the basic question: if I spend that much on a security technology or process what is the benefit for security ? In technical terms this means doing a Cost vs Benefit Analysis (CBA). CBA can be used in security to correlate the total cost of security to increased information or software security assurance. Dan Geer covers well this analysis as related to data security in his book " Economics and Strategies of Data Security ". By analogy, in the case of software security, "Bang For The Buck" decision spending need to take into account the total security costs that is all failure costs such as the total cost of failing as business impact as well the total cost of finding, fixing, testing and deploying the security defect. Only then, the total cost of software security (the BUCK) can be compared against the BANG that is the increased level of software security assurance. The general law for bang for the buck is that you are getting the most bang for the buck when your total costs (cost of failure/data loss/fraud + cost of security/countermeasure) reaches a minimum. As failure costs decrease exponentially, the "anticipation costs" that you take proactively by spending in software security initiatives will increase. From risk management perspective this means that the total software security costs decrease up to a minimum to raise again as you keep spending on security measures. You will reach an optimal cost vs. benefit where more spending in anticipation cost (e.g. countermeasures) will not provide the most benefit (e.g. spending more in countermeasures then the value of the assets that the countermeasure is supposed to protect) This optimal spending for anticipation costs (proactive countermeasures) is about 40% of your failure costs (to be exact 37% according to Gordon and Loeb research: The Economics of Information Security Investment ). According to the empirical law of the most bang for the buck applied to software failure costs vs. assumption costs it is therefore fair to assume that you get the most of security by spending 37% of what your software failure costs are. Assume for example software security failures costs are $ 10 ML it would be reasonable for an organization to spend as much as $ 3.7 ML in acquiring software security tools and technology, develop new software security process as well as in new software security training activities.
  • Per un business case piu significativo e’ quantificare le perdite come ONERI per ogni perdita di dati sensibili
  • E’ possibile correlare tali perdite a dati statistici sugli attachi
  • E sulle vulnerabilita.
  • Usando questi dati e’ possibile stimare per esempio il costo in termini di impatto di una perdita di dati sensibilie (carte di credito) via SQL injection A similar math can be factored using Van Geer data of 4.5% of data loss probability (FTC 2003 data) x 655 $/person cost Can we correlate this losses somehow, the cost for loss with the fact that among all breaches 13 % are from wbe and 19% use SQL injection attack vectors you can come out with 0.025 that is 2.5 % as the probability that an identify theft will occurr through the web channel because of a SQL injection attack. The cost is the cost per incident x record loss for internal (pokemon is 200 $) or you can just factor the cost per re-issance of the card This multipled for the cost per incident/records you can came out with 241 ML for 14 million records for 130 ML is 2.5 BILLION According to the case in NJ SQL injection is considered cause of Hannaford Assume that according that 2003 FTC data the potential loss per identity theft incident is $ 655 per incident. Assume you are serving via your web site a population of 4 million customers, the potential loss of losing your customer data such as credit card accounts for example would be of $ 2,6 Billion and with probability of identity theft occurrence of 4.6 % (also FTC data) the projected loss for your company could be $ 120 ML for which 14% or $ 16 ML would be the cost of data losses via the web channel alone. Assume that according that 2003 FTC data the potential loss per identity theft incident is $ 655 per incident. Assume you are serving via your web site a population of 4 million customers, the potential loss of losing your customer data such as credit card accounts for example would be of $ 2,6 Billion and with probability of identity theft occurrence of 4.6 % (also FTC data) the projected loss for your company could be $ 120 ML for which 14% or $ 16 ML would be the cost of data losses via the web channel alone. With these assumptions based upon publicly disclosed data losses, a security program (that include both information and application security) that cost as much as $ 16 ML would be justified for a company with a customer base of 4 million on-line customers for example.
  • Come numeri siamo vicini….
  • Un fattore che non si stima sono le perdite indirette intangibili per esempio sulle azioni in conseguenza alla pubblicazione di un incidente si tenga presente che nel caso di HPY l’exploit citiato nei procdeeednigns rigrairda SQL inection che e’ diretta consegienza di unsecure software
  • Un altra metodologia consiste nel valutare le perdite annue come fattore di esposizione alle perdite all;occorenza dell’ evento e dell’impatto dell evento (SLE)
  • Nel caso in esame e’ possibile stimare SLE e ALE per SQL injection tenendo conto della probabilita stimata 2.5 % di un evento per un sito che serve 3 millioni di utenti A quantitative risk assessment can also be used to determine the extent on which a software security initiative can reduce risk from potential losses. The correlation has to take into account the probability of the event and the loss that the event can cause. This is difficult to quantify in general for software security issues since assumes a cause-effect between vulnerability exploit and financial impact. Nevertheless it can be used for rough estimates. Assume a web application that delivers banking services for example and that the loss caused by an event such as denial of service impact on-line transactions for 3 million customers with an average of $ 20 per transaction: the loss per single DOS event (SLE) is $ 60 ML. Assume that the probability that a new SQL injection vulnerability would cause a denial of service is 30% (Annualized Rate of Occurrence) then the Annual Loss Expected (ALE) is $ 1.8 ML. If the cost of the new security countermeasures that will stop the security incident is less than $ 1.8 M than the organization should implement it.
  • Un altro fattore molto discusso a livello di esecutives e’ il ROSI che quantifica in termini di investimento in sicurezza il ritorno come risparmio. Secondo studi e’ possibile stimare il risparmio delle attivita di security engineering nelle diverse fasi per esempio
  • Un altra stima usando la formula di risparmi sul impatto delle perdite dopo che la soluzione di secure software e stata addottata in riferimento al cost della soluzione stessa. Un ROSI negativo non e’ giustificabile mentre uno positivo lo e’ wuanto sia giustificabile ha valore comparativo rispetto a diverse attivita.
  • Da punto di vista dei busienss cases un metrica in supporto e la PAGELLA BILANCIATA il cui goal e’ allineare software security ad altri obbiettivi aziendali come finanza clienti, processi e capacita interne di formazione etc. Alcuni obbiettivi di questa metrica scorecard sono mostrati qui It is used by companies to assess performance of a target process against factors that are outside the traditional time and effort costing. There are different views: the financial view tries to answer the question, to succeed financially, how should we appear to our shareholders Background The balanced scorecard is a strategic planning and management system that is used extensively in business and industry, government, and nonprofit organizations worldwide to align business activities to the vision and strategy of the organization, improve internal and external communications, and monitor organization performance against strategic goals. It was originated by Drs. Robert Kaplan (Harvard Business School) and David Norton as a performance measurement framework that added strategic non-financial performance measures to traditional financial metrics to give managers and executives a more 'balanced' view of organizational performance. 
  • Per ogni area ci sono tre metriche: costi tecnici di software security engineering, costi di trends costi relativi al budget allocato etc
  • In analisi se si ha una methologia si puo’ vendere il giaccio agli eskimesi
  • I plan to start at 2.30 and finish at 3 PM
  • most of software is tested after being build
  • Transcript

    • 1. How to Create a Business Case for Software Security Initiatives Marco Morana OWASP Lead TISO Citigroup
    • 2. Status Quo of Software Security Spending
    • 3. Making the Business Cases: Essentials
      • Secure Software Engineering Awareness
        • “ Security involves making sure things work, not in the presence of random faults, but in the face of an intelligent and malicious adversary trying to ensure that things will fail in the worst possible way at the worst possible time … again and again”
      • Prepare for Executive Management FAQs:
        • Why I spend money on software security?
        • How much I should spend ?
        • What my competitors are doing?
        • How I am doing at my vulnerabilities?
        • How I get the most bang for the buck ?
    • 4. Main Factors Driving Software Security Adoption ?
    • 5. Lessons From the Court Room 170 million card and ATM numbers used sql injection and packet sniffers
    • 6. Lessons From Law Enforcement (FBI)
      • Attack “xp_cmdshell on MSQL server to upload sniffers to capture CC transactions and ATM PINs from DB, HSM
      • Disable xp_cmdshell,
      • Deny extended URL,
      • escape special characters such as “”,
      • Use store procedures,
      • Run SQL Server and IIS under non-privilege,
      • Do not use “sa” hardcoded,
      • Lock account on mainframes against brute force
      • Use minimum privileges on AD/SQL server, restrict access
      • Use proxy server for internet access,
      • Implement firewall rules
      • Ensure HSM do not take commands with PIN in the clear
    • 7. Defect Management/Costs Measurements
      • Process Metrics
      • Is code validated against security coding standards?
      • Is design
      • of developers trained, using organizational security best practice technology, architecture and processes
      • Management Metrics
      • % of applications rated “business-critical” that have been security tested
      • % of projects that where developed with the SDL
      • % of security issues identified by lifecycle phase
      • % of issues whose risk has been accepted
      • % of security issues being fixed
      • Average time to correct vulnerabilities
      • Business impact of critical security incidents.
      Most of my vulnerabilities are coding and design issues But are mostly found during pen test in UAT The cost of fixing them in UAT is 10 X during coding (unit tests)
    • 8. Analysts/Researchers Opinions
      • “ 75% of security breaches happen at the application layer”- Gartner
      • “ Over 70 % of security vulnerabilities exist at the application layer, not the network layer” – Gartner
      • “ If only 50 percent of software vulnerabilities were removed prior to production … costs would be reduced by 75 percent ” - Gartner
      • “ Correction of security flaws at the requirement level is up to 100 times less the cost of correction of security flaws in fielded software” – Fortify
    • 9. Why Using Metrics And Maturity Models?
      • Use vulnerability metrics to articulate software security needs/opportunities
        • Point to software security root causes
        • Identify vulnerability trends
        • Analyze needs for improvements
        • Use maturity models to provide visibility on the organization’s security capabilities
        • Assess organization capability levels
        • Set goals and needs to reach the goals
        • Provide the roadmap
    • 10. Vulnerability Taxonomies and Trends Am I getting better ? Where ?
    • 11. Software Security Metrics Business Cases
      • Business Managers : shows that projects are on schedule and moving on target and testing cycles for vulnerabilities are shorter translating in cost savings
      • Information Security Officers: show that we are getting better on reporting compliance and manage risk reduction
      • Developer Leads : show that developers are getting better to write secure software when provided with secure coding training and tools
    • 12. Software Security Maturity Models: SAMM, BSIMM
        • Source
        • Source SAMM :
    • 13. Activities, Objectives and Capability Levels Use this as a yardstick to compare software security practices with other organizations
        • Source BSIMM
    • 14. The Software Security Maturity Curve (CMM) Highest effort/cost is required here
    • 15. Cost vs. Benefit Analysis (CBA)
        • Purpose is to weight the cost of software security initiative vs. the benefits
        • CBRatio = COST of initiative
        • BENEFIT of initiative
        • Need to cost quantify factors and compare them (to compare apples with apples) for example:
          • COSTs:
            • Secure software engineering costs for training, new processes and tools
          • BENEFITs:
            • Reduced costs in fixing with patching, lessen business impact of exploits
    • 16. Assumption Costs and Failure Costs of the Software Security Initiative
      • Assumption Costs (proactive):
        • Cost of acquiring tools, standards and processes to develop secure software
        • Cost of hiring and/or training a software security team
        • Costs for implement security features (e.g. estimate possible as function of KLOC)
      • Failure Costs (reactive):
        • Cost of develop and/or deploy patches
        • Cost of incident response
        • Cost of vulnerability exploits resulting in data breach, fraud, denial of service, quantifiable damage to the organization
      The most difficult to estimate
    • 17. Assumption Costs vs. Failure Costs No countermeasures, cost of failure (e.g. data breach is high) Cost is low but CBRatio >1 CBRatio <1 but room for improvement Optimal Spending, around 37% (Gordon and Loeb)
    • 18. Data Loss Liabilities Estimates
      • Consider FTC data (2003)
        • 4.6 % of US population suffered identity fraud
        • Companies spent 3 * 10^8 hours repairing the troubles caused + $ 5 Billion dollar spent out of pocket
        • Minimum wage of 5.15 $/hr (in 2003)
        • 10 Million people involved
          • P = 4.6 %
          • L = 3 x 10^8 x $ 5.15/hr + $ 5 * 10^9 = $ 655/victim
            • 10^7 victims
      • My annual liability (P X L) for each data theft victim is $ 30.11
      SOURCE: Dan. E. Geer, Economics & Strategies of Data Security
    • 19. Data Losses As Web Breaches ( SOURCE: Open Security Foundation Data Loss Statistics
    • 20. Which Vulnerabilities Are Exploited? (WHID) SOURCE: Breach Security The WHID 2009, August 2009
    • 21. Estimating SQL Injection Attack Liability
      • Probability of attack by type and attack vector incident (identity theft) data :
        • 13 % of incidents involve breaches of web channel (
        • 19 % of incidents use SQL injection as attack vector (WHID)
        • P = 0.13 x 0.19 = 0.025 (2.5 %)
      • Estimate data loss for this attack:
        • $ 655 per identity theft victim (2003 FTC data)
        • 94 million individual records stolen (TJX incident)
        • L = 94 X 10^6 x 0.025 x 655 = $ 1.5 Billion
    • 22. Reporting of Losses in Quarterly Earnings
      • The cyber attack on the retailer Marshalls and TJ Maxx (94 Million CCN reported in Jan/2007): after-tax cash charge of approximately $118 million , or $.25 per share.
      • The company increased its estimate of pre-tax charges for the compromise to nearly $216 million .
      • According to some experts, TJX may have to spend in the end a total between $ 500 Million to $ 1 Billion (, including non compliance fees (e.g. PCI-DSS) litigation fees and government fines.
    • 23. Another Way to Look at Business Impact Of Data Breaches : Drop in Stock Price
          • 130 ML CCN loss
          • (reported January 20 2009)
    • 24. Quantitative Risk Analysis
      • Goal:
        • Justify spending to improve security by assigning an objective monetary value to risk
      • Risk Analysis Methodology:
        • Determine the Exposure Factor: Percentage of asset loss caused by identified threat (e.g. 20%)
        • Determine Single Loss Expectancy (SLE):
        • EF x the value of assets (e.g. $ 1 ML * 30%= $ 200 K)
        • Estimate Annualized Rate of Occurrence (ARO): twice in ten years 2/10=0.2, 1 every year =1
        • Determine the Annualized Loss Expectancy (ALE) ALE = SLE x ARO = $ 40 K
    • 25. Use Quantitative Risk Analysis to Estimate Annual Loss Due to SQL Injection Exploit
      • Exposure Factor (likelihood) of data loss via SQL injection attack: 2.5%
        • Based upon datalossdb and WHID calculated probability data)
      • SLE (EF x Value Assets) : $ 43 Million
        • Asset Value: assume SQL injection attack will cause fraud for 3 million credit card accounts (on-line web site for major bank) at a 580 $/account (use SANS data)
      • ARO : 40 % (four every 10 years)
      • ALE (ALO X SLE) : = $ 17 Million
    • 26. ROSI Of Secure Software Initiatives
      • ROSI (Return Of Security Investment)
        • ROSI = Savings (Avoided loss) /Total Cost Of Solution
      • Goal:
        • Answer the question on how much I can save by investing in Software Security
      • According to previous studies (Soo Hoo-IBM):
        • For every 100,000 $ spent in software security I save:
          • $21,000 (21%) when defects are fixed and identified during design
          • $15,000 (15%) when defects are fixed during implementation
          • $12,000 (12%) when defects are fixed during testing
    • 27. Using ROSI to justify software security investments
      • ROSI = [( ALE X % Risk Mitigation)- SCost] SCost
      • Calculation example:
        • ALE : $ 17 Million, risk exposure for SQL injection
        • Risk Mitigation: 75 % of risk mitigated by software security solution source code analysis, filtering
        • SCost: $ 4 Million, Total Cost of Ownership (TCO) software security solution
        • Savings = $ 8.75 Million, loss prevention savings
        • ROSI = 210 %
          • N egative = investment not justifiable
          • Null = no return on investment
          • Positive = justifiable as compared with other solutions
    • 28. Security Software Assurance Metrics: Balanced Scorecards
          • Correlation of budget with risk assessment and cost/benefits
          • Improved results of software security processes and
          • operations
          • Reduced calls to CSR for reporting on security issues
          • Growth in assessed security processes & training activities
    • 29. Software Security Metrics In Support Of Business Cases
      • Metrics of technical value
        • Costs for testing and fixing vulnerabilities
        • Percent security requirements satisfied
        • Percent developers with software sec. certifications
      • Metrics of comparative value
        • TCO of software security activities vs. unit revenue
        • Secure software engineering costs vs. patching costs
      • Metrics of business value
        • Estimate for vulnerability & risk assessment costs
        • Budget to address gaps in software sec. processes
        • Costs for security certifications per business unit
    • 30. Come on is not so hard..
    • 31. In Summary
      • Rationale For Software Security Business Case
      • Preparing the Business Case
        • Maturity Models
        • Metrics and Measurements
      • Making the Business Case
        • Software Security Assurance Awareness
        • Failure Costs vs. Assumption Costs
        • Qualitative Risk Assessments
        • Return Of Security Investment (ROSI)
        • Performance Measurement Metrics
      • Questions & Answers
    • 32. Thanks for listening, further references
      • Applied Software Measurement: Assuring Productivity and Quality
      • PCI-Data Security Standard (PCI DSS)
      • A CISO’s Guide to Application Security
      • PCI
      • ROSI
      • Gartner study on phising: )
      • UC Berkeley Center for Law and Technology on identity theft
    • 33. Further references con’t
      • Gartner 2004 Press Release
      • Making The Business Case For Software Assurance
      • SEI Capability Maturity Model Integration CMMi
    • 34. Further references con’t
      • Software Assurance Maturity Model
      • The Software Security Framework (SSF)
      • National Information Assurance Glossary
      • Dan. E. Geer, Economics & Strategies of Data Security
    • 35. Further references con’t
      • Open Security Foundation, Data Loss Statistics
      • The WHID 2009 BI-Annual Report, August 2009
      • Quantitative Risk Analysis Step-By-Step
      • Breach Worse Than Reported..
    • 36. Further references con’t
      • Estimating Benefits from Investing in Secure Software Development
      • Return On Security Investment (ROSI)
      • Models for Assessing the Cost and Value of Software Assurance