Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Risk Management Metrics That Matter


Published on

Security risk management metrics and methodology. Includes examples of good and bad security metrics using a risk-based approach.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Risk Management Metrics That Matter

  1. 1. Risk Management Metrics that Matter
  2. 2. Ed Bellis • Co-founder and CTO at Kenna Security, an automated risk & vulnerability intelligence platform • Orbitz CISO for 6 years • 20+ years Info Security experience including Bank of America, CSC, E&Y • Contributing Author Beautiful Security • Frequent speaker at events such as… About Me
  3. 3. Warning This presentation contains large amounts of data used for the purpose of proving an information security theory. No marketers were harmed during the making of this presentation.
  4. 4. You Are What You Measure
  6. 6. Inherent Risk Residual Risk Know & Measure the Difference vs. Hint: This is NOT a math formula
  7. 7. Inherent Risk: 80 Please Don’t Do This! Control Effectiveness: 50% X Residual Risk: 40
  9. 9. Do This Instead 1. Calculate Risk 2. Identify Potential Key Controls 3. ReCalculate Risk
  10. 10. The Language Barrier *source: Cyber Balance Sheet - The Cyentia Institute
  11. 11. The Language Barrier *source: Cyber Balance Sheet - The Cyentia Institute What the CISO perceives as important versus what the BoD believes is important often don’t match and often neither are actually given.
  12. 12. The Language Barrier *source: Cyber Balance Sheet - The Cyentia Institute
  13. 13. But First… Threats, Vulnerabilities & Risks.. oh my!
  14. 14. But First… Some Definitions Threat: A negative scenario you want to avoid. Threat Actor: the agent that makes the threat happen. Vulnerabilities: a weakness that can be exploited. Risk: a negative scenario you want to avoid combined with its probability & impact.
  15. 15. FAIR Example: Risk Taxonomy
  16. 16. Integrate or Die
  17. 17. Operationalizing Security Risk Management Measurement + Integration
  18. 18. Risk Management Decision Making
  19. 19. Selecting the Right Metrics for Risk Management Risks > Counts Results > Work Quantitative Where Possible
  20. 20. Know Your Assets Some Useful Metrics 1.External Asset Coverage 2.Internal Asset Coverage 3.Time to Discover
  21. 21. Know Your Business Some useful metrics here include: 1. System Susceptibility 1. Value to Attackers 2. Vulnerabilities 2. Time to Compromise: How long would it take to compromise any of the key controls for these assets and applications? 3. Threat Accessibility 1. Access Points and Attack Surface 4. Threat Actor Capability 1. Tools 2. Resources c. 3. Techniques Does Your Threat Model Include Alexa Ratings?
  22. 22. Know Your Risk Some Useful Metrics 1.Risk by Asset 2.Risk by Business Unit 3.Trending Risk over Time 4.Mean Time to Risk Reduction *use targets/goals and mature to SLAs
  23. 23. Know Your Resources Some Useful Metrics 1.Budget Spent on Security Remediation 2.Risk Carried Above Tolerance Level 3.Hours spent per Security Solution
  24. 24. Know Your Direction Some Useful Metrics 1.Risk Reduction by Group Over Time 2.Risk Goal/SLA by Group 3.Cumulative Risk Accepted Over Time
  25. 25. Some Not So Useful Metrics 1. Measuring Work AKA “atta boy metrics” Number of Vulnerabilities Closed Number of Patches Deployed Number of Incidents Responded to
  26. 26. Some Not So Useful Metrics 2. Measuring Counts “vanity metrics” Number of Packets Dropped Number of Malware Detections Number of IDS Alerts
  27. 27. Some Not So Useful Metrics 3. Averages can be a Fool’s Errand Average Age of Vulnerability Average Time to Discover Average Time to Respond Hint: Averages are skewed by outliers. Medians are your friend.
  28. 28. Aging Can Incent Wrong Behavior
  29. 29. Remember This?
  30. 30. Your Coworkers Have Day Jobs Too Leverage Existing Tools • Bug Trackers • Trouble Ticketing • Configuration Management • Continuous Integration & Deployment Bonus Points: Leverage Existing Tools for Security Purposes
  31. 31. Your Coworkers Have Day Jobs Too Leverage Existing Processes • Change Management • Bug Fixing • Design Reviews • QA Testing • Continuous Integration
  32. 32. The Payoff Operationalizing Security Risk Management Security Teams Operations Teams Development Teams Executive Management Common Language Distinct Objectives Efficiency Effectiveness
  33. 33. References FAIR Risk Taxonomy: Cyber Balance Sheet: Risk Management Metrics That Matter: 2017/03/creating-risk-management-metrics-that-matter/
  34. 34. Q&A