Old Presentation on Security Metrics 2005

Slide 1: Metrics: Optimizing Security Operations Performance Dr Anton Chuvakin Chief Security Strategist November 2005

Slide 2: WARNING! This is an old presentation; I am publishing it in the hopes that it will be useful for somebody. I do not necessarily share all the views I held back in 2005! And I ramble less now 

Slide 3: Agenda • Catalysts for security performance metrics • Why security metrics are essential • Types of metrics and what to measure • Challenges surrounding metrics • The metrics lifecycle

Slide 4: Catalysts For Security Performance Metrics • Enterprise compliance and governance initiatives • Need to manage security as a strategic business process • Pressure to demonstrate the efficiency of security technology and related efforts

Slide 5: Managing Information Security Strategically • Compliance has shifted mindset from devices to process • Focus on continuous optimization – Conduct security assessment – Define policies, process, metrics, and baselines – Collect data; measure results against baselines over time – Identify process and policy weaknesses – Refine policy, process, and metrics

Slide 6: Creating a Culture of Measurement: Recommendations • Define security policies and relevant metrics via security assessment • Capture all relevant security operational data • Generate reports off the data to measure success and identify performance gaps • Integrate security performance management data with enterprise compliance & governance initiatives

Slide 7: Why Security Metrics? “You can’t manage what you can’t measure” You need to know if… • your security strategies are successful • you require more resources • there are in-efficiencies • you are compliant

Slide 8: Types of Metrics • Technical eg: Percentage of systems with virus protection • Process eg: Average virus incident resolution time • Risk eg: Risk of viruses for a specific business unit

Slide 9: Criteria for Good Metrics When determining the metrics that you will track, be SMART… Specific Measurable Attainable Repeatable Time-dependent And… your results need to be ACTIONABLE !

Slide 10: Metrics and Baselines What is the difference? Baselines are metrics that are compared to aggregated information about the past • Common baselines – Yesterday (“NASDAQ is up by a 100”) – Average (“This took us 3 more time than the average case”) – Maximum (“The traffic is spiking 300% over the maximum!”) • Baselines enjoy the ease of interpretation!

Slide 11: Perceived Effectiveness of Metrics Source: CSO Magazine, 2005

Slide 12: Key Challenges with Security Metrics • “Let’s make up some metrics…” – Metrics must be based on policy to be effective • Where is the data? – Difficulties in acquiring raw data for measurement • So many metrics, so little time… – Which indicators should we compute and use? • What does it mean? – Interpreting the changes in measures parameters

Slide 13: Future of Security Metrics Metrics and Crystal Ball Gazing Don’t Mix, but… • “Information Security Management Metrics and Measurement” (ISO 27004) in 2007-2009 • Increased adoption of NIST 800-55 • Legal and compliance drivers for standard metrics • Growth of “best practices”-based metrics • Convergence of security vendor metrics • More automation of metrics

Slide 14: Conclusion • Security needs to be managed as a strategic business process (bla-bla-bla ) • Enterprise compliance and governance initiatives are driving security performance metrics • Establishing a security metrics program is essential to determining if – your security strategies are successful – you are compliant • In the future metrics will be based on standards

Slide 15: Thanks for Viewing Dr Anton Chuvakin http://www.chuvakin.org Also see my blog at www.securitywarrior.org