Security Metrics [2008]


Published on

An introduction and overview presentation I gave in 2008 to the Northern UK Security Group in Leeds.

Published in: Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Security Metrics [2008]

  1. 1. 14/07/2008Security MetricsPhil Huggins
  2. 2. 14/07/2008 Security MetricsPage 2Core TextSecurity Metrics : Replacing Fear, Uncertainty and DoubtAndy Jaquith, 20070-321-34998-9
  3. 3. 14/07/2008 Security MetricsPage 3Recommended Texts
  4. 4. 14/07/2008 Security MetricsPage 4Growing field► Areas of interest► Software security► Modelling► Benchmarking► Return on investment► Breach data► Standards► ISO / IEC 27004► NIST SP800-55► Communities►►► Cybersecurity KTN
  5. 5. 14/07/2008 Security MetricsPage► Open mailing list and wiki► Active community► Established by Andy Jaquith► Runs the US based Metricon and MiniMetricon eventseach year
  6. 6. 14/07/2008 Security MetricsPage► New open community group► Established by Elizabeth Nichols► Sharing metrics definitions, learning and data► Early days, big ideas
  7. 7. 14/07/2008 Security MetricsPage 7Cybersecurity KTN – Metrics SIG► UK Knowledge Trading Networks established by DTI► Promoting collaboration between industry, academia andgovernment► Metrics Special Interest Group has focused on thedelivery of the Internet Threat Exposure (ITE) Index► Threat and Countermeasure focused metric of exposure► Appears to be aimed at less sophisticated securitypractitioners► Risk assessment-lite ?► Currently being developed in an open group
  8. 8. 14/07/2008 Security MetricsPage 8Standards► NIST SP800 – 55► Exhaustive list of possible security metrics to measure► 99 pages► No real sense of what is a useful metric► Defines useful characteristics to describe a metric► Performance Goal, Performance Objective, Metric, Purpose,Implementation Evidence, Frequency, Formula, Data Source,Indicator► ISO/IEC 27004► Currently in draft / closed group► Metrics covering the performance of an ISMS as defined in 27001and 27002
  9. 9. 14/07/2008 Security MetricsPage 9Types of Security Metrics► Risk Metrics► Compliance Metrics► Operational Metrics► Quality Metrics► Management Metrics► Business Metrics► Confusion among practitioners
  10. 10. 14/07/2008 Security MetricsPage 10Focus problems► Technical Focus► “What do we count?”► Business Focus► “What do we need to do and why?”► Counting is the mechanical foundation► The business wants the story the numbers tell► Metrics are not the answer to funding problems
  11. 11. 14/07/2008 Security MetricsPage 11Other common problems► Managing to the metric► No longer focused on the result► Measuring emerging threats► Measuring last years breaches
  12. 12. 14/07/2008 Security MetricsPage 12Questions from the board► Am I safe?► Can I take responsibility for the actions of my company?► Who handles my data?► Who am I doing business with?► Are they accountable?
  13. 13. 14/07/2008 Security MetricsPage 13Metricon practitioners top 10 metrics► Data volumes transmitted to competition► Coverage metrics► Availability of business systems► End user perception of security► Legal fees paid out► Total cost of information security► Information asset value► Count of events on systems► Security control success rate► Cost of security monitoring and reporting
  14. 14. 14/07/2008 Security MetricsPage 14Balanced Security Scorecards► Complete:► People, Process, Technology, Budgeting, Innovation,Organisational Planning, Operations► Traditionally include four primary perspectives:► Financial► Customer► Internal Processes► Learning and Growth► Jaquith has a comprehensive chapter on balancedsecurity scorecards in his book
  15. 15. 14/07/2008 Security MetricsPage 15Geer’s Scorecard► Finance► Cost of data security per transaction► Downtimes lost to attack by attack class► Data flow per transaction and source► Budget correlation with risk measures► Process► % of critical systems under a DR plan► % of critical systems obeying the security policy► MTBF & MTTR for security incidents► Frequency of security team internal consultations► Latency to obey security change orders by department
  16. 16. 14/07/2008 Security MetricsPage 16Geer’s Scorecard► Learning and growth► % of job reviews involving security► % of security workers with training► Ratio of B.U. security staff to central security staff► Timely new system security consultations► % of programs with budgeted security► Customer► % of SLAs with security standards► % of tested external facing applications► Number of non-employees with access► % of data secure by default► % of customer data outside the data centre
  17. 17. 14/07/2008 Security MetricsPage 17GE Global experience► Metrics to drive behaviour► Scorecard approach► Business unit drill down and comparison views► Communication plan was key► Built a custom system piecemeal over several years► Started with manual data, automated over time► Now moving to a common platform► Monolithic vs Composite data sources► Centralised vs Business unit data sources
  18. 18. 14/07/2008 Security MetricsPage 18Dept of Veterans Affairs’ experience► Didn‟t have common definitions of:► What IT Security was► What better IT security looked like► The value of security► Identified the security events that drove perception ofsecurity► Focused on the frequency and impact of those events► Did not ignore uncertainty!► Results-focused
  19. 19. 14/07/2008 Security MetricsPage 19Intel’s experience► Developed predictive model for future security incidents► Used to provide ROI on „reduce the occurrence‟ controlsNOT „reduce the effect‟ controls► Needed to gather current state data first in order toidentify „Annual Rate of Occurrence‟► 2 years of data from 20+ global locations► Needed to estimate „Single Loss Expectancy‟ value fortarget environment► Identified limited target groups to pilot controls in first tomeasure results► Needed a LOT of data► 87% accurate predictions over a 12 month period
  20. 20. 14/07/2008 Security MetricsPage 20Verizon 2008 Data Breach InvestigationsReport► 500 Investigations over 4 Years► 18% of breaches were the result of an unpatched system► 90% of unpatched breaches had had patches publiclyavailable for 6 months or more► No more would have been prevented by a patch cycleshorter than a month► There is a lot of useful data in this report
  21. 21. 14/07/2008 Security MetricsPage 21Dan Geer’s counterpoint► We are losing► The bad guys are in it for the money► Attackers costs are continually falling► Need to start measuring „attack metrics‟► Focus on increasing their cost of attack► More cost effective to redirect than to resist
  22. 22. 14/07/2008 Security MetricsPage 22Marcus Ranum’s counterpoint► Statistics only work where:► Population is large► Problems are common and widely shared► Aggressors act consistently► The only scores that matter are 0% and 100%► Security is not „risk management‟ it is „complexitymanagement‟
  23. 23. 14/07/2008Thank