Meaningfull security metrics


Published on

A presentation slides given at NetFocus 2011 in Bournemouth.

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This model is used to link security technologies reference model and blueprints to business requirementsAll security technology must support at least one information security process otherwise should be deployedBy linking requirements to policies to processes and to technologies we can be assured that technologies we deploy are justifiable and, at the same time, we know there should be no gapsInformation Security is a journey not a project and needs to be treated accordinglyInformation Security Policy is driven by business, legal and regulatory requirements which then mandates what security processes must and should be implementedIT Security policy is based on ISF Standard of Good Practice (SoGP) which maps to major regulatory and international standardsSecurity processes are run by People using Technology and report to Information Security Centre where data is correlated, normalised and available for management decisions, all in appropriate level of detail for audienceThe effectiveness of security processes is measured by Internal security metrics that are based on accepted best practice metrics, hence Nokia’s information security status can be compared with other companies
  • Meaningfull security metrics

    1. 1. Meaningful and useful Security metrics<br />Vladimir Jirasek<br /><br />5st Oct 2011<br />
    2. 2. About me<br />Security professional (11 years), current work at Nokia as Enterprise Security architect<br />Founding member and steering group member of (Common Assurance Maturity Model) CAMM (<br />Director of Research, CSA UK & Ireland<br />I love reading books: thrillers (Clive Cusler) and business management (Jo Owen)<br />
    3. 3. I will cover three topics today<br />Information Security Model<br />Metrics for CIO<br />Metrics for Operations manager<br />Metrics for CISO<br />Metrics for CEO and the Board<br />
    4. 4. Security model – business drives security<br />Feedback: update business requirements<br />International security standards<br />Governance<br />Input<br />Line Management<br />Security management<br />Correction of security processes<br />Laws & Regulations<br />Product Management<br />Process framework<br />Policy framework<br />Metrics framework<br />Define<br />Information Security Metrics objectives<br />Information Security Processes<br />Information Security policies<br />Program Management<br />Compliancerequirements<br />Measured by<br />Mandate<br />Input<br />Inform<br />Information Security standards<br />Risk & Compliance<br />IT GRC<br />Business objectives<br />Technology<br />People<br />Services<br />Assurance<br />Information Security guidelines<br />Define<br />External security metrics<br />Business impact<br />Auditors<br />Measure security maturity<br />Execute security controls<br />Define security controls<br />Define<br />Security management<br />Business & information risks<br />Security intelligence<br />Security Services<br />Security Professionals<br />Input<br />Security threats<br />
    5. 5. Security metrics characteristics<br />Measurable<br />Objective<br />Quantitative (ideally)<br />Meaningful<br />With KPIs attached – know what is good and bad<br />Linked to business objectives – money speaks<br />
    6. 6. Metrics for CIO – (1) Policy compliance and control maturity<br />
    7. 7. Metrics for CIO – (2) Value at risk*<br />Input<br />Asset values<br />Maturity of controls<br />System weaknesses<br />Threat information<br />Output – most likely (probability distribution) £ value of total exposure that IT organisation is exposed to<br />Inspiration in BASEL II<br />Work in progress<br />* Eq most likely Total Exposure<br />
    8. 8. Metrics for Ops manager<br />The morning dilema: “Can I have a coffee or is there something urgent to fix?”<br />Suggested metrics:<br />A number/percentage of systems outside SLA for fixing security weaknesses (both patches and configuration errors) – details of highly critical offenders – sorted by value at risk<br />Security incidents that resulted in breached SLA (SLA is both time and £ value) <br />And of course: Value at Risk<br />Quiz: Is “A number of critical vulnerabilities good metric?”<br />Answer: Not on its own!<br />
    9. 9. Metrics for CISO<br />Gartner: by 2014 IT GRC and eGRC will merge in 70% of organisations. Likely head: CISO<br />Relevant metrics:<br />Value at Risk – includes IT and other departments<br />Compliance matrix ( same as for CIO)<br />Annual risk reduction - Difference between VaR now and last year compared to money spent<br />
    10. 10. Showing value for money<br />End year review: We have spent more than the risk reduction but there were no incidents!<br />VaR can also increase with new business processes and changes in regulatory and threat landscape.<br />
    11. 11. Metrics for CEO and board<br />Total exposure (£) = Value at Risk indicator<br />Unmanaged risk = likelihood there are risks that we do not know about = inverse of eGRC maturity<br />
    12. 12. How do I know I have good metrics – metrics of metrics<br />Decision effectiveness approach<br />% of important management decisions that can be or have been influenced by double learning (i.e. revision and refinement of targets, measures, criteria, etc.)<br />Investment approach<br />% of security metrics costs for “exploratory/testing” vs. total metrics cost<br />Speed<br />Cycle time from “Sense” to “Respond” for changing security metrics and management procedures.<br />% of metrics that are collected and calculated automatically<br />Cost<br />Cost of changing security metrics and management procedures as % of total security management costs.<br />Error<br />% of security metrics that do not tie to any decisions or decision processes (over-shoot)<br />% of decisions that have inadequate metrics support (under-shoot)<br />% of metrics which have significant number of false signals<br />
    13. 13. Summary<br />Metrics need to include monetary value otherwise the business leaders will not understand why the metrics are collected and presented<br />Security (and GRC in general) are here to keep the company risk at acceptable level – that needs to be measured<br />Link security metrics to policy which is linked to business objectives<br />Boards do not like “un-managed risk”<br />Measure the metrics<br />