This document discusses meaningful security metrics for various stakeholders. It recommends metrics that measure policy compliance, control maturity, and value at risk for CIOs. For operations managers, it suggests metrics that track systems outside of SLAs and security incidents breaching SLAs. For CISOs, suggested metrics include value at risk, compliance, and annual risk reduction compared to spending. For CEOs and boards, total exposure and unmanaged risk are recommended metrics. It also provides characteristics for effective security metrics and metrics for evaluating the metrics themselves.

1. Meaningful and useful Security metricsVladimir JirasekAbout.me/jirasek5st Oct 2011

2. About meSecurity professional (11 years), current work at Nokia as Enterprise Security architectFounding member and steering group member of (Common Assurance Maturity Model) CAMM (common-assurance.com)Director of Research, CSA UK & IrelandI love reading books: thrillers (Clive Cusler) and business management (Jo Owen)

3. I will cover three topics todayInformation Security ModelMetrics for CIOMetrics for Operations managerMetrics for CISOMetrics for CEO and the Board

4. Security model – business drives securityFeedback: update business requirementsInternational security standardsGovernanceInputLine ManagementSecurity managementCorrection of security processesLaws & RegulationsProduct ManagementProcess frameworkPolicy frameworkMetrics frameworkDefineInformation Security Metrics objectivesInformation Security ProcessesInformation Security policiesProgram ManagementCompliancerequirementsMeasured byMandateInputInformInformation Security standardsRisk & ComplianceIT GRCBusiness objectivesTechnologyPeopleServicesAssuranceInformation Security guidelinesDefineExternal security metricsBusiness impactAuditorsMeasure security maturityExecute security controlsDefine security controlsDefineSecurity managementBusiness & information risksSecurity intelligenceSecurity ServicesSecurity ProfessionalsInputSecurity threats

5. Security metrics characteristicsMeasurableObjectiveQuantitative (ideally)MeaningfulWith KPIs attached – know what is good and badLinked to business objectives – money speaks

6. Metrics for CIO – (1) Policy compliance and control maturity

7. Metrics for CIO – (2) Value at risk*InputAsset valuesMaturity of controlsSystem weaknessesThreat informationOutput – most likely (probability distribution) £ value of total exposure that IT organisation is exposed toInspiration in BASEL IIWork in progress* Eq most likely Total Exposure

8. Metrics for Ops managerThe morning dilema: “Can I have a coffee or is there something urgent to fix?”Suggested metrics:A number/percentage of systems outside SLA for fixing security weaknesses (both patches and configuration errors) – details of highly critical offenders – sorted by value at riskSecurity incidents that resulted in breached SLA (SLA is both time and £ value) And of course: Value at RiskQuiz: Is “A number of critical vulnerabilities good metric?”Answer: Not on its own!

9. Metrics for CISOGartner: by 2014 IT GRC and eGRC will merge in 70% of organisations. Likely head: CISORelevant metrics:Value at Risk – includes IT and other departmentsCompliance matrix ( same as for CIO)Annual risk reduction - Difference between VaR now and last year compared to money spent

10. Showing value for moneyEnd year review: We have spent more than the risk reduction but there were no incidents!VaR can also increase with new business processes and changes in regulatory and threat landscape.

11. Metrics for CEO and boardTotal exposure (£) = Value at Risk indicatorUnmanaged risk = likelihood there are risks that we do not know about = inverse of eGRC maturity

12. How do I know I have good metrics – metrics of metricsDecision effectiveness approach% of important management decisions that can be or have been influenced by double learning (i.e. revision and refinement of targets, measures, criteria, etc.)Investment approach% of security metrics costs for “exploratory/testing” vs. total metrics costSpeedCycle time from “Sense” to “Respond” for changing security metrics and management procedures.% of metrics that are collected and calculated automaticallyCostCost of changing security metrics and management procedures as % of total security management costs.Error% of security metrics that do not tie to any decisions or decision processes (over-shoot)% of decisions that have inadequate metrics support (under-shoot)% of metrics which have significant number of false signals

13. SummaryMetrics need to include monetary value otherwise the business leaders will not understand why the metrics are collected and presentedSecurity (and GRC in general) are here to keep the company risk at acceptable level – that needs to be measuredLink security metrics to policy which is linked to business objectivesBoards do not like “un-managed risk”Measure the metrics