SlideShare a Scribd company logo
1 of 25
SECURITY METRICS



 A presentation developed by Cydney Davis, Senior Technical Write
What are Metrics?
2


    A method which facilitates decision-making and
    improved performance and accountability through
    collection, analysis and reporting of performance-
    related data.

     Information Security metrics must be:
     • based on Information Security performance goals and
       objectives
     • useful for reduction and management of risks

     • readily obtainable and replicable

     • useful for tracking performance and directing resources

     • able to yield quantifiable information
What is our Mission/Goal?
3

    It is critical that we use metrics that are relevant to
    our organization and to the mission we are
    measuring.

    But first, we have to determine:
    • Where we are (Baseline)
    • Where we are going (End Goals)
    • Who/what relies on us? (Users/Management)
    • What do they need/expect? (Reports/Assurance)
    • What are we trying to prove?
    • What are we trying to solve?
    • What are we trying to improve?
How can we use Metrics?
4


       Communicate Performance
       Drive Performance Improvement
       Measure Effectiveness of Security Controls
       Help Diagnose Problems
       Provide Effective Decision-making Support
       Increase Accountability
       Guide Resource Allocation
       Demonstrate the state of compliance
       Facilitate Benchmark Comparisons
Metrics can help determine:
5



    • the number of resources it takes to accomplish security
        goals
    • justifiability for financing new security measures
    • If the company is getting its money’s worth
    • If the company is managing risk appropriately*
    • what Information Security needs to do to improve
        Security
        ˉ administration/processes/procedures/policies/person
             nel/enhancements/technology/etc.)
    • where we are with comparisons to peers regarding to
        standards, best practices, execution and results of
        security measures
    *The residual risk that a company is willing to take based on; business needs,
    budget limits, industry regulations/requirements and other criteria.
6
    Building the
    Security Metrics Program
Executive Focus
7



     “The heart of it is that if a business process
     cannot be measured in one way or another,
     we likely ought to cast it off as wasted effort.”
             Comment from a CEO to an anonymous Information Security Profession




    Translation:
        Why do it if we can’t prove/justify its value?
         (time, money, effort, results and actions)
Good Metrics Guidelines
8



    • Consistently Measured
       • apples to apples/same time same place
    • Cheap to Produce (Time-wise)
    • Yield Quantifiable Information
    • Contextually Specific – who
    • Expressed using at least 2 units of measure or data po
Metrics Program Success
9
                        Criteria
    Identify incident trends important to key senior managers,
    stakeholders and to the InfoSec Mission from a management
    perspective.*

    Provide consistent information that adds value and is actionable by:

       • Tracking changes on a consistent basis.

       • Focusing on what's important in our business

       • Developing a few value indicators that we can track with a high
         degree of reliability

       • Doing some service benchmarking with our peers.

       *This is the first and most important decision
Basic Information Security
10
             Measures
     Anti-malware       Firewalls            Asset
                                             Management
     Intrusion          Anti-SPAM            Patch
     Detection                               Management
     and Prevention
     Vulnerability      Unified Threat       Application
     Management         Management           Security
                                             Scanners
     Databases          Website Statistics   Network Access
                                             Control
     System Integrity   Operating            Data Leakage
     Checking           Systems              Protection
     Configuration      Secure Web           Web Application
     Hardening          Gateways             Firewalls
     Mobile Data        Media Sanitation     Storage
     Protection                              Encryption
Formula for Deriving True
11
                     Meaning
     WHY we need WHAT we need  WHO we are
     to measure it to measure measuring it for


        •   Financial              DATA            •   C-Level
        •   Governance             DATA            •   Board of Directors
        •   Legal                  DATA            •   Marketing Releases
        •   Regulatory             DATA            •   Industry Report
        •   Directive              DATA            •   General Staff



     Determine how the information will be analyzed, interpreted and
     used!
12




     “Good metrics facilitate discussion, insight
      and analysis...”
Metrics Program - Components
13

       Program Component

        Define the metrics program goal(s) and objectives


        Decide which metrics to generate


        Develop strategies for generating the metrics


        Establish benchmarks and targets


        Determine how the metrics will be reported


        Create an action plan and act on it


        Establish a formal program review/refinement cycle
High Level Process Steps
14

  Obtain management input, agreement and support for the implementation of a strong
     metrics program.
  Review our organization’s mission statements, policies, plans, procedures, goals and
     objectives, and assess them against legislative and regulatory requirements, as well as
     against effectiveness goals.
  Describe how we will achieve company and department goals
       List milestones, dates and quantifiable objectives against which to map progress.
  Select appropriate, quantifiable effectiveness metrics to indicate baseline, interim and
     final success.
  Gather the metrics.
  Analyze and present the results to management and key stakeholders.
  Recommend that management make decisions based on the metrics, and plan the
     execution of these decisions. * Metrics are often referred to as “decision support.”
  Evaluate the outcome of decisions against goals. This should be done from a perspective of
                                                                     *The real value of a metrics program
Project Plan Overview
15
16   Metrics Versus Numbers
Good metrics are those that are
17
     SMART;
            • Specific
            • Measurable
            • Attainable
            • Repeatable,
            • Time-dependent

     Truly useful metrics indicate the degree to which
     security goals are being met – and they drive
     actions that need to be taken to improve our
     overall security goals.
Metrics? Or Just Numbers?
18

     Exhibit A - This set of numbers can give us a sense of the overall health of anti-virus
     defenses and can show trends over time; but the information is not actionable in
     any way and will not serve as a meaningful diagnostic tool.

              SO WHAT??? = False sense of security without more knowledge
Good Metrics = Numbers with
                     Relevance
19

Exhibit B displays the same measurements as Exhibit A. By drilling down into the data we can begin to
understand which locations are struggling with this activity. This in turn will help us choose where to focus in
order to improve the performance of our organization. This kind of actionable intelligence is valuable and it
can really drive performance improvement and provide information that is actionable to a productive end.

                           Example Metrics showing
                           RELEVANCE
                      Percentage of computers with current anti-virus definitions


     City A                                                                                            99.4 %



     City B                                                                                  94.7 %



     City C                                                                         89.8 %


          50 %     55 %      60 %     65 %      70 %      75 %     80 %      85 %     90 %      95 %     100 %
Good Metrics = Actionable
20

                       Percentage of computers with current anti-virus definitions
        CITY A                                                           99.4 %

        City B                                                      94.7 %

        City C
                                                              89.8 %

                 50 % 55 % 60 % 65 % 70 % 75 % 80 % 85 % 90 % 95 % 100 %

Example Question: Why is one location so much farther behind in implementation?

Possible Reasons: Understaffed
                 Limited Bandwidth
                 More staff traveling that previous years

Possible Actions:      Hire additional staff
                     Share resources if the implementation MUST be done by xxx date
                     Set different schedules for each location for future projects
Presenting and Interpreting Data
21
                 Reports
       Visually Appealing                   Visually Appealing
                                        Interpreted and Actionable

           _______% improved
                                         _______% improved

                               from _______ and that means _________ .

                               What we need is ______ based on

                               requirements for __________ . Going

                               forward we should consider doing

                               ___________ .
22
     Measuring for value not
     numbers
     Examples to work with
     Defining, refining and Interpreting data/results for the intended audience
EXAMPLE Metric : Baseline Defenses
                Coverage
                     (Antivirus, Antispyware, Firewall, etc)
23


 Measurement of how well we are protecting our enterprise against
 the most basic information security threats.

 Just Numbers: ________ %

 What would an additional relevant value be that we can use to have
 SMART data?

 Metrics: ________ %
        Increase since (prior month/inception/year over year/etc.)
        Device Type
        Location
        Length of time it took to detect
EXAMPLE Metric : Legitimate E-Mail Traffic
                     Analysis
24

Legitimate e-mail traffic analysis is a family of metrics including incoming and outgoing
traffic volume, incoming and outgoing traffic size, and traffic flow between our company and
others.

By monitoring legitimate e-mail flow over time, we can learn where to set alarm points.

Numbers:
Compare the amount of good and junk e-mail that we are receiving
____ percent good
____ percent junk

What would an additional relevant value be that we can use to have SMART data?

Metrics
       ____ percent good
       ____ percent junk
       Quarterly/Annually/Since inception/Current Month
       Since adding the _________ criteria
       Received from _________ types/organizations
       Sent During ____________ (AM/PM – Holidays , etc.)
       Junk Detected Quicker _______ (first time/second time)
Conclusion
25

      By presenting information in a sufficiently granular way we can inject business relevance
      into the exhibits. Producing a benchmark is also a powerful approach to performance
      improvement.


                                         Percentage of computers with current anti-virus definitions
                          City A                                                                           99.4 %

                          City B                                                                    94.7 %

                          City C
                                                                                             89.8 %

                                   50 % 55 % 60 % 65 % 70 % 75 % 80 % 85 % 90 % 95 % 100 %




     Frequently this level of visibility will spark a competitive fire in those being measured. Professional pride will
     drive most people to make sure they are found among the high performers on your report.

More Related Content

What's hot

SOC presentation- Building a Security Operations Center
SOC presentation- Building a Security Operations CenterSOC presentation- Building a Security Operations Center
SOC presentation- Building a Security Operations CenterMichael Nickle
 
From SIEM to SOC: Crossing the Cybersecurity Chasm
From SIEM to SOC: Crossing the Cybersecurity ChasmFrom SIEM to SOC: Crossing the Cybersecurity Chasm
From SIEM to SOC: Crossing the Cybersecurity ChasmPriyanka Aash
 
Optimizing Security Operations: 5 Keys to Success
Optimizing Security Operations: 5 Keys to SuccessOptimizing Security Operations: 5 Keys to Success
Optimizing Security Operations: 5 Keys to SuccessSirius
 
Cybersecurity Assessment Framework - Slideshare.pptx
Cybersecurity Assessment Framework - Slideshare.pptxCybersecurity Assessment Framework - Slideshare.pptx
Cybersecurity Assessment Framework - Slideshare.pptxAzra'ee Mamat
 
Next-Gen security operation center
Next-Gen security operation centerNext-Gen security operation center
Next-Gen security operation centerMuhammad Sahputra
 
Insight into SOAR
Insight into SOARInsight into SOAR
Insight into SOARDNIF
 
Cybersecurity Incident Management Powerpoint Presentation Slides
Cybersecurity Incident Management Powerpoint Presentation SlidesCybersecurity Incident Management Powerpoint Presentation Slides
Cybersecurity Incident Management Powerpoint Presentation SlidesSlideTeam
 
Roadmap to security operations excellence
Roadmap to security operations excellenceRoadmap to security operations excellence
Roadmap to security operations excellenceErik Taavila
 
Cybersecurity Attack Vectors: How to Protect Your Organization
Cybersecurity Attack Vectors: How to Protect Your OrganizationCybersecurity Attack Vectors: How to Protect Your Organization
Cybersecurity Attack Vectors: How to Protect Your OrganizationTriCorps Technologies
 
Vulnerability Management
Vulnerability ManagementVulnerability Management
Vulnerability Managementasherad
 
Cyber Security: Threat and Prevention
Cyber Security: Threat and PreventionCyber Security: Threat and Prevention
Cyber Security: Threat and Preventionfmi_igf
 
The Next Generation of Security Operations Centre (SOC)
The Next Generation of Security Operations Centre (SOC)The Next Generation of Security Operations Centre (SOC)
The Next Generation of Security Operations Centre (SOC)PECB
 
Threat Hunting - Moving from the ad hoc to the formal
Threat Hunting - Moving from the ad hoc to the formalThreat Hunting - Moving from the ad hoc to the formal
Threat Hunting - Moving from the ad hoc to the formalPriyanka Aash
 
When and How to Set up a Security Operations Center
When and How to Set up a Security Operations CenterWhen and How to Set up a Security Operations Center
When and How to Set up a Security Operations CenterKomand
 
Rothke secure360 building a security operations center (soc)
Rothke   secure360 building a security operations center (soc)Rothke   secure360 building a security operations center (soc)
Rothke secure360 building a security operations center (soc)Ben Rothke
 
Security Operations Center (SOC) Essentials for the SME
Security Operations Center (SOC) Essentials for the SMESecurity Operations Center (SOC) Essentials for the SME
Security Operations Center (SOC) Essentials for the SMEAlienVault
 
NIST cybersecurity framework
NIST cybersecurity frameworkNIST cybersecurity framework
NIST cybersecurity frameworkShriya Rai
 
Cyber Security Maturity Assessment
 Cyber Security Maturity Assessment Cyber Security Maturity Assessment
Cyber Security Maturity AssessmentDoreen Loeber
 

What's hot (20)

SOC presentation- Building a Security Operations Center
SOC presentation- Building a Security Operations CenterSOC presentation- Building a Security Operations Center
SOC presentation- Building a Security Operations Center
 
Soc
SocSoc
Soc
 
From SIEM to SOC: Crossing the Cybersecurity Chasm
From SIEM to SOC: Crossing the Cybersecurity ChasmFrom SIEM to SOC: Crossing the Cybersecurity Chasm
From SIEM to SOC: Crossing the Cybersecurity Chasm
 
Optimizing Security Operations: 5 Keys to Success
Optimizing Security Operations: 5 Keys to SuccessOptimizing Security Operations: 5 Keys to Success
Optimizing Security Operations: 5 Keys to Success
 
Cybersecurity Assessment Framework - Slideshare.pptx
Cybersecurity Assessment Framework - Slideshare.pptxCybersecurity Assessment Framework - Slideshare.pptx
Cybersecurity Assessment Framework - Slideshare.pptx
 
Next-Gen security operation center
Next-Gen security operation centerNext-Gen security operation center
Next-Gen security operation center
 
Information Security and the SDLC
Information Security and the SDLCInformation Security and the SDLC
Information Security and the SDLC
 
Insight into SOAR
Insight into SOARInsight into SOAR
Insight into SOAR
 
Cybersecurity Incident Management Powerpoint Presentation Slides
Cybersecurity Incident Management Powerpoint Presentation SlidesCybersecurity Incident Management Powerpoint Presentation Slides
Cybersecurity Incident Management Powerpoint Presentation Slides
 
Roadmap to security operations excellence
Roadmap to security operations excellenceRoadmap to security operations excellence
Roadmap to security operations excellence
 
Cybersecurity Attack Vectors: How to Protect Your Organization
Cybersecurity Attack Vectors: How to Protect Your OrganizationCybersecurity Attack Vectors: How to Protect Your Organization
Cybersecurity Attack Vectors: How to Protect Your Organization
 
Vulnerability Management
Vulnerability ManagementVulnerability Management
Vulnerability Management
 
Cyber Security: Threat and Prevention
Cyber Security: Threat and PreventionCyber Security: Threat and Prevention
Cyber Security: Threat and Prevention
 
The Next Generation of Security Operations Centre (SOC)
The Next Generation of Security Operations Centre (SOC)The Next Generation of Security Operations Centre (SOC)
The Next Generation of Security Operations Centre (SOC)
 
Threat Hunting - Moving from the ad hoc to the formal
Threat Hunting - Moving from the ad hoc to the formalThreat Hunting - Moving from the ad hoc to the formal
Threat Hunting - Moving from the ad hoc to the formal
 
When and How to Set up a Security Operations Center
When and How to Set up a Security Operations CenterWhen and How to Set up a Security Operations Center
When and How to Set up a Security Operations Center
 
Rothke secure360 building a security operations center (soc)
Rothke   secure360 building a security operations center (soc)Rothke   secure360 building a security operations center (soc)
Rothke secure360 building a security operations center (soc)
 
Security Operations Center (SOC) Essentials for the SME
Security Operations Center (SOC) Essentials for the SMESecurity Operations Center (SOC) Essentials for the SME
Security Operations Center (SOC) Essentials for the SME
 
NIST cybersecurity framework
NIST cybersecurity frameworkNIST cybersecurity framework
NIST cybersecurity framework
 
Cyber Security Maturity Assessment
 Cyber Security Maturity Assessment Cyber Security Maturity Assessment
Cyber Security Maturity Assessment
 

Viewers also liked

The Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your StoryThe Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your StoryPriyanka Aash
 
Top 10 Essentials for Building a Powerful Security Dashboard
Top 10 Essentials for Building a Powerful Security DashboardTop 10 Essentials for Building a Powerful Security Dashboard
Top 10 Essentials for Building a Powerful Security DashboardTripwire
 
Sudarsan Jayaraman - Open information security management maturity model
Sudarsan Jayaraman  - Open information security management maturity modelSudarsan Jayaraman  - Open information security management maturity model
Sudarsan Jayaraman - Open information security management maturity modelnooralmousa
 
Cybersecurity Metrics: Reporting to BoD
Cybersecurity Metrics: Reporting to BoDCybersecurity Metrics: Reporting to BoD
Cybersecurity Metrics: Reporting to BoDPranav Shah
 
Measuring Success - Security KPIs
Measuring Success - Security KPIsMeasuring Success - Security KPIs
Measuring Success - Security KPIsH Contrex
 
Developing Metrics and KPI (Key Performance Indicators
Developing Metrics and KPI (Key Performance IndicatorsDeveloping Metrics and KPI (Key Performance Indicators
Developing Metrics and KPI (Key Performance IndicatorsVictor Holman
 
25 KPIs Every Manager Needs To Know
25 KPIs Every Manager Needs To Know25 KPIs Every Manager Needs To Know
25 KPIs Every Manager Needs To KnowBernard Marr
 
Practical Measures for Measuring Security
Practical Measures for Measuring SecurityPractical Measures for Measuring Security
Practical Measures for Measuring SecurityChris Mullins
 
Values: A Manager's Guide
Values: A Manager's GuideValues: A Manager's Guide
Values: A Manager's GuideCynthia Scott
 
Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...
Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...
Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...NJVC, LLC
 
The Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your StoryThe Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your StoryPriyanka Aash
 
Link Reclamation Strategies
Link Reclamation Strategies Link Reclamation Strategies
Link Reclamation Strategies patrickstox
 
Everyone Screws Up HTTPS
Everyone Screws Up HTTPSEveryone Screws Up HTTPS
Everyone Screws Up HTTPSpatrickstox
 
Discovering Values: The Key to Unlocking Employee Engagement
Discovering Values: The Key to Unlocking Employee EngagementDiscovering Values: The Key to Unlocking Employee Engagement
Discovering Values: The Key to Unlocking Employee EngagementCynthia Scott
 
Nabil Malik - Security performance metrics
Nabil Malik - Security performance metricsNabil Malik - Security performance metrics
Nabil Malik - Security performance metricsnooralmousa
 
Metrics & Analytics That Matter - Steve Krull, CEO, Be Found Online
Metrics & Analytics That Matter - Steve Krull, CEO, Be Found OnlineMetrics & Analytics That Matter - Steve Krull, CEO, Be Found Online
Metrics & Analytics That Matter - Steve Krull, CEO, Be Found OnlineBrightEdge Technologies
 
Lean Workbench For Creating And Tracking Metrics That Matter
Lean Workbench For Creating And Tracking Metrics That MatterLean Workbench For Creating And Tracking Metrics That Matter
Lean Workbench For Creating And Tracking Metrics That MatterJennifer Rubinovitz
 

Viewers also liked (20)

The Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your StoryThe Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your Story
 
Top 10 Essentials for Building a Powerful Security Dashboard
Top 10 Essentials for Building a Powerful Security DashboardTop 10 Essentials for Building a Powerful Security Dashboard
Top 10 Essentials for Building a Powerful Security Dashboard
 
Sudarsan Jayaraman - Open information security management maturity model
Sudarsan Jayaraman  - Open information security management maturity modelSudarsan Jayaraman  - Open information security management maturity model
Sudarsan Jayaraman - Open information security management maturity model
 
Cybersecurity Metrics: Reporting to BoD
Cybersecurity Metrics: Reporting to BoDCybersecurity Metrics: Reporting to BoD
Cybersecurity Metrics: Reporting to BoD
 
Measuring Success - Security KPIs
Measuring Success - Security KPIsMeasuring Success - Security KPIs
Measuring Success - Security KPIs
 
Developing Metrics and KPI (Key Performance Indicators
Developing Metrics and KPI (Key Performance IndicatorsDeveloping Metrics and KPI (Key Performance Indicators
Developing Metrics and KPI (Key Performance Indicators
 
25 KPIs Every Manager Needs To Know
25 KPIs Every Manager Needs To Know25 KPIs Every Manager Needs To Know
25 KPIs Every Manager Needs To Know
 
Practical Measures for Measuring Security
Practical Measures for Measuring SecurityPractical Measures for Measuring Security
Practical Measures for Measuring Security
 
Values: A Manager's Guide
Values: A Manager's GuideValues: A Manager's Guide
Values: A Manager's Guide
 
Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...
Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...
Health IT Cyber Security HIPAA Summit Presentation: Metrics and Continuous Mo...
 
Security Development Lifecycle Tools
Security Development Lifecycle ToolsSecurity Development Lifecycle Tools
Security Development Lifecycle Tools
 
The Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your StoryThe Measure of Success: Security Metrics to Tell Your Story
The Measure of Success: Security Metrics to Tell Your Story
 
Link Reclamation Strategies
Link Reclamation Strategies Link Reclamation Strategies
Link Reclamation Strategies
 
Everyone Screws Up HTTPS
Everyone Screws Up HTTPSEveryone Screws Up HTTPS
Everyone Screws Up HTTPS
 
Discovering Values: The Key to Unlocking Employee Engagement
Discovering Values: The Key to Unlocking Employee EngagementDiscovering Values: The Key to Unlocking Employee Engagement
Discovering Values: The Key to Unlocking Employee Engagement
 
Helpdesk
HelpdeskHelpdesk
Helpdesk
 
Nabil Malik - Security performance metrics
Nabil Malik - Security performance metricsNabil Malik - Security performance metrics
Nabil Malik - Security performance metrics
 
Measuring Effectiveness
Measuring EffectivenessMeasuring Effectiveness
Measuring Effectiveness
 
Metrics & Analytics That Matter - Steve Krull, CEO, Be Found Online
Metrics & Analytics That Matter - Steve Krull, CEO, Be Found OnlineMetrics & Analytics That Matter - Steve Krull, CEO, Be Found Online
Metrics & Analytics That Matter - Steve Krull, CEO, Be Found Online
 
Lean Workbench For Creating And Tracking Metrics That Matter
Lean Workbench For Creating And Tracking Metrics That MatterLean Workbench For Creating And Tracking Metrics That Matter
Lean Workbench For Creating And Tracking Metrics That Matter
 

Similar to Security Metrics Program

Presenting Metrics to the Executive Team
Presenting Metrics to the Executive TeamPresenting Metrics to the Executive Team
Presenting Metrics to the Executive TeamJohn D. Johnson
 
Strategic governance performance_management_systems
Strategic governance performance_management_systemsStrategic governance performance_management_systems
Strategic governance performance_management_systemsRamsés Gallego
 
Gregs BI Presentation
Gregs BI PresentationGregs BI Presentation
Gregs BI Presentationflyjock1
 
Business Value Measurements and the Solution Design Framework
Business Value Measurements and the Solution Design FrameworkBusiness Value Measurements and the Solution Design Framework
Business Value Measurements and the Solution Design FrameworkLeo Barella
 
Bi in telecom through kpi’s
Bi in telecom through kpi’sBi in telecom through kpi’s
Bi in telecom through kpi’sSai Venkatesh
 
La Importancia del Análisis de la Información
La Importancia del Análisis de la InformaciónLa Importancia del Análisis de la Información
La Importancia del Análisis de la InformaciónNexolution
 
Software Security Metrics
Software Security MetricsSoftware Security Metrics
Software Security MetricsCigital
 
Accelerate Mobile Success with a Mobile Center of Excellence
Accelerate Mobile Success with a Mobile Center of ExcellenceAccelerate Mobile Success with a Mobile Center of Excellence
Accelerate Mobile Success with a Mobile Center of ExcellenceXamarin
 
Take your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena SoftwareTake your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena SoftwareSerena Software
 
2016 Risk Management Workshop
2016 Risk Management Workshop2016 Risk Management Workshop
2016 Risk Management WorkshopStacy Willis
 
IBM Decision Server Insights
IBM Decision Server InsightsIBM Decision Server Insights
IBM Decision Server InsightsAlain Neyroud
 
Achieving Compliance Through Security
Achieving Compliance Through SecurityAchieving Compliance Through Security
Achieving Compliance Through SecurityEnergySec
 
Old Presentation on Security Metrics 2005
Old Presentation on Security Metrics 2005Old Presentation on Security Metrics 2005
Old Presentation on Security Metrics 2005Anton Chuvakin
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The MetricsTeamQualityPro
 
managed-services-buying-guide
managed-services-buying-guidemanaged-services-buying-guide
managed-services-buying-guideMarie Peters
 
Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018
Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018
Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018Association for Project Management
 
ServiceNow Performance Analytics for Security Operations
ServiceNow Performance Analytics for Security OperationsServiceNow Performance Analytics for Security Operations
ServiceNow Performance Analytics for Security OperationsJade Global
 
Cyber risk management-white-paper-v8 (2) 2015
Cyber risk management-white-paper-v8 (2) 2015Cyber risk management-white-paper-v8 (2) 2015
Cyber risk management-white-paper-v8 (2) 2015Accounting_Whitepapers
 
La Digital Transformation ha un nuovo alleato: Value Stream Management
La Digital Transformation ha un nuovo alleato: Value Stream ManagementLa Digital Transformation ha un nuovo alleato: Value Stream Management
La Digital Transformation ha un nuovo alleato: Value Stream ManagementEmerasoft, solutions to collaborate
 

Similar to Security Metrics Program (20)

Presenting Metrics to the Executive Team
Presenting Metrics to the Executive TeamPresenting Metrics to the Executive Team
Presenting Metrics to the Executive Team
 
Strategic governance performance_management_systems
Strategic governance performance_management_systemsStrategic governance performance_management_systems
Strategic governance performance_management_systems
 
Gregs BI Presentation
Gregs BI PresentationGregs BI Presentation
Gregs BI Presentation
 
Business Value Measurements and the Solution Design Framework
Business Value Measurements and the Solution Design FrameworkBusiness Value Measurements and the Solution Design Framework
Business Value Measurements and the Solution Design Framework
 
Bi in telecom through kpi’s
Bi in telecom through kpi’sBi in telecom through kpi’s
Bi in telecom through kpi’s
 
La Importancia del Análisis de la Información
La Importancia del Análisis de la InformaciónLa Importancia del Análisis de la Información
La Importancia del Análisis de la Información
 
Software Security Metrics
Software Security MetricsSoftware Security Metrics
Software Security Metrics
 
Accelerate Mobile Success with a Mobile Center of Excellence
Accelerate Mobile Success with a Mobile Center of ExcellenceAccelerate Mobile Success with a Mobile Center of Excellence
Accelerate Mobile Success with a Mobile Center of Excellence
 
A9 schubert
A9 schubertA9 schubert
A9 schubert
 
Take your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena SoftwareTake your code and quality to the next level by Serena Software
Take your code and quality to the next level by Serena Software
 
2016 Risk Management Workshop
2016 Risk Management Workshop2016 Risk Management Workshop
2016 Risk Management Workshop
 
IBM Decision Server Insights
IBM Decision Server InsightsIBM Decision Server Insights
IBM Decision Server Insights
 
Achieving Compliance Through Security
Achieving Compliance Through SecurityAchieving Compliance Through Security
Achieving Compliance Through Security
 
Old Presentation on Security Metrics 2005
Old Presentation on Security Metrics 2005Old Presentation on Security Metrics 2005
Old Presentation on Security Metrics 2005
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The Metrics
 
managed-services-buying-guide
managed-services-buying-guidemanaged-services-buying-guide
managed-services-buying-guide
 
Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018
Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018
Adam Suchley - Predictive Delivery Assurance - APM Assurance SIG Conference 2018
 
ServiceNow Performance Analytics for Security Operations
ServiceNow Performance Analytics for Security OperationsServiceNow Performance Analytics for Security Operations
ServiceNow Performance Analytics for Security Operations
 
Cyber risk management-white-paper-v8 (2) 2015
Cyber risk management-white-paper-v8 (2) 2015Cyber risk management-white-paper-v8 (2) 2015
Cyber risk management-white-paper-v8 (2) 2015
 
La Digital Transformation ha un nuovo alleato: Value Stream Management
La Digital Transformation ha un nuovo alleato: Value Stream ManagementLa Digital Transformation ha un nuovo alleato: Value Stream Management
La Digital Transformation ha un nuovo alleato: Value Stream Management
 

Security Metrics Program

  • 1. SECURITY METRICS A presentation developed by Cydney Davis, Senior Technical Write
  • 2. What are Metrics? 2 A method which facilitates decision-making and improved performance and accountability through collection, analysis and reporting of performance- related data. Information Security metrics must be: • based on Information Security performance goals and objectives • useful for reduction and management of risks • readily obtainable and replicable • useful for tracking performance and directing resources • able to yield quantifiable information
  • 3. What is our Mission/Goal? 3 It is critical that we use metrics that are relevant to our organization and to the mission we are measuring. But first, we have to determine: • Where we are (Baseline) • Where we are going (End Goals) • Who/what relies on us? (Users/Management) • What do they need/expect? (Reports/Assurance) • What are we trying to prove? • What are we trying to solve? • What are we trying to improve?
  • 4. How can we use Metrics? 4  Communicate Performance  Drive Performance Improvement  Measure Effectiveness of Security Controls  Help Diagnose Problems  Provide Effective Decision-making Support  Increase Accountability  Guide Resource Allocation  Demonstrate the state of compliance  Facilitate Benchmark Comparisons
  • 5. Metrics can help determine: 5 • the number of resources it takes to accomplish security goals • justifiability for financing new security measures • If the company is getting its money’s worth • If the company is managing risk appropriately* • what Information Security needs to do to improve Security ˉ administration/processes/procedures/policies/person nel/enhancements/technology/etc.) • where we are with comparisons to peers regarding to standards, best practices, execution and results of security measures *The residual risk that a company is willing to take based on; business needs, budget limits, industry regulations/requirements and other criteria.
  • 6. 6 Building the Security Metrics Program
  • 7. Executive Focus 7 “The heart of it is that if a business process cannot be measured in one way or another, we likely ought to cast it off as wasted effort.” Comment from a CEO to an anonymous Information Security Profession Translation: Why do it if we can’t prove/justify its value? (time, money, effort, results and actions)
  • 8. Good Metrics Guidelines 8 • Consistently Measured • apples to apples/same time same place • Cheap to Produce (Time-wise) • Yield Quantifiable Information • Contextually Specific – who • Expressed using at least 2 units of measure or data po
  • 9. Metrics Program Success 9 Criteria Identify incident trends important to key senior managers, stakeholders and to the InfoSec Mission from a management perspective.* Provide consistent information that adds value and is actionable by: • Tracking changes on a consistent basis. • Focusing on what's important in our business • Developing a few value indicators that we can track with a high degree of reliability • Doing some service benchmarking with our peers. *This is the first and most important decision
  • 10. Basic Information Security 10 Measures Anti-malware Firewalls Asset Management Intrusion Anti-SPAM Patch Detection Management and Prevention Vulnerability Unified Threat Application Management Management Security Scanners Databases Website Statistics Network Access Control System Integrity Operating Data Leakage Checking Systems Protection Configuration Secure Web Web Application Hardening Gateways Firewalls Mobile Data Media Sanitation Storage Protection Encryption
  • 11. Formula for Deriving True 11 Meaning WHY we need WHAT we need WHO we are to measure it to measure measuring it for • Financial DATA • C-Level • Governance DATA • Board of Directors • Legal DATA • Marketing Releases • Regulatory DATA • Industry Report • Directive DATA • General Staff Determine how the information will be analyzed, interpreted and used!
  • 12. 12 “Good metrics facilitate discussion, insight and analysis...”
  • 13. Metrics Program - Components 13 Program Component  Define the metrics program goal(s) and objectives  Decide which metrics to generate  Develop strategies for generating the metrics  Establish benchmarks and targets  Determine how the metrics will be reported  Create an action plan and act on it  Establish a formal program review/refinement cycle
  • 14. High Level Process Steps 14  Obtain management input, agreement and support for the implementation of a strong metrics program.  Review our organization’s mission statements, policies, plans, procedures, goals and objectives, and assess them against legislative and regulatory requirements, as well as against effectiveness goals.  Describe how we will achieve company and department goals  List milestones, dates and quantifiable objectives against which to map progress.  Select appropriate, quantifiable effectiveness metrics to indicate baseline, interim and final success.  Gather the metrics.  Analyze and present the results to management and key stakeholders.  Recommend that management make decisions based on the metrics, and plan the execution of these decisions. * Metrics are often referred to as “decision support.”  Evaluate the outcome of decisions against goals. This should be done from a perspective of *The real value of a metrics program
  • 16. 16 Metrics Versus Numbers
  • 17. Good metrics are those that are 17 SMART; • Specific • Measurable • Attainable • Repeatable, • Time-dependent Truly useful metrics indicate the degree to which security goals are being met – and they drive actions that need to be taken to improve our overall security goals.
  • 18. Metrics? Or Just Numbers? 18 Exhibit A - This set of numbers can give us a sense of the overall health of anti-virus defenses and can show trends over time; but the information is not actionable in any way and will not serve as a meaningful diagnostic tool. SO WHAT??? = False sense of security without more knowledge
  • 19. Good Metrics = Numbers with Relevance 19 Exhibit B displays the same measurements as Exhibit A. By drilling down into the data we can begin to understand which locations are struggling with this activity. This in turn will help us choose where to focus in order to improve the performance of our organization. This kind of actionable intelligence is valuable and it can really drive performance improvement and provide information that is actionable to a productive end. Example Metrics showing RELEVANCE Percentage of computers with current anti-virus definitions City A 99.4 % City B 94.7 % City C 89.8 % 50 % 55 % 60 % 65 % 70 % 75 % 80 % 85 % 90 % 95 % 100 %
  • 20. Good Metrics = Actionable 20 Percentage of computers with current anti-virus definitions CITY A 99.4 % City B 94.7 % City C 89.8 % 50 % 55 % 60 % 65 % 70 % 75 % 80 % 85 % 90 % 95 % 100 % Example Question: Why is one location so much farther behind in implementation? Possible Reasons: Understaffed Limited Bandwidth More staff traveling that previous years Possible Actions: Hire additional staff Share resources if the implementation MUST be done by xxx date Set different schedules for each location for future projects
  • 21. Presenting and Interpreting Data 21 Reports Visually Appealing Visually Appealing Interpreted and Actionable _______% improved _______% improved from _______ and that means _________ . What we need is ______ based on requirements for __________ . Going forward we should consider doing ___________ .
  • 22. 22 Measuring for value not numbers Examples to work with Defining, refining and Interpreting data/results for the intended audience
  • 23. EXAMPLE Metric : Baseline Defenses Coverage (Antivirus, Antispyware, Firewall, etc) 23 Measurement of how well we are protecting our enterprise against the most basic information security threats. Just Numbers: ________ % What would an additional relevant value be that we can use to have SMART data? Metrics: ________ % Increase since (prior month/inception/year over year/etc.) Device Type Location Length of time it took to detect
  • 24. EXAMPLE Metric : Legitimate E-Mail Traffic Analysis 24 Legitimate e-mail traffic analysis is a family of metrics including incoming and outgoing traffic volume, incoming and outgoing traffic size, and traffic flow between our company and others. By monitoring legitimate e-mail flow over time, we can learn where to set alarm points. Numbers: Compare the amount of good and junk e-mail that we are receiving ____ percent good ____ percent junk What would an additional relevant value be that we can use to have SMART data? Metrics ____ percent good ____ percent junk Quarterly/Annually/Since inception/Current Month Since adding the _________ criteria Received from _________ types/organizations Sent During ____________ (AM/PM – Holidays , etc.) Junk Detected Quicker _______ (first time/second time)
  • 25. Conclusion 25 By presenting information in a sufficiently granular way we can inject business relevance into the exhibits. Producing a benchmark is also a powerful approach to performance improvement. Percentage of computers with current anti-virus definitions City A 99.4 % City B 94.7 % City C 89.8 % 50 % 55 % 60 % 65 % 70 % 75 % 80 % 85 % 90 % 95 % 100 % Frequently this level of visibility will spark a competitive fire in those being measured. Professional pride will drive most people to make sure they are found among the high performers on your report.

Editor's Notes

  1. First let’s agree to the definition and rationale for Metrics and their true intent.
  2. Asking some basic questions will help determine the direction of the program.
  3. Metrics can be used in a variety of ways – it is important to understand, evaluate and decide how the metrics need to be used before launching a metrics program.
  4. From a management (C-level, VP and AVP) level, why do it if we can’t prove its value or the expenditure?
  5. Let’s look at a few qualities that will make our data useable and will provide the answer to cost and value.
  6. These are some of the elements that most Information Security teams use as a basis for metrics reports – a lot of teams report on all of these and others as well. But measuring these may not provide the value that management wants or needs. Some of the data is back-end data that will only be meaningful to the InfoSec team – it’s important to begin the process of separation, evaluation, presentation and relevance to the intended audience(s).
  7. We just looked at a list of data points that can be measured and tracked – for each one that is selected, we should be able to know the answers for the What, Who and Why and then set them by priority.
  8. *While 93.4% looks good on the surface and the trend seems upward overall, Without comparison points, those numbers don’t provide effectiveness over time, how many virus’ were known, how many were unknown/new, how quickly they were caught, where they originated, etc.
  9. Once we determine which data points are relevant and have added at least 2 data points, we then have to look at how we present the reports – visually pleasing – and apply interpretation to those metrics. It’s important to present information in such a way that the audience does not have to try to figure out on their own what the significance of the result shows – and the benefit of this is that now we can have actionable results.Example for hiring: We only have a 50% improvement from last year at this time and that means that we are behind schedule of wanting to be 100% complete by years’ end. What we need is 2 more headcount based on requirements for implementation as agreed in the IRMAC meeting and supported by senior management. Going forward we should consider hiring additional staff before setting a tight schedule.Example for positive reinforcement: We have a 50% improvement from last year at this time and that means that we are ahead of schedule of wanting to be 100% complete by years’ end. What we need is to continue this program based on requirements for implementation as agreed in the IRMAC meeting and supported by senior management. Going forward we should consider doing things in this same manner since it is so effective.