Metrics & Maturity Cartagena, 5 de Agosto de 2009 © ISM3 Consortium 2009

Goals, Objectives, Thresholds, Processes, Value, Management, Metrics, Maturity

Goals

The motivation is to achieve broad needs, wanted p ositive outcomes , e.g: meeting commitments.

Goal: Success

Goal: Quality

Goal: Security

Obligations

The motivation is to prevent unwanted negative outcomes, e.g: breach of contract.

Goals & Obligations Business Goals Obligations Success What we want to What we have to Quality As well as we want to As well as we have to Security As reliably as we want to As reliably as we have to

Success: Achieve Mission Quality: Achieve Vision Security: Existence

Success: Have Reputation Quality: Grow Reputation Security: Maintain Reputation

Success: Have Brand Quality: Grow Brand Security: Maintain Brand

Success: Have Revenue Quality: Grow Revenue Security: Maintain Revenue

Success: Get Talent Quality: Foster Talent Security: Retain Talent

Success: Comply with Social Responsibilities

Success: Comply with Regulations

Success: Comply with Contracts

Success: Comply with Ethics

Technical Technical Obligations Success: S ystems are as free of weaknesses as possible, Systems are visible to trusted systems only, Systems that need to be visible to not trusted systems are the least visible possible, Systems run trusted services only, The electricity, temperature and humidity where systems operate exceeds the systems needs.

Objectives: Specific things that have to happen for Goals and Obligations to be met

Objectives: Specific things that have to happen for Goals and Obligations to be met

Targets: A threshold for an Objective to be considered successful.

Sample Objectives Paying the payroll on the 1st of every month; Paying all incoming invoices within a certain time frame; Paying taxes in time; Invoicing all products and services provided; Delivering the products and services when and where committed by the organization; Keeping all necessary records to pass any audit successfully (i.e., tax audit, software license audit, etc.)

Paying the payroll on the 1st of every month;

Paying all incoming invoices within a certain time frame;

Paying taxes in time;

Invoicing all products and services provided;

Delivering the products and services when and where committed by the organization;

Keeping all necessary records to pass any audit successfully (i.e., tax audit, software license audit, etc.)

Payroll Objectives Payroll Contribute to Goals Contribute to Obligations Success Reward your work Because we have a contract Quality Broken down so you understand each item With the minimum fields mandated Security No excuses not to We will get fined if we regularly fail

Payroll Thresholds Payroll Goals Obligations Success No later than the 2 nd commercial day of the month No later than the latest date permitted by labour law Quality Understood by 95% of freshly recruited employees without help Get reported less than once a year Security Successful 97% of the months in ten years Get fined less than once every five years

Sample Information Security Objectives Goals -> “Secrets should be accessible to authorized users only” Obligations -> “Repositories with Personal information have to be registered with the Data Protection agency” Technical Obligations -> “Systems are as free of weaknesses as possible”

Goals -> “Secrets should be accessible to authorized users only”

Obligations -> “Repositories with Personal information have to be registered with the Data Protection agency”

Technical Obligations -> “Systems are as free of weaknesses as possible”

Deal with goals, coordination and provision of resources; Deal with the design and implementation of the ISM system, objectives and management of resources; Deal with achieving defined objectives by means of technical processes. Management Levels Strategic Practices Tactical Practices Operational Practices

Deal with goals, coordination and provision of resources;

Deal with the design and implementation of the ISM system, objectives and management of resources;

Deal with achieving defined objectives by means of technical processes.

Generic Goals Prevent and mitigate incidents that could jeopardize the organization’s property and the output of products and services that rely on information systems; Optimise the use of information, money, people, time and infrastructure.

Prevent and mitigate incidents that could jeopardize the organization’s property and the output of products and services that rely on information systems;

Optimise the use of information, money, people, time and infrastructure.

Strategic Goals Provides leadership and coordination of: Information security; Physical security; Workplace security (outside scope of ISM3); Interaction with organizational units; Reviews and improves the information security management system; Provides resources for information security; Defines relationships with other organisations; Defines Security Objectives consistent with organizational goals and obligations, protecting stakeholders interests; Sets the organizational scheme of delegation.

Provides leadership and coordination of:

Information security;

Physical security;

Workplace security (outside scope of ISM3);

Interaction with organizational units;

Reviews and improves the information security management system;

Provides resources for information security;

Defines relationships with other organisations;

Defines Security Objectives consistent with organizational goals and obligations, protecting stakeholders interests;

Sets the organizational scheme of delegation.

Tactical Goals Provide feedback to Strategic Management; Define the environment for Operational Management. Define Security Targets; Select appropriate processes to achieve the Security Targets; Manage budget, people and other resources allocated to information security

Provide feedback to Strategic Management;

Define the environment for Operational Management.

Define Security Targets;

Select appropriate processes to achieve the Security Targets;

Manage budget, people and other resources allocated to information security

Operational Goals Provide feedback to Tactical Management; Identify and protect assets; Protection and support of information systems throughout their lifecycle; Management of the security measures lifecycle; Apply allocated resources efficiently and effectively; Carry out processes for incident prevention, detection and mitigation (both real time and following an incident).

Provide feedback to Tactical Management;

Identify and protect assets;

Protection and support of information systems throughout their lifecycle;

Management of the security measures lifecycle;

Apply allocated resources efficiently and effectively;

Carry out processes for incident prevention, detection and mitigation (both real time and following an incident).

Process Template

Process Description: The activity performed in the process. Documentation: Policies, Procedures and Templates Process Definitions needed to describe and perform the process. Process Owner: Every process should have one and no more than one process owner.

Description: The activity performed in the process.

Documentation: Policies, Procedures and Templates Process Definitions needed to describe and perform the process.

Process Owner: Every process should have one and no more than one process owner.

Process Inputs: Inputs to the process. (Inputs in italics or obtained from sources other than documents) Work Products: Results of the process. (Work Products in italics are work products other than documents) Value: How the process contributes to specific and generic goals.

Inputs: Inputs to the process. (Inputs in italics or obtained from sources other than documents)

Work Products: Results of the process. (Work Products in italics are work products other than documents)

Value: How the process contributes to specific and generic goals.

Value

Managing is achieving goals with the resources available for it. There are specific activities for management that we will call “Management Practices”. Management

Managing is achieving goals with the resources available for it.

There are specific activities for management that we will call “Management Practices”.

Testing: Assessment of whether process outputs are as expected when test data is put in. Mana gement Practices

Testing: Assessment of whether process outputs are as expected when test data is put in.

Monitoring: Checking whether the outputs of the process and the resources used are within normal range. Management Practices

Monitoring: Checking whether the outputs of the process and the resources used are within normal range.

Improving: Making changes in the process to make it more suitable for the purpose, or to reduce usage of resources. Management Practices

Improving: Making changes in the process to make it more suitable for the purpose, or to reduce usage of resources.

Planning. Organizing and forecasting the amount, assignment and milestones of tasks, resources, budget, deliverables and performance of a process. Management Practices

Planning. Organizing and forecasting the amount, assignment and milestones of tasks, resources, budget, deliverables and performance of a process.

Assessment. How well the process matches the organization's needs and compliance goals. Management Practices

Assessment. How well the process matches the organization's needs and compliance goals.

Audit. Whether the process inputs, activities and results match their documentation. Management Practices

Audit. Whether the process inputs, activities and results match their documentation.

Certify: Whether the process inputs, process documentation, activities and results comply with a pre-defined standard, law or regulation. Management Practices

Certify: Whether the process inputs, process documentation, activities and results comply with a pre-defined standard, law or regulation.

Benefits realization: Show how achieving security objectives contributes to achieving business objectives. Management Practices

Benefits realization: Show how achieving security objectives contributes to achieving business objectives.

The more sophisticated your management practices, the higher your capability. Management and Capability

The more sophisticated your management practices, the higher your capability.

It is possible to assess the capability of a process checking the metrics used to manage it. Management You can perform few management practices without metrics.

It is possible to assess the capability of a process checking the metrics used to manage it.

You can perform few management practices without metrics.

Metrics A quantitative measurement that can be interpreted in the context of a series of previous or equivalent measurements

A quantitative measurement that can be interpreted in the context of a series of previous or equivalent measurements

Ty pes of Process Metrics Activity: Number of outputs produced and their mean age.

Activity: Number of outputs produced and their mean age.

Types of Process Metrics Scope: Percentage of all inputs producers covered by this process.

Scope: Percentage of all inputs producers covered by this process.

Types of Process Metrics Unavailability: Number, frequency and duration of interruptions in the normal operation of the process.

Unavailability: Number, frequency and duration of interruptions in the normal operation of the process.

Types of Process Metrics Effectiveness: Number, mean time between inputs and percentage of Inputs that produce an Output.

Effectiveness: Number, mean time between inputs and percentage of Inputs that produce an Output.

Types of Process Metrics Efficiency: Ratio between the number of outputs submitted and the available resources for this process in actual use.

Efficiency: Ratio between the number of outputs submitted and the available resources for this process in actual use.

Types of Process Metrics Load: Percentage of resources reserved for the process in actual use.

Load: Percentage of resources reserved for the process in actual use.

Types of Process Metrics Quality: Measure of the fitness for purpose of the outputs.

Quality: Measure of the fitness for purpose of the outputs.

Description of what is measured How is the metric measured How often is the measurement taken How are the thresholds calculated Current range of values considered normal for the metric Best possible value of the metric Units of measurement Metrics Specification

Description of what is measured

How is the metric measured

How often is the measurement taken

How are the thresholds calculated

Current range of values considered normal for the metric

Best possible value of the metric

Units of measurement

Sample: Optimizing Cobit: “Good Practices are followed and automated” CMMI: “An optimizing process is a quantitatively managed process that is improved based on an understanding of the common causes of variation inherent in the process. The focus of an optimizing process is on continually improving the range of process performance through both incremental and innovative improvements”

Cobit: “Good Practices are followed and automated”

CMMI: “An optimizing process is a quantitatively managed process that is improved based on an understanding of the common causes of variation inherent in the process. The focus of an optimizing process is on continually improving the range of process performance through both incremental and innovative improvements”

3 – Definición Objetiva de Madurez

Value

Goals

THANK YOU

ISM3 Maturity Levels (examples) ISM3 Basic Level - Significant risk reduction from technical threats, for a minimum investment in essential ISM processes. For organizations with low Information Security Targets in low risk environments. ISM3 SMEs Level - Highest risk reduction from technical threats, for a significant investment in Information Security processes. For organizations with high Information Security Targets in normal or high-risk environments. ISM3 Military Level - Highest risk reduction from technical and internal threats, for a high and optimized investment in Information Security processes. For organizations affected by specific requirements (such as utilities, and financial institutions) with high Information Security Targets in normal or high-risk environments.

ISM3 Basic Level - Significant risk reduction from technical threats, for a minimum investment in essential ISM processes.

For organizations with low Information Security Targets in low risk environments.

ISM3 SMEs Level - Highest risk reduction from technical threats, for a significant investment in Information Security processes.

For organizations with high Information Security Targets in normal or high-risk environments.

ISM3 Military Level - Highest risk reduction from technical and internal threats, for a high and optimized investment in Information Security processes.

For organizations affected by specific requirements (such as utilities, and financial institutions) with high Information Security Targets in normal or high-risk environments.

ISM3 Capability Levels Capability Level Metrics Requirements Enabled Managed Practices Basic Documentation Audit and Certify. Defined Basic, plus Activity, Scope, Unavailability and Effectiveness Basic, plus Test Managed Defined, plus Load Defined, plus Monitor, Benefits Realization, Planning and removing weaknesses before they produce incidents, and getting feedback on the result of changes. Controlled Managed, plus Quality Managed, plus Assessment and removing bottlenecks that hamper performance. Optimized Controlled, plus Efficiency Controlled, plus finding points of diminishing return and making trade-offs.

Use case – Malware Management Use case – ISM3-less management Motivation: Clean viruses or your business will sink. Objective: No system should get a virus ever Activity: Install antivirus on personal computers, servers, mail servers, add antivirus functionality to firewalls, add antispyware, antitrojan, antirookit to the mix. Policy: Prevent any USB, DVD, to touch any company system without being searched for viruses. Success criterion: When no system gets ever a virus. Continuous improvement: Add more antimalware controls (Tripwire, CORE, etc)

Use case – ISM3-less management

Motivation: Clean viruses or your business will sink.

Objective: No system should get a virus ever

Activity: Install antivirus on personal computers, servers, mail servers, add antivirus functionality to firewalls, add antispyware, antitrojan, antirookit to the mix.

Policy: Prevent any USB, DVD, to touch any company system without being searched for viruses.

Success criterion: When no system gets ever a virus.

Continuous improvement: Add more antimalware controls (Tripwire, CORE, etc)

Use Case – ISM3-style management Motivation: Unfortunately systems, specially Windows and malware prone. We should invest proportionally to the damage they can make. Goal: Systems should accomplish their business role with or without malware. Activity: Install antimalware in vulnerable systems. Measure activity, scope, update and availability of antimalware. Consider other measures, like using less malware prone systems. Policy: Use in every system the antimalware protection that will detect malware and prevent the system from failing to play its business role. Success criterion: When protected system play their business role without interruption or degradation. Continuous improvement: Use metrics to improve the antimalware protection and use those with better effectively and ROI. Use case – Malware Management

Use Case – ISM3-style management

Motivation: Unfortunately systems, specially Windows and malware prone. We should invest proportionally to the damage they can make.

Goal: Systems should accomplish their business role with or without malware.

Activity: Install antimalware in vulnerable systems. Measure activity, scope, update and availability of antimalware. Consider other measures, like using less malware prone systems.

Policy: Use in every system the antimalware protection that will detect malware and prevent the system from failing to play its business role.

Success criterion: When protected system play their business role without interruption or degradation.

Continuous improvement: Use metrics to improve the antimalware protection and use those with better effectively and ROI.