How To Set Security Awareness Strategic Goals, KPIs and Metrics

© Terranova Worldwide Corporation 2019. All rights reserved.
Security Awareness Objectives & Metrics
An Overview and Discussion
Theo Zafirakos | CISO / Professional Services

Discussion Topics
• Strategic Goals
• Objectives, KPIs & Metrics
• Reporting & Optimizing

KPI
A key performance indicator is used to measure program performance and success.
They should be aligned to the objectives of the program.
Metric
A metric is a number within a KPI that helps track the progress and performance of
the program.
All KPIs are metrics, but not all metrics are KPIs.
Definitions – KPI vs Metric

Strategic Goals

Implementing a Measurement Framework
5
Increase Awareness
Structured Learning
Testing and Verifications
Continuous Awareness
Measure
Ad hoc
messages
Newsletters
Fliers
Mandatory
CBTs
Role Based
Risk Based
Phishing
Simulations
Quiz
Audits
Reinforcement
Lunch-n-Learn
Micro- and
Nano-learnings
Objectives
KPIs
Metrics

Why are you deploying a security awareness program?
What are you trying to achieve?
• Strategic goals must clearly identify what you aim to achieve.
• Goals must be identified so a plan can be laid out with all the steps required get
there.
• Goals must be aligned with the Information Security program and even the business
strategy of the organization.
• Clearly defined goals will help decision makers approve and support your program.
Introduction

Have you defined security awareness strategic goals for your program?
• Yes, as part of the Information Security strategic goals
• Yes
• No
• I don't know
Poll Question 1

Your security awareness goals could be in
any or all of these three categories:
• Risks and Behaviors
• to reduce risk and foster behavioral
changes
• Security Culture
• to instill or reinforce a culture of
security
• Compliance Obligations
• to ensure compliance with your
organization’s security obligations
Strategic Goals

• Reduce human errors
• Encourage everyone at your organization
to adopt security best practices
• Reduce human-related security incidents
• Address the rapid changes in your
organization’s threat landscape
Strategic Goals – Risks and Behaviors

Risks and Behaviors
Minimize enterprise risks and costs from cybersecurity threats by empowering employees
with the motivation and knowledge they need to employ good behaviors.
Strategic Goals – Example 1

Strategic Goals – Security Culture
• Demonstrate the importance of
information security
• Mobilize managers to become security
and awareness ambassadors
• Change attitudes about towards security
• Get people to consider the security
implications of their actions
Opinions Beliefs
Views Feelings
Culture

Security Culture
Enable employees in making safe cyber security decisions in their day to day activities by
creating a security-aware culture.

• Meet legal, regulatory, or industry awareness compliance obligations
• Fulfill contractual agreements regarding security and privacy awareness clauses
• Enforce organizational security policies and standards
Strategic Goals – Compliance Obligations

Compliance Obligations
Minimize corporate liability, risks and costs arising from non-compliance for the protection
of personal information or other regulated information.

Clearly defined and concrete goals are essential.
They will allow you to plan strategically and to
develop an awareness program that is focused on
producing tangible results.

Objectives, KPIs & Metrics Program/ Campaign Results
Participation Rates
Metrics and KPIs DataMetrics and KPIsMetrics and KPIs
Participation RatesParticipation Rates
DataData
Program/ Campaign ResultsProgram/ Campaign Results

An LMS can be used to measure the participation rate to online training activities and to
determine the percentage of users who have completed the courses successfully.
Examples of metrics
• % of participation per course
• % completion per course
• % failed by course
• % not completed by course
• % not started by course
Online Training

• The only metric we need to capture is how many persons completed the
mandatory on-line training.
Most Common Mistake
This does not demonstrate whether or
not the students understand the
materials, or if they are putting what
they learned into practice.

• Once the program’s long-term strategic goals have been defined, specific objectives for
each of the campaigns must be determined.
• A program consists of several smaller campaigns, each designed to meet its own set of
objectives.
• Enough time to measure campaign success and implement an action plan if objectives
and goals are not met.
Campaign Objectives

Objective – Training Participation
20
KPI Metric Effectiveness Indicator
Users are aware of cyber
security risks and controls
Percentage of participants
who have completed
training
Increase in attendance

How often do you track online course participation statistics?
• Several times throughout the campaign.
• Only at the end of the campaign.
• When management requests participation reports.
• Once a year.
• We don't track or capture participation statistics.
Poll Question 2

Measures are identified in three areas:
• Compliance: Report on how many users have assisted in awareness activities to
demonstrate that the program is being followed by the users.
• Behavior/Knowledge: Report on what knowledge the users have retained as a result of
awareness activities.
• Culture: Report on whether users are applying what they learned in the awareness
activities.
KPIs

Objective – Defend Against Phishing
24
who have completed
training
Compliance Behavior Culture
KPI
Metric
Effectiveness
Indicator
All employees have
received training on the
Phishing attack method
Training participation
rates
Increase in the number of
users that participate to
online training
Reduction in the number
of incidents that result
from an email attack
Recorded malware
infections or other
incidents as a result of
phishing
Reduction in the number
of users that opened
attachments in real or
simulated phishing
attempts
Increase in number of
employees report
phishing activity to the
Service Desk
Reported phishing attacks
(e.g. simulations)
users who reported real
or simulated phishing
attempts

Objective – Comply with Privacy Regulations
25
who have completed
training
Compliance Behavior Culture
KPI
Metric
Effectiveness
Indicator
Employees are informed
on privacy principles,
laws and regulations
Number of employees
who have completed the
training on time
Decrease in the number
of follow-ups required
to achieve 100%
participation
Personal information is
handled in accordance to
privacy principles, laws
and regulations
Number of violations
related to privacy
regulations
Decrease in the number
of incidents or complaints
Employees take
responsibility for the
protection of personal
data
Number of
questions/notifications
that may have prevented
a data breach
inquiries related to the
proper protection of
regulated data

• Time required to recover an infected computer
• Time spent handling security incident related calls handled by the Service Desk or Security
Operations
• Unproductive time spent waiting for a computer to be recovered
• Time required to recover an infected server
• Productivity impact and revenue loss if a critical server is infected
• Time required to restore an encrypted file share
• Reputation and client confidence loss
Return on Investment (ROI)

Which subjective indicators are useful in observing that security behaviors are improving?
• Office chatter about the security awareness program
• Local information security representatives start to surface in various regions or departments
• Informal discussions are occurring about topics within the security awareness program
• Senior management supports information security program
• None of the above
Poll Question 3

Compare results to objectives
Goal
Objective
Measure
Activities Activities Activities
Measure
Activities Activities
Objective
Measure
Activities Activities
Unexpected
Positive or
Negative effect

• Associate a specific objective with each KPI
• Allow some time to measure progress (positive or negative)
• Do not select KPIs that require significant effort to implement or collect
• Focus on metrics to measure that are the most relevant
• Report and act on the results
Considerations

• How often do you report program KPIs and metrics?
• To who do you report program performance?
• How often do you adjust selected KPIs and target objectives?
Considerations

Typical activities:
• Analyze metrics
• Compare results with campaign objectives and program goals
• Identify improvement opportunities
• Identify new objectives
• Conduct a post-mortem meeting
Optimize – How do you optimize your program?

• Metrics and KPIs should be tracked prior to program/campaign deployment
to set a baseline against which future results will be compared.
• Participation rates should be monitored as soon as online training is
launched, and on an ongoing basis to track progress and trends.
• Give your program enough time to track culture and behavior improvements
(1 year or more).
Summary

Example metrics 1
33
Topic Objectives Metrics Mode of Measure
xx% of employees that acquired
the required knowledge during the
introduction (score > xx%)
Post-training Quiz
xx% of employees who completed
the course within the time limit
LMS reports
xx% of employees satisfied with
course content (satisfaction rating
> x)
Satisfaction Survey
Email
Reduce the risk of information leakage
/ data breach through email
Decrease the number of
occurrences of business messages
sent to a personal email (Gmail,
Yahoo, etc.) (baseline vs post-
training)
Survey
Audits
xx% of employees that know how
to create passwords that follow
the guidelines
Survey
xx% of employees that create
passwords that follow the
guidelines
Audits
Introductions to Information Security
Introduction
Employees have a basic understanding
of information security
Passwords
Ensure access to business information
and technology is secured with strong
passwords

Example metrics 2
34
Topic Objectives Metrics Mode of Measure
Reduction in the number of
instances employees use the
Internet for personal purposes
Audit
HTTP Logs
instances employees download
files from the Internet for personal
purposes
Audit
HTTP Logs
employees who post proprietary,
personal content or sensitive
information on the web
Audit
Reported incidents
Reduction in corporate bandwidth
usage by employees streaming
media online
Survey
Audit (network traffic)
Confidentiality on the Web
Reduce the number of corporate
system or data compromises as a
result of unsafe web surfing behaviors
% of sample employees know
features of a secure web page
when conducting financial
transactions or posting personal or
confidential information
Phishing simulations
Survey
Internet
Responsible Use of the Internet at Work
Reduce the number of policy
violations related to acceptable use of
the corporate Internet service

HOW DOES YOUR CLICK RATE STACK UP?
Register for The Gone Phishing Tournament
today to find out.
October 2019
FREE REGISTRATION (Tournament will take place on week of October 21st)
TerranovaSecurity.com/GonePhishingTournament

Questions

How To Set Security Awareness Strategic Goals, KPIs and Metrics

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to How To Set Security Awareness Strategic Goals, KPIs and Metrics

Similar to How To Set Security Awareness Strategic Goals, KPIs and Metrics (20)

Recently uploaded

Recently uploaded (20)

How To Set Security Awareness Strategic Goals, KPIs and Metrics

Editor's Notes