Smu seminar 2014_03_26 v3

Fundamentals Matter – A Brief
Introduction to Risk Analysis for
Information Security
Southern Methodist University, March 26, 2014
Heather Goodnight, President
Patrick Florer, CTO
Risk Centric Security, Inc.
www.riskcentricsecurity.com
Authorized reseller of ModelRisk from Vose Software
Risk Centric Security, Inc. Confidential and Proprietary .
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Risk Analysis for the 21st Century®

• Introductions
• What we are going to talk about
o Why Fundamentals Matter / Current State
o Definitions
• Risk and the Risk Landscape
• Possibility and Probability
• Measurement
• Variability and Uncertainty
• Precision vs. Accuracy
• Scales of Measurement: Qualitative vs. Quantitative
• Not Enough Data
• Monte Carlo Simulation
• Modeling Expert Opinion and PERT distributions
Agenda

Heather Goodnight is an accomplished Global Sales and Business Development
Consultant. Over the years, her unique, practical insight into problems of risk and
opportunity have provided important guidance for organizations both large and
small. She is a cofounder of Risk Centric Security and currently serves as President
of the Corporation. In 2010, she was appointed to the RIM Council (Responsible
Information Council) of the Ponemon Institute. In addition to her role at Risk
Centric Security, she serves as Business Development Manager at Triumfant, Inc.,
a vendor of advanced anti-malware products.
Patrick Florer has worked in information technology for almost 35 years. For 17
years, he worked a parallel track in medical outcomes research, analysis, and the
creation of evidence-based guidelines for medical treatment. His roles have
included IT operations, programming, and systems analysis. From 1986 until now,
he has worked as an independent consultant, helping customers with strategic
development, analytics, risk analysis, and decision analysis. He is a cofounder of
Risk Centric Security and currently serves as Chief Technology Officer. He is a
member of the Ponemon Institute RIM council. In 2012, he was appointed
Distinguished Fellow of the Ponemon Institute.
Introductions

The Current State of Confusion …
.

ROI IRR
EPS
EMV
EBITDA
≠
Often leads to this …

What is Risk?
Risk Centric Security, Inc. Confidential and Proprietary.

What Risk Isn’t!
Vulnerability Threat

Risk = Frequency x Impact
Frequency
Impact
Risk

Risk and Opportunity

Possibility and Probability: Possibility

Possibility and Probability: Probability

What is a Measurement?

Properties of Measurement
Validity
Reproducibility
Detail

Sources of Error in Measurement?
Random Error
Errors from Bias

Variability and Uncertainty
Variability
Uncertainty

Precision and Accuracy

Scales of Measurement
Qualitative Quantitative

Qualitative Scales
Nominal/Categorical
IntervalOrdinal
HIGH - Red
MEDIUM - Orange
LOW - Green
First, Second, Third … On a scale of …

Quantitative/Ratio Scales
1, 2, 3, 4, 5, 6, … n

Problems with Qualitative Scales
My Scale
High
Medium
Low
Red
Orange
Green
Your Scale
High
Medium
Low
Red
Orange
Yellow
Green
(RED – GREEN + MEDIUM) / Somewhat Likely
= ???
Mismatched Scales
Meaningless Calculations
Assessor Disagreements

Problems with Qualitative Scales
Boundary Problems
$2.5M Loss Exposure = Moderate = Yellow
$7.5M Aggregate Loss Exposure = not so Moderate !
Issues with Loss of Information

Quantitative Scales
2 + 2 - 1 = 3
360 * 10 = 3,600
Sqrt(25) = 5
f(x) = y
etc.

Qualitative Methods - Problems
Difficulty with arithmetic and statistical operations
From ISO 17999

On a scale of 1 to 5,
where 1 = least and 5 = most,
please rate …
Likert scale (From Wikipedia, the free encyclopedia)
When responding to a Likert questionnaire item, respondents
specify their level of agreement or disagreement …
In so doing, Likert scaling assumes that distances
on each item are equal …

Data
Good Data Bad Data
Big Data
Little Data

How much data is enough data?
How do I get to the mall?
How do we build this?
vs.

Data from Calibrated Estimates
More often than you might think, the data we have to work with
comes from Subject Matter Experts (SME’s).
How can we improve the accuracy of these SME’s – to a 90%
confidence level?
With calibration.
Example: How much does an iPhone 5s weigh?

Monte Carlo Simulation
The average = $12,500
$2,500 $12,500 $32,000
The range is:
The distributions are:

Monte Carlo Simulation

The Beta Pert Calculator
Minimum:
What is the least or lowest (best or worst) numerical
estimate that you believe to be reasonable? This will be the
smallest number you come up with.
Most Likely:
What is the most likely or most probable numerical estimate
in your opinion? This number must fall between the
minimum and maximum. It may equal either the minimum
or the maximum, but should not equal both

Maximum:
What is the greatest or highest (best or worst)
numerical estimate that you believe to be
reasonable?
Note that “best” or “worst” case estimates could be
either minimum or maximum values, depending upon
the scenario.
In a risk / loss exposure scenario, lower is better, so the
minimum represents the lowest loss, or best outcome.
The maximum represents the highest loss, or worst
outcome.
In a sales or opportunity scenario, it’s the reverse:
lower is not better, so the minimum represents the
worst case. Higher is better, so the maximum
represents the best case.

Confidence:
On a scale that includes “Very Low”, “Low”, “Average”,
“High”, and “Very High”, how confident are you in the
accuracy of your estimates?
This parameter controls the sampling around the most likely
value, and thereby also controls the height of the histogram
or slope of the cumulative plot.
For most analyses, using “Average” for the confidence
parameter works well. In this instance, “Average” really
means having no strong feeling about the matter – being
evenly divided between under-confidence and over-
confidence.

Percentile Tables

Percentile Tables
1% of values are <= 10,044 and 99% are > 10,044
The 50th percentile has another name - it’s
called the Median.
The Median is the mid-point in a list of values -
half of the values in the list are less and half
are greater than the Median.

Histogram

Cumulative Plot

Thank you !
Heather Goodnight
President and Cofounder
Patrick Florer
CTO and Co-founder
Risk Centric Security, Inc
patrick@riskcentricsecurity.com
214.828.1172
Authorized reseller of ModelRisk from Vose Software
Risk Analysis for the 21st Century ®

”We don’t have enough data!” - Sources
Open Security Foundation: datalossdb and osvdb
http://www.opensecurityfoundation.org/
Office of Inadequate Security:
http://www.databreaches.net/
Identity Theft Resource Center:
http://www.idtheftcenter.org/
ISACA: www.isaca.org
ISSA: www.issa.org

Mitre Corporation: www.mitre.org
OWASP: http://owasp.com/index.php/Main_Page
Privacy Rights Clearing House:
http://www.privacyrights.org/
SANS: www.sans.org
The Ponemon Institute: www.ponemon.org

Conference procedings: Black Hat, RSA, Source
Conferences, BSides
Internet tools:
Search engines: Google, Bing, Yahoo, Ask.com
Trend Analyzers:
Google trends:
http://www.google.com/trends
Twitter Trends: www.trendistic.com
Amazon:
http://www.metricjunkie.com/

Securitymetrics.org – mailing list
Society of Information Risk Analysts (SIRA)
Books:
How to Measure Anything – Hubbard
The Failure of Risk Management – Hubbard
Risk Analysis: A Quantitative Guide – Vose
Clinical Epidemiology and Biostatistics – Kramer
Data-Driven Security: Analysis, Visualization and Dashboards – Jacobs and
Rudis

Smu seminar 2014_03_26 v3

Recommended

Recommended

More Related Content

What's hot

What's hot (8)

Viewers also liked

Viewers also liked (19)

Similar to Smu seminar 2014_03_26 v3

Similar to Smu seminar 2014_03_26 v3 (20)

Recently uploaded

Recently uploaded (20)

Smu seminar 2014_03_26 v3