Smu seminar 2014_03_26 v3

359 views
295 views

Published on

Fundamental concepts and definitions for risk analysis, measurement, probability, scales of measurement, and data

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
359
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Smu seminar 2014_03_26 v3

  1. 1. Fundamentals Matter – A Brief Introduction to Risk Analysis for Information Security Southern Methodist University, March 26, 2014 Heather Goodnight, President Patrick Florer, CTO Risk Centric Security, Inc. www.riskcentricsecurity.com Authorized reseller of ModelRisk from Vose Software Risk Centric Security, Inc. Confidential and Proprietary . Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Risk Analysis for the 21st Century®
  2. 2. • Introductions • What we are going to talk about o Why Fundamentals Matter / Current State o Definitions • Risk and the Risk Landscape • Possibility and Probability • Measurement • Variability and Uncertainty • Precision vs. Accuracy • Scales of Measurement: Qualitative vs. Quantitative • Not Enough Data • Monte Carlo Simulation • Modeling Expert Opinion and PERT distributions Agenda Risk Centric Security, Inc. Confidential and Proprietary . Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  3. 3. Heather Goodnight is an accomplished Global Sales and Business Development Consultant. Over the years, her unique, practical insight into problems of risk and opportunity have provided important guidance for organizations both large and small. She is a cofounder of Risk Centric Security and currently serves as President of the Corporation. In 2010, she was appointed to the RIM Council (Responsible Information Council) of the Ponemon Institute. In addition to her role at Risk Centric Security, she serves as Business Development Manager at Triumfant, Inc., a vendor of advanced anti-malware products. Patrick Florer has worked in information technology for almost 35 years. For 17 years, he worked a parallel track in medical outcomes research, analysis, and the creation of evidence-based guidelines for medical treatment. His roles have included IT operations, programming, and systems analysis. From 1986 until now, he has worked as an independent consultant, helping customers with strategic development, analytics, risk analysis, and decision analysis. He is a cofounder of Risk Centric Security and currently serves as Chief Technology Officer. He is a member of the Ponemon Institute RIM council. In 2012, he was appointed Distinguished Fellow of the Ponemon Institute. Introductions Risk Centric Security, Inc. Confidential and Proprietary . Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  4. 4. The Current State of Confusion … . Risk Centric Security, Inc. Confidential and Proprietary . Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  5. 5. ROI IRR EPS EMV EBITDA ≠ Often leads to this … Risk Centric Security, Inc. Confidential and Proprietary . Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  6. 6. What is Risk? Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  7. 7. What Risk Isn’t! Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Vulnerability Threat
  8. 8. Risk = Frequency x Impact Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Frequency Impact Risk
  9. 9. Risk and Opportunity Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  10. 10. Possibility and Probability: Possibility Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  11. 11. Possibility and Probability: Probability Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  12. 12. What is a Measurement? Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  13. 13. Properties of Measurement Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Validity Reproducibility Detail
  14. 14. Sources of Error in Measurement? Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Random Error Errors from Bias
  15. 15. Variability and Uncertainty Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Variability Uncertainty
  16. 16. Precision and Accuracy Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  17. 17. Scales of Measurement Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Qualitative Quantitative
  18. 18. Qualitative Scales Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Nominal/Categorical IntervalOrdinal HIGH - Red MEDIUM - Orange LOW - Green First, Second, Third … On a scale of …
  19. 19. Quantitative/Ratio Scales Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. 1, 2, 3, 4, 5, 6, … n
  20. 20. Problems with Qualitative Scales Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. My Scale High Medium Low Red Orange Green Your Scale High Medium Low Red Orange Yellow Green (RED – GREEN + MEDIUM) / Somewhat Likely = ??? Mismatched Scales Meaningless Calculations Assessor Disagreements
  21. 21. Problems with Qualitative Scales Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Boundary Problems $2.5M Loss Exposure = Moderate = Yellow $2.5M Loss Exposure = Moderate = Yellow $2.5M Loss Exposure = Moderate = Yellow $7.5M Aggregate Loss Exposure = not so Moderate ! Issues with Loss of Information
  22. 22. Quantitative Scales Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. 2 + 2 - 1 = 3 360 * 10 = 3,600 Sqrt(25) = 5 f(x) = y etc.
  23. 23. Qualitative Methods - Problems Difficulty with arithmetic and statistical operations From ISO 17999 Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  24. 24. Qualitative Methods - Problems Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  25. 25. Qualitative Methods - Problems On a scale of 1 to 5, where 1 = least and 5 = most, please rate … Likert scale (From Wikipedia, the free encyclopedia) When responding to a Likert questionnaire item, respondents specify their level of agreement or disagreement … In so doing, Likert scaling assumes that distances on each item are equal … Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  26. 26. Data Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Good Data Bad Data Big Data Little Data
  27. 27. How much data is enough data? Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. How do I get to the mall? How do we build this? vs.
  28. 28. Data from Calibrated Estimates Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. More often than you might think, the data we have to work with comes from Subject Matter Experts (SME’s). How can we improve the accuracy of these SME’s – to a 90% confidence level? With calibration. Example: How much does an iPhone 5s weigh?
  29. 29. Monte Carlo Simulation Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. The average = $12,500 $2,500 $12,500 $32,000 The range is: The distributions are:
  30. 30. Monte Carlo Simulation Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  31. 31. The Beta Pert Calculator Minimum: What is the least or lowest (best or worst) numerical estimate that you believe to be reasonable? This will be the smallest number you come up with. Most Likely: What is the most likely or most probable numerical estimate in your opinion? This number must fall between the minimum and maximum. It may equal either the minimum or the maximum, but should not equal both Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  32. 32. The Beta Pert Calculator Maximum: What is the greatest or highest (best or worst) numerical estimate that you believe to be reasonable? Note that “best” or “worst” case estimates could be either minimum or maximum values, depending upon the scenario. In a risk / loss exposure scenario, lower is better, so the minimum represents the lowest loss, or best outcome. The maximum represents the highest loss, or worst outcome. In a sales or opportunity scenario, it’s the reverse: lower is not better, so the minimum represents the worst case. Higher is better, so the maximum represents the best case. Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  33. 33. The Beta Pert Calculator Confidence: On a scale that includes “Very Low”, “Low”, “Average”, “High”, and “Very High”, how confident are you in the accuracy of your estimates? This parameter controls the sampling around the most likely value, and thereby also controls the height of the histogram or slope of the cumulative plot. For most analyses, using “Average” for the confidence parameter works well. In this instance, “Average” really means having no strong feeling about the matter – being evenly divided between under-confidence and over- confidence. Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  34. 34. The Beta Pert Calculator Percentile Tables Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  35. 35. The Beta Pert Calculator Percentile Tables 1% of values are <= 10,044 and 99% are > 10,044 10% of values are <= 11,120 and 90% are > 11,120 20% of values are <= 11,658 and 80% are > 11,658 50% of values are <= 13,025 and 50% are > 13,025 The 50th percentile has another name - it’s called the Median. The Median is the mid-point in a list of values - half of the values in the list are less and half are greater than the Median. Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  36. 36. The Beta Pert Calculator Histogram Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  37. 37. The Beta Pert Calculator Cumulative Plot Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  38. 38. Thank you ! Heather Goodnight President and Cofounder Patrick Florer CTO and Co-founder Risk Centric Security, Inc patrick@riskcentricsecurity.com 214.828.1172 Authorized reseller of ModelRisk from Vose Software Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved. Risk Analysis for the 21st Century ®
  39. 39. ”We don’t have enough data!” - Sources Open Security Foundation: datalossdb and osvdb http://www.opensecurityfoundation.org/ Office of Inadequate Security: http://www.databreaches.net/ Identity Theft Resource Center: http://www.idtheftcenter.org/ ISACA: www.isaca.org ISSA: www.issa.org Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  40. 40. ”We don’t have enough data!” - Sources Mitre Corporation: www.mitre.org OWASP: http://owasp.com/index.php/Main_Page Privacy Rights Clearing House: http://www.privacyrights.org/ SANS: www.sans.org The Ponemon Institute: www.ponemon.org Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  41. 41. ”We don’t have enough data!” - Sources Conference procedings: Black Hat, RSA, Source Conferences, BSides Internet tools: Search engines: Google, Bing, Yahoo, Ask.com Trend Analyzers: Google trends: http://www.google.com/trends Twitter Trends: www.trendistic.com Amazon: http://www.metricjunkie.com/ Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
  42. 42. ”We don’t have enough data!” - Sources Securitymetrics.org – mailing list Society of Information Risk Analysts (SIRA) Books: How to Measure Anything – Hubbard The Failure of Risk Management – Hubbard Risk Analysis: A Quantitative Guide – Vose Clinical Epidemiology and Biostatistics – Kramer Data-Driven Security: Analysis, Visualization and Dashboards – Jacobs and Rudis Risk Centric Security, Inc. Confidential and Proprietary. Copyright © 2014 Risk Centric Security, Inc . All rights reserved.

×