Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data Driven Risk Management

374 views

Published on

In this presentation, Joe and Brian contrast traditional risk assessment with some emerging techniques that use internal and market risk event (incident) data to drive a more accurate risk model.

Presentation by:
Joe Crampton, VP – Applications, Resolver Inc.
Brian Link, CIA, VP – GRC Strategy & Partnerships, Resolver Inc.

Published in: Business
  • Be the first to comment

Data Driven Risk Management

  1. 1. I am the VP Product at Resolver. @jcrampton joe@resolver.com
  2. 2. Participants will leave with.. Awareness of a variety of risk assessment techniques An introduction to machine learning as a risk assessment tool Knowledge of what type of risk assessment is right for what scenario.
  3. 3. What are the different techniques people employ? How does risk management work at most companies? What are the strengths and weaknesses of these techniques? What are some of the emerging techniques?
  4. 4. Risk An event that may impact your objectives. Frequency / Likelihood The probability of a risk event occurring. Impact The magnitude of risk event on your organization should it occur. Control Any action that reduces the impact and/or likelihood of the risk. Vulnerability / Control Effectiveness How well prepared are we for this risk?
  5. 5. Rating Definition Extreme • Financial loss of $X million or more • International long-term negative media coverage; game-changing loss of market share • Significant prosecution and fines, litigation including class actions, incarceration of leadership • Significant injuries or fatalities to employees or third parties, such as customers or vendors • Multiple senior leaders leave Major • … Moderate • … Minor • … Incidental • Financial loss up to $X million • Local media attention quickly remedied • Not reportable to regulator • No injuries to employees or third parties, such as customers or vendors • Isolated staff dissatisfaction Source: Coso.org
  6. 6. Rating Annual Frequency Probability in life of asset or project Frequent Up to once in 2 years or more > 90% Likely Once in 2 years up to once in 25 years 65%-90% Possible Once in 25 years up to once in 50 years 35%-65% Unlikely Once in 50 years up to once in 100 years 10%-35% Rare Once in 100 years or less <10% Source: Coso.org
  7. 7. Rating Definition Vey High • No scenario planning performed • Lack of enterprise level/process level capabilities to address risks • Responses not implemented • No contingency or crisis management plans in place High • … Medium • … Low • … Very Low • Real options deployed to maximize strategic flexibility • High enterprise level/process level capabilities to address risks • Redundant response mechanisms in place and regularly tested for critical risks • Contingency and crisis management plans in place and rehearsed regularly Source: Coso.org
  8. 8. 1. Divide into two groups 2. One group at a time we’re going to look at a math problem. 3. You’ll have 5 seconds to look at the problem and estimate an answer in your head 4. Remember your answer
  9. 9. CLOSE YOUR EYES
  10. 10. CLOSE YOUR EYES
  11. 11. The actual result. How close were you?
  12. 12. Mean Guess = 2,250Mean Guess = 512
  13. 13. The availability heuristic is a mental shortcut that relies on immediate examples that come to a given person's mind when evaluating a specific topic, concept, method or decision.
  14. 14. Contributing Factor Risk Event (n-1) Risk Event (n+1) Impact Risk Event
  15. 15. Low Employee Morale Unauthorized Access IP Theft Additional IT LoadAsset Theft Service Downtime
  16. 16. Low Employee Morale Unauthorized Access IP Theft Additional IT LoadAsset Theft Service Downtime
  17. 17. Low Employee Morale Unauthorized Access IP Theft Additional IT LoadAsset Theft Service Downtime I Employee Sat Survey I Access Control Monitoring
  18. 18. Low Employee Morale Unauthorized Access IP Theft Additional IT LoadAsset Theft Service Downtime I Employee Sat Survey I Access Control Monitoring C C C C C
  19. 19. Incident Whenever an incident occurs, we link it to risk? RISK Incident Incident Incident
  20. 20. Speak the language of the business. Understand the impact on objectives Improved risk assessment accuracy. Factual justification for assessment. Identify emerging risks Confirm / disprove existing risks Understand root causes Target controls where failures are identified.
  21. 21. 1. Risk events that are frequent enough to produce data, you don’t need huge numbers, but the more the better 2. Risk events where the past is representative of the future 3. You have or can get the data
  22. 22. Analytics Machine Learning
  23. 23. Parking tickets in the City of Toronto Jan 1 – Dec 31, 2016
  24. 24. Parking Tickets Impact Likelihood
  25. 25. Minimum ticket = $20 Mean ticket = $30 Maximum ticket = $450
  26. 26. Fewest tickets issued at 5 AM Most tickets issued at 12PM
  27. 27. Fewest tickets issued on Sunday Most tickets Tuesday - Friday
  28. 28. Spatial Analysis shows areas of concentration of tickets. We can observe a higher density of parking tickets in the core
  29. 29. Can we derive the LIKELIHOOD of getting a ticket?
  30. 30. 1. Risk events that are frequent enough to produce data, you don’t need huge numbers, but the more the better 2. Risk events where the past is representative of the future 3. You have or can get the data 4. You can establish a baseline (data about when the risk didn’t happen)
  31. 31. Machine Learning is giving computers the ability to learn without being explicitly programmed y = 1/4x + 5 y = Θx + offset
  32. 32. Sq ft (x) Sale Price (y) House 1 2300 $750,000 House 2 850 $400,000 House 3 1420 $625,000 y = 0.2329x + 236.86 (offset) y = 0.2329x + 236.86 $0 $100 $200 $300 $400 $500 $600 $700 $800 $900 0 500 1000 1500 2000 2500 HOUSE PRICE BY SQ FT
  33. 33. Sq ft (x) Sale Price (y) House 1 2300 $750k House 2 850 $400k House 3 1420 $625k House 4 2700 ? y = 0.2329x + 236.86 $0 $100 $200 $300 $400 $500 $600 $700 $800 $900 0 500 1000 1500 2000 2500 HOUSE PRICE BY SQ FT 0.2329(2700) + 236.86 $866k
  34. 34. y = 0.2431x + 224.44 $0 $100 $200 $300 $400 $500 $600 $700 $800 $900 $1,000 0 1000 2000 3000 HOUSE PRICE BY SQ FTSq ft (x) Sale Price (y) House 1 2300 $750k House 2 850 $400k House 3 1420 $625k House 4 2700 $890k
  35. 35. Sq Feet (x1) # of Bedrooms (x2) # of Bathrooms (x3) Local School Rating (x4) … Sale Price (y) House 1 2300 2 3 65 $750k House 2 850 1 1 97 $400k House 3 1420 2 2 14 $625k y = Θ1x1 + Θ2x2 + Θ3x3 + Θ4x4 + … + offset
  36. 36. Machine Learning is giving computers the ability to learn without being explicitly programmed
  37. 37. Size (x1) Color (x2) … Probability Based on a Decision Boundary Classification Type (y) Tumor 1 20 Brown 82% Malignant Malignant 18% Benign Tumor 2 5 Black 5% Malignant Benign 95% Benign Tumor 3 10 Red 1% Malignant Benign 99% Benign
  38. 38. ▪ We’ve partnered with a company called Advisen who builds and maintains a list of public risk events ▪ While they don’t have all risk events, the list is impressive, and we will treat it as representative of the risks that could occur
  39. 39. ▪ Model Type: NUMERIC PREDICTION – LINEAR & LOGISTIC REGRESSION ▪ Predicts (output): $ impact of a risk ▪ Features (inputs): risk type and company data: revenue, # of employees, location, industry.
  40. 40. The majority of risk predictions are within $10M of the actual risk Some of these risks are very large (largest risk in the DB is >$2B)
  41. 41. AWS Machine Learning Model to Predict Risk Impact
  42. 42. ▪ Model Type: CLASSIFICATION ▪ Predicts: Likelihood of a Risk Type ▪ Features: Revenue, # of employees, location, industry ▪ Returns: Most a probability of each risk Type
  43. 43. AWS Machine Learning Model to Predict Risk Likelihood
  44. 44. The Results - automated Risk Prediction and Impact for a known set of Risk Types
  45. 45. Start collecting data Use bow ties to understand risks better Use Indicators and Incidents to feed data into your models
  46. 46. Effective when there is no relevant data present. Typically used for strategic risks that have never occurred before. Subject to bias and errors in estimation. Effective when there is large risk event (incident) data set that can be analyzed by a subject matter expert. Data needs to be interpreted and aligned to risk events. Effective to predict a specific value or category of a given risk event. Requires similar data to analytics and skill to build the model. Highly effective in predicting the outcome of a potential event with many dimensions.
  47. 47. @jcrampton joe@resolver.com

×