Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ISSA ORM 2012 June 20 v0.3

852 views

Published on

The pres for ISSA NL chapter, June 20 2012

Published in: Business, Economy & Finance
  • Be the first to comment

  • Be the first to like this

ISSA ORM 2012 June 20 v0.3

  1. 1. Operations Risk‘Management’ ISSA NL Eurojust Den Haag, June 20 2012 Jurgen van der Vlugt
  2. 2. Agenda Intro ORM The Totalitarian Dictatorship of the Perfected Bureaucracy Was Nun?Operations Risk Management ISSA June 20 2012
  3. 3. • Jurgen = Ir.drs. J. van der Vlugt RE CISA CRISC• Maverisk Consultancy, IS Audit and Advisory services (KPMG, ABN AMRO, Noordbeek, Achmea, ABN AMRO 322 (F16) sqn, RNLAF Vlb Leeuwarden-Noord)• (IS) Audit, (Info)Security, Y2k, BCM, ERM/ORM• ISSA, NOREA: Various committees Operations Risk Management ISSA June 20 2012
  4. 4. YouInterruptions,Please!• WIP• Contestable content (Hi Darryl! ) Operations Risk Management ISSA June 20 2012
  5. 5. Agenda Intro → ORM The Totalitarian Dictatorship of the Perfected Bureaucracy Was Nun?Operations Risk Management ISSA June 20 2012
  6. 6. Infosec; traditionally bottom-up Operations Risk Management ISSA June 20 2012
  7. 7. B2 • 5 / 95 pp. Mention of ‘O’ (incl ToC) • ‘Guidance’ → Hobson’s choice … … → Catch-22 (zie verderop) • Loss db driven (stats) • Amateur mistakes: • Event = 1 Cause, 1 Effect … At best: ± n:1:m • Non-orthogonal categories, weak definitions • No time aspect, no feedback loops • Modeling: Figure it out yourself • Wrong modelOperations Risk Management ISSA June 20 2012
  8. 8. (Intermission: Turf wars) Many small errors; easily undone or insignificantFreq Material (significant) damage; will occur frequently Ops (but is not ‘routine’) Los ses Break-the-business incidents; organization will not survive the hit Security Incidents Threats to continuity Impact Operations Risk Management ISSA June 20 2012
  9. 9. ‘Risk’ ‘Methodology’• Risk = Chance x Impact (H/M/L, 3/5-scale) Initiële auditissues Forecast ultimo 2011 1 2 3 4 4 3 5 9 7 8 6 9 Kans Kans 6 2 7 1 Impact Impact Operations Risk Management ISSA June 20 2012
  10. 10. Risk ‘methodologu’• 1 Kans Shame!• … per? Year? Transaction? Nanosecond?• 1 Impact Shame!• … Only financial? Reputation, etc.?• H x H = 25 Shame!• 3xM=H Shame!• ’16’ > ’12’ Shame!• Who estimates ‘H’; how and with what evidence?• No-one corrects that? Operations Risk Management ISSA June 20 2012
  11. 11. n:m and feedback, and time, continuity Operations Risk Management ISSA June 20 2012
  12. 12. ‘In control’ …?Operations Risk Management ISSA June 20 2012
  13. 13. Wait, there’s moreOperations Risk Management ISSA June 20 2012
  14. 14. Wait… even moreIn particular, for any consistent,effectively generated formaltheory that proves certain basicarithmetic truths, there is anarithmetical statement that istrue, but not provable in the theory.Kurt GödelNo matter how perfect you try toprotect, infosec incidents willhappenYours Truly Operations Risk Management ISSA June 20 2012
  15. 15. ‘Turkey before Thanksgiving’Operations Risk Management ISSA June 20 2012
  16. 16. Don’t start on cost issuesWhat was it astronaut JohnGlenn said went through his mindas he awaited lift-off?"Youre thinking youre sitting ontop of the most complex machineever built by man, with a millionseparate components, allsupplied by the lowest bidder." Operations Risk Management ISSA June 20 2012
  17. 17. Attempting functions ∫ ( Chance × Impact ) ∑( Costs of countermeasures )For many series of functions and parameters, impact estimateranges (…), variable sets of countermeasuresIncluding variable degrees of effectiveness, with vague notionsof risk appetites in some backs of minds(I’ll come back to that later) Operations Risk Management ISSA June 20 2012
  18. 18. Yes but …: your arguments1. Yes we know all that. Nothing’s perfect.2. The assumptions are reasonable.3. The assumptions don’t really matter.4. The assumptions are conservative.5. You cannot prove the assumptions are wrong.6. We only do what everyone else does.7. The decision maker is better off with us than without us.8. The models are not completely useless.9. You gotta make the best of the data you’ve got.10. You need assumptions to make progress.11. The models deserve the benefit of the doubt.12. Models and assumptions don’t do any harm so why bother …?© David Freedman (in Nassim Taleb’s Black Swan) Operations Risk Management ISSA June 20 2012
  19. 19. Operations Risk Management ISSA June 20 2012
  20. 20. Operational Risk (≡ ..?) ‘Management’Evaluate design & Analysis Monitor & react set-up Operational Risk Problem Management Mgt Incidents ORAP Inherent Controls Risk indicators for analysis risks (Problems) R(S)A (K)ORC KRI Incident (+Audit) (Mgt) (Mgt) Mgt Insu- Designed, Tuning, Near rance Selected for Mandatory misses CLD Mgt efficiency Corrective KRI actions values Incidents Indemnities Process Breach Operations Risk Management ISSA June 20 2012
  21. 21. Agenda Intro ORM → The Totalitarian Dictatorship of the Perfected Bureaucracy Was Nun?Operations Risk Management ISSA June 20 2012
  22. 22. 3LoD quod nonVery, very basically Surprise! Operations Risk Management ISSA June 20 2012
  23. 23. Operations Risk Management ISSA June 20 2012
  24. 24. Operations Risk Management ISSA June 20 2012
  25. 25. (Defense in Depth) …?Operations Risk Management ISSA June 20 2012
  26. 26. Not to mention 1937 ..!Operations Risk Management ISSA June 20 2012
  27. 27. ResultOperations Risk Management ISSA June 20 2012
  28. 28. The Illusion of Being In Control Hey, Darryl again !)Operations Risk Management ISSA June 20 2012
  29. 29. (Intermission: Mandatory Reading) Operations Risk Management ISSA June 20 2012
  30. 30. Be my guestOperations Risk Management ISSA June 20 2012
  31. 31. You of course know better than the Dakota Operations Risk Management ISSA June 20 2012
  32. 32. →Operations Risk Management ISSA June 20 2012
  33. 33. Agenda Intro ORM The Totalitarian Dictatorship of the Perfected Bureaucracy → Was Nun?Operations Risk Management ISSA June 20 2012
  34. 34. Was nun ...? (I)Operations Risk Management ISSA June 20 2012
  35. 35. Was nun … ? (II)In theory, nothing works, In practice, everything works,and but no-one knows why.Everyone knows why. We have in our organisation a combination of theory and practice. Operations Risk Management ISSA June 20 2012
  36. 36. Was Nun …? (III)• Alternative approaches from the risk perspective → Much better modeling• Alternative approaches from a trust angle (Qualitative approaches) → Yikes!• Alternative approaches from the (info)sec field → Doing much better what needs to be done Operations Risk Management ISSA June 20 2012
  37. 37. Modeling in rk s oW re sprog = Operations Risk Management ISSA June 20 2012
  38. 38. Some pointers; what quant helps out?• (F)actors • ‘Threat’ factors, maybe or maybe not also being • ‘Control’ factors, maybe or maybe not also being • ‘Vulnerability’ factors• Continuously (! in time) variable qua • Chance • Severity/size • Impacts (mult.) on (variable #) other factors • Feedback (var. #, impact, time lags) on other factors Operations Risk Management ISSA June 20 2012
  39. 39. Which should lead to:• All sorts of continuous functions, continuously variable (time, parameters) → ‘normal’ Markov chains don’t work• Bootstrapping parameter estimations → lots of data required• Modeling the unk unk’s; good luck Operations Risk Management ISSA June 20 2012
  40. 40. Consumes a lot of time …• Is all required data available?• Are the models developed yet, and tested for robustness …? (re parameter sensitivity ++)• What if reality turns out to be uncontrollable ..? (Koot&Bie, 1977)• Ow well, we’ll just sit and wait …?• And if we don’t get ‘it’ done: “Inzicht, doorzicht en op tijd een banaan.” Management ≡ Decision making with limited information! Operations Risk Management ISSA June 20 2012
  41. 41. In the mean time• Do the right thing right• Stress Operations Risk Management ISSA June 20 2012
  42. 42. Doing the right things rightOperations Risk Management ISSA June 20 2012
  43. 43. That is complex enough in itself Operations Risk Management ISSA June 20 2012
  44. 44. The new worldOperations Risk Management ISSA June 20 2012
  45. 45. And of course: StressOperations Risk Management ISSA June 20 2012
  46. 46. (RNLAF 323sqn vlb Leeuwarden-Zuid)Operations Risk Management ISSA June 20 2012
  47. 47. We do that already, in infosec (?)• Data- and system oriented CIA Requirements, tests• Defence in Depth: ( )• Monitoring, pentesting, fallback testing, etc. Operations Risk Management ISSA June 20 2012
  48. 48. And for the risk managers in the room … Operations Risk Management ISSA June 20 2012
  49. 49. Bruce SchneierOperations Risk Management ISSA June 20 2012
  50. 50. ResultaatOperations Risk Management ISSA June 20 2012
  51. 51. Top-down and bottom-up• And/or middle-out• Don’t switch over but continuously all the way• Re-think trust-/control-models• Do The Right Thing• Be certain there’ll be defectors• Against diffusion of accountability,• Watch Coase’s ceiling Operations Risk Management ISSA June 20 2012
  52. 52. High demandsOperations Risk Management ISSA June 20 2012
  53. 53. Agenda Intro ORM The Totalitarian Dictatorship of the Perfected Bureaucracy → Was Nun?Operations Risk Management ISSA June 20 2012
  54. 54. Summing up• Our (O)RM methods are wrong (not a bit) • Enthousiastically down a blind alley • False view on reality → • Wrong risk management. And you know it!• Totalitarian dictatorship of the perfected bureaucracy doesn’t help against anything & gives (also) false sense of In Control• Are you part of that ..?• Let’s ‘pre-emptively’ build some methodology bottom-up Operations Risk Management ISSA June 20 2012
  55. 55. Solution: less, moreOperations Risk Management ISSA June 20 2012
  56. 56. Yes, the methodology is Work In Progress, hence … Operations Risk Management ISSA June 20 2012
  57. 57. That was all. Thank you. Hope you enjoy(ed) the ride Operations Risk Management ISSA June 20 2012
  58. 58. Operations Risk Management ISSA June 20 2012
  59. 59. Contact detailsJurgen van der Vlugt,Maverisk Consultancy, IS Audit and Advisory services:• Jvdvlugt åt maverisk døt nl• LinkedIn, Twitter (etc.etc.)• Tel +31-(0)6-206.648.23• www.maverisk.nlMotivate yourself! www.despair.com/viewall.html Operations Risk Management ISSA June 20 2012
  60. 60. (Even More Mandatory Reading) Operations Risk Management ISSA June 20 2012
  61. 61. The End, really. Unintentionally left blank. Really, this was not the plan. The plan called forlots of stuff here. But noooo, it had to turn out blank. Darn. Operations Risk Management ISSA June 20 2012

×