ISSA ORM 2012 June 20 v0.3

  • 453 views
Uploaded on

The pres for ISSA NL chapter, June 20 2012

The pres for ISSA NL chapter, June 20 2012

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
453
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen Had ook kunnen heten: Op het verkeerde paard gewed Een doodlopende straat in Eind zoek, al zoek
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011 Niks is perfect maar weinig is zo gebrekkig en fout als uw modellen. De aannames zijn niet redelijk. En een aap gooit beter dartpijltjes (geen bias). Als ze er niet toe doen, niet doen punt. En ze doen er wel toe, anders hebt u nooit een functioneel model. Conservatief ten opzichte van ..? En waarom niet accuraat boven conservatief (biased). En als ze niet accuraat maar conservatief zijn, hebben odellen dus geen realiteitsgehalte. Conservatisme kan eenvoudig leiden tot onjuiste conclusies. Uw annames worden oneindig eenvoudiger aangetoond verkeerd te zijn dan dat ze juist zijn. Ík heb geen bewijslast, maar u! Geldt ook indien niet ‘bewijs’ maar ‘aannemelijkheid’ wordt gevraagd. Dus als iedereen in het water springt, springt u erachteraan? CYA is niet goed genoeg… Ah, de valse profeet. Is de beslisser beter af als hij wordt mis leid …? Oh jawel dat zijn ze wel want ze misleiden tot u weet welke delen wél zouden werken. Waarom dan de rest niet weggegooid? Of gebruik een horoscoop, die bezweert ook een hoop onzekerheid. Garbage in, garbage out. En je best is wellicht gewoon niet goed genoeg zelfs als de data correct zouden zijn. Volledigheid, iemand? Ja. Maar doe dan wel de juiste aannames en wees rücksichtlos in de beoordeling van hun waarheidsgehalte, én bepaal de variabiliteit in uitkomsten bij variatie van aannames. Doet u dat, ooit? Hoezo? Het zijn geen babies. Het zijn hulpmiddelen. Het kwaad schuilt in de misleiding van uw klanten, in des keizers nieuwe kleren gezet. Vlieg van Schiphol naar O’Hare met brandstof en plattegrond van Eelde!
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen The Future of Risk Management / Where Will Risk Management Go ..? ISSA Interntional Conference Baltimore October 2011
  • Operations Risk 'Management'Van plank misslaan naar spijker op de kop ISSA NL Eurojust Den Haag June 20 2012ISACA Roundtable 5 maart 2012 Breukelen

Transcript

  • 1. Operations Risk‘Management’ ISSA NL Eurojust Den Haag, June 20 2012 Jurgen van der Vlugt
  • 2. Agenda Intro ORM The Totalitarian Dictatorship of the Perfected Bureaucracy Was Nun?Operations Risk Management ISSA June 20 2012
  • 3. • Jurgen = Ir.drs. J. van der Vlugt RE CISA CRISC• Maverisk Consultancy, IS Audit and Advisory services (KPMG, ABN AMRO, Noordbeek, Achmea, ABN AMRO 322 (F16) sqn, RNLAF Vlb Leeuwarden-Noord)• (IS) Audit, (Info)Security, Y2k, BCM, ERM/ORM• ISSA, NOREA: Various committees Operations Risk Management ISSA June 20 2012
  • 4. YouInterruptions,Please!• WIP• Contestable content (Hi Darryl! ) Operations Risk Management ISSA June 20 2012
  • 5. Agenda Intro → ORM The Totalitarian Dictatorship of the Perfected Bureaucracy Was Nun?Operations Risk Management ISSA June 20 2012
  • 6. Infosec; traditionally bottom-up Operations Risk Management ISSA June 20 2012
  • 7. B2 • 5 / 95 pp. Mention of ‘O’ (incl ToC) • ‘Guidance’ → Hobson’s choice … … → Catch-22 (zie verderop) • Loss db driven (stats) • Amateur mistakes: • Event = 1 Cause, 1 Effect … At best: ± n:1:m • Non-orthogonal categories, weak definitions • No time aspect, no feedback loops • Modeling: Figure it out yourself • Wrong modelOperations Risk Management ISSA June 20 2012
  • 8. (Intermission: Turf wars) Many small errors; easily undone or insignificantFreq Material (significant) damage; will occur frequently Ops (but is not ‘routine’) Los ses Break-the-business incidents; organization will not survive the hit Security Incidents Threats to continuity Impact Operations Risk Management ISSA June 20 2012
  • 9. ‘Risk’ ‘Methodology’• Risk = Chance x Impact (H/M/L, 3/5-scale) Initiële auditissues Forecast ultimo 2011 1 2 3 4 4 3 5 9 7 8 6 9 Kans Kans 6 2 7 1 Impact Impact Operations Risk Management ISSA June 20 2012
  • 10. Risk ‘methodologu’• 1 Kans Shame!• … per? Year? Transaction? Nanosecond?• 1 Impact Shame!• … Only financial? Reputation, etc.?• H x H = 25 Shame!• 3xM=H Shame!• ’16’ > ’12’ Shame!• Who estimates ‘H’; how and with what evidence?• No-one corrects that? Operations Risk Management ISSA June 20 2012
  • 11. n:m and feedback, and time, continuity Operations Risk Management ISSA June 20 2012
  • 12. ‘In control’ …?Operations Risk Management ISSA June 20 2012
  • 13. Wait, there’s moreOperations Risk Management ISSA June 20 2012
  • 14. Wait… even moreIn particular, for any consistent,effectively generated formaltheory that proves certain basicarithmetic truths, there is anarithmetical statement that istrue, but not provable in the theory.Kurt GödelNo matter how perfect you try toprotect, infosec incidents willhappenYours Truly Operations Risk Management ISSA June 20 2012
  • 15. ‘Turkey before Thanksgiving’Operations Risk Management ISSA June 20 2012
  • 16. Don’t start on cost issuesWhat was it astronaut JohnGlenn said went through his mindas he awaited lift-off?"Youre thinking youre sitting ontop of the most complex machineever built by man, with a millionseparate components, allsupplied by the lowest bidder." Operations Risk Management ISSA June 20 2012
  • 17. Attempting functions ∫ ( Chance × Impact ) ∑( Costs of countermeasures )For many series of functions and parameters, impact estimateranges (…), variable sets of countermeasuresIncluding variable degrees of effectiveness, with vague notionsof risk appetites in some backs of minds(I’ll come back to that later) Operations Risk Management ISSA June 20 2012
  • 18. Yes but …: your arguments1. Yes we know all that. Nothing’s perfect.2. The assumptions are reasonable.3. The assumptions don’t really matter.4. The assumptions are conservative.5. You cannot prove the assumptions are wrong.6. We only do what everyone else does.7. The decision maker is better off with us than without us.8. The models are not completely useless.9. You gotta make the best of the data you’ve got.10. You need assumptions to make progress.11. The models deserve the benefit of the doubt.12. Models and assumptions don’t do any harm so why bother …?© David Freedman (in Nassim Taleb’s Black Swan) Operations Risk Management ISSA June 20 2012
  • 19. Operations Risk Management ISSA June 20 2012
  • 20. Operational Risk (≡ ..?) ‘Management’Evaluate design & Analysis Monitor & react set-up Operational Risk Problem Management Mgt Incidents ORAP Inherent Controls Risk indicators for analysis risks (Problems) R(S)A (K)ORC KRI Incident (+Audit) (Mgt) (Mgt) Mgt Insu- Designed, Tuning, Near rance Selected for Mandatory misses CLD Mgt efficiency Corrective KRI actions values Incidents Indemnities Process Breach Operations Risk Management ISSA June 20 2012
  • 21. Agenda Intro ORM → The Totalitarian Dictatorship of the Perfected Bureaucracy Was Nun?Operations Risk Management ISSA June 20 2012
  • 22. 3LoD quod nonVery, very basically Surprise! Operations Risk Management ISSA June 20 2012
  • 23. Operations Risk Management ISSA June 20 2012
  • 24. Operations Risk Management ISSA June 20 2012
  • 25. (Defense in Depth) …?Operations Risk Management ISSA June 20 2012
  • 26. Not to mention 1937 ..!Operations Risk Management ISSA June 20 2012
  • 27. ResultOperations Risk Management ISSA June 20 2012
  • 28. The Illusion of Being In Control Hey, Darryl again !)Operations Risk Management ISSA June 20 2012
  • 29. (Intermission: Mandatory Reading) Operations Risk Management ISSA June 20 2012
  • 30. Be my guestOperations Risk Management ISSA June 20 2012
  • 31. You of course know better than the Dakota Operations Risk Management ISSA June 20 2012
  • 32. →Operations Risk Management ISSA June 20 2012
  • 33. Agenda Intro ORM The Totalitarian Dictatorship of the Perfected Bureaucracy → Was Nun?Operations Risk Management ISSA June 20 2012
  • 34. Was nun ...? (I)Operations Risk Management ISSA June 20 2012
  • 35. Was nun … ? (II)In theory, nothing works, In practice, everything works,and but no-one knows why.Everyone knows why. We have in our organisation a combination of theory and practice. Operations Risk Management ISSA June 20 2012
  • 36. Was Nun …? (III)• Alternative approaches from the risk perspective → Much better modeling• Alternative approaches from a trust angle (Qualitative approaches) → Yikes!• Alternative approaches from the (info)sec field → Doing much better what needs to be done Operations Risk Management ISSA June 20 2012
  • 37. Modeling in rk s oW re sprog = Operations Risk Management ISSA June 20 2012
  • 38. Some pointers; what quant helps out?• (F)actors • ‘Threat’ factors, maybe or maybe not also being • ‘Control’ factors, maybe or maybe not also being • ‘Vulnerability’ factors• Continuously (! in time) variable qua • Chance • Severity/size • Impacts (mult.) on (variable #) other factors • Feedback (var. #, impact, time lags) on other factors Operations Risk Management ISSA June 20 2012
  • 39. Which should lead to:• All sorts of continuous functions, continuously variable (time, parameters) → ‘normal’ Markov chains don’t work• Bootstrapping parameter estimations → lots of data required• Modeling the unk unk’s; good luck Operations Risk Management ISSA June 20 2012
  • 40. Consumes a lot of time …• Is all required data available?• Are the models developed yet, and tested for robustness …? (re parameter sensitivity ++)• What if reality turns out to be uncontrollable ..? (Koot&Bie, 1977)• Ow well, we’ll just sit and wait …?• And if we don’t get ‘it’ done: “Inzicht, doorzicht en op tijd een banaan.” Management ≡ Decision making with limited information! Operations Risk Management ISSA June 20 2012
  • 41. In the mean time• Do the right thing right• Stress Operations Risk Management ISSA June 20 2012
  • 42. Doing the right things rightOperations Risk Management ISSA June 20 2012
  • 43. That is complex enough in itself Operations Risk Management ISSA June 20 2012
  • 44. The new worldOperations Risk Management ISSA June 20 2012
  • 45. And of course: StressOperations Risk Management ISSA June 20 2012
  • 46. (RNLAF 323sqn vlb Leeuwarden-Zuid)Operations Risk Management ISSA June 20 2012
  • 47. We do that already, in infosec (?)• Data- and system oriented CIA Requirements, tests• Defence in Depth: ( )• Monitoring, pentesting, fallback testing, etc. Operations Risk Management ISSA June 20 2012
  • 48. And for the risk managers in the room … Operations Risk Management ISSA June 20 2012
  • 49. Bruce SchneierOperations Risk Management ISSA June 20 2012
  • 50. ResultaatOperations Risk Management ISSA June 20 2012
  • 51. Top-down and bottom-up• And/or middle-out• Don’t switch over but continuously all the way• Re-think trust-/control-models• Do The Right Thing• Be certain there’ll be defectors• Against diffusion of accountability,• Watch Coase’s ceiling Operations Risk Management ISSA June 20 2012
  • 52. High demandsOperations Risk Management ISSA June 20 2012
  • 53. Agenda Intro ORM The Totalitarian Dictatorship of the Perfected Bureaucracy → Was Nun?Operations Risk Management ISSA June 20 2012
  • 54. Summing up• Our (O)RM methods are wrong (not a bit) • Enthousiastically down a blind alley • False view on reality → • Wrong risk management. And you know it!• Totalitarian dictatorship of the perfected bureaucracy doesn’t help against anything & gives (also) false sense of In Control• Are you part of that ..?• Let’s ‘pre-emptively’ build some methodology bottom-up Operations Risk Management ISSA June 20 2012
  • 55. Solution: less, moreOperations Risk Management ISSA June 20 2012
  • 56. Yes, the methodology is Work In Progress, hence … Operations Risk Management ISSA June 20 2012
  • 57. That was all. Thank you. Hope you enjoy(ed) the ride Operations Risk Management ISSA June 20 2012
  • 58. Operations Risk Management ISSA June 20 2012
  • 59. Contact detailsJurgen van der Vlugt,Maverisk Consultancy, IS Audit and Advisory services:• Jvdvlugt åt maverisk døt nl• LinkedIn, Twitter (etc.etc.)• Tel +31-(0)6-206.648.23• www.maverisk.nlMotivate yourself! www.despair.com/viewall.html Operations Risk Management ISSA June 20 2012
  • 60. (Even More Mandatory Reading) Operations Risk Management ISSA June 20 2012
  • 61. The End, really. Unintentionally left blank. Really, this was not the plan. The plan called forlots of stuff here. But noooo, it had to turn out blank. Darn. Operations Risk Management ISSA June 20 2012