0
Risk Management
Time to blow it up and start over?

@alexhutton
Met E.T. Jaynes
probability theory, the logic of
science
Kuhn’s Protoscience
 A stage in the development of a science
that is described by:

• somewhat random fact gathering
  (ma...
only the wisest and stupidest of
men never change
Confucius
Destroy GRC
Musings of a Risk Management
Deconstructivist
A feeling of diss-connect
between GRC and Security
let’s talk governance
governance, without metrics &
models, is superstition
governance, with metrics &
models, describes capability to
manage ri...
Why does what you
execute on and how
you execute matter?
governance, without metrics & models,
is superstition
governance, with metrics & models,
describes capability to manage ri...
not sucking eggs at security is a
good idea
compliance*, without metrics, is
superstition
compliance*, with metrics, is risk
management


                          *(...
But “GRC” Risk
Management

Find issue, call
issue bad, fix issue,
hope you don’t find
it again...
What is risk?
a. Risk is notional
b. Risk is tangible
Problems with “tangible”

- complex systems, complexity
science

- usefulness outside of the very
specific

- measurements...
How Complex Systems Fail
(Being a Short Treatise on the Nature of Failure; How Failure
is Evaluated; How Failure is Attrib...
Catastrophe requires multiple failures
single point failures are not enough..

The array of defenses works. System operati...
Complex systems run in degraded mode.

Post-accident attribution accident to a ‘root
cause’ is fundamentally wrong.

All p...
Problems with “notional”

- becomes difficult to extract wisdom - we
want a “Gross Domestic Product”

- unable to be defen...
from Mark Curphey’s SecurityBullshit
What is risk?
uses of “risk”

- engineering
         - complex systems says “no”
- financial
          - no 110% return on your firewall...
our standards say:

Find issue, call
issue bad, fix issue,
hope you don’t find
it again...
Managing risk means aligning the
capabilities of the organization, and
the exposure of the organization
with the tolerance...
evidence based medicine, meet information security



     What is evidence-based risk
           management?

        a d...
Loss Landscape




                                              Threat Landscape



                            risk

Ass...
Loss Landscape

                                                                  a balanced
                             ...
Loss Landscape                                              a balanced
                                                   ...
The Achilles heel again, lack of
             data
Models and data
sharing
Good Lord Of The Dance, something a
vendor might actually help you with
Verizon Incident Sharing Framework
             it’s open*!



                            * kinda
Verizon has shared data
-   2009 –
    over 600
    cases


-   2010 –
    between
    1000 &
    1400
Verizon is sharing our
framework
What is the Verizon Incident Sharing (VerIS)
Framework?

 - A means   to create metrics
   from the incident narrative
   ...
What makes up the VerIS framework?


 -   Demographics
 -   Incident Classification
     -   Event Modeling (a4)

 -   Dis...
Cybertrust Security




                      demographics   -   company industry

                                     - ...
Cybertrust Security




                       incident classification                                                   -...
Cybertrust Security




                      incident classification
                      a4 event model


             ...
Cybertrust Security




                      discovery & mitigation   -   incident timeline
                             ...
Cybertrust Security




                      Impact classification   -   impact
                                         ...
Cybertrust Security




                      incident narrative                     incident metrics

                   ...
Cybertrust Security
                           case studies                         data set

                            ...
Cybertrust Security
                           data set                       knowledge & wisdom

                        ...
Cybertrust Security
                           threat modeling

                                                          ...
Cybertrust Security
                           threat modeling

                                                          ...
Cybertrust Security
                           impact modeling

                                                          ...
Cybertrust Security
                           impact modeling

                                                          ...
Problems:

Data sharing, incidents, privacy

Failures vs. Successes
(where management capability helps)




Talking to the...
Successes:

Bridge the gap
(IRM becomes tactically actionable based on threat/attack modeling)

(Capability measurements b...
Requirements:
Data Sets

Models

Technology

Sciences - complexity, management/TQM/Probability/
Game Theory, biomimicry...
Risk Management - Time to blow it up and start over? - Alex Hutton
Risk Management - Time to blow it up and start over? - Alex Hutton
Upcoming SlideShare
Loading in...5
×

Risk Management - Time to blow it up and start over? - Alex Hutton

2,506

Published on

Now that the industry is trying to formalize the concept of risk management into neat little compartments like standards (ISO 27005/31000), certifications (CRISC) and products (GRC) guess what? We're doing it wrong. Fundamentally wrong. This talk will discuss why all this current risk management stuff is goofy and what sort of alternatives we have that might help us understand our ability to protect, our tendancy towards failure, and how to match that up with what management will stomach.

Published in: Business, Economy & Finance
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,506
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
97
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Risk Management - Time to blow it up and start over? - Alex Hutton"

  1. 1. Risk Management Time to blow it up and start over? @alexhutton
  2. 2. Met E.T. Jaynes probability theory, the logic of science
  3. 3. Kuhn’s Protoscience A stage in the development of a science that is described by: • somewhat random fact gathering (mainly of readily accessible data) • a “morass” of interesting, trivial, irrelevant observations • A variety of theories (that are spawned from what he calls philosophical speculation) that provide little guidance to data gathering
  4. 4. only the wisest and stupidest of men never change Confucius
  5. 5. Destroy GRC Musings of a Risk Management Deconstructivist
  6. 6. A feeling of diss-connect between GRC and Security
  7. 7. let’s talk governance
  8. 8. governance, without metrics & models, is superstition governance, with metrics & models, describes capability to manage risk
  9. 9. Why does what you execute on and how you execute matter?
  10. 10. governance, without metrics & models, is superstition governance, with metrics & models, describes capability to manage risk measurably good governance practices (can/will) reduce risk measurably good governance is simply a description of capability to manage risk
  11. 11. not sucking eggs at security is a good idea
  12. 12. compliance*, without metrics, is superstition compliance*, with metrics, is risk management *(regulatory)
  13. 13. But “GRC” Risk Management Find issue, call issue bad, fix issue, hope you don’t find it again...
  14. 14. What is risk?
  15. 15. a. Risk is notional b. Risk is tangible
  16. 16. Problems with “tangible” - complex systems, complexity science - usefulness outside of the very specific - measurements - lots of belief statements
  17. 17. How Complex Systems Fail (Being a Short Treatise on the Nature of Failure; How Failure is Evaluated; How Failure is Attributed to Proximate Cause; and the Resulting New Understanding of Patient Safety) Richard I. Cook, MD Cognitive technologies Laboratory University of Chicago http://www.ctlab.org/documents/How %20Complex%20Systems %20Fail.pdf
  18. 18. Catastrophe requires multiple failures single point failures are not enough.. The array of defenses works. System operations are generally successful. Overt catastrophic failure occurs when small, apparently innocuous failures join to create opportunity for a systemic accident. Each of these small failures is necessary to cause catastrophe but only the combination is sufficient to permit failure. Put another way, there are many more failure opportunities than overt system accidents. Most initial failure trajectories are blocked by designed system safety components. Trajectories that reach the operational level are mostly blocked, usually by practitioners. Complex systems contain changing mixtures of failures latent within them. The complexity of these systems makes it impossible for them to run without multiple flaws being present. Because these are individually insufficient to cause failure they are regarded as minor factors during operations. Eradication of all latent failures is limited primarily by economic cost but also because it is difficult before the fact to see how such failures might contribute to an accident. The failures change constantly because of changing technology, work organization, and efforts to eradicate failures.
  19. 19. Complex systems run in degraded mode. Post-accident attribution accident to a ‘root cause’ is fundamentally wrong. All practitioner actions are gambles. Human expertise in complex systems is constantly changing Change introduces new forms of failure. Views of ‘cause’ limit the effectiveness of defenses against future events.
  20. 20. Problems with “notional” - becomes difficult to extract wisdom - we want a “Gross Domestic Product” - unable to be defended - pseudo-scientific - lots of belief statements
  21. 21. from Mark Curphey’s SecurityBullshit
  22. 22. What is risk?
  23. 23. uses of “risk” - engineering - complex systems says “no” - financial - no 110% return on your firewall - medical - requires data
  24. 24. our standards say: Find issue, call issue bad, fix issue, hope you don’t find it again...
  25. 25. Managing risk means aligning the capabilities of the organization, and the exposure of the organization with the tolerance of the data owners - Jack Jones
  26. 26. evidence based medicine, meet information security What is evidence-based risk management? a deconstructed, notional view of risk
  27. 27. Loss Landscape Threat Landscape risk Asset Landscape Controls Landscape
  28. 28. Loss Landscape a balanced scorecard? Asset Landscape Threat Landscape risk Controls Landscape
  29. 29. Loss Landscape a balanced scorecard? capability (destroys “g” introducing quality management & mgmt. Asset Landscape Threat Landscape science elements into infosec) risk exposure change “compliance” simply becomes a factor of loss landscape and/or operating as a Controls Landscape control group for comparative data
  30. 30. The Achilles heel again, lack of data
  31. 31. Models and data sharing Good Lord Of The Dance, something a vendor might actually help you with
  32. 32. Verizon Incident Sharing Framework it’s open*! * kinda
  33. 33. Verizon has shared data
  34. 34. - 2009 – over 600 cases - 2010 – between 1000 & 1400
  35. 35. Verizon is sharing our framework
  36. 36. What is the Verizon Incident Sharing (VerIS) Framework? - A means to create metrics from the incident narrative - how Verizon creates measurements for the DBIR - how *anyone* can create measurements from an incident - http://securityblog.verizonbusiness.com/wp-content/uploads/ 2010/03/VerIS_Framework_Beta_1.pdf
  37. 37. What makes up the VerIS framework? - Demographics - Incident Classification - Event Modeling (a4) - Discovery & Mitigation - Impact Classification - Impact Modeling
  38. 38. Cybertrust Security demographics - company industry - company size - geographic location - of business unit in incident - size of security department
  39. 39. Cybertrust Security incident classification - agent error misuse - what acts against us malware hacking environmental external - asset social action physical - what the agent acts against internal agent asset confidentiality possession - action partner availability - what the agent does to the type attribute utility asset function authenticity integrity - attribute - the result of the agent’s action against the asset
  40. 40. Cybertrust Security incident classification a4 event model the series of events (a4) creates an “attack model” 1 > 2 > 3 > 4 > 5
  41. 41. Cybertrust Security discovery & mitigation - incident timeline - discovery method evidence sources + - - control capability - corrective action - most straightforward manner in which the incident could be prevented - the cost of preventative controls
  42. 42. Cybertrust Security Impact classification - impact categorization - sources of Impact $ (direct, indirect) - similar to iso 27005/FAIR - impact estimation - distribution for amount of impact - impact qualification - relative impact rating
  43. 43. Cybertrust Security incident narrative incident metrics discovery demographics incident classification (a4) impact classification + & mitigation 1> 2> 3> 4 > 5 $$$
  44. 44. Cybertrust Security case studies data set discovery demographics incident classification (a4) impact classification + & mitigation a 1> 2> 3> 4 > 5 $$$ b 1> 2> 3> 4 > 5 + $$$ c 1> 2> 3> 4 > 5 + $$$ d 1> 2> 3> 4 > 5 + $$$ e 1> 2> 3> 4 > 5 + $$$ f 1> 2> 3> 4 > 5 + $$$
  45. 45. Cybertrust Security data set knowledge & wisdom discovery demographics incident classification (a4) impact classification + & mitigation a 1> 2> 3> 4 > 5 $$$ b 1> 2> 3> 4 > 5 + $$$ c 1> 2> 3> 4 > 5 + $$$ d 1> 2> 3> 4 > 5 + $$$ e 1> 2> 3> 4 > 5 + $$$ f 1> 2> 3> 4 > 5 + $$$
  46. 46. Cybertrust Security threat modeling discovery demographics incident classification (a4) impact classification + & mitigation a 1> 2> 3> 4 > 5 $$$ b 1> 2> 3> 4 > 5 + $$$ c 1> 2> 3> 4 > 5 + $$$ d 1> 2> 3> 4 > 5 + $$$ e 1> 2> 3> 4 > 5 + $$$ f 1> 2> 3> 4 > 5 + $$$
  47. 47. Cybertrust Security threat modeling discovery demographics incident classification (a4) impact classification + & mitigation a 1> 2> 3> 4 > 5 $$$ b 1> 2> 3> 4 > 5 + $$$ c 1> 2> 3> 4 > 5 + $$$ d 1> 2> 3> 4 > 5 + $$$ e 1> 2> 3> 4 > 5 + $$$ f 1> 2> 3> 4 > 5 + $$$
  48. 48. Cybertrust Security impact modeling discovery demographics incident classification (a4) impact classification + & mitigation a 1> 2> 3> 4 > 5 $$$ b 1> 2> 3> 4 > 5 + $$$ c 1> 2> 3> 4 > 5 + $$$ d 1> 2> 3> 4 > 5 + $$$ e 1> 2> 3> 4 > 5 + $$$ f 1> 2> 3> 4 > 5 + $$$
  49. 49. Cybertrust Security impact modeling discovery demographics incident classification (a4) impact classification + & mitigation a 1> 2> 3> 4 > 5 $$$ b 1> 2> 3> 4 > 5 + $$$ c 1> 2> 3> 4 > 5 + $$$ d 1> 2> 3> 4 > 5 + $$$ e 1> 2> 3> 4 > 5 + $$$ f 1> 2> 3> 4 > 5 + $$$
  50. 50. Problems: Data sharing, incidents, privacy Failures vs. Successes (where management capability helps) Talking to the business owner (might still need a “tangible approach here, but pseudo-actuarial data can help - we still want a GDP)
  51. 51. Successes: Bridge the gap (IRM becomes tactically actionable based on threat/attack modeling) (Capability measurements bridged to notional increase/decrease in risk) (complex system problems addressed by showing multiple sources of causes) Accurate, notional likelihood Accurate tangible impact
  52. 52. Requirements: Data Sets Models Technology Sciences - complexity, management/TQM/Probability/ Game Theory, biomimicry...
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×