Adversary ROI: Evaluating Security from the Threat Actor’s Perspective

7,920 views
6,778 views

Published on

This is a current version of a presentation that @JoshCorman and I have given at RSA US 2012, GFIRST 2012 and RSA Europe 2012 on adversary centric security models. We've gotten a number of request for the slides, so we've posted them here.

The security community has failed for years to determine return on investment (ROI) or Return on Security Investment (ROSI). It’s failed as you can’t evaluate security efficacy without assessing the adversary’s perspective. Updated from the highly rated RSA US 2012 session, we’ll discuss the “Adversary ROI” model and provide mappings for different threat actors, ranging from organized to chaotic.

For more resources on Adversary ROI, visit http://elevatedsecurity.blogspot.com/2012/10/adversary-roi-resources.html

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,920
On SlideShare
0
From Embeds
0
Number of Embeds
247
Actions
Shares
0
Downloads
0
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide
  • Economics is the study of how society allocates scarce resources and goods. A well managed Info/Cyber/Security/Assurance program requires intelligent allocation of scarce resources–we can not protect everythingWe can’t build the entire airplane out of the “black box”
  • Most organizations, especially large corporations, use Return on Investment (ROI) to make investment decisionsROI gave us a framework to talk to non-security people about the benefits of security investments
  • When our attack surfaces approach infinity, its easier to manage threatsCONTROL QUOTIENTMost security programs focused solely on vulnerability management, which necessary but insufficientTechnology changes at high rate of speed making vulnerability a moving targetAdversary community changes faster than defendersAttacks quickly move to the most porous layerEnd users likely to remain a significant vulnerability
  • Classes of actors can be identified (and even particular actors in some cases)Capabilities can be estimated (and potentially managed by working Governments and Law Enforcement)Motive can be analyzed via “Adversary ROI”
  • Rorschach Test: http://en.wikipedia.org/wiki/Rorschach_testWe see in Anonymous what we WANT to see.. We project. Our perceptions say more about us than they do about the multitude of subgroups/causes in Anonymous.
  • Serenity prayer
  • The 2009 REPORT… has data from calendar year 2008… make sense?VERBALLY – or maybe add… the 2012 report on 2011 incidents… saw 3 or 5 or 6 (single digit) that were patchable
  • Adversary ROI: Evaluating Security from the Threat Actor’s Perspective

    1. Adversary ROI: Evaluating Securityfrom the Threat Actor’s Perspective Josh Corman David Etue Director, Security Intelligence VP, Corp Dev Strategy @joshcorman @djetue
    2. About Joshua Corman @joshcorman – Director of Security Intelligence for Akamai Technologies • Former Research Director, Enterprise Security [The 451 Group] • Former Principal Security Strategist [IBM ISS] – Industry: • Faculty: The Institute for Applied Network Security (IANS) • 2009 NetworkWorld Top 10 Tech People to Know • Co-Founder of “Rugged Software” www.ruggedsoftware.org • BLOG: www.cognitivedissidents.com – Things I’ve been researching: • Compliance vs Security • Disruptive Security for Disruptive Innovations • Chaotic Actors • Espionage • Security Metrics
    3. About David Etue @djetue – VP, Corporate Development Strategy at SafeNet • Former Cyber Security Practice Lead [PRTM Management Consultants] (now PwC) • Former VP Products and Markets [Fidelis Security Systems] • Former Manager, Information Security [General Electric Company] – Industry: • Faculty: The Institute for Applied Network Security (IANS) • Leads Washington Relations for Cyber Security Forum Initiative • Certified Information Privacy Professional (CIPP/G) – Cyber things that interest me: • Adversary innovation • Social media security • Applying intelligence cycle / OODA loop in cyber • Supply chain security
    4. Agenda Context Why ROI and ROSI have failed us… Adversary ROI Categorizing Threat Actors Application in the Real World
    5. CONTEXT
    6. We Have Finite Resources…We CanNot Protect Everything! “Black Box” Lufthansa Airbus A380 D-AIMC with the name "Peking" at Stuttgart http://commons.wikimedia.org/wiki/File:Fdr_sidefront.jpg Lasse Fuss http://commons.wikimedia.org/wiki/File:Lufthansa_A380_D-AIMC.jpg
    7. Consequences: Value & Replaceability http://blog.cognitivedissidents.com/2011/10/24/a-replaceability-continuum/
    8. Misplaced Focus“With the breach-a-week over the last twoyears, the key determinate was nothing YOUdid… but rather was WHO was after you.”
    9. WHY ROI AND ROSI HAVE FAILED US…
    10. Why ROI failed… Expected Returns Cost of InvestmentROI Cost of Investment at Net Present Value for an organization’s required Rate of Return• Most security people aren’t finance experts• Typically applied in a vacuum• No actual no profit from security investments• Doesn’t determine efficacy of security investment or commensurate investment levels
    11. From the Failure of ROI comes ROSI• Return on Security Investment (ROSI) created as a well intentioned way to apply risk metrics to ROI Risk Exposure % Risk Mitigated Solution Cost ROSI Solution Cost• Problems: – Attack surface is approaching infinity (not a real number) – “Risk Mitigated” can be both subjective and objective – Lacks accuracy (see @djbphaedrus Accuracy vs. Precision…)
    12. Practical Application of ROSI
    13. Examples of Failures...
    14. The Adversary Doesn’t Care AboutYour ROI/ROSI• Adversaries don’t care if you spend 4% or 12% of your IT budget on security• Adversaries are results oriented• Adversaries care if *they* can get a return on investment from an attack, not you…
    15. ADVERSARY ROI
    16. Why Adversary ROI• Adversaries want assets - vulnerabilities are a means• Our attack surface is approaching infinity• Adversaries have scarce resources too
    17. Adversary ROI Came About ByLooking at RiskA risk requires a threat and a vulnerability thatresults in a negative consequence Current State Proposed State? Threat Vulnerability Consequence We have finite resources, and must optimize the entire risk equation for our success!
    18. What is a “Threat”?A Threat is an Actorwith a Capabilityand a Motive Threats Are A “Who”, Not a “What”
    19. Solely Managing Vulnerabilities WillNever WinExploit for New Vulnerability Attacker Adoption Early Adopters Early Majority Late Majority Laggards
    20. Solely Managing Vulnerabilities WillNever Win Vendor Starts Technology Added toExploit for New Solution Solution Declared “Best Compliance Vulnerability Development Available Practice” Regulations Attacker Defender Adoption Adoption 0 Early Adopters Early Majority Late Majority Laggards Extensive Lag Between Attack Innovation, Solution, and Adoption
    21. Value Favors the Attacker Are you prepared to address a funded nation state targeting your highest value intellectual property?Attacker Gains Typical IT Security Budget (1-12% of IT Budget) Information Classification Public Sensitive Sensitive Highly Replicable Irreplaceable
    22. The Adversary ROI EquationAdversary ROI = ( ) Value of Assets Compromised + Cost of Attack Value [ Adversary Value of Operational Impact ] - the Attack Cost of the Attack Probability of X Success Deterrence - Measures (% Chance of Getting Caught x Cost of Getting Caught)
    23. Adversary ROI Example: Bicycle Theft OR
    24. CATEGORIZING THREAT ACTORS
    25. Dogma: You Don’t Need To Be FasterThan the Bear… 25
    26. A Modern Pantheon of AdversaryClasses Actor Classes Organized Script States Competitors Terrorists “Hactivists” Insiders Auditors Crime Kiddies Motivations Financial Industrial Military Ideological Political Prestige Target Assets Intellectual Cyber Core BusinessCredit Card #s Web Properties PII / Identity Property Infrastructure Processes Impacts Reputational Personal Confidentiality Integrity Availability Methods“MetaSploit” DoS Phishing Rootkit SQLi Auth Exfiltration Malware Physical
    27. Profiling a Particular Actor Actor Classes Organized Script States Competitors Terrorists “Hactivists” Insiders Auditors Crime Kiddies Motivations Financial Industrial Military Ideological Political Prestige Target Assets Intellectual Cyber Core BusinessCredit Card #s Web Properties PII / Identity Property Infrastructure Processes Impacts Reputational Personal Confidentiality Integrity Availability Methods“MetaSploit” DoS Phishing Rootkit SQLi Auth Exfiltration Malware Physical
    28. Script Kiddies (aka Casual Adversary) Skiddie Profit, Prestige CCN/Fungible Confidentially, Reputat ion “MetaSploit”, SQLi, Phi shing 28
    29. Organized Crime Organized Crime Profit Fungible, Banking Confidentially Malware, Botnets, Rootkits
    30. Adaptive Persistent Adversaries State/Espionage Industrial/Military Confidentially, Reputation Intellectual Property Trade Secrets Infrastructure Custom Malware, SpearPhishing, Physical, ++
    31. Hactivists Chaotic Actors Chaotic Actor Ideological and/or LULZ Web Properties, Individuals, Polic y Availability, Confidentiality, Reputation, Personal DoS, SQLi, Phishing
    32. Auditors Auditor QSA Profit Credit Card #s Distraction, Fines CheckList
    33. Compare and Contrast Threat Actors Casual State QSA Chaotic Actor Org Crime Attacker APT/APA Reputation, IP, Trade CCNs Dirty Laundry Secrets,Asset Focus CCNs CCNs… Banking DDoS/Availabi National Fungible $ lity Security DataTimeframe Annual Anytime Flash Mobs Continuous Long Cons Target NA LOW HIGH LOW HIGHStickinessProbability 100% MED ? HIGH ? “Impact” Annual $ 1 and done Relentless Varies Varies
    34. Attacker Power - HD Moore’s Law• Moore’s Law: Compute power doubles every 18 months• HDMoore’s Law: Casual Attacker Strength grows at the rate of MetaSploit
    35. HDMoore’s Law 100 90 80 Success Rate (%) 70 Adversary Classes 60 Espionage Organized Crime 50 APT/APA Chaotic Actors 40 Organized Crime Casual Attacker Anon/Lulz Auditor/Assessor 30 Casual 20 QSA 10 x 1 2 3 4 5 6 7 8 9 10 11 12 Defender “SecureOns” http://blog.cognitivedissidents.com/2011/11/01/intro-to-hdmoores-law/
    36. HDMoore’s Law (continued) HDMoore’s Law 100 90 80 Success Rate (%) 70 Adversary Classes 60 Espionage Organized Crime 50 APT/APA Chaotic Actors 40 Organized Crime Casual Attacker Anon/Lulz Auditor/Assessor 30 Casual 20 QSA 10 x 1 2 3 4 5 6 7 8 9 10 11 12 Defender “SecureOns” http://blog.cognitivedissidents.com/2011/11/01/intro-to-hdmoores-law/
    37. HDMoore’s Law (continued) HDMoore’s Law 100 90 80 Success Rate (%) 70 Adversary Classes 60 Espionage Organized Crime 50 APT/APA Chaotic Actors 40 Organized Crime Casual Attacker Anon/Lulz Auditor/Assessor 30 Casual 20 QSA 10 x 1 2 3 4 5 6 7 8 9 10 11 12 Defender “SecureOns” http://blog.cognitivedissidents.com/2011/11/01/intro-to-hdmoores-law/
    38. HDMoore’s Law (continued) HDMoore’s Law 100 90 80 Success Rate (%) 70 Adversary Classes 60 Espionage Organized Crime 50 APT/APA Chaotic Actors 40 Organized Crime Casual Attacker Anon/Lulz Auditor/Assessor 30 Casual 20 QSA 10 x 1 2 3 4 5 6 7 8 9 10 11 12 Defender “SecureOns” http://blog.cognitivedissidents.com/2011/11/01/intro-to-hdmoores-law/
    39. HDMoore’s Law (continued) HDMoore’s Law 100 90 80 Success Rate (%) 70 Adversary Classes 60 Espionage Organized Crime 50 APT/APA Chaotic Actors 40 Organized Crime Casual Attacker Anon/Lulz Auditor/Assessor 30 Casual 20 QSA 10 x 1 2 3 4 5 6 7 8 9 10 11 12 Defender “SecureOns” http://blog.cognitivedissidents.com/2011/11/01/intro-to-hdmoores-law/
    40. HDMoore’s Law (continued) HDMoore’s Law 100 90 80 Success Rate (%) 70 Adversary Classes 60 Espionage Organized Crime 50 APT/APA Chaotic Actors 40 Organized Crime Casual Attacker Anon/Lulz Auditor/Assessor 30 Casual 20 QSA 10 x 1 2 3 4 5 6 7 8 9 10 11 12 Defender “SecureOns” http://blog.cognitivedissidents.com/2011/11/01/intro-to-hdmoores-law/
    41. APPLICATION IN THE REAL WORLD
    42. Does it Matter Who is Attacking? Was #18 in overall DBIRTop Threat Action Types used to steal INTELLECTUAL PROPERTY AND CLASSIFIED INFORMATION by number ofbreaches - (excludes breaches only involving payment card data, bank account information, personal information, etc)Source: Verizon Business Security Blog (post-DBIR), 2011http://securityblog.verizonbusiness.com/2011/06/23/new-views-into-the-2011-dbir/
    43. Impacting Adversary ROI It is typically not desirable to make your assets lessAdversary ROI = valuable ( ) Value of Assets Compromised + Cost of Attack Value ( Adversary Value of Operational Impact )- the Attack Cost of the Attack Probability of X Increase adversary Success “Work Effort” Deterrence - Measures (% Chance of Getting Caught x Cost of Getting Caught) Impact of getting caught is Ability to respond typically a government issue and recover key
    44. Who Are You Playing Against?
    45. False Flags http://www.flickr.com/photos/pierre_tourigny/367078204/
    46. VZ DBIR Patching: Evolving Adversary TTPs “Let’s Patch Faster!” 2008 2009 2010 22% Patchable 6 of 90 Patchable ZERO Patchable (not 90%) 6.66% [0] Barking up the wrong tree?Source: Verizon Business Data Breach Investigations Report (DBIR), Years 2009-2011
    47. SQLi We spend under $500m Source: 2011 Verizon Business Data Breach Investigations Report (DBIR)
    48. 2011: Attacks Density (4Realz DBIR Style) “Only 55 of the 630 possible events have a value greater than 0…90% of the threat space was not in play at all” Source: 2011 Verizon Business Data Breach Investigations Report (DBIR)
    49. 2012: Attacks Density (4Realz DBIR Style) “Only 22 of the 315 possible events have a value greater than 0…93.1% of the threat space was not in play at all” Source: 2012 Verizon Business Data Breach Investigations Report (DBIR)
    50. 2011 VZ DBIR: Non-CCN Asset TypeBreakdown 2009 2010 Delta 141 incidents 761 incidentsIntellectual Property 10 41 + 31National Security Data 1 20 + 19Sensitive Organizational 13 81 + 68System Information ZERO 41 + 41Source: 2010 & 2011 Verizon Business Data Breach Investigations Report (DBIR)
    51. 2012 VZ DBIR: Non-CCN Asset TypeBreakdown 2009 2010 Delta 141 incidents 761 incidentsIntellectual Property 10 41 + 31National Security Data 1 20 + 19Sensitive Organizational 13 81 + 68System Information ZERO 41 + 41Source: 2012 Verizon Business Data Breach Investigations Report (DBIR)
    52. Think About Work Effort/Factor What Do You Look Like To Different Adversaries?
    53. Real Life Example from a DefenseIndustrial Base Company Who Are The Threats? What Do They Want? What Are Their TTPs? Deployed Specific Technology and Processes—Forced Adversary to Change TTPs Or Target Other Organizations
    54. Real Life Technology Examples Work Effort Respond and Recover• WebLabyrinth • FOG Computing http://code.google.com/p/weblabyrinth/ http://sneakers.cs.columbia.edu:8080/fog/• SCIT: Self Cleansing • Honeyports Intrusion Tolerance http://honeyports.sourceforge.net/ http://cs.gmu.edu/~asood/scit/ Photo - http://www.flickr.com/photos/shannonholman/2138613419 *Neither presenter has any affiliation with these technologies*
    55. Adversary ROI – Getting Non-SecurityExecutives Involved• What protected or sensitive information do we have?• What adversaries desire the information and why?• What is the value of the information to the organization?• How would the adversary value it?• What are the adversaries capabilities?• What controls protect the information?
    56. How To Apply To EnrichCurrent Security Investments• Enrich incident response – Increase aim of incident responders – Detect false flags• Enrich Security Information and Event Management (SIEM) – Cluster assets or methods by adversary class - new "pivots" to interpret security events• Enrich Budgeting – More precision in how you apply investment
    57. Apply: Final Thoughts• Start with a blank slate!• Engage non-security people• Identify your most likely adversaries• Obtain/share adversary centric intel – Threat Intelligence – Brand/chatter monitoring – Information sharing• Simulate adversary-driven scenarios – Table tops/roll playing (w/ Crisis Management) – Adversary-Centric Penetration Testing
    58. Thank You / Contact Josh Corman David Etue @joshcorman @djetue blog.cognitivedissidents.com profile.david.etue.net Actor Classes Motivations Target Assets Impacts Methods

    ×