Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Adversary Emulation using CALDERA

795 views

Published on

Presentation that describes how CALDERA can be used for automated adversary emulation.

Published in: Technology
  • Be the first to comment

Adversary Emulation using CALDERA

  1. 1. Network Security – 11 September 2019ErikVan Buggenhout Automated adversary emulation using Caldera ErikVan Buggenhout - NVISO @erikvabu - @nvisosecurity
  2. 2. Network Security - Automated Adversary Emulation using Caldera 2 WHO AM I? • Lead Author & Instructor SEC599 • Lead Author SEC699 (*NEW*) • Instructor & co-author SEC560 • Co-founder • Offices in Brussels, Frankfurt, Munich • Adversary emulation
  3. 3. Network Security - Automated Adversary Emulation using Caldera 3 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  4. 4. Network Security - Automated Adversary Emulation using Caldera 4 THIS IS NOT ADVERSARY EMULATION “Creative” RedTeamVuln Scanning like an APT
  5. 5. Network Security - Automated Adversary Emulation using Caldera 5 DEFINING ADVERSARY EMULATION Adversary emulation is an activity where security experts emulate how an adversary operates. The ultimate goal, of course, is to improve how resilient the organization is versus these adversary techniques. Both red and purple teaming can be considered as adversary emulation. Adversary activities are described using TTPs (Tactics, Techniques & Procedures). These are not as concrete as, for example, IOCs, but they describe how the adversary operates at a higher level. Adversary emulation should be based on TTPs. As such, a traditional vulnerability scan or internal penetration test that is not based on TTPs should not be considered adversary emulation. TTP Adversary emulation should be performed using a structured approach, which can be based on a kill chain or attack flow. MITRE ATT&CK is a good example of such a standard approach. ATT&CK
  6. 6. Network Security - Automated Adversary Emulation using Caldera 6 PENETRATION TEST VS ADVERSARY EMULATION PENETRATIONTEST Identify and exploit vulnerabilities on a (series of) system(s) to assess security Focused on a specific scope (typically an application or network range) ADVERSARY EMULATION Assess how resilient an organization is versus a certain adversary / threat actor Focused on the execution of a scenario (typically defined by a number of flags) VS Both Penetration Tests and Adversary Emulation engagements have value. It’s however important to know the difference and the results you can expect! Typically tests both prevention & detection (so is less valuable if there is no blue team) Primarily tests prevention, typically less focus on detection
  7. 7. Network Security - Automated Adversary Emulation using Caldera 7 RED TEAM VS PURPLE TEAM REDTEAM A red team involves emulation of a realistic threat actor (usingTTPs) In a typical red team, interaction with the blue team is limited (red vs blue) PURPLETEAM A purple team involves emulation of a realistic threat actor (usingTTPs) In a typical purple team, interaction with the blue team is maximized (collaboration) VS Both Red Team and Purple Team engagements have value. It’s however important to know the difference and the results you can expect! The goal of the purple team is to improve how well the blue team prevents & detects The goal of the red team is to assess how well the blue team prevents & detects
  8. 8. Network Security - Automated Adversary Emulation using Caldera 8 WHAT IS MITRE ATT&CK? "MITRE ATT&CK™ is a globally-accessible knowledge base of adversary tactics and techniques based on real-world observations.The ATT&CK knowledge base is used as a foundation for the development of specific threat models and methodologies in the private sector, in government, and in the cybersecurity product and service community.” – MITRE ATT&CK website Tactics are used to describe high-levels attack steps used by an adversary. These can be compared to the “steps” in the Lockheed Martin Cyber Kill Chain © Tactics & Techniques MITRE ATT&CK assumes breach and thus the “first” tactic is initial intrusion. Any activity performed before is covered by the PRE-ATT&CK framework. How a certain tactic is executed is described by a variety of techniques. For every technique, MITRE ATT&CK includes a description, detection & prevention recommendations and known threat actors who use the technique.
  9. 9. Network Security - Automated Adversary Emulation using Caldera 9 WHAT IS MITRE ATT&CK? TACTICS TECHNIQUES
  10. 10. Network Security - Automated Adversary Emulation using Caldera 10 WHAT DETAILS ARE AVAILABLE FOR A TECHNIQUE? Source: https://attack.mitre.org/techniques/T1122/ Known adversaries that use the technique General Info High-Level Description
  11. 11. Network Security - Automated Adversary Emulation using Caldera 11 WHAT DETAILS ARE AVAILABLE FOR A TECHNIQUE? Source: https://attack.mitre.org/techniques/T1122/ How to prevent? How to detect?
  12. 12. Network Security - Automated Adversary Emulation using Caldera 12 LEVERAGING MITRE ATT&CK ATT&CK for Adversary Emulation When organizing adversary emulation (such as red or purple team exercises), the emulation plan can be based on MITRE ATT&CK.This facilitates tracking & reporting. ATT&CK for Detection Capability The overall detection capability of an organization can be mapped to MITRE ATT&CK.This facilitates for example reporting on the maturity / scope of the SOC. ATT&CK forThreat Intelligence When consuming or generatingThreat Intelligence, observed adversary behavior can be mapped to MITRE ATT&CK. Several platforms support this mapping (e.g. MISP has a MITRE ATT&CK mapping). ATT&CK for Defense Prioritization In addition to measuring the detection coverage using MITRE ATT&CK, we can do the same for preventive controls.What MITRE ATT&CK techniques do we actively block? Organzations should leverage MITRE ATT&CK as the common language!
  13. 13. Network Security - Automated Adversary Emulation using Caldera 13 SOME COMMON ATT&CK PITFALLS #1 Consider all ATT&CK techniques equal Given the size of the ATT&CK matrix, it’s impossible to (a) prevent or (b) detect all techniques.You only have limited resources and should thus prioritize! #2 Misjudge your coverage Most ATT&CK techniques are not “Boolean”. It’s possible that you detect or block certain variations of a technique, but others not. Scoring should thus be fine-grained. #3 Consider ATT&CK as the “holy trinity” ATT&CK is a valuable tool, but it’s not a silver bullet. Recognize that, for some use cases,ATT&CK is not perfect. Furthermore, not everything is documented in ATT&CK.
  14. 14. Network Security - Automated Adversary Emulation using Caldera 14 WHAT TECHNIQUES SHOULD WE PRIORITIZE? So how do I know what techniques are most important? Criteria #1 Overall popularity of the technique The overall popularity of an ATT&CK technique is a good indicator of how important it is to cover it (using either preventive or detective controls). In January 2019, MITRE & Red Canary released a presentation where they highlighted 7 key techniques! Furthermore, many vendors provide “ATT&CK Heat Maps” where they describe what techniques they most frequently observe. Criteria #2 Relevance of threat actors for your organization Next to the overall “popularity” of a technique, there is of course another factor: Is the technique known to be used by an adversary that is interested in your organization? ATT&CK has information on what techniques are used by what actors. In order to figure out what threat actors are relevant for your industry or organization, it helps to follow up on threat intelligence reports.
  15. 15. Network Security - Automated Adversary Emulation using Caldera 15 ATT&CK USE CASE 1 - ADVERSARY EMULATION – PLANS To demonstrate the potential of ATT&CK, MITRE developed an emulation plan for APT3 (it’s mainly used as a showcase for MITRE ATT&CK). It’s a great example of how the ATT&CK framework can be leveraged to develop a concrete action plan to emulate a specific adversary. Source: https://attack.mitre.org/resources/adversary-emulation-plans/
  16. 16. Network Security - Automated Adversary Emulation using Caldera 16 BUILDING AN ADVERSARY EMULATION PLAN What threat actors (and related adversary techniques) are relevant to the organization? What techniques does the organization believe are covered by security controls? What techniques does the organization believe are detected by monitoring use cases? How much time & effort will be spent during the engagement? During both red team and purple team engagements, building a good adversary emulation plan is crucial to success.The emulation plan should mimic an actual adversary and can include distinct phases. In MITRE’s APT3 emulation plan, the following phases are distinguished: 1. Set up adversary infrastructure (e.g. C2) and obtain initial execution (Initial Access) 2. Internal discovery, privilege escalation and lateral movement (Lateral movement) 3. Collection, staging and exfiltration (Action on Objectives) So what techniques should you select as part of your plan?There’s a few criteria to take into account;
  17. 17. Network Security - Automated Adversary Emulation using Caldera 17 EXAMPLE OF AN EMULATION PLAN EMULATION PLAN FOR APT-28 PHASE 1 PHASE 2 PHASE 3 Initial Access T1192 - Spearphishing Link Execution T1086 - PowerShell Persistence T1122 - COM Hijacking Privilege Escalation T1078 -Valid Accounts Defense Evasion T1107 - File Deletion Lateral Movement T1075 – PassThe Hash Not every plan needs to cover every single tactic! Improvise! Exfiltration T1041 - Exfil over C&C
  18. 18. Network Security - Automated Adversary Emulation using Caldera 18 ATT&CK USE CASE 2 – DETECTION COVERAGE Do we have the right log sources? If we want to assess our overall detection coverage, we first need to assess what ATT&CK coverage we can achieve with the logs that we are currently generating and collecting. Malware Archeology, Olaf Hartong’s Sysmon configuration and DeTACCT (by Rabobank) are initiatives that aim to provide this visibility. SIGMA Do we have the right use cases? Next to having the right log sources, we also need use cases / signatures that can automatically alert when they are triggered. SIGMA provides an open source / generic format to develop vendor-agnostic use cases.We will leverage community SIGMA rules and develop our own during this week! Note: Manual threat hunting should compliment automated detection
  19. 19. Network Security - Automated Adversary Emulation using Caldera 19 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  20. 20. Network Security - Automated Adversary Emulation using Caldera 20 ADVERSARY EMULATION STACK Adversary emulation can typically take two different forms: • Automated / scripted emulation of a (number of) specific MITRE ATT&CK techniques • Manual, full-stack emulation according to an adversary emulation plan Different tools exist that can help emulate the two objectives listed above! Automated / scripted emulation Manual, full-stack, emulation METTA
  21. 21. Network Security - Automated Adversary Emulation using Caldera 21 ATOMIC RED TEAM When trying to “quickly” test detection of specifics techniques, we can use Atomic RedTeam to emulate certain ATT&CK techniques.All Atomic Red Team tests are portable and light-weight and allow for easy execution!
  22. 22. Network Security - Automated Adversary Emulation using Caldera 22 ENDGAME RED TEAM AUTOMATION RTA builds up local IoCs through Python scripts mapped with the MITRE ATT&CK Framework.
  23. 23. Network Security - Automated Adversary Emulation using Caldera 23 Uber Metta limits its field of interaction toVirtualBox and Vagrant machines. UBER METTA
  24. 24. Network Security - Automated Adversary Emulation using Caldera 24 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  25. 25. Network Security - Automated Adversary Emulation using Caldera 25 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  26. 26. Network Security - Automated Adversary Emulation using Caldera 26 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  27. 27. Network Security - Automated Adversary Emulation using Caldera 27 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  28. 28. Network Security - Automated Adversary Emulation using Caldera 28 METASPLOIT: ATT&CK PLUGIN PRAETORIAN-CODE/ PURPLE-TEAM-ATTACK-AUTOMATION
  29. 29. Network Security - Automated Adversary Emulation using Caldera 29 COVENANT Covenant is a .NET alternative to PowerShell Empire. It uses the same flow between listeners and launchers with a slight terminology variant.
  30. 30. Network Security - Automated Adversary Emulation using Caldera 30 SLIVER BISHOPFOX/SLIVER1. Compile Binary 2. Start Listener 3. Reverse Shell Drop Unique Binary
  31. 31. Network Security - Automated Adversary Emulation using Caldera 31 SLIVER Adversary Sliver DNS C2 Unique Binary Unique Binary Sliver compiles a unique obfuscated binary per target.
  32. 32. Network Security - Automated Adversary Emulation using Caldera 32 SLIVER Adversary Sliver DNS C2 Unique Binary Unique Binary Each host is identified by the custom obfuscated C2 domain used.
  33. 33. Network Security - Automated Adversary Emulation using Caldera 33 SLIVER: CANARY TOKENS Adversary Sliver DNS C2 Unique Binary Each binary is equipped with plaintext canary DNS tokens. BlueTeam $ strings $ nslookup canary.c2.adversary.org
  34. 34. Network Security - Automated Adversary Emulation using Caldera 34 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  35. 35. Network Security - Automated Adversary Emulation using Caldera 35 WHAT IS MITRE CALDERA? Caldera is a tool built by MITRE, with the express purpose of doing adversary emulation. It requires a bit of setup (as a server and clients need to be installed), it will actively "attack" target systems by deploying custom backdoors. Caldera’s attack steps are fully linked to the ATT&CK framework techniques! Local MITRE ATT&CK Caldera Attack GUI
  36. 36. Network Security - Automated Adversary Emulation using Caldera 36 MITRE CALDERA CHAIN ResetView Add toView
  37. 37. Network Security - Automated Adversary Emulation using Caldera 37 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - GROUPS
  38. 38. Network Security - Automated Adversary Emulation using Caldera 38 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - ABILITIES
  39. 39. Network Security - Automated Adversary Emulation using Caldera 39 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - FACTS
  40. 40. Network Security - Automated Adversary Emulation using Caldera 40 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - ADVERSARIES
  41. 41. Network Security - Automated Adversary Emulation using Caldera 41 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH – OPERATIONS
  42. 42. Network Security - Automated Adversary Emulation using Caldera 42 Plugin Type Notes Sandcat Required Cross-platform RAT plugin. SSL Required SSL reverse proxy plugin – Depends on HAProxy. GUI Required Main GUI and authentication plugin. Chain Required Advanced GUI plugin (groups, abilities, adversaries, operations, …) Stockpile Required Abilities and adversary database plugin. Caltack Optional MITRE ATT&CK local website plugin. Adversary Deprecated – Breaking Original Caldera mode plugin. MITRE CALDERA INCLUDED PLUGINS
  43. 43. Network Security - Automated Adversary Emulation using Caldera 43 MITRE CALDERA’S ABILITIES Ability An ability describes a suite of actions achieving a small goal.
  44. 44. Network Security - Automated Adversary Emulation using Caldera 44 Adversary MITRE CALDERA’S ADVERSARIES Ability An adversary describes a malicious actor equipped with abilities.
  45. 45. Network Security - Automated Adversary Emulation using Caldera 45 Adversary MITRE CALDERA’S PHASES Multiple abilities can be grouped in a phase. P1 Ability I Ability II
  46. 46. Network Security - Automated Adversary Emulation using Caldera 46 Adversary MITRE CALDERA’S ADVERSARIES Multiple phases describe an adversary. P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV
  47. 47. Network Security - Automated Adversary Emulation using Caldera 47 MITRE CALDERA’S INFECTED HOSTS A newly infected host, by the Sandcat plugin, joins a predefined group. Windows Groups
  48. 48. Network Security - Automated Adversary Emulation using Caldera 48 MITRE CALDERA’S GROUPS Multiple newly infected hosts can have different initial groups. Windows OS X Linux Groups
  49. 49. Network Security - Automated Adversary Emulation using Caldera 49 MITRE CALDERA’S GROUPS Different groups can contain multiple hosts with multiple operating systems. Foo Bar Windows OS X Linux Groups
  50. 50. Network Security - Automated Adversary Emulation using Caldera 50 MITRE CALDERA’S OPERATIONS Adversary P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV Foo Bar Windows OS X Linux Ability PowerShell OS X Shell Linux Shell Operation Groups
  51. 51. Network Security - Automated Adversary Emulation using Caldera 51 MITRE CALDERA’S OPERATIONS Adversary P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV Foo Bar Windows OS X Linux Ability PowerShell OS X Shell Linux Shell Operation Groups
  52. 52. Network Security - Automated Adversary Emulation using Caldera 52 MITRE CALDERA’S OPERATIONS Adversary P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV Foo Bar Windows OS X Linux Ability PowerShell OS X Shell Linux Shell Operation Groups
  53. 53. Network Security - Automated Adversary Emulation using Caldera 53 MITRE CALDERA’SVARIABLES OS X Ability PowerShell OS X Shell Linux Shell Operation Variable Injection server The Caldera server address. group The target host’s group. files The payload’s temporary host’s group.
  54. 54. Network Security - Automated Adversary Emulation using Caldera 54 Object Creation Modification Notes Groups Run-Time Possible – Automatic Created automatically when an infected host beacon’s the server. Abilities Hard-Coded Possible – Needs Caldera restart Created manually inside the Stockpile plugin. Facts Hard-Coded Possible Created manually inside the Stockpile plugin and CRUD-able through the GUI. Adversaries Hard-Coded Possible Created manually inside the Stockpile plugin and CRUD-able through the GUI. Operations Run-Time Possible Created manually inside the GUI. MITRE CALDERA’S OBJECT CREATIONS https://github.com/mitre/caldera/wiki
  55. 55. Network Security - Automated Adversary Emulation using Caldera 55 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  56. 56. Network Security - Automated Adversary Emulation using Caldera 56 QUESTIONS? Need help / support? info@nviso.eu Want to learn more? Join SEC599 or SEC699 www.sans.org

×