Successfully reported this slideshow.
Your SlideShare is downloading. ×

Adversary Emulation using CALDERA

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 56 Ad
Advertisement

More Related Content

Slideshows for you (20)

Similar to Adversary Emulation using CALDERA (20)

Advertisement
Advertisement

Adversary Emulation using CALDERA

  1. 1. Network Security – 11 September 2019ErikVan Buggenhout Automated adversary emulation using Caldera ErikVan Buggenhout - NVISO @erikvabu - @nvisosecurity
  2. 2. Network Security - Automated Adversary Emulation using Caldera 2 WHO AM I? • Lead Author & Instructor SEC599 • Lead Author SEC699 (*NEW*) • Instructor & co-author SEC560 • Co-founder • Offices in Brussels, Frankfurt, Munich • Adversary emulation
  3. 3. Network Security - Automated Adversary Emulation using Caldera 3 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  4. 4. Network Security - Automated Adversary Emulation using Caldera 4 THIS IS NOT ADVERSARY EMULATION “Creative” RedTeamVuln Scanning like an APT
  5. 5. Network Security - Automated Adversary Emulation using Caldera 5 DEFINING ADVERSARY EMULATION Adversary emulation is an activity where security experts emulate how an adversary operates. The ultimate goal, of course, is to improve how resilient the organization is versus these adversary techniques. Both red and purple teaming can be considered as adversary emulation. Adversary activities are described using TTPs (Tactics, Techniques & Procedures). These are not as concrete as, for example, IOCs, but they describe how the adversary operates at a higher level. Adversary emulation should be based on TTPs. As such, a traditional vulnerability scan or internal penetration test that is not based on TTPs should not be considered adversary emulation. TTP Adversary emulation should be performed using a structured approach, which can be based on a kill chain or attack flow. MITRE ATT&CK is a good example of such a standard approach. ATT&CK
  6. 6. Network Security - Automated Adversary Emulation using Caldera 6 PENETRATION TEST VS ADVERSARY EMULATION PENETRATIONTEST Identify and exploit vulnerabilities on a (series of) system(s) to assess security Focused on a specific scope (typically an application or network range) ADVERSARY EMULATION Assess how resilient an organization is versus a certain adversary / threat actor Focused on the execution of a scenario (typically defined by a number of flags) VS Both Penetration Tests and Adversary Emulation engagements have value. It’s however important to know the difference and the results you can expect! Typically tests both prevention & detection (so is less valuable if there is no blue team) Primarily tests prevention, typically less focus on detection
  7. 7. Network Security - Automated Adversary Emulation using Caldera 7 RED TEAM VS PURPLE TEAM REDTEAM A red team involves emulation of a realistic threat actor (usingTTPs) In a typical red team, interaction with the blue team is limited (red vs blue) PURPLETEAM A purple team involves emulation of a realistic threat actor (usingTTPs) In a typical purple team, interaction with the blue team is maximized (collaboration) VS Both Red Team and Purple Team engagements have value. It’s however important to know the difference and the results you can expect! The goal of the purple team is to improve how well the blue team prevents & detects The goal of the red team is to assess how well the blue team prevents & detects
  8. 8. Network Security - Automated Adversary Emulation using Caldera 8 WHAT IS MITRE ATT&CK? "MITRE ATT&CK™ is a globally-accessible knowledge base of adversary tactics and techniques based on real-world observations.The ATT&CK knowledge base is used as a foundation for the development of specific threat models and methodologies in the private sector, in government, and in the cybersecurity product and service community.” – MITRE ATT&CK website Tactics are used to describe high-levels attack steps used by an adversary. These can be compared to the “steps” in the Lockheed Martin Cyber Kill Chain © Tactics & Techniques MITRE ATT&CK assumes breach and thus the “first” tactic is initial intrusion. Any activity performed before is covered by the PRE-ATT&CK framework. How a certain tactic is executed is described by a variety of techniques. For every technique, MITRE ATT&CK includes a description, detection & prevention recommendations and known threat actors who use the technique.
  9. 9. Network Security - Automated Adversary Emulation using Caldera 9 WHAT IS MITRE ATT&CK? TACTICS TECHNIQUES
  10. 10. Network Security - Automated Adversary Emulation using Caldera 10 WHAT DETAILS ARE AVAILABLE FOR A TECHNIQUE? Source: https://attack.mitre.org/techniques/T1122/ Known adversaries that use the technique General Info High-Level Description
  11. 11. Network Security - Automated Adversary Emulation using Caldera 11 WHAT DETAILS ARE AVAILABLE FOR A TECHNIQUE? Source: https://attack.mitre.org/techniques/T1122/ How to prevent? How to detect?
  12. 12. Network Security - Automated Adversary Emulation using Caldera 12 LEVERAGING MITRE ATT&CK ATT&CK for Adversary Emulation When organizing adversary emulation (such as red or purple team exercises), the emulation plan can be based on MITRE ATT&CK.This facilitates tracking & reporting. ATT&CK for Detection Capability The overall detection capability of an organization can be mapped to MITRE ATT&CK.This facilitates for example reporting on the maturity / scope of the SOC. ATT&CK forThreat Intelligence When consuming or generatingThreat Intelligence, observed adversary behavior can be mapped to MITRE ATT&CK. Several platforms support this mapping (e.g. MISP has a MITRE ATT&CK mapping). ATT&CK for Defense Prioritization In addition to measuring the detection coverage using MITRE ATT&CK, we can do the same for preventive controls.What MITRE ATT&CK techniques do we actively block? Organzations should leverage MITRE ATT&CK as the common language!
  13. 13. Network Security - Automated Adversary Emulation using Caldera 13 SOME COMMON ATT&CK PITFALLS #1 Consider all ATT&CK techniques equal Given the size of the ATT&CK matrix, it’s impossible to (a) prevent or (b) detect all techniques.You only have limited resources and should thus prioritize! #2 Misjudge your coverage Most ATT&CK techniques are not “Boolean”. It’s possible that you detect or block certain variations of a technique, but others not. Scoring should thus be fine-grained. #3 Consider ATT&CK as the “holy trinity” ATT&CK is a valuable tool, but it’s not a silver bullet. Recognize that, for some use cases,ATT&CK is not perfect. Furthermore, not everything is documented in ATT&CK.
  14. 14. Network Security - Automated Adversary Emulation using Caldera 14 WHAT TECHNIQUES SHOULD WE PRIORITIZE? So how do I know what techniques are most important? Criteria #1 Overall popularity of the technique The overall popularity of an ATT&CK technique is a good indicator of how important it is to cover it (using either preventive or detective controls). In January 2019, MITRE & Red Canary released a presentation where they highlighted 7 key techniques! Furthermore, many vendors provide “ATT&CK Heat Maps” where they describe what techniques they most frequently observe. Criteria #2 Relevance of threat actors for your organization Next to the overall “popularity” of a technique, there is of course another factor: Is the technique known to be used by an adversary that is interested in your organization? ATT&CK has information on what techniques are used by what actors. In order to figure out what threat actors are relevant for your industry or organization, it helps to follow up on threat intelligence reports.
  15. 15. Network Security - Automated Adversary Emulation using Caldera 15 ATT&CK USE CASE 1 - ADVERSARY EMULATION – PLANS To demonstrate the potential of ATT&CK, MITRE developed an emulation plan for APT3 (it’s mainly used as a showcase for MITRE ATT&CK). It’s a great example of how the ATT&CK framework can be leveraged to develop a concrete action plan to emulate a specific adversary. Source: https://attack.mitre.org/resources/adversary-emulation-plans/
  16. 16. Network Security - Automated Adversary Emulation using Caldera 16 BUILDING AN ADVERSARY EMULATION PLAN What threat actors (and related adversary techniques) are relevant to the organization? What techniques does the organization believe are covered by security controls? What techniques does the organization believe are detected by monitoring use cases? How much time & effort will be spent during the engagement? During both red team and purple team engagements, building a good adversary emulation plan is crucial to success.The emulation plan should mimic an actual adversary and can include distinct phases. In MITRE’s APT3 emulation plan, the following phases are distinguished: 1. Set up adversary infrastructure (e.g. C2) and obtain initial execution (Initial Access) 2. Internal discovery, privilege escalation and lateral movement (Lateral movement) 3. Collection, staging and exfiltration (Action on Objectives) So what techniques should you select as part of your plan?There’s a few criteria to take into account;
  17. 17. Network Security - Automated Adversary Emulation using Caldera 17 EXAMPLE OF AN EMULATION PLAN EMULATION PLAN FOR APT-28 PHASE 1 PHASE 2 PHASE 3 Initial Access T1192 - Spearphishing Link Execution T1086 - PowerShell Persistence T1122 - COM Hijacking Privilege Escalation T1078 -Valid Accounts Defense Evasion T1107 - File Deletion Lateral Movement T1075 – PassThe Hash Not every plan needs to cover every single tactic! Improvise! Exfiltration T1041 - Exfil over C&C
  18. 18. Network Security - Automated Adversary Emulation using Caldera 18 ATT&CK USE CASE 2 – DETECTION COVERAGE Do we have the right log sources? If we want to assess our overall detection coverage, we first need to assess what ATT&CK coverage we can achieve with the logs that we are currently generating and collecting. Malware Archeology, Olaf Hartong’s Sysmon configuration and DeTACCT (by Rabobank) are initiatives that aim to provide this visibility. SIGMA Do we have the right use cases? Next to having the right log sources, we also need use cases / signatures that can automatically alert when they are triggered. SIGMA provides an open source / generic format to develop vendor-agnostic use cases.We will leverage community SIGMA rules and develop our own during this week! Note: Manual threat hunting should compliment automated detection
  19. 19. Network Security - Automated Adversary Emulation using Caldera 19 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  20. 20. Network Security - Automated Adversary Emulation using Caldera 20 ADVERSARY EMULATION STACK Adversary emulation can typically take two different forms: • Automated / scripted emulation of a (number of) specific MITRE ATT&CK techniques • Manual, full-stack emulation according to an adversary emulation plan Different tools exist that can help emulate the two objectives listed above! Automated / scripted emulation Manual, full-stack, emulation METTA
  21. 21. Network Security - Automated Adversary Emulation using Caldera 21 ATOMIC RED TEAM When trying to “quickly” test detection of specifics techniques, we can use Atomic RedTeam to emulate certain ATT&CK techniques.All Atomic Red Team tests are portable and light-weight and allow for easy execution!
  22. 22. Network Security - Automated Adversary Emulation using Caldera 22 ENDGAME RED TEAM AUTOMATION RTA builds up local IoCs through Python scripts mapped with the MITRE ATT&CK Framework.
  23. 23. Network Security - Automated Adversary Emulation using Caldera 23 Uber Metta limits its field of interaction toVirtualBox and Vagrant machines. UBER METTA
  24. 24. Network Security - Automated Adversary Emulation using Caldera 24 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  25. 25. Network Security - Automated Adversary Emulation using Caldera 25 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  26. 26. Network Security - Automated Adversary Emulation using Caldera 26 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  27. 27. Network Security - Automated Adversary Emulation using Caldera 27 INFECTION MONKEY SOURCE: GUARDICORE.COM/INFECTIONMONKEY
  28. 28. Network Security - Automated Adversary Emulation using Caldera 28 METASPLOIT: ATT&CK PLUGIN PRAETORIAN-CODE/ PURPLE-TEAM-ATTACK-AUTOMATION
  29. 29. Network Security - Automated Adversary Emulation using Caldera 29 COVENANT Covenant is a .NET alternative to PowerShell Empire. It uses the same flow between listeners and launchers with a slight terminology variant.
  30. 30. Network Security - Automated Adversary Emulation using Caldera 30 SLIVER BISHOPFOX/SLIVER1. Compile Binary 2. Start Listener 3. Reverse Shell Drop Unique Binary
  31. 31. Network Security - Automated Adversary Emulation using Caldera 31 SLIVER Adversary Sliver DNS C2 Unique Binary Unique Binary Sliver compiles a unique obfuscated binary per target.
  32. 32. Network Security - Automated Adversary Emulation using Caldera 32 SLIVER Adversary Sliver DNS C2 Unique Binary Unique Binary Each host is identified by the custom obfuscated C2 domain used.
  33. 33. Network Security - Automated Adversary Emulation using Caldera 33 SLIVER: CANARY TOKENS Adversary Sliver DNS C2 Unique Binary Each binary is equipped with plaintext canary DNS tokens. BlueTeam $ strings $ nslookup canary.c2.adversary.org
  34. 34. Network Security - Automated Adversary Emulation using Caldera 34 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  35. 35. Network Security - Automated Adversary Emulation using Caldera 35 WHAT IS MITRE CALDERA? Caldera is a tool built by MITRE, with the express purpose of doing adversary emulation. It requires a bit of setup (as a server and clients need to be installed), it will actively "attack" target systems by deploying custom backdoors. Caldera’s attack steps are fully linked to the ATT&CK framework techniques! Local MITRE ATT&CK Caldera Attack GUI
  36. 36. Network Security - Automated Adversary Emulation using Caldera 36 MITRE CALDERA CHAIN ResetView Add toView
  37. 37. Network Security - Automated Adversary Emulation using Caldera 37 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - GROUPS
  38. 38. Network Security - Automated Adversary Emulation using Caldera 38 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - ABILITIES
  39. 39. Network Security - Automated Adversary Emulation using Caldera 39 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - FACTS
  40. 40. Network Security - Automated Adversary Emulation using Caldera 40 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH - ADVERSARIES
  41. 41. Network Security - Automated Adversary Emulation using Caldera 41 MITRE CALDERA CHAIN – INTERFACE WALKTHROUGH – OPERATIONS
  42. 42. Network Security - Automated Adversary Emulation using Caldera 42 Plugin Type Notes Sandcat Required Cross-platform RAT plugin. SSL Required SSL reverse proxy plugin – Depends on HAProxy. GUI Required Main GUI and authentication plugin. Chain Required Advanced GUI plugin (groups, abilities, adversaries, operations, …) Stockpile Required Abilities and adversary database plugin. Caltack Optional MITRE ATT&CK local website plugin. Adversary Deprecated – Breaking Original Caldera mode plugin. MITRE CALDERA INCLUDED PLUGINS
  43. 43. Network Security - Automated Adversary Emulation using Caldera 43 MITRE CALDERA’S ABILITIES Ability An ability describes a suite of actions achieving a small goal.
  44. 44. Network Security - Automated Adversary Emulation using Caldera 44 Adversary MITRE CALDERA’S ADVERSARIES Ability An adversary describes a malicious actor equipped with abilities.
  45. 45. Network Security - Automated Adversary Emulation using Caldera 45 Adversary MITRE CALDERA’S PHASES Multiple abilities can be grouped in a phase. P1 Ability I Ability II
  46. 46. Network Security - Automated Adversary Emulation using Caldera 46 Adversary MITRE CALDERA’S ADVERSARIES Multiple phases describe an adversary. P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV
  47. 47. Network Security - Automated Adversary Emulation using Caldera 47 MITRE CALDERA’S INFECTED HOSTS A newly infected host, by the Sandcat plugin, joins a predefined group. Windows Groups
  48. 48. Network Security - Automated Adversary Emulation using Caldera 48 MITRE CALDERA’S GROUPS Multiple newly infected hosts can have different initial groups. Windows OS X Linux Groups
  49. 49. Network Security - Automated Adversary Emulation using Caldera 49 MITRE CALDERA’S GROUPS Different groups can contain multiple hosts with multiple operating systems. Foo Bar Windows OS X Linux Groups
  50. 50. Network Security - Automated Adversary Emulation using Caldera 50 MITRE CALDERA’S OPERATIONS Adversary P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV Foo Bar Windows OS X Linux Ability PowerShell OS X Shell Linux Shell Operation Groups
  51. 51. Network Security - Automated Adversary Emulation using Caldera 51 MITRE CALDERA’S OPERATIONS Adversary P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV Foo Bar Windows OS X Linux Ability PowerShell OS X Shell Linux Shell Operation Groups
  52. 52. Network Security - Automated Adversary Emulation using Caldera 52 MITRE CALDERA’S OPERATIONS Adversary P1 Ability I Ability II P2 Ability III P3 Ability IV AbilityV Foo Bar Windows OS X Linux Ability PowerShell OS X Shell Linux Shell Operation Groups
  53. 53. Network Security - Automated Adversary Emulation using Caldera 53 MITRE CALDERA’SVARIABLES OS X Ability PowerShell OS X Shell Linux Shell Operation Variable Injection server The Caldera server address. group The target host’s group. files The payload’s temporary host’s group.
  54. 54. Network Security - Automated Adversary Emulation using Caldera 54 Object Creation Modification Notes Groups Run-Time Possible – Automatic Created automatically when an infected host beacon’s the server. Abilities Hard-Coded Possible – Needs Caldera restart Created manually inside the Stockpile plugin. Facts Hard-Coded Possible Created manually inside the Stockpile plugin and CRUD-able through the GUI. Adversaries Hard-Coded Possible Created manually inside the Stockpile plugin and CRUD-able through the GUI. Operations Run-Time Possible Created manually inside the GUI. MITRE CALDERA’S OBJECT CREATIONS https://github.com/mitre/caldera/wiki
  55. 55. Network Security - Automated Adversary Emulation using Caldera 55 TOPICS FORTODAY What is adversary emulation? Tools of the trade MITRE Caldera Demo: MITRE Caldera
  56. 56. Network Security - Automated Adversary Emulation using Caldera 56 QUESTIONS? Need help / support? info@nviso.eu Want to learn more? Join SEC599 or SEC699 www.sans.org

Editor's Notes

  • Welcome to SANS Security SEC699: Advanced Purple Team Tactics – Adversary Emulation for Breach Prevention & Detection.

    <TO ADD>

    Erik Van Buggenhout
    evanbuggenhout@nviso.be
    www.nviso.be

    James Shewmaker
    @sans.org
    www.sans.org

    Update: E01
  • Leveraging ATT&CK: The Navigator
    As part of many tools, MITRE has developed the ATT&CK Navigator. It's a web application that represents the ATT&CK techniques in a dynamic fashion. It can be used to select specific techniques based on a threat group (e.g. select all APT-28 techniques), after which modifications and annotations can be made. It can be an excellent central resource for an organization to obtain a "quick look" at how well they are doing versus the different techniques. It's open-source and can thus be self-hosted!

    A demo instance of the ATT&CK navigator can be found here: https://mitre.github.io/attack-navigator/enterprise/

    The source code can be found here:
    https://github.com/mitre/attack-navigator
  • Leveraging ATT&CK: The Navigator
    As part of many tools, MITRE has developed the ATT&CK Navigator. It's a web application that represents the ATT&CK techniques in a dynamic fashion. It can be used to select specific techniques based on a threat group (e.g. select all APT-28 techniques), after which modifications and annotations can be made. It can be an excellent central resource for an organization to obtain a "quick look" at how well they are doing versus the different techniques. It's open-source and can thus be self-hosted!

    A demo instance of the ATT&CK navigator can be found here: https://mitre.github.io/attack-navigator/enterprise/

    The source code can be found here:
    https://github.com/mitre/attack-navigator
  • Leveraging ATT&CK: The Navigator

    Do we have the right use cases?
    Having the right log sources is a start. Ideally however, we’d also like to have use cases / signatures that can automatically alert when they are triggered. SIGMA provides an open source / generic format to develop vendor-agnostic use cases. We will leverage community SIGMA rules and develop our own during this week! Note: Not all detection can be automated using a signature. Manual threat hunting efforts may be required to identify complex / stealth ATT&CK techniques.
  • What is MITRE Caldera?
    Caldera is one off the most promising open-source adversary emulation tools built by MITRE. It requires a bit of setup (as a server and clients need to be installed), it will actively "attack" target systems by deploying custom backdoors. Caldera’s attack steps are fully linked to the ATT&CK framework techniques!

    It includes a full-blown local implementation of the MITRE ATT&CK framework and has a GUI that allows us to configure and run adversary emulation campaigns. A fair warning though: Caldera is under continuous development and thus quickly changes! We will illustrate this by developing our own module during the upcoming Caldera lab! As Caldera will be the main tool we will use for automated adversary emulation through SEC699, we will spend quite some time walking through its different features in this section!

    All Caldera documentation can be found at https://github.com/mitre/caldera.
  • MITRE Caldera Chain
    MITRE Caldera is fully-customizable, all of its features are structured in plugins. A set of basic plugins are shipped by default to provide a functioning boilerplate.

    The Chain plugin, as observed in the above image, provides us with a GUI to manage our operations. Although quite rudimental at the moment, the Chain plugin works by pinning sub-views to our main view which enables us to manage groups, facts, adversaries and operations. Once our view is over-populated, a “clear” button provides us with the ability to reset the view.
  • MITRE Caldera Chain – Interface Walkthrough – Groups
    The “Groups” view shows all systems that are currently “connected” to Caldera (i.e. that have the Caldera client running). We can create and structure groups ourselves. Note that a system can be part of different groups at the same time! On the slide above, we have the following systems

    dc-01 (member of the “windows” group)
    sql-01 (member of the “windows” group)
    ubuntu18-01 (member of the “linux” group)
    win10-01 (member of the “windows” group)
    win10-02 (member of the “windows” group)
    win19-01 (member of the “windows” group)
    win19-02 (member of the “windows” group)
    Workstation (member of the “windows” group)

    We will further discuss how these groups can be used later in this section!
  • MITRE Caldera Chain – Interface Walkthrough – Abilities
    The abilities view is a very interesting one! This is the core of Caldera’s emulation capabilities. An ability can be thought of as an implementation of a specific ATT&CK technique. As you can see in the slide, the following information is available for an ability

    GUID identified for the ability
    A link to the ATT&CK tactic and technique
    A name and description
    The platform that it is valid for
    A command that is to be ran to emulate the technique
    If applicable, clean-up activities

    We will further discuss how these abilities can be used later in this section!
  • MITRE Caldera Chain – Interface Walkthrough – Facts
    The Facts enable us to rely on variables to customize our operations. Although these are currently set statically (e.g. usernames, passwords,…), Caldera intends to support dynamic variables retrieved from previous outputs through regex expressions and full line retrievals.

    Each fact can be granted a score influencing its precedence over similar variables while individual variables can be temporarily disabled through blacklisting.
  • MITRE Caldera Chain – Interface Walkthrough – Adversaries
    The adversaries view is where we configure adversary profiles. This is where we could for example develop a threat actor that can afterwards be emulated. As part of the adversary profile, we create a number of phases that consist of different techniques that are to be executed!
  • MITRE Caldera Chain – Interface Walkthrough – Operations
    Operations are what we really seek in Caldera. An operation runs an adversary (a.k.a. threat actor; chain of actions/abilities) against a group of predefined infected hosts.
    One notable feature is Caldera’s ability to clean up behind itself, avoiding payloads and temporary files from staying on the targeted systems after use.

    Besides the clean-up, Caldera offers stealth-related options, one of which is the ability to automatically obfuscate its operation. One more advanced option is the Jitter which gives us the ability to customize the noise made during the operation by setting both the minimal and maximal duration between two C2 polls.

    On the right-hand side, we can see the results of the operation as it runs!
  • This page intentionally left blank.
  • Abilities which can’t be applied on a specific OS will just be skipped.
  • With the same principles, Abilities targeting all OS’es can be executed as part of an operation on a single targeted OS
  • This page intentionally left blank.

×