This is the slides from keynote presentation at BSidesTampa 2015. A recording of the talk can be found at: https://www.youtube.com/watch?v=751bkSD2Nn8&t=1m35s
My Keynote from BSidesTampa 2015 (video in description)
1. The Need for Proactive Threat
Hunting
Andrew Case
@attrc
2. Who Am I?
• Core Volatility developer
• Co-Author “Art of Memory Forensics”
• Lead-investigator on large-scale investigations
• Performed many RE efforts, pentests, and
source code audits
• BSidesNola (New Orleans) Co-Organizer
3. Why is Threat Hunting Needed?
• Many opaque components of the information
infrastructure
• You are combating a creative and adaptive
adversary and thus you need a creative and
adaptive analyst to find them
• Statistics have shown that people are
compromised for years without noticing
4. What is Threat Hunting?
• Searching for adversaries without a particular
indicator
• Dedicating time and resources to deep
analysis of potentially compromised resources
• See [1] for great commentary by Sean Mason
and [2] for several posts by Jack Crook
5. What are its Benefits?
• Makes the organization proactive against
attackers
• Quickly find gaps in system and application
configurations
• Defenders more familiar with their own
environment and infrastructure
• Documentation leads to organizational
knowledge
6. Gaining Familiarity
• Understanding and defining “normal” in order
to detect anomalous behavior and attributes
• “normal” is unique to a particular organization
and even subsets within the organization
– “normal” of a web server is quite different than
the system of Joe in accounting
• Unfamiliarity with “normal” leads to
extremely ineffective response
7. Running Processes
• If your analysts were given a list of every process
running on a system in your environment, how
many of them could definitively rule each as
normal or abnormal?
• How would this be judged?
– Name of the process?
– Path to the executable on disk?
– Parent process?
• Patrick Olsen has gone through great lengths to
document this [5]
8. Process Privileges
• What privileges do each process run as?
• Do any 3rd party programs abuse privileges or
grant themselves higher privileges than
necessary?
• Do you know which of your users run as local
admin?
9. Network Activity
• Which applications should be listening for
network connections?
• Which applications should talk on the
network?
• Is there any ingress/egress filtering?
– Has it been disabled or tampered with by
malware/attackers?
10. Kernel Drivers
• Kernel drivers have full access to entirety of a
system and its resources
• A default Windows 7 install loads over 100 kernel
drivers
• Two of the following drivers are normal, two are
Stuxnet, do your analysts know which?
– MRxCls
– MRxDAV
– MRxNet
– MRxSMB
11. Persistence Mechanisms
• More than just AutoRun Keys
• “Beyond the Run Key”, 26 (now 27) part and
counting blog post series by Adam Blaszczyk
on forensics of Windows persistence
mechanisms [4]
12. Scheduled Tasks
• Default Windows 7 install has numerous
scheduled tasks by default
• 3rd party applications create them to check for
updates, run maintenance scripts, and so on
• Adversaries also leave scheduled tasks
behind…
• Triggers: [6]
13. Services
• Like Scheduled Tasks, many will be installed by
default and 3rd party applications will create
their own
• Sophisticated threat groups also like to install
services
– Can be very manual and time consuming to detect
malicious services, even with memory forensics
analysis
14. Anti-Virus & HIPs
• Often act like malware to gain visibility into
the system
• Need to be filtered/whitelisted from any
rootkit detection tools
• Changes in AV/HIPs algorithms requires
changes in filters
15. Documentation is Org Knowledge
• Team members should not live in a silo
– “normal” should be documented in a way that
other team members can access
• Documentation outlives employees leaving
and scales during incidents
– Do not allow “Brents” to be created
• If your entire IR team mutinied tomorrow,
how long would it take for new hires to regain
all the departing knowledge?
16. What is the End Result?
• Proactive detection of threats
• Effective detection and response
• IR teams that deeply understand their
environment
• Organizational knowledge that continues to
grow and survives generations of employees
17. How Do You Get There?
• The executives need to understand the value
of a properly prepared IR team
• The IR team must be elevated to the status of
the IT Security team and be just as an integral
a part of the organization’s ongoing IT flow
18. Security vs IR
• Security teams are positioned during all parts
of the IT process while IR is used only during
incidents
• This leads to IR staff not being effectively
utilized and not being an on-going part of the
organization
19. IT Security Pre-Deployment
• Baseline testing of gold images
– Security evaluations done well before production
use
• Application development
– Secure SDLC
• Secure DevOps
– Incorporating security into cloud deployments
– Richard Mogull does great work in this space [3]
21. IR is Embedded Into Nothing
• It is always after the fact
• This leaves knowledge gaps and forces on-the-
spot learning during incidents
• How do we fix this?
22. Incorporating the IR Team Pre-Deployment
• As security reviews gold images, the IR team
should be building baselines and looking for
logging misconfigurations that prevent full
forensic exploitation
• Applications should be developed and
configured so that all relevant activity is
logged and recoverable
23. Incorporating the IR Team Post-Deployment
• Continuous:
– Threat hunting
– Documentation of changes to systems and
applications
– Incorporation of new forensics artifacts into
analysis processes
24. Incident Preparedness
• IT security has dedicated systems for
vulnerability scanning, application testing, etc.
• IR teams need dedicated, pre-configured
systems to effectively hunt as well as respond
to incidents
25. Incident Preparedness Essentials
• Network monitoring
• Dedicated storage servers
• Deployable acquisition/sampling tools and
agents
• Analysis servers with real processing power
• Without these and others, response will be
chaotic, underpowered, and likely ineffective
26. Utilizing Documentation
• As the IR team becomes embedded,
everything it learns should be documented
• If done correctly, everything that is known
from a forensics perspective about a system
and its applications will be readily available to
all team members
27. Documentation into Internal Training
• New hires can be pointed to documentation of
all assets on the network and their forensic
value and artifacts
• Ongoing internal training can focus on new
artifacts discovered during all phases of the IR
team’s involvement
• A great post by Jack Crook that covers this
topic [7]
28. Helping Outside Parties
• Only a handful of organizations can
completely handle major breaches internally
• Giving organized access to 3rd party analysts
makes their effort more effective and
accomplished in a shorter amount of time
• To accomplish this, documentation and
analysis infrastructure must be setup before a
breach
30. Spending: Security vs IR Preparedness
• If “Shell Shock 2” were to be released right
now would you feel better knowing your
systems were fully patched (hence vulnerable)
or that you had a fully prepared IR team that
can handle the outbreak effectively?
• Does your organization’s resource allocation
reflect your feelings on this?
31. Steps – Preparing for a Hunt
1. Free up your IR teams time to prepare for
hunting
2. Create a plan that will lead to documentation
of all your hunting and response efforts
3. Start small
4. Refine
5. Move to team wide hunts
6. Create challenges and internal training based
on real events
32. Steps – Embedding IR into IT
1. Convince executives of the need
2. Update policy to ensure IR has a hand in
ongoing operations just like security does
3. Document everything learned
4. Incorporate what is learned into analysis
during hunts and incident handling
33. Conclusions
• Threat hunting is one of the best tools
available to organizations in order to stay
ahead of adversaries
• You should aim to minimize the space
attackers can work where you will not find
them
• Don’t wait on a vendor or the FBI to notify you
of breaches – be active and find them
yourself!