1.
Hunting:
Defense Against The Dark Arts

2.
New slides, who dis?
Danny Akacki
Hunting: Defense Against The Dark Arts 2
Who: @DAkacki
(in conjunction with and on behalf of
@find_evil & @StephenHinck)
What: Hunter of things
Where: Fortune blah blah blah
About:
• Mandiant x2
• GE Capital
• Long, sordid love affair with Philly.
• Enthusiastic hugger.
#HumanZoo

3.
Hunting: Defense Against The Dark Arts 3
Problem Set
• Find Evil
• Find Ways for Evil to do Evil Things
• Drive maturation of monitoring & detection capabilities

4.
Hunting: Defense Against The Dark Arts 4
Traditional Detection vs. Hunting
Not
❌ Tools
❌ Alerts
❌ Automation

5.
Hunting: Defense Against The Dark Arts 5
Hunting As A Methodology.
• Think layers.
• Linear.
• Iterative.
• Hypothesis driven.

6.
Threat Hunting Loop
Hunting: Defense Against The Dark Arts 6
https://sqrrl.com/solutions/cyber-threat-hunting

7.
Building a Hunt Program
Hunting: Defense Against The Dark Arts 7
"Understanding is the first step to acceptance,
and only with acceptance can there be
recovery.“
— Albus Dumbledore

8.
Hunt Program
Mature detection capabilities
Use Cases + Playbooks
Guiding processes for SOC / CIRT
Technology & Tools
Operationally-driven and requirements-based
SOC + CIRT
Security operations and incident response
Formalized Security Program
Chartered and backed by an executive sponsor
Hunting: Defense Against The Dark Arts 8
Hunting Capability Pyramid
Must be this
tall to ride 

9.
Hunting: Defense Against The Dark Arts 9
http://blog.sqrrl.com/the-cyber-hunting-maturity-model
Hunting Maturity Model

10.
Building a Hunt Program
Hunting: Defense Against The Dark Arts 10
1. Establish executive sponsorship and mission charter/objectives
2. Establish and implement enterprise logging strategy
3. Aggregate, centralize, and process data
4. Make data available within a (fast) searchable interface
5. Drive maturity
• Develop Use Cases
• Are we getting the right data?
• Review tooling and associated requirements
• Reintegrate hunt mission data to security operations

11.
Hunting + IR  Detection Maturation
Hunting: Defense Against The Dark Arts 11
HUNT SOC DETECT
IR USE CASE
Ongoing Hunt
Missions
Feed Incident
Response activities
IR outcomes affect
SecOps
Lessons Learned
incorporated to
SecOps
Detection
capability
improvement
Evil
Non-Evil Risk

12.
Fantastic Use Cases and How To Make Them
Hunting: Defense Against The Dark Arts 12
• Scenarios to help solve/uncover problems and guide your thinking.
• Can be simple or complex
• Helps to identify data / capability requirements and gaps
• Aligned to an attacker lifecycle: Kill Chain or ATT&CK
• Contains Internal TTP used to achieve the Use Case Objectives
• Data – What should we collect to detect events of interest?
• Tools – What can we use to handle our Data?
• Logic – How can we best leverage both our Data and Tools?

13.
Use Cases: Slide 2: Detection Bugaloo
Hunting: Defense Against The Dark Arts 13
Incident
Events of Interest
Detection Use Case
Events of Interest, ex.
1. $Endpoint1 seen making
DNS requests for known
bad domain
2. HTTP Proxy sees
$Endpoint1 requesting
binary with unknown MD5
3. Network logs show
periodic suspicious
communications from
$Endpoint1 to multiple
new hosts in unlikely
countries

14.
Use Case Design Tree: Objective
Hunting: Defense Against The Dark Arts 14

15.
Use Case Design Tree: Tools & Capabilities
Hunting: Defense Against The Dark Arts 15

16.
Hunt Mission Outcomes
Hunting: Defense Against The Dark Arts 16
•Benefit: Activity shown not to be present
•Next Step: Evaluate hunt mission effectiveness
No Detection
•Benefits: Activity shown to be present
Hunt mission effectiveness validated
Identify best practice / compliance issues
•Next Step: Escalate as appropriate, monitor to closure
Detection:
Non-Malicious
•Benefits: Activity shown to be present
Hunt mission effectiveness validated
Identify security incidents
•Next Step: Escalate as appropriate, monitor to closure
Detection:
Malicious

17.
Sorting Out Your Data
Hunting: Defense Against The Dark Arts 17
"Not Slytherin, eh? Are you sure? You could be great, you know."

18.
Data Sources
- Remote Access
- Web Proxy
- IDS / IPS
- Email
- WAF
- DNS
- DHCP
- NetFlow
- Firewall
- Router / Switch
- Wireless Infrastructure
- Agents
- Antivirus
- Operating Systems
- Active Directory
- File, Print, Database
- Other Services
External Feeds
- Paid, Free, OSINT
Internal Feeds
- Recon Data
- Threat/Risk Models
- IR Lessons Learned
- Critical Asset Inventory
- Identity & Access
Management (IAM)
- Scheduled Service
Interruptions
- Terminated Users
- Acceptable Use Policy
- Employee Work Hours
- Physical Access Logs
Security
Network
Endpoint
IT
Threat Intel
HR
Hunting: Defense Against The Dark Arts 18

19.
Two Types of Events
Hunting: Defense Against The Dark Arts 19
1. Observed  Originated from a device that handled the event in some way
2. Synthetic  Generated through automated analysis of event data

20.
What is the Right Data?
Hunting: Defense Against The Dark Arts 20
• Original source data wherever possible
• Ensure the presence of important metadata
• Generally, observed events > synthetic events
• Synthetic events can provide useful context in the form of analytics
• Logs must enable pivoting
• Minimum - one extractable / consistent data point to correlate log sources

21.
Ready the Spells!
Hunting: Defense Against The Dark Arts 21
• Understand the network
• Learn critical assets
• Develop enterprise logging strategy
• Ensure data sources use consistent time settings; implement NTP, use GMT (or UTC)
• Plug in to asset, change, and configuration management processes
• Account for other organizational use cases
• IT Operations
• Forensics / Incident Response
• Compliance / Audit
• Clean up the hunt dataset
• Normalization
• De-duplication
• Parsing
• Enrich and contextualize the dataset...!

22.
Event Enrichment
Hunting: Defense Against The Dark Arts 22
• Internally-sourced Intelligence
• Attack Trees
• Red Team / Penetration test output
• TTPs from previous incidents
• Deviances from baselines / Expected behavior
• Organizational risk profile / Threat context
• Externally-sourced Intelligence
• Paid subscriptions
• OSINT
• Free feeds
• Passive DNS, WHOIS, etc.
• Geographical data
• ISAC, Infragard, etc.
• Context
• Environmental
• Refer to "Data Source" slide
• Previous hunt and IR output
• Malware analysis
• Analytics, Ex:
• Geo-infeasibility
• Beacon detection
• DNS entropy
• Data exfiltration

23.
Tools of the Trade
Hunting: Defense Against The Dark Arts 23
"It is important to fight, and fight again, and keep fighting, for only
then could evil be kept at bay, though never quite eradicated"
— Albus Dumbledore

24.
Criteria for a Working Hunt Platform
Hunting: Defense Against The Dark Arts 24
• Rapid search with high quality UI and / or API
• Stacking
• Group and reduce the dataset to more easily identify outliers
• Improves feasibility of analyzing large environments
• Pivoting
• Move laterally through the dataset
• See the whole picture
• Nice to Have
• Tagging and Enrichments
• Intelligence Integration Support
• Automation: Rules & Alerting

25.
All About The Galleons
Hunting: Defense Against The Dark Arts 25
• Budget!
• Driven by Operational Requirements
• Tool/Vendor Selection Process
• Evaluation Success Criteria
• Multiple Tools: Diverse Perspectives
• Free and Open Source Software!
• NXLog
• Sysmon
• Moloch
• Wireshark
• Bro Network Security Monitor
• ELK Stack (ElasticSearch, Logstash, Kibana)
• Security Onion Linux Distribution– Da Real MVP
+ a bunch of other stuff not listed here...

26.
Analysis
Hunting: Defense Against The Dark Arts 26
"We teachers are rather good at magic, you
know."

27.
Sample Hypotheses to Drive Hunt Missions
Hunting: Defense Against The Dark Arts 27
1. Sensitive corporate data stored only in
approved locations
2. Large or extended outbound data transfers
meet business needs
3. Reconnaissance activities against DMZ hosts
provide advance warning of pending malicious
activity
4. VPN logins by users are geographically
feasible
5. Domain controller baselines are simple and
deviations rarely occur
6. Service credentials are used only in expected
ways and for their appropriate services
7. Web proxies are appropriately configured to
block suspicious traffic
8. Services communicate using secure,
encrypted protocols
9. Tunneling HTTP traffic and other proxy
avoidance techniques are not allowed in or
out of our network
10. The use of management tools (such as
PSExec) occurs only within approved change
windows
11. Endpoints are not added to the network
without infosec visibility

28.
More Data, More Problems
Hunting: Defense Against The Dark Arts 28
"Dobby is... free."

29.
Hunting: Defense Against The Dark Arts 29
Evil vs. Ways for Evil to do Evil Things

30.
1. Remote Access
Hunting: Defense Against The Dark Arts 30
Hypothesis: Remote access to our environment is conducted using approved means
Discovery:
• Remote access is occurring over multiple protocols to / from unapproved hosts
• VNC to / from production network
• RDP to domain controllers from DMZ
• Evidence of unapproved remote access utilities such as LogMeIn, GoToMyPC, etc
Recommendation:
• Evaluate unapproved connections for mitigation or for risk acceptance
• Ensure that risk accepted software is fully patched and up to date
• Implement strong encryption, jump boxes / VPN ACLs, and two-factor authentication where possible

31.
2. Data Storage
Hunting: Defense Against The Dark Arts 31
Hypothesis: Corporate data is only stored in approved locations
Discovery:
• Sensitive corporate data stored on unencrypted and infected external media
• Unrestricted use of common cloud data storage providers
• Unmanaged source code repositories (intellectual property)
Recommendation:
• Evaluate DLP implementation and allowed web proxy categories
• Consider establishing formalized agreement with a cloud storage provider
• Bring unmanaged data stores under management in support of development teams

32.
3. Proxy Infrastructure
Hunting: Defense Against The Dark Arts 32
Hypothesis: Our proxy infrastructure is properly configured
Discovery:
• Not blocking known malicious categories
• Not blocking executable downloads
• Proxies not logging all necessary protocol metadata
• Ex. User Agent, Status Code, Byte Counts, X-Forward-For, etc.
Recommendation:
• Validate security operations' requirements of proxy infrastructure
• Re-evaluate proxy configurations for appropriate changes
• Ensure security operations are looped in to the change management process

33.
4. Approved Protocols
Hunting: Defense Against The Dark Arts 33
Hypothesis: Protocols transiting our network are secure and approved for use
Discovery:
• Various insecure protocols identified in use across the network
• Unencrypted: Telnet, FTP
• Deprecated: SNMP v2, cleartext SMTP
• Risky: IRC, TOR / i2p
Recommendation:
• Identify opportunities to deploy secured versions of protocols
• FTP  SFTP
• Telnet  SSH
• SNMP v2  SNMP v3, etc.
• Evaluate implementation of risk detection and mitigation strategies

34.
5. Approved Clients
Hunting: Defense Against The Dark Arts 34
Hypothesis: Internet access is achieved using known and approved client software
Discovery:
• Suspicious user-agents identified - indicating potential latent infections
• Extremely out of date software, including: client browsers, Flash, and Java
Recommendation:
• Begin incident response procedures to evaluate and triage endpoints
• Evaluate consistency of patch and vulnerability management processes

35.
6. Privilege Management
Hunting: Defense Against The Dark Arts 35
Hypothesis: Account management is rooted in best practice
Discovery:
• Service accounts used for unrelated purposes or shared by users
• Regular and privileged users with non-specific accounts
• Direct privileged logins without approved privilege escalation process (e.g. sudo)
• Suspicious usernames that do not conform to the organizational standard
• User account belonging to terminated user active on the network
Recommendation:
• Evaluate suspicious or ambiguous accounts for mitigation or for risk acceptance
• Ensure security operations are tied into the HR termination workflow
• Update organizational username standard and privilege management processes

36.
7. Security Architecture
Hunting: Defense Against The Dark Arts 36
Hypothesis: Event logs provide information needed to validate control effectiveness
Discovery:
• Non-security specific appliances with disabled security functionality
• Ex. Cisco ASA scan detection disabled
• Security specific appliances improperly placed
• Bro NSM placed post-proxy, post-NAT
Recommendation:
• Evaluate IT systems for security value (non-traditional security appliances)
• Ex. Network devices
• Modify configuration and placement of systems to meet requirements

37.
8. Process Execution
Hunting: Defense Against The Dark Arts 37
Hypothesis: Endpoints only execute processes required for business functions
Discovery:
• Obfuscated PowerShell execution
• Mimikatz and other persistence toolkit execution
• Suspicious filenames/paths/registry entries, etc.
• Users installing browser toolbars and miscellaneous adware/spyware
Recommendation:
• Call the IR Team 
• Adjust detections / controls to rapidly detect and prevent future occurrences

38.
9. DNS
Hunting: Defense Against The Dark Arts 38
Hypothesis: DNS resolutions occur within the bounds of best practices
Discovery:
• "Weird" protocol deviations/padded packets suggesting exfil or C&C
• Uncontrolled resolutions that are not forced through corporate infrastructure
• Resolutions for unusual or risky domains
• Ex. Dynamic DNS domains, domains appearing to be algorithmically generated
• Initial resolutions for suspicious domains + subsequent unusual communication
Recommendation:
• Harden organizational DNS infrastructure
• Ex. Implement DNSSEC, prevent zone transfers, etc.
• Configure perimeter devices to only accept DNS requests from corporate DNS
• Implement protocol anomaly detection to identify protocol misuse

39.
Thinking Ahead
Hunting: Defense Against The Dark Arts 39
"The one with the power to
vanquish the Dark Lord
approaches..."
— Sybill Trelawney

40.
Ensuring Successful Outcomes
Hunting: Defense Against The Dark Arts 40
• Goals
• Reduce attack surface
• Harden the environment
• Improve detection and monitoring
• Don't bother hunting without using the outputs!
• Lessons Learned / AAR
• Feedback loop on IR processes
• Create new or improve existing detections
• Metrics
• Cannot improve what is not measured
• The absence of something is still something
• Most metrics will trend upwards before they come down
• 'Time to Detect' and other metrics will trend downward over time

41.
Hunt Methodology: From Art to Science
Hunting: Defense Against The Dark Arts 41
Begin evolution from intuitive art to a more rigorously structured science

42.
Show of Hands...
Hunting: Defense Against The Dark Arts 42

43.
Resources
Hunting: Defense Against The Dark Arts 43
FireEye Threat Analytics Platform: Hunting at Scale
https://www.fireeye.com/products/threat-analytics-platform.html
MITRE: Adversarial Tactics, Techniques & Common Knowledge
https://attack.mitre.org
The Threat Hunting Project: Compendium of useful resources
http://www.threathunting.net
Loggly: Helpful logging guidelines
https://www.loggly.com/intro-to-log-management
Security Onion: Peel back the layers of your network
https://securityonion.net

44.
Resources
Hunting: Defense Against The Dark Arts 44
The Bro Network Security Monitor
https://www.bro.org/
Jack Crook: Finding Bad
http://findingbad.blogspot.com/
Sqrrl Blog
http://blog.sqrrl.com/
The Elastic Stack
https://www.elastic.co/products
Of 2 Minds – How Fast and Slow Thinking Shape Perception and Choice
https://www.scientificamerican.com/article/kahneman-excerpt-thinking-fast-and-slow/

45.
FIN
Hunting: Defense Against The Dark Arts 45
In Conclusion:
• Building a program is hard, building a capability less so.
• The tools and knowledge are out there.
• Context is king.
• Silo’s will kill you. Share early, share often.
• While you’re here, meet someone new. Strike up a conversation. This is what it’s all
about.
• Completely unrelated and completely related at the same time. Be kind to one
another. Work is hard, life is harder. Give hugs.

46.
Happy Hunting!