This document discusses using the Elastic Stack (Elasticsearch, Logstash, Kibana) for threat hunting. It begins with an introduction to the speaker and their company, Network Intelligence. It then provides an overview of threat hunting and why it is important for early detection of attacks. The bulk of the document focuses on explaining the Elastic Stack tools and how they can be used together to ingest, parse, index and visualize log data for threat hunting purposes. Specific techniques like searching for IOCs and analyzing patterns in logs are discussed.

1. THREAT HUNTING
USING OPEN-SOURCE TECHNOLOGY (ELASTIC STACK)
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 1
SACON 2019

2. ABOUT ME
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15/02/2019 2
Head of Research
and Development
@ Network
Intelligence
Been part of the
infosec community
for 10+ years
Ex-Null Mumbai
Moderator
Presented at
Bsides, Malcon, CSI,
OWASP,
SecurityBytes
@washalsec
wasim.halani@

3. ABOUT NETWORK INTELLIGENCE
¡ Global cybersecurity provider founded in 2001
¡ More than 550 team members and 8 offices across the globe
¡ Offer services across 5 broad spectrums
¡ Assessment
¡ GRC
¡ Professional Services
¡ MSSP
¡ Cybersecurity Trainings.
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 3
We believe that cybersecurity is not a destination, it is a journey and we partner with our clients to address the dynamic cybersecurity threat landscape.

4. WHAT?
Threat Hunting is the use of traditional and non-
traditional techniques to proactively look for
evidence of an attack taking place within the
network.
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 415/02/2019

5. WHAT?
¡ A different Perspective (of looking at logs) and Approach (to analyzing logs)
¡ Asking questions:
¡ Did it happen?
¡ How did it happen?
¡ When did it happen?
¡ Where all did it occur?
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 515/02/2019

6. WHY?
¡ The annualVerizon Data Breach Report 2018 68% breaches took months or longer to
discover
¡ Which means, organizations investing millions in cyber-security are not detecting attacks on
time
¡ Traditional SIEM/SOC setups are geared more towards standard use cases
¡ Compliance monitoring
¡ Change monitoring
¡ Firewall changes
¡ Operational activities
¡ Network interface up/down
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 6

7. DIFFERENCES AND OVERLAPS
Security
Monitoring
Incident
Response
Threat
Hunting
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 7
Search queries, hi-fi rules,TI
Event investigation, IOC extraction,
retro-search

8. HOW?
Threat
Hunting
People
Skills
Tools
Data
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 8
¡ Requires management support and the dedicated
people for the job
¡ Skills define the hunters ability to ask right questions
¡ Tools provide ability to query and interact with the
data
¡ Data sources provide visibility
¡ You need the right data
¡ You need to know your data

9. THREAT HUNTING APPROACHES
¡ Structured
¡ Hypothesis driven
¡ Unstructured
¡ Anomaly and pattern driven
¡ Hypothesis: a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation.
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 9

10. BELK ELASTIC STACK PRIMER
ELASTICSEARCH, LOGSTASH, KIBANA, BEATS
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 10

11. WHY ELASTIC?
¡ Why not!?
¡ Open-source engine
¡ Scalable cluster
¡ Great visualization capabilities - #secviz
¡ Fast search and analytics
Remember, it’s not about the tool, but how you use it
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 11

12. STACK OVERVIEW
¡ Beats
¡ Log shippers - Windows events, System status, Network traffic
¡ Elasticsearch
¡ Data storage, search engine
¡ Logstash
¡ Log management component. Ingest, Process, Output
¡ Kibana
¡ Create visualizations and dashboards from data stored in Elasticsearch
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 1215/02/2019

13. 15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 13
ELK :ARCHITECTURE

14. ELASTICSEARCH
¡ Created by Shay Banon (~2010)
¡ Elasticsearch is a distributed NoSQL datastore, RESTful search and analytics engine
¡ Based on Apache Lucene
¡ Open-source search engine library
¡ API architecture
¡ Uses inverted index data-structure
Raw Data Stored Indexed Data
Blogpost ID Blog Tags Tags IDs
22 appsec appsec 501,340,22
340 netsec,appsec mobile 501
501 appsec, mobile netsec 340
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 1415/02/2019

15. IMPORTANT TERMS/CONCEPTS
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15

16. NODES & CLUSTER
¡ A cluster is a collection of one or more nodes (servers) that together holds your entire data and provides
federated indexing and search capabilities across all nodes.
¡ A cluster is identified by a unique name which by default is "elasticsearch".
¡ A node is a single server that is part of your cluster, stores your data, and participates in the cluster’s indexing and
search capabilities.
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 1615/02/2019

17. INDEX & SHARDS
¡ An index is a collection of documents that have somewhat similar characteristics. E.g. customer data, product
catalog, order data
¡ An index is identified by a name (that must be all lowercase)
¡ There is no limit to how many documents you can store on each index
¡ Large Index size => Elasticsearch performance issues
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 1715/02/2019

18. DOCUMENTS
¡ A document is a basic unit of information that can be indexed. E.g. document for a single customer, another
document for a single product, and yet another for a single order.
¡ document is expressed in JSON
Sample document
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 1815/02/2019

19. FIELDS
¡ Fields are the smallest individual unit of data in Elasticsearch.
¡ Each field has a defined type and contains a single piece of data
¡ a Boolean
¡ string
¡ array expression
¡ A collection of fields are together a single Elasticsearch document.
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 1915/02/2019

20. LOGSTASH
¡ Integrated log management framework
¡ log collection
¡ centralization
¡ parsing
¡ storage
¡ Written in JRuby
¡ Runs in JVM
¡ Multiple input mechanism
¡ TCP/UDP
¡ Files
¡ Syslog
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2015/02/2019

21. LOGSTASH: CONFIGURATION
¡ input {}
¡ filter {}
¡ output {}
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2115/02/2019

22. LOGSTASH:
PLUGINS
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15/02/2019 22

23. IMPORTANT FILTER PLUGINS
¡ date
¡ mutate
¡ grok
¡ kv
¡ geoip
¡ metrics
¡ translate
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2315/02/2019

24. UNDERSTANDING GROK
¡ Parse arbitrary text and structure it.
¡ parse crappy unstructured log data into something structured and query-able.
¡ Logstash ships with many patterns by default.You can find them here
¡ https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns
¡ http://grokdebug.herokuapp.com
¡ http://grokconstructor.appspot.com/
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2415/02/2019

25. GROK BASICS
¡ The syntax for a grok pattern is %{SYNTAX:SEMANTIC}
¡ The SYNTAX is the name of the pattern that will match your text.
¡ The SEMANTIC is the identifier you give to the piece of text being matched.
¡ 3.44
¡ SYNTAX: NUMBER || SEMANTIC: duration
¡ 1.3.44.55
¡ SYNTAX: IP || SEMANTIC: clientip
¡ GROK Filter
¡ %{NUMBER:duration} %{IP:client}
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2515/02/2019

26. GROK EXAMPLE
¡ Log Event
¡ 55.3.244.1 GET /index.html 15824 0.043
¡ Grok Filter
¡ %{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2615/02/2019

27. KIBANA
¡ Visualization platform
¡ Tightly integrated with Elasticsearch
¡ http://localhost:5601/app/kibana
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2715/02/2019

28. NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2815/02/2019

29. NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 2915/02/2019

30. FILEBEAT
¡ Lightweight file log shipper
¡ Basically does a ‘tail’ of the log file
¡ Automatically sends data appended in real-time
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15/02/2019 30

31. WINLOGBEAT
¡ Lightweight windows events shipper
¡ Automatically sends data appended in real-time
¡ Can filter specific event IDs
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15/02/2019 31

32. DEMO
• INGESTING LOGS
• CREATING
VISUALIZATIONS
• ANALYZING DATA
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15/02/2019 32

33. USE-CASE BUILDING
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 33

34. LET’S SEARCH
¡ Integrate with Threat Intelligence (IoC)
¡ “mimikatz.exe”
¡ Specific Hashes/Filenames/IP-addresses (not found in TI feed)
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 34

35. LET’S THINK PATTERNS
¡ Web Logs
¡ Too many HTTP 200 responses for same pages
¡ Too many HTTP 404 error messages from same IP but different pages
¡ DNS Logs
¡ Spike in NXDomain responses
¡ Firewall
¡ Multiple denied attempts from one machine against one/more machines on different ports
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 35

36. LET’S BUILD A HYPOTHESIS
¡ An attacker has compromised a machine using macro-based Powershell payload sent over email
¡ We need
¡ Endpoint logs
¡ Authentication
¡ Process execution
¡ Network connection
¡ Email gateway logs
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 36

37. LET’S BUILD A HYPOTHESIS
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 37
1. Filter for winword.exe processes in endpoint logs
2. Search for events with winword.exe as parent process and powershell.exe as child process
3. Trace back the to original CLI i.e. find the file which contained the payload
4. Identify the path – downloads or email or unknown (?)
1. If email temp folder then look in Exchange logs for other potential targets or payloads

38. MITRE FRAMEWORKS
¡ Enterprise ATT&CK Matrix
¡ CAR
15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 38

39. NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15/02/2019 39

40. 15/02/2019NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 40

41. THANKYOU!
CONTACT:
@WASHAL
NETWORK INTELLIGENCE INDIA PVT. LTD. | SECURITY ARCHITECTURE CONFERENCE (SACON), 2019 15/02/2019 41