MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

4,128 views

Published on

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,128
On SlideShare
0
From Embeds
0
Number of Embeds
283
Actions
Shares
0
Downloads
80
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • Example of typical cover slide.
  • MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

    1. 1. MOLOCH: SEARCHFOR FULLPACKET CAPTURE
    2. 2. It is a Great Horned Owl Project Logo
    3. 3. WHYTHE OWL? Owls are silent hunters that go after RATs. We think that’s pretty cool. 3
    4. 4. HISTORY LESSON WHYAOL BUILT MOLOCH
    5. 5. WHAT IS MOLOCH? 5 Moloch is an open source, scalable IPv4 packet capture indexing and database system, built using open source technologies. • A simple web GUI is provided for browsing, searching, viewing and exporting PCAP data. • Web APIs are accessible if you wish to design your own GUI or directly grab PCAP with various command line tools for further analysis or processing. • Find it on AOL’s GitHub page: https://github.com/aol/moloch It’s like AOL Search for PCAP repositories!
    6. 6. WHAT IS MOLOCH NOT? 6 NOT IDS: NO ALERTS NOT IPV6 (Today) NOT SLOW NOT CLOSED NOT EXPENSIVE
    7. 7. WHYUSE MOLOCH? 7 Real-time capture of network traffic for forensic and investigative purposes • Combine the power of Moloch with other indicators (intelligence feeds, alerting from IDS/anti-virus) to empower your analysts to quickly and effectively review actions on the network to determine the validity/threat. • Review past network traffic for post compromise investigations. Static PCAP repository • Import large collections of PCAP that were created by malware. • Import collections of PCAP from Capture The Flag events. • Custom tagging of data at time of import.
    8. 8. THE PIECES OF MOLOCH 8 CAPTURE • A C application that sniffs the network interface, parses the traffic, and creates the Session Profile Information (SPI data) and writes it to disk. DATABASE • Elasticsearch is used for storing and searching through the SPI data generated by the capture component. VIEWER • A web interface that allows for GUI and API access from remote hosts to browse or query SPI data and retrieve stored PCAP.
    9. 9. THE PIECES OF MOLOCH: CAPTURE 9 Libnids based daemon written in C Can be used to sniff network interface for live capture Can be called from CLI to do manual imports Parses layers 3-7 to create SPI data • Spits them out to the Elasticsearch cluster. A lot like making owl pellets!
    10. 10. THE PIECES OF MOLOCH: DATABASE 11 Elasticsearch (http://www.elasticsearch.org) • Powered by Apache Lucene (http://lucene.apache.org) • Requests over HTTP(s) • Results returned in JSON Nosql • Network traffic doesn’t fit the mold for relational DBs. Documented oriented • Great for lots and lots of network sessions. Automatic sharding across multiple hosts • At the time, we skipped SOLR because it couldn’t run distributed. Fast, scalable, all that goodness
    11. 11. THE PIECES OF MOLOCH: VIEWER 12 Node.js based application • Event driven server side JavaScript platform. • Based on Chrome’s JavaScript runtime. • Comes with its own HTTP server and easy JSON for communication. Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP. GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, and command line tools is easy. • Easy automation to retrieve PCAP or sessions of interest.
    12. 12. THE PIECES OF MOLOCH: VIEWER 13 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
    13. 13. THE PIECES OF MOLOCH: VIEWER 14 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
    14. 14. ARCHITECTUREOF MOLOCH: DATAFLOW 15
    15. 15. ARCHITECTUREOF MOLOCH: MULTINODE WITH CLUSTER 16
    16. 16. ARCHITECTUREOF MOLOCH: SCALE 17 Packets Captured Kilobytes Saved Sessions Saved Example: Moloch Capture Documents Disk Storage (MB) Example: Elasticsearch
    17. 17. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 18 IP • Source • Destination • Ports • Protocol HTTP • Method • Status Codes • Headers • Content Type DNS • IP Address • Hostnames
    18. 18. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 19 SSL/TLS • Cert Elements: • Common Name • Serial Number • Alt Names SSH • Client Name • Public Key • Port IRC • Channel Name • Hostname
    19. 19. MOLOCH: CAPTURE CREATING SPI DATA 20
    20. 20. MOLOCH: CAPTURE CREATING SPI DATA 21
    21. 21. MOLOCH: CAPTURE CREATING SPI DATA 22
    22. 22. MOLOCH: CAPTURE CREATING SPI DATA 23
    23. 23. MOLOCH: CAPTURE CREATING SPI DATA 24
    24. 24. MOLOCH: DEMO 25
    25. 25. MOLOCH: QUESTIONS? 26

    ×