MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

4,455 views

Published on

Published in: Technology

MOLOCH: Search for Full Packet Capture (OA Cyber Summit)

  1. 1. MOLOCH: SEARCHFOR FULLPACKET CAPTURE
  2. 2. It is a Great Horned Owl Project Logo
  3. 3. WHYTHE OWL? Owls are silent hunters that go after RATs. We think that’s pretty cool. 3
  4. 4. HISTORY LESSON WHYAOL BUILT MOLOCH
  5. 5. WHAT IS MOLOCH? 5 Moloch is an open source, scalable IPv4 packet capture indexing and database system, built using open source technologies. • A simple web GUI is provided for browsing, searching, viewing and exporting PCAP data. • Web APIs are accessible if you wish to design your own GUI or directly grab PCAP with various command line tools for further analysis or processing. • Find it on AOL’s GitHub page: https://github.com/aol/moloch It’s like AOL Search for PCAP repositories!
  6. 6. WHAT IS MOLOCH NOT? 6 NOT IDS: NO ALERTS NOT IPV6 (Today) NOT SLOW NOT CLOSED NOT EXPENSIVE
  7. 7. WHYUSE MOLOCH? 7 Real-time capture of network traffic for forensic and investigative purposes • Combine the power of Moloch with other indicators (intelligence feeds, alerting from IDS/anti-virus) to empower your analysts to quickly and effectively review actions on the network to determine the validity/threat. • Review past network traffic for post compromise investigations. Static PCAP repository • Import large collections of PCAP that were created by malware. • Import collections of PCAP from Capture The Flag events. • Custom tagging of data at time of import.
  8. 8. THE PIECES OF MOLOCH 8 CAPTURE • A C application that sniffs the network interface, parses the traffic, and creates the Session Profile Information (SPI data) and writes it to disk. DATABASE • Elasticsearch is used for storing and searching through the SPI data generated by the capture component. VIEWER • A web interface that allows for GUI and API access from remote hosts to browse or query SPI data and retrieve stored PCAP.
  9. 9. THE PIECES OF MOLOCH: CAPTURE 9 Libnids based daemon written in C Can be used to sniff network interface for live capture Can be called from CLI to do manual imports Parses layers 3-7 to create SPI data • Spits them out to the Elasticsearch cluster. A lot like making owl pellets!
  10. 10. THE PIECES OF MOLOCH: DATABASE 11 Elasticsearch (http://www.elasticsearch.org) • Powered by Apache Lucene (http://lucene.apache.org) • Requests over HTTP(s) • Results returned in JSON Nosql • Network traffic doesn’t fit the mold for relational DBs. Documented oriented • Great for lots and lots of network sessions. Automatic sharding across multiple hosts • At the time, we skipped SOLR because it couldn’t run distributed. Fast, scalable, all that goodness
  11. 11. THE PIECES OF MOLOCH: VIEWER 12 Node.js based application • Event driven server side JavaScript platform. • Based on Chrome’s JavaScript runtime. • Comes with its own HTTP server and easy JSON for communication. Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP. GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, and command line tools is easy. • Easy automation to retrieve PCAP or sessions of interest.
  12. 12. THE PIECES OF MOLOCH: VIEWER 13 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
  13. 13. THE PIECES OF MOLOCH: VIEWER 14 Nodejs based application • Event driven server side JavaScript platform • Based on Chrome’s JavaScript runtime • Comes with its own HTTP server and easy JSON for communication Web based GUI • Browsing / searching / viewing / exporting SPI data and PCAP GUI and API use URIs • All calls are done using URIs so integration with SEIMs, consoles, command line tools is easy. • Easy automation to retrieve pcap or sessions of interest.
  14. 14. ARCHITECTUREOF MOLOCH: DATAFLOW 15
  15. 15. ARCHITECTUREOF MOLOCH: MULTINODE WITH CLUSTER 16
  16. 16. ARCHITECTUREOF MOLOCH: SCALE 17 Packets Captured Kilobytes Saved Sessions Saved Example: Moloch Capture Documents Disk Storage (MB) Example: Elasticsearch
  17. 17. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 18 IP • Source • Destination • Ports • Protocol HTTP • Method • Status Codes • Headers • Content Type DNS • IP Address • Hostnames
  18. 18. MOLOCH: SPI-DATATYPES SESSION PROFILE INFORMATION 19 SSL/TLS • Cert Elements: • Common Name • Serial Number • Alt Names SSH • Client Name • Public Key • Port IRC • Channel Name • Hostname
  19. 19. MOLOCH: CAPTURE CREATING SPI DATA 20
  20. 20. MOLOCH: CAPTURE CREATING SPI DATA 21
  21. 21. MOLOCH: CAPTURE CREATING SPI DATA 22
  22. 22. MOLOCH: CAPTURE CREATING SPI DATA 23
  23. 23. MOLOCH: CAPTURE CREATING SPI DATA 24
  24. 24. MOLOCH: DEMO 25
  25. 25. MOLOCH: QUESTIONS? 26

×