Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moloch: Recent Changes & Upcoming Features - Andy Wick, Sr Principal Architect, Oath & Elyse Rinne, Software Dev Engineer, Oath

263 views

Published on

Presented at the 2nd Annual Moloch (https://molo.ch/) Conference on November 1st, 2018. Moloch is a large-scale, open source, full packet capturing, indexing, and database system.

Overview:
Since the last MolochON (https://molo.ch/on), many new features have been added to Moloch. We will review some of these features and demo how to use them. We will also discuss a few desired upcoming features.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Moloch: Recent Changes & Upcoming Features - Andy Wick, Sr Principal Architect, Oath & Elyse Rinne, Software Dev Engineer, Oath

  1. 1. Recent Changes
  2. 2. Some of the big changes this year ● Moloch 1.0 ● Capture stability ● Full IPv6 support ● ES 6 support ● Parliament Alerting ● Packet Search
  3. 3. Moloch 1.0 ● Previously field names were terrible, new names are so beautiful ● Unfortunately required a painful reindexing ● Removed all analyzed fields ○ We’ve gotten feedback this is bad, planning to add back for Moloch 2.0 ● ES 5 & ES 6 Support ● Switch to the new Maxmind API and 2 character country codes
  4. 4. Capture ● Many new classifiers: dhcp, dhcpv6, splunk, isakmp, ntp, ... ● OUI lookups ● Can reload oui, geo, rules without restarting ● Can decode many new VPNs ● Suricata plugin ● Autogenerated ES Ids
  5. 5. Capture Stability ● Require gnu99 compiler now ● 1.5/1.6 have numerous stability fixes ● Sanitize ○ New option for clang/gcc ○ Memory, integer overflow, and other checks ○ Runs on every commit now ○ Working on running in lab and production setting ● Cppcheck ○ Static analysis ○ Working to integrate into build system
  6. 6. Suricata Plugin ● Reads eve.json or alerts.json from disk ● Able to enrich moloch sessions since Suricata writes right away, and moloch is delayed ● Not a Suricata UI ● Only works when Moloch can read the files as they are written
  7. 7. Suricata Screenshot
  8. 8. Wise ● Handle multiple WISE servers better ● Support any field ● Splunk data source ● Easier to create views/sources ● Support more than 255 fields
  9. 9. Viewer ● Angular to Vue.js (performance improvements) ● Stats pages for Indices, Tasks, and Shards! ● Packet Search ● Shared Views ● Keyboard shortcuts
  10. 10. DEMO
  11. 11. Upcoming Changes
  12. 12. Building/Releases ● Last year had 4 build systems! ● Currently 3 build systems: ○ Vagrant - Releases ○ Vagrant - Nightly (Will be removed Dec 1st) ○ Screwdriver - builds on commits and pull requests ● Move to screwdriver for all builds ● Use bintray for ppa/repos
  13. 13. Moloch 2.0 - Ideas ● ES 6.x required ● Add field analyzers back ● New visualizations ○ Connections tab rewrite ○ Flow view ● Viewer/Multiviewer merge - Selectable clusters to search ● New Parsers: SIP, IMAP, ... ● Users “rethink” and Parliament ● History of Observed Data Indicators ● Tshark json view
  14. 14. Open source hygiene ● Adding a Contributor License Agreement (CLA) to github commits ● Adding a Code of Conduct to the github project ● Encourage code contributors from outside of Oath ● Goal of adding an external main committer ● Encourage github issues, feature requests, pull requests, wiki additions/revisions
  15. 15. PARLIAMENT
  16. 16. QUESTIONS?

×