This document provides an overview and demonstration of using open source tools for security information and event management (SIEM). It begins with an introduction to SIEM and the ELK stack (Elasticsearch, Logstash, Kibana) for data aggregation, correlation, alerting and dashboards. The document demonstrates using Logstash to parse Apache logs and load them into Elasticsearch. It also discusses clustering and sizing requirements. Finally, it introduces Wazuh as an open source SIEM solution built on OSSEC and the ELK stack.

1. 1 Open Source SIEM in 2017Geneva Open Source Meetup 20170629 – Café Voisins
Jérôme Steunenberg
Clément Hampaï
Société romande spécialisée dans les solutions
d'infrastructure, de développement web et
logiciels sur mesure et de data intelligence
https://www.meetup.com/fr-
FR/Geneve-Open-Source-Meetup/
https://www.meetup.com/fr-
FR/Lausanne-Open-Source-Meetup/
Merci Café Voisins!

2. 2 ProgrammeGeneva Open Source Meetup 20170629 – Café Voisins
18h30 : Accueil des participants
19h : Présentation ELK/SIEM/Wazuh
20h15 : Q&A
20h30 : Buvons un verre !

3. 3 Open Source SIEM in 2017By Clever Net Systems

4. 4 Open Source SIEMWhat is SIEM ?
SIEM
=
Security Information and Event Management
=
SIM (security information management /
long-term log management)
+
SEM (security event management / real-time
monitoring)

5. 5 Open Source SIEMCapabilities of SIEM
Data aggregation: exhaustive, comprehensive and consolidated centralization of logs
Correlation: event linking through common attributes in order to extract meaning from raw data
Alerting: automatic analysis of correlated data or raw events turned into alerts
Dashboards: centralized high-level overview of data
Compliance: automatic gathering of compliance data, reporting on level of compliance
Retention: retention of data due to compliance requirements and/or for long term analysis
Forensic analysis: study of what happened

6. 6 Open Source SIEMWhich events do we correlate ?
Logs
• Syslogs / Windows WMI event logs / Network and firewall logs
• Application & DB logs
Scan results
• File integrity checking
• Registry keys integrity checking (Windows)
• Signature based malware / rootkits detection
• Antivirus software logs
Behavioral monitoring
• Netflow, Ntop, Nagios, Centreon, etc.
• Application behaviour (multiple logins, etc...)
Threat detection
• HIDS & NIDS
• Needs threat DB (Snort, Suricata, OSSEC, etc.)
• Signature & Anomaly based
Vulnerability assessment
• OpenVAS, Metasploit, Aircrack, Nessus, etc.
• Compliance scanners (PCI-DSS, CIS, etc.)

7. 7 Open Source SIEMVery incomplete OSS & proprietary vendor landscape

8. 8 The ELK stackData centralization and correlation
Logstash Elasticsearch Kibana
Beats
Ingest,
transform and
stash
Visualize and
navigate data
Distributed,
RESTful search
and analytics
engine
Lightweight
data shipper
https://www.elastic.co/guide/en/logstash/current/input-plugins.html

9. 9 The ELK stackElastic components
Open Source (free to use)
• Logstash (collector / transformer)
• Elasticsearch (full-text indexing)
• Kibana (analysis interface)
• Beats (data shipper)
(previously known as logstash-forwarder)
Proprietary plugins (X-Pack)
• Security (prev. Shield) - access protection
• Alerting (prev. Watcher)
• Monitoring (prev. Marvel)
• Reporting
• Graph
• Machine learning
Costs
• By JVM, not by daily data quantity (Splunk)
• Yearly
• Two different levels
• Need three licences for a cluster
• Licences comes with engineering & support

10. 10 The ELK stackParse Apache access logs with Logstash

11. 11 The ELK stackParse Apache access logs with Logstash
Original logs
178.194.37.205 - - [10/Feb/2017:16:00:12 +0100] "POST /wp-admin/admin-ajax.php HTTP/1.1" 200 102
"https://www.clevernetsystems.com/wp-admin/post.php?post=5674&action=edit" "Mozilla/5.0 (X11; Fedora; Linux x86_64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
54.205.244.176 - - [10/Feb/2017:16:00:23 +0100] "GET /monitoring-mysql-replication-with-munin/feed/ HTTP/1.1" 200 887
"http://www.google.com" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43
Safari/537.31"
108.61.68.156 - - [10/Feb/2017:16:00:25 +0100] "GET /installing-rhel-packages-without-network-connection/ HTTP/1.1" 200
14379 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71
Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /recruitment/ HTTP/1.1" 200 9093 "-" "Mozilla/5.0 (Windows NT 6.1)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /wp-content/themes/enfold/css/grid.css?ver=2 HTTP/1.1" 200 2050
"https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/css/base.css?ver=2 HTTP/1.1" 200 3990
"https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/js/aviapopup/magnific-popup.css?ver=1
HTTP/1.1" 200 1914 "https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"

12. 12 The ELK stackParse Apache access logs with Logstash
Parsed logs

13. 13 The ELK stackDemo
i ELK demo
20 minutes
Technologies :

14. 14 The ELK stackDemo architecture

15. 15 The ELK stackDemo architecture

16. 16 The ELK stackClustering & scalability
Initial empty state
First index creation
Additional replication node

17. 17 The ELK stackClustering & scalability
Horizontal scaling – shard reallocation
number_of_replicas = 2

18. 18 The ELK stackSizing
Sizing requirements for 100GB / day of raw data
It’s impossible to estimate the hardware and disk requirements.
A large number of factors come into play.
These numbers will turn out to be completely false.
• 4 nodes (3 ES nodes + 1 Logstash / Kibana node)
• 8 cores per node + 64GB per node (32GB for the JVM, 32GB for the system)
• Virtual or physical nodes
• SSD disks preferably
• Only local storage (local to the node, or local to the hypervisor, no SAN!)
• Disk space requirements vary depending on amount of daily data and retention policy
• Multiply disk space requirements by 1.5 with regards to raw data
• Multiply by number_of_replicas
Ex: 100GB / day and 3 months retention with 2 replicas = 27TB

19. 19 WazuhWazuh (OSSEC + ELK) as an OSS SIEM solution

20. 20 WazuhOSSEC architecture

21. 21 WazuhDemo
i Wazuh demo
15 minutes
Technologies :