The document discusses open source Security Information and Event Management (SIEM) systems, highlighting their capabilities such as data aggregation, correlation, alerting, and compliance monitoring. It focuses on the ELK Stack (Elasticsearch, Logstash, Kibana), detailing its components and functionalities in handling logs and threat detection. Additionally, it covers hardware requirements for scaling and setup considerations for effectively utilizing SIEM solutions.

1. 1 Open Source SIEM in 2017By Clever Net Systems

2. 2 Open Source SIEMWhat is SIEM ?
SIEM
=
Security Information and Event Management
=
SIM (security information management /
long-term log management)
+
SEM (security event management / real-time
monitoring)

3. 3 Open Source SIEMCapabilities of SIEM
Data aggregation: exhaustive, comprehensive and consolidated centralization of logs
Correlation: event linking through common attributes in order to extract meaning from raw data
Alerting: automatic analysis of correlated data or raw events turned into alerts
Dashboards: centralized high-level overview of data
Compliance: automatic gathering of compliance data, reporting on level of compliance
Retention: retention of data due to compliance requirements and/or for long term analysis
Forensic analysis: study of what happened

4. 4 Open Source SIEMWhich events do we correlate ?
Logs
• Syslogs / Windows WMI event logs / Network and firewall logs
• Application & DB logs
Scan results
• File integrity checking
• Registry keys integrity checking (Windows)
• Signature based malware / rootkits detection
• Antivirus software logs
Behavioral monitoring
• Netflow, Ntop, Nagios, Centreon, etc.
• Application behaviour (multiple logins, etc...)
Threat detection
• HIDS & NIDS
• Needs threat DB (Snort, Suricata, OSSEC, etc.)
• Signature & Anomaly based
Vulnerability assessment
• OpenVAS, Metasploit, Aircrack, Nessus, etc.
• Compliance scanners (PCI-DSS, CIS, etc.)

5. 5 Open Source SIEMVery incomplete OSS & proprietary vendor landscape

6. 6 The ELK stackData centralization and correlation
Logstash Elasticsearch Kibana
Beats
Ingest,
transform and
stash
Visualize and
navigate data
Distributed,
RESTful search
and analytics
engine
Lightweight
data shipper
https://www.elastic.co/guide/en/logstash/current/input-plugins.html

7. 7 The ELK stackElastic components
Open Source (free to use)
• Logstash (collector / transformer)
• Elasticsearch (full-text indexing)
• Kibana (analysis interface)
• Beats (data shipper)
(previously known as logstash-forwarder)
Proprietary plugins (X-Pack)
• Security (prev. Shield) - access protection
• Alerting (prev. Watcher)
• Monitoring (prev. Marvel)
• Reporting
• Graph
• Machine learning
Costs
• By JVM, not by daily data quantity (Splunk)
• Yearly
• Two different levels
• Need three licences for a cluster
• Licences comes with engineering & support

8. 8 The ELK stackParse Apache access logs with Logstash

9. 9 The ELK stackParse Apache access logs with Logstash
Original logs
178.194.37.205 - - [10/Feb/2017:16:00:12 +0100] "POST /wp-admin/admin-ajax.php HTTP/1.1" 200 102
"https://www.clevernetsystems.com/wp-admin/post.php?post=5674&action=edit" "Mozilla/5.0 (X11; Fedora; Linux x86_64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
54.205.244.176 - - [10/Feb/2017:16:00:23 +0100] "GET /monitoring-mysql-replication-with-munin/feed/ HTTP/1.1" 200 887
"http://www.google.com" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43
Safari/537.31"
108.61.68.156 - - [10/Feb/2017:16:00:25 +0100] "GET /installing-rhel-packages-without-network-connection/ HTTP/1.1" 200
14379 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71
Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /recruitment/ HTTP/1.1" 200 9093 "-" "Mozilla/5.0 (Windows NT 6.1)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:50 +0100] "GET /wp-content/themes/enfold/css/grid.css?ver=2 HTTP/1.1" 200 2050
"https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/css/base.css?ver=2 HTTP/1.1" 200 3990
"https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/41.0.2228.0 Safari/537.36"
198.27.68.101 - - [10/Feb/2017:16:00:51 +0100] "GET /wp-content/themes/enfold/js/aviapopup/magnific-popup.css?ver=1
HTTP/1.1" 200 1914 "https://www.clevernetsystems.com/recruitment/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"

10. 10 The ELK stackParse Apache access logs with Logstash
Parsed logs

11. 11 The ELK stackDemo
i ELK demo
20 minutes
Technologies :

12. 12 The ELK stackClustering & scalability
Initial empty state
First index creation
Additional replication node

13. 13 The ELK stackClustering & scalability
Horizontal scaling – shard reallocation
number_of_replicas = 2

14. 14 The ELK stackSizing
Sizing requirements for 100GB / day of raw data
It’s impossible to estimate the hardware and disk requirements.
A large number of factors come into play.
These numbers will turn out to be completely false.
• 4 nodes (3 ES nodes + 1 Logstash / Kibana node)
• 8 cores per node + 64GB per node (32GB for the JVM, 32GB for the system)
• Virtual or physical nodes
• SSD disks preferably
• Only local storage (local to the node, or local to the hypervisor, no SAN!)
• Disk space requirements vary depending on amount of daily data and retention policy
• Multiply disk space requirements by 1.5 with regards to raw data
• Multiply by number_of_replicas
Ex: 100GB / day and 3 months retention with 2 replicas = 27TB

15. 15 WazuhWazuh (OSSEC + ELK) as an OSS SIEM solution

16. 16 WazuhDemo
i Wazuh demo
10 minutes
Technologies :