Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

The Elastic Stack as a SIEM Slide 1 The Elastic Stack as a SIEM Slide 2 The Elastic Stack as a SIEM Slide 3 The Elastic Stack as a SIEM Slide 4 The Elastic Stack as a SIEM Slide 5 The Elastic Stack as a SIEM Slide 6 The Elastic Stack as a SIEM Slide 7 The Elastic Stack as a SIEM Slide 8 The Elastic Stack as a SIEM Slide 9 The Elastic Stack as a SIEM Slide 10 The Elastic Stack as a SIEM Slide 11 The Elastic Stack as a SIEM Slide 12 The Elastic Stack as a SIEM Slide 13 The Elastic Stack as a SIEM Slide 14 The Elastic Stack as a SIEM Slide 15 The Elastic Stack as a SIEM Slide 16 The Elastic Stack as a SIEM Slide 17 The Elastic Stack as a SIEM Slide 18 The Elastic Stack as a SIEM Slide 19 The Elastic Stack as a SIEM Slide 20 The Elastic Stack as a SIEM Slide 21 The Elastic Stack as a SIEM Slide 22 The Elastic Stack as a SIEM Slide 23 The Elastic Stack as a SIEM Slide 24 The Elastic Stack as a SIEM Slide 25 The Elastic Stack as a SIEM Slide 26 The Elastic Stack as a SIEM Slide 27 The Elastic Stack as a SIEM Slide 28 The Elastic Stack as a SIEM Slide 29 The Elastic Stack as a SIEM Slide 30 The Elastic Stack as a SIEM Slide 31 The Elastic Stack as a SIEM Slide 32 The Elastic Stack as a SIEM Slide 33 The Elastic Stack as a SIEM Slide 34 The Elastic Stack as a SIEM Slide 35 The Elastic Stack as a SIEM Slide 36 The Elastic Stack as a SIEM Slide 37 The Elastic Stack as a SIEM Slide 38 The Elastic Stack as a SIEM Slide 39 The Elastic Stack as a SIEM Slide 40 The Elastic Stack as a SIEM Slide 41 The Elastic Stack as a SIEM Slide 42 The Elastic Stack as a SIEM Slide 43 The Elastic Stack as a SIEM Slide 44 The Elastic Stack as a SIEM Slide 45 The Elastic Stack as a SIEM Slide 46 The Elastic Stack as a SIEM Slide 47 The Elastic Stack as a SIEM Slide 48 The Elastic Stack as a SIEM Slide 49 The Elastic Stack as a SIEM Slide 50 The Elastic Stack as a SIEM Slide 51 The Elastic Stack as a SIEM Slide 52 The Elastic Stack as a SIEM Slide 53 The Elastic Stack as a SIEM Slide 54 The Elastic Stack as a SIEM Slide 55 The Elastic Stack as a SIEM Slide 56 The Elastic Stack as a SIEM Slide 57 The Elastic Stack as a SIEM Slide 58 The Elastic Stack as a SIEM Slide 59 The Elastic Stack as a SIEM Slide 60 The Elastic Stack as a SIEM Slide 61 The Elastic Stack as a SIEM Slide 62 The Elastic Stack as a SIEM Slide 63 The Elastic Stack as a SIEM Slide 64 The Elastic Stack as a SIEM Slide 65 The Elastic Stack as a SIEM Slide 66 The Elastic Stack as a SIEM Slide 67 The Elastic Stack as a SIEM Slide 68 The Elastic Stack as a SIEM Slide 69 The Elastic Stack as a SIEM Slide 70 The Elastic Stack as a SIEM Slide 71
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

8 Likes

Share

Download to read offline

The Elastic Stack as a SIEM

Download to read offline

Video: https://www.youtube.com/watch?v=v69kyU5XMFI

A talk I gave at the Philly Security Shell meetup 2019-02-21 on how the Elastic Stack works and how you can use it for indexing and searching security logs. Tools I mentioned: Github repo with script and demo data - https://github.com/SecHubb/SecShell_Demo Cerebro - https://github.com/lmenezes/cerebro Elastalert - https://github.com/Yelp/elastalert For info on my SANS teaching schedule visit: https://www.sans.org/instructors/john... Twitter: https://twitter.com/SecHubb

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

The Elastic Stack as a SIEM

  1. 1. The Elastic Stack as a SIEM Philly Security Shell 2019
  2. 2. Who Am I? John Hubbard [@SecHubb] • Previous SOC Lead @ GlaxoSmithKline • Certified SANS Instructor • Author • SEC450: Blue Team Fundamentals – Security Analysis and Operations • SEC455: SIEM Design & Implementation (Elasticsearch as a SIEM) • Instructor • SEC511: Continuous Monitoring & Security Operations • SEC555: SIEM with Tactical Analytics • Mission: Make life awesome for the blue team • Data for this talk: https://github.com/SecHubb/SecShell_Demo
  3. 3. What is a SIEM? • A central log repository that enriches logs and assists threat detection • Components • Log Sources • Log Aggregator • Log Storage & Indexing • Search & Viz. Interface + Alerting Engine Log Sources Log Aggregation / Queue Log Storage & Indexing Search, Visualization, & Alerting John Hubbard [@SecHubb] 3
  4. 4. What is the Elastic Stack? • Open source, real-time search and analytics engine • Made up of 4 pieces: collection, ingestion, storage, and visualization John Hubbard [@SecHubb] 4
  5. 5. History of Elastic 2010 Created by Shay Bannon Recipe search engine for his wife in culinary school Inspired by Minority Report 2012 Elastic Co. Founded 2019 Used by Wikipedia, Stack Overflow, GitHub, Netflix, LinkedIn, … One of the most popular projects on GitHub Iterating versions rapidly with awesome new features John Hubbard [@SecHubb] 5
  6. 6. Elastic Stack vs. SIEM Log Sources Log Aggregation / Queue Log Storage & Indexing Search, Visualization, & Alerting John Hubbard [@SecHubb] 6
  7. 7. Elastic stack as a SIEM Used for many different use cases • NOT a SIEM out of the box • Not in the magic quadrant as one • Can do the things a SIEM does Gartner's definition of a SIEM: "supports threat detection and security incident response through the real- time collection and historical analysis of security events from a wide variety of event and contextual data sources. It also supports compliance reporting and incident investigation through analysis of historical data from these sources." John Hubbard [@SecHubb] 7
  8. 8. Elasticsearch as a SIEM • Collects, indexes, and stores high volumes of logs • Functional visualizations and dashboards • Reporting and alerting • Log enrichment through plugins • Compatible with almost every format • Log retention settings • Anomaly detection via machine learning • RBAC securable John Hubbard [@SecHubb] 8
  9. 9. Elastic Stack Overview Raw Logs Raw Logs Log Ingestion & Parsing Log Storage Search & Visualization John Hubbard [@SecHubb] 9
  10. 10. Winlogbeat Beats Agents Lightweight log agents written in Go • Filebeat • Winlogbeat • Packetbeat • Auditbeat • Functionbeat • Journalbeat • Community Beats FilebeatPacketbeat John Hubbard [@SecHubb] 10
  11. 11. Elasticsearch Architecture John Hubbard [@SecHubb] 11
  12. 12. Clusters, Nodes, and Indices Cluster Node Indices John Hubbard [@SecHubb] 12
  13. 13. Index Creation Across Time Firewall-2018-01 Firewall-2018-02 Firewall-2018-03 IDS-2018-01 IDS-2018-02 IDS-2018-03 John Hubbard [@SecHubb] 13
  14. 14. Shards and Documents Index Shards Documents John Hubbard [@SecHubb] 14
  15. 15. John Hubbard [@SecHubb] 15
  16. 16. Reason 1: Schema on Ingest Many SIEMs: Schema applied at search time Elasticsearch: Schema applied at ingestion John Hubbard [@SecHubb] 16
  17. 17. Reason 2: Data is distributed Index Shards Nodes John Hubbard [@SecHubb] 17
  18. 18. Shard Types Primary Shards • Like RAID 0 – Need all shards to make the whole index Replica Shards • Like RAID 1 • Each primary shard has arbitrary number of copies • Each copy can be polled to balance search load John Hubbard [@SecHubb] 18
  19. 19. Shards • All shards belong to and make up an index • Enables arbitrary horizontal scaling • Spread evenly across all available hardware • Designated a Primary or Replica Primary Shard 1 Primary Shard 2 Primary Shard 3 Replica Shard 1 Replica Shard 2 Replica Shard 3 Full Index Data John Hubbard [@SecHubb] 19
  20. 20. Primaries and Replicas Copy 2 Shards Nodes P0 P1 R0 R1 Copy 1 John Hubbard [@SecHubb] 20
  21. 21. Primaries and Replicas Copy 2 Shards Nodes P0 P1 R0 R1 Copy 1 Copy 3 R0 R1 John Hubbard [@SecHubb] 21
  22. 22. Balancing Writes Incoming Logs Shards Nodes P0 P1 P2 P3 P4 P5 John Hubbard [@SecHubb] 22
  23. 23. Balancing Searches Search Requests Shards Nodes P0 R0 R0 R0 R0 R0 John Hubbard [@SecHubb] 23
  24. 24. Balancing Searches: multi-shard Search 2 Shards Nodes P0 P1 R0 R1 Search 1 Search 3 R0 R1 John Hubbard [@SecHubb] 24
  25. 25. Documents to Fields Document Single Log (Converted to JSON by Logstash) Fields John Hubbard [@SecHubb] 25
  26. 26. Documents • Indices hold documents in serialized JSON objects • 1 document = 1 log entry • Contains "field : value" pairs • Metadata • _index – Index the document belongs to • _id – unique ID for that log • _source – parsed log fields
  27. 27. Fields and Mappings • Field – A key-value pair inside a document • username: admin • hostname: web-server1 • Mapping - Defines information about the fields • Think "database schema" • The data type for each field (integer, ip, keyword, etc.) John Hubbard [@SecHubb] 27
  28. 28. Key Concept: Keyword vs. Text String datatypes are either text or keyword, or both! • Keyword indexes the exact values • Example: Usernames, ID numbers, tags, FQDNs • Binary search results – full exact matches, or not • Text type breaks things up into pieces • Example: "http://www.mywebmail.com/mailbox/mail1.htm" • Allows searching for "http", "www.mywebmail.com", "mailbox", "mail1.htm" • Fed through an "analyzer" • This data type cannot be aggregated / visualized John Hubbard [@SecHubb] 28
  29. 29. http://www.mywebmail.com/mailbox/mail1.htm Text Data Type Example Character Filter http www.mywebmail.com mailbox mail1.htm Tokenizer http www.mywebmail.com mailbox mail1.htm Tokens can be searched for
  30. 30. Where Tokens Go: Inverted Index Lucene builds "inverted index" of tokens in text field data Doc 1: "The woman is walking down the street." Doc 2: "The man is walking into the store." Tokens Doc 1 Doc 2 the x x woman x is x x walking x x down x street x man x into x store x John Hubbard [@SecHubb] 30
  31. 31. Elasticsearch instance Elasticsearch Term Summary Shard Lucene Cluster = Multiple Nodes Segment Segment Segment Shard Lucene Segment Segment Segment Index Shard Lucene Segment Segment Segment Shard Lucene Segment Segment Segment Index Node Holds one log type Partial index Search engine "Inverted index"
  32. 32. Kibana John Hubbard [@SecHubb] 32
  33. 33. Kibana Interface • Discover - Search and explore data • Visualize - Create graphs and charts • Dashboard – Display a collection of saved items • Timelion – Unique time series data visualization • Canvas – New visualization type • Machine Learning – Ponies and magic • Infrastructure – Monitor all Metricbeats • Logs – Watch logs streaming from Filebeat • Dev Tools – Console for API access • Monitoring – Health of your cluster/agents/logstash • Management – Manage the cluster
  34. 34. Using the Discover Tab Histogram Document data Field list Index pattern Time filter
  35. 35. Discover Tab Details Field must exist Add as column Filter out this field value Filter for this field value Data type Move left/right Remove this column Sort by this column Show document
  36. 36. Index Patterns • Kibana must be told to show an index for searching • Searching can be performed on more than 1 index at once Example usage: • "*" - Search ALL indices • "firewall-*" • "firewall-pfsense-*" • "firewall-pfsense-2019-*" • "alexa-top1M" John Hubbard [@SecHubb] 36
  37. 37. Visualization Types
  38. 38. Creating Visualizations • Metrics: What to calculate • Buckets: How to group it "I want to see <metric> per <bucket>" • "Total bytes" • "Total bytes per username" • "Request count, bytes per HTTP method" • "Requests per user per site" John Hubbard [@SecHubb] 38
  39. 39. Bucket Options • Date Histogram (time) • Date Range • Filters • Histogram • IPv4 Range • Range • Significant Terms • Terms (log fields) John Hubbard [@SecHubb] 39
  40. 40. Visualization Demo John Hubbard [@SecHubb] 40
  41. 41. Default Elasticsearch Security Elasticsearch is completely open by default John Hubbard [@SecHubb] 41
  42. 42. Options for Security •N00b mode: nginx reverse proxy with basic auth •Better: •Best: John Hubbard [@SecHubb] 42
  43. 43. Logstash John Hubbard [@SecHubb] 43
  44. 44. Logstash • Free, developed and maintained by Elastic • Integrates with Beats • Integrates with Elasticsearch • Tons of plugins • Easy to learn and use • Built-in buffering • Back-pressure support
  45. 45. Logstash – Ingestion Workhorse Syslog TCP UDP Other
  46. 46. Routing to Logstash Logstash01 Logstash02 Logstash03 Load Balancer
  47. 47. Input -> Filter -> Output Logstash has 3 components: • Input - Methods to listen for and accept logs • Filter - Filters, parses, and enriches logs • Output - Sends logs to another system or program Input plugins Filter plugins Output plugins Logstash Pipeline Log source Log destination
  48. 48. Logstash Config Files John Hubbard [@SecHubb] 48 For our premade configs, see: https://github.com/HASecuritySolutions/Logstash
  49. 49. Data Ingestion Demo John Hubbard [@SecHubb] 49
  50. 50. Input Plugins • Input receives logs in multiple formats • Key plugins: • Common options – beats, syslog, file, http, tcp, udp, elasticsearch • Database – jdbc, sqlite • Message Brokers – kafka, redis, rabbitmq Input plugins Filter plugins Output plugins
  51. 51. Filter Plugins • Filter section parses, filters, and enriches logs • Key plugins: • Parsing - csv, grok, kv, json, syslog_pri, xml, date • Log filtering - drop • Enrichment - dns, elasticsearch, geoip, mutate, rest, oui, useragent, tld, and ruby Input plugins Filter plugins Output plugins
  52. 52. Output Plugins • Output steers parsed logs to multiple destinations • Key plugins: • elasticsearch – For storage • stdout – for debugging and development • 3rd party applications - email, irc, csv, kafka, rabbitmq, graphite, google_cloud_storage, jira, nagios, pagerduty, sns, tcp/udp Input plugins Filter plugins Output plugins
  53. 53. Traditional Logging - Syslog <81>Feb 21 14:43:13 logparse sudo: jhubbard : 1 incorrect password attempt ; TTY=pts/1 ; PWD=/var/log ; USER=root ; COMMAND=/bin/su • PRI = <81> Time/date = Jan 4 14:43:13 Source host = logparse Source process = sudo Message = jhubbard : 1 incorrect password attempt ; TTY=pts/1 ; PWD=/var/log ; USER=root ; COMMAND=/bin/su
  54. 54. The Problems With Syslog • Unstructured syslog is the worst • Wrong regex? No parsing • No pre-made regex? No parsing • Poor regex? Poor performance = Low EPS • Unparsed logs means your analytics don't work! • Grok plugin in Logstash eases pain of writing statements • Gives pre-made regexs a name • Use the name, statement becomes readable and dependable • Ideally new log formats should be used when available
  55. 55. Log Standardization Better log formats are becoming more prevalent • Comma Separated Values (CSV) • Key-Value pairs (KV) • JavaScript Object Notation (JSON) Logstash has plugins for these log formats • csv, kv, and json
  56. 56. csv - Filter Plugin Delimited values can be automatically extracted csv { columns => ["src_ip","src_port","dst_ip", "method","virtual_host","uri"] } "10.4.55.1","50001","8.8.8.8","GET" ,"sec455.com","/page.php"
  57. 57. kv - Filter Plugin Syslog is still the most common transport method • Syslog message portion is not standardized • Standardization inside syslog message is becoming more common Example: Firewall log message uses key : value pairs kv { value_split => "=" field_split => " " } Example log message: src_ip=10.0.01 src_port=50001 dst_ip=8.8.8.8 dst_port=53 policyid=17 action=allow
  58. 58. kv + Logstash: Easing syslog pain <81>Jan 4 14:43:13 logparse sudo: jhubbard : 1 incorrect password attempt ; TTY=pts/1 ; PWD=/var/log ; USER=root ; COMMAND=/bin/su Applying Logstash config: input { syslog {} } filter { kv {} } "severity" => 1, "syslog_severity_code" => 5, "syslog_facility" => "user-level", "syslog_facility_code" => 1, "program" => "sudo", "message" => "jhubbard : 1 incorrect password attempt ; TTY=pts/1 ; PWD=/var/log ; USER=root ; COMMAND=/bin/sun", "priority" => 81, "logsource" => "logparse", "USER" => "root", "syslog_severity" => "notice", "@timestamp" => 2017-01-04T19:43:13.000Z, "TTY" => "pts/1", "COMMAND" => "/bin/sun", "PWD" => "/var/log", "facility" => 10, "severity_label" => "Alert", "facility_label" => "security/authorization"
  59. 59. json - Filter Plugin The easiest…the json plugin json { source => "message"} } That's all! Windows logs have lots of fields, let JSON handle it!
  60. 60. Full Elastic Stack In a Nutshell 1. Send things to Logstash via agents or forwarding 2. Parse them in whatever way you want 3. Send them to Elasticsearch for storage 4. Query Elasticsearch via Kibana John Hubbard [@SecHubb] 60
  61. 61. Default Ports :9200 :5601 :9300 HTTP HTTP :5044
  62. 62. Dual Stack SIEM John Hubbard [@SecHubb] 62
  63. 63. Logstash to Multiple SIEMs Logs Commercial SIEM Elasticsearch
  64. 64. Logstash Log Pulling Commercial SIEM Elasticsearch Logs Pull
  65. 65. Message Broker to SIEM Logs Commercial SIEM Elasticsearch Log Agent
  66. 66. The Full Layout John Hubbard [@SecHubb] 66 https://www.elastic.co/assets/blt2614227bb99b9878/architecture-best-practices.pdf
  67. 67. Hardware Backup Slides John Hubbard [@SecHubb] 67
  68. 68. CPU and Memory • How much CPU and memory are required? Memory will run out first • Use as much as possible • 8GB+ per node • 64GB = sweet spot (Java limitations) • <=31GB dedicated to Java max • /etc/elasticsearch/jvm.options file CPU – multi-core/node, 64bit • More cores better than faster speed Heap OS / Lucene Node RAM <=31GB John Hubbard [@SecHubb] 68 All other RAM See: https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
  69. 69. Networking • You can never have too much bandwidth! • Moving 50GB shards node to node • Returning large query results • Restoring from backup • Network Setup: • 1GB is required • 10GB is better! • Minimize latency • Jumbo frames enabled John Hubbard [@SecHubb] 69
  70. 70. Hard Drives • Disk speed for logging clusters is VERY important • Lots of hard drives for high IO, not one big one • RAID0 setup, replica shards take care of availability John Hubbard [@SecHubb] 70
  71. 71. Thanks! John Hubbard [@SecHubb] 71
  • SeidDegu1

    Sep. 14, 2021
  • bamaemmanuel

    Jan. 13, 2021
  • WasiqRehman

    Jul. 9, 2019
  • AshokLokhande2

    Jul. 9, 2019
  • hteixe

    Jun. 19, 2019
  • spacemonkey3e1

    May. 12, 2019
  • Nikmidu

    Apr. 15, 2019
  • cheehe

    Mar. 13, 2019

Video: https://www.youtube.com/watch?v=v69kyU5XMFI A talk I gave at the Philly Security Shell meetup 2019-02-21 on how the Elastic Stack works and how you can use it for indexing and searching security logs. Tools I mentioned: Github repo with script and demo data - https://github.com/SecHubb/SecShell_Demo Cerebro - https://github.com/lmenezes/cerebro Elastalert - https://github.com/Yelp/elastalert For info on my SANS teaching schedule visit: https://www.sans.org/instructors/john... Twitter: https://twitter.com/SecHubb

Views

Total views

2,196

On Slideshare

0

From embeds

0

Number of embeds

229

Actions

Downloads

164

Shares

0

Comments

0

Likes

8

×