2. 15+ Years Information Security
10+ Years Financial Services
2 Years Director Security Operations
Bill Ebel
3. Discuss My Experience:
- Coming to the decision to change SIEM
- Why we opted for Splunk with ES vs just core
Splunk
- How we did it?
Provide some pointers for a successful
deployment.
Agenda
5. 5
Overall a pain in the …
Noise Machine
Too Difficult
Compliance Checkbox
Ignored
Too Expensive
6. Common SIEM Features and Services
• Data aggregation
• Event Correlation
• Automated Alerting
• Dashboards
• Compliance Reporting
• Log Retention
• Asset Discovery
SIEM
7. Gartner:
Pitfall 1: Failure to Plan Before Buying
Pitfall 2: Failure to Define Scope
Pitfall 3: Overly Optimistic Scoping
Pitfall 4: Monitoring Noise
Pitfall 5: Lack of Sufficient Context
Pitfall 6: Insufficient Resources
Deployment Challenges
8. My 5 year old SIEM (in leaders quadrant)
• Difficult to Maintain and Support
• Extremely Slow Search
• Poor Vendor Support
• No Active User Community
• Not Used Outside of Information Security
• Poor Development Capabilities
• End-of-Life
Ended up being centralized log management
Loathed by all that used it
My Old SIEM
9. Need access to more log sources
High performance searching
More advanced searching and analysis
Must support very high volume of data
• Firewall > 3.5K e/s.
• Web Logs> 1.6K e/s
• Proxy > 1.6K e/s
Integration/enrichment with threat intel data
Need a more agile/nimble integration and
development
New Requirements
10. IT Operations already supported and
maintained a large Splunk deployment.
• Several important logs were already being
indexed and normalized.
• High performance deployment
• Being broadly adopted by other teams
Decision was made to partner with IT
Operations to test the waters
We Already Had Splunk
11. Keep existing SIEM running
Setup a new search head for Information Security
Configure a few devices to ship logs to Splunk
• Firewall
• Proxy
Normalize Events (CIM)
Install Some Useful Apps for the event sources
Provide Analysts basic Splunk training
Let them go to town
Testing The Waters
12. Analysts much happier with performance.
• Example Simple proxy search over 30 days for a user
• Old SIEM: 2 Days
• Splunk: >10 min
• More than 1 user could search at a time.
Analysts could easily correlate events across different
sources
Analysts could easily enrich events with external
lookups and intelligence data
Results
14. Moved sources one at a time.
• Focused on CIM compatibility (normalization)
• Brought in Splunk Apps to give Analysts baseline
searches and dashboards (as available)
• Corrected issues with apps and add-ons
• Corrected issues with varying log formats
• Open source to Analysts.
Core Splunk was our SIEM
• Immediate value
• Analyst write their own searches and alerts
• We had a ton more work
Moved all logs to Splunk
15. Enough knowledge to be dangerous
Not enough knowledge to be smart
Thoughts on ES:
• Why pay for this?
• We can write our own searches
• We already have several alerts based on our own
correlations
• We can make our own dashboards
Build or Buy
16. Migrated most SIEM log sources to Splunk
Engaged Splunk Professional Services to perform a
“ES Readiness Review”
Scope (2 weeks):
• Evaluate available sources for CIM compliance
• Correct any issues with CIM compliance and log inputs
• Install Splunk Enterprise Security and light it up
• Enable a few basic correlations.
1 Month to let us play
Enterprise Security POC
17.
18. ES was much more than core Splunk:
• Seeing how searches were created and all of the
different dashboards opened my eyes.
• ES provides a good template for how it should be done.
• Wouldn’t have used accelerated data models in
correlation searches.
• Alerts would have went into email and may not have
been tracked (i.e. notable)
• Would have taken months to recreate
• Existing correlations provide a good starting point for
creating more customized and advanced rules
Ignorance is Bliss
19. Incident Review is critical for a SOC
• Provides single pane of glass for all items that must
be reviewed by SOC
• Correlation searches are written to a notable event
• Notable events can be assigned workflow stages.
Provides a solid foundation for Incident Review
workflow.
Allows me to better manage what is open and who is
working it.
Incident Review
20. Ability to enrich log data with asset and
identity information.
• Simplifies a tracking down owners of a server
• Simplifies identifying managers, users etc
Event Investigator
• Quickly visualize all events for a user or system
Asset Management
21. Provide much faster searches across data
• Minimizes the time to detect issues
• Speeds up the time to cross reference out logs
with Threat Intelligence feeds
Data Models
23. 23
• Partner with other teams that may already have or could use Splunk
– More sources better for IOC hunting
– If possible have another team support infrastructure. Be a consumer of Splunk.
• Focus on CIM compliance (aka normalization) first with your sources
– Correct issues with logs first
– Prioritize highest value logs first
• Train analysts as early as possible
– Search, Pivot, Data Model
• Start small with use cases (correlations)
– Turn them all off get
– Turn one on high value correlation and tune
– Build a response plan for the notable
– Start staff following procedure
– Rinse and repeat with another high value correlation
• Take it easy with the threat feeds
– Turn on what you trust
– Assign confidence to your feeds
– Consider looking backwards not forward for IOC activity
– Consider Third Party Threat Intel Platform
• Data models are your friend
– Leverage these for IOC searches or search across large sets of logs
– Pivot is an easy way for analyst to work with data
– Consider changing default constraints to speed up build and pivot