Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Why Cybersecurity Needs Big
Data & Intro to Apache Metron
James Sirota, Director Security Solutions
March 2017
Michael Sch...
2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Michael Schiebel,
Cybersecurity Strategy
www.linkedin.com/in/michael...
3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Agenda
 Why Cybersecurity Needs Big Data
 Intro to Apache Metron
...
4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Why Cybersecurity
Needs Big Data
5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Digital World Generates Big Data That Security Teams Need to Process
6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Existing Cyber ...
7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Cybersecurity Journey
Single View into Security Risk
Free data from ...
8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Metron at Capital One
CapOne uses HDF to ingest log data into...
9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Introduction to Apache
Metron
10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Metron Journey
Jan 2016
OpenSOC
renamed
Metron
Dec 2016
Accepted in...
11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
DataServicesandIntegrationLayer
Search and
Dashboarding
Portal
Secu...
12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
How Big Data Experts
Can Help Security
Teams
13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
A Day in the Life of An Analyst:
• Too many disparate tools
• Too m...
14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Problem Posed (For Existing Tools)
Security
Information
Management
...
15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Problem Posed (For Bad Guys)
Advanced
Persistent
Threat
Script
Kidd...
16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Case Study: Accelerate
Investigation of a
Phishing Attack
17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
The “Threat Story” the Workflow Told….
18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
The Challenges faced by the SOC Analyst to Create this Story…
Chall...
19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Old School vs. New School Security Controls
Email
Security
Rules
Fi...
20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Apache Metron Resources
http://hortonworks.com/apache/metron/
https...
21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Questions?
Upcoming SlideShare
Loading in …5
×

Why Cybersecurity Needs Big Data

1,487 views

Published on

The connected world creates a rate and volume of streaming cybersecurity data that is unprecedented, and attacks are increasingly sophisticated and multifaceted. Yet it is unreasonably time-consuming for security personnel to piece together data from multiple systems to assess the true nature of a single threat across an enterprise.

Learn how big data and data science teams can help information security teams improve threat detection with machine learning and real-time streaming analytics. You will hear from, Michael Schiebel, cybersecurity strategist and James Sirota, Apache Metron committer and Director of Security Solutions at Hortonworks on how to apply big data technology to prevent cybercrime.

Leveraging big data and machine learning, Apache Metron can help detect phishing attacks such as the Yahoo security breach by Russian spies. See how Apache Metron accelerates the process of investigating a phishing attack – slides 16-19. You can also learn more about Apache Metron here: https://hortonworks.com/apache/metron/ and join the community at user-subscribe@metron.incubator.apache.org or dev-subscribe@metron.incubator.apache.org

View the on-demand webinar: https://hortonworks.com/webinar/why-cybersecurity-needs-big-data/

Published in: Technology
  • Be the first to comment

Why Cybersecurity Needs Big Data

  1. 1. Why Cybersecurity Needs Big Data & Intro to Apache Metron James Sirota, Director Security Solutions March 2017 Michael Schiebel, Cybersecurity Strategy
  2. 2. 2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Michael Schiebel, Cybersecurity Strategy www.linkedin.com/in/michaelschiebel/ James Sirota Director Security Solutions www.linkedin.com/in/jsirota/ Anna Yong Cybersecurity Product Marketing www.linkedin.com/in/4everfusion/
  3. 3. 3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Agenda  Why Cybersecurity Needs Big Data  Intro to Apache Metron  How big data experts can help IT security teams  Case Study: Accelerating Investigation of a Phishing Attack
  4. 4. 4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Why Cybersecurity Needs Big Data
  5. 5. 5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Digital World Generates Big Data That Security Teams Need to Process
  6. 6. 6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Existing Cyber Security Solutions Don’t Scale to the Challenge 82% of breaches happened in minutes 8 months: Average time an advanced security breach goes unnoticed 70%-80% of breaches are first detected by a 3rd party. 2016 Verizon Data Breach Investigations Report Current security tools installed in the data center can’t handle volume of data & threats from everywhere
  7. 7. 7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Cybersecurity Journey Single View into Security Risk Free data from security tools Correlate and discover threats Operational efficiency and governance Predictive insights using machine learning Single unified view of enterprise risk & security posture. Innovate Renovate Single Holistic View Historical Records OPEX Reduction Security Tool Ingest Digital Protection Fraud Prevention Public Data Capture ACTIVE ARCHIVE DATA DISCOVERY PREDICTIVE ANALYTICS Cyber Security Machine Data Risk Modeling
  8. 8. 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache Metron at Capital One CapOne uses HDF to ingest log data into their cyber security data lake and uses Apache Metron to detect threats that cannot be detected by traditional cyber security tools https://youtu.be/Nffx8SKn7l4?t=1h37m50s
  9. 9. 9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Introduction to Apache Metron
  10. 10. 10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Metron Journey Jan 2016 OpenSOC renamed Metron Dec 2016 Accepted into Apache Incubation Oct. 2015 Hortonworks, Mantech, B23 press release Sept 2014 First release of OpenSOC Beta by Cisco April 2014 OpenSOC in production June 2014 OpenSOC Community Edition July 2015 Cisco stops supporting OpenSOC March 2016 First Apache Release
  11. 11. 11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved DataServicesandIntegrationLayer Search and Dashboarding Portal Security Data Vault Community Analytical Models Provisioning, Management and Monitoring ModulesReal-time Processing Cyber Security Engine Telemetry Parsers Enrichment Threat Intel Alert Triage Indexers and Writers Cyber Security Stream Processing Pipeline Apache Metron: Incubating Project TelemetryIngestBuffer Telemetry Data Collectors Real-time Enrich / Threat Intel Streams Performance Network Ingest Probes / OtherMachine Generated Logs (AD, App / Web Server, firewall, VPN, etc.) Security Endpoint Devices (Fireye, Palo Alto, BlueCoat, etc.) Network Data (PCAP, Netflow, Bro, etc.) IDS (Suricata, Snort, etc.) Threat Intelligence Feeds (Soltra, OpenTaxi, third-party feeds) Telemetry Data Sources
  12. 12. 12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved How Big Data Experts Can Help Security Teams
  13. 13. 13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved A Day in the Life of An Analyst: • Too many disparate tools • Too many alerts to process • Too much noise • How to connect the dots of the relevant data points together?
  14. 14. 14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Problem Posed (For Existing Tools) Security Information Management System (SIEM) • I am prohibitively expensive • I have vendor lock-in • I can’t deal with big data • I am not open • I am not extensible enough Legacy Point Tools • I was built for 1995 • I am super specialized • I don’t scale horizontally • I have a proprietary format • You need a PhD to operate me Behavioral Analytics Tools (UEBA) • I have a limited # of models • I am not trained on YOUR data • I am built by a small startup
  15. 15. 15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Problem Posed (For Bad Guys) Advanced Persistent Threat Script Kiddie • I am very unique in a way I do things • I live on your network for about 300 days • I know what I am after and I look for it, slowly • Your rules will not detect me, I am too smart • I impersonate a legitimate user, but I don’t act like one Apache Metron can take everything that is known about me and check for it in real time Apache Metron can model historical behavior of whoever I am impersonating and flag me as I try to deviate • My techniques are predictable and known • My attack vectors are also known • I fumble around a lot • I set off a large number of alerts • You are not the only person I’ve attacked • I brag about what I did or will do Repeatable Patterns Unique Patterns
  16. 16. 16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Case Study: Accelerate Investigation of a Phishing Attack
  17. 17. 17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved The “Threat Story” the Workflow Told….
  18. 18. 18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved The Challenges faced by the SOC Analyst to Create this Story… Challenge • The analyst had to jump from the SIEM to more than 7 different tools that took up valuable time. • It took more than 24 hours across 2 SOC shifts to investigate, determine scope, remediate and do further forensics/investigation. • Half of my time was spending getting the context needed for me to create the story • The threat was detected too late. Instead of detecting the incident on 4/9, the threat should have been detected on 3/20 when the attacker spoofed Sonja’s email address Need • Want a Centralized View of my data so I don’t have to jump around and learn other tools Eliminate manual tasks to investigate a case • Need to discover bad stuff quicker • Need the System to create the context for me in real-time • The current static rules in the SIEM didn’t detect the threat. Need smart analytics based on: • User Sonja hasn’t used corp gmail in the last 3 months • User Sonja can’t login from Ireland and Southern Cali at the same time
  19. 19. 19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Old School vs. New School Security Controls Email Security Rules Firewall Rules IDS Rules Sandbox Rules DLP RulesOld School -> (1-1) New School -> (1-*) Email Classifier Alerts Triage Malware Family Classifier Network Behavior Classifier UEBA System
  20. 20. 20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache Metron Resources http://hortonworks.com/apache/metron/ https://metron.incubator.apache.org/ https://www.meetup.com/futureofdata- london/events/237165504/
  21. 21. 21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Questions?

×