Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Near Real-Time Outlier Detection and Interpretation

733 views

Published on

Near Real-Time Outlier Detection and Interpretation

Published in: Technology
  • Be the first to comment

Near Real-Time Outlier Detection and Interpretation

  1. 1. © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. June 28, 2016 Near Real-time Outlier Detection and Interpretation An Hadoop Based Approach Hadoop Summit 2016 Bob Thorman Principal – Technology Security AT&T Chief Security Organization
  2. 2. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. 2 Presentation Outline: Brief Context of the Problem of Cyber Threats in our industry Recent History of AT&T Cyber Threat Capabilities Hadoop Based Approach to Threat Analytics Platform Cyber Threat Detection and Interpretation Insider Threat
  3. 3. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. The Problem of Cyber Threats in Our Industry A Brief Context
  4. 4. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. 4 Network Scale • ~1M Authenticated users • ~800K user oriented devices • ~1100 security devices on the network (FW, IDS, etc.) • Approximately 5B network events per day – Firewall, Proxy, IDS, SIEM, etc. Facing Alarming Trends Bridging to the Internet • Next Slides The Problem of Cyber Threats in Our Industry
  5. 5. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. 5 Distributed Reflection DoS (DrDoS) Attack Evolution Attack activity trending up Oct 2013  1900/udp: SSDP  123/udp: NTP  19/udp: chargen  0/udp: packet fragmentation  53/udp: DNS (some legitimate)30 months shown
  6. 6. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. Recent History of AT&T Cyber Threat Protection Capabilities A Need for Big Data
  7. 7. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. History of AT&T Cyber Threat Protection Capabilities Chief Security Office – 2002 Program concept for millions of records per day – 2005 Program concept tens of millions of records per day – 2016 Big Data concept for tens of billions events/day – 2017 Big Data concepts for trillions events/day Major Big Data Development Milestone – 2008 Beginnings of Accumulo, an implementation of Google™ Bigtable – 2011 Accumulo open sourced to Apache Software Foundation – 2013 AT&T initiates Threat Analytics modernization project – 2014 AT&T initiates deployment of Hadoop-based Threat Analytics Platform Cyber Threat Protection Platform Architecture Evolution – Next slides 7
  8. 8. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. Threat Platform of Yesterday SIEM 8 Source/processing/analytics DBMS/SAN Query
  9. 9. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. Threat Detection and Interpretation Process 9 Architectural Component Ingestion Outlier Detection1 Spark Streaming Detectors1 R Analytics1 Web UI Dashboards Custom Alerting Framework1 Threat Operations 1Area of focus for automation
  10. 10. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. An Hadoop Based Approach to Threat Analytics Platform Securing AT&T with Hadoop
  11. 11. Detecting and Interpreting Cyber Threats at AT&T © 2016 AT&T Intellectual Property. All rights reserved. AT&T, Globe logo, Mobilizing Your World and DIRECTV are registered trademarks and service marks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks are the property of their respective owners. AT&T Proprietary (Internal Use Only). Not for use or disclosure outside the AT&T companies except under written agreement. Today’s Platform Details Using An Hadoop Based Platform for Log Management, Threat Analysis, Reporting AT&T approach to use of Hadoop in a Threat Analysis Platform SIEM Raw logs Events, Intelligence, Alarms, Threats Results, Reports, Analytics Source Processing Threat Analytics Platform UI/Visual/Report 11

×