Work real quick through agenda Just set the stage for an Hadoop based threat analytics platform that has NRT capabilities
Set the stage for how a typical network in this industry and how much work there is for securing it.
Presents an industry problem, not an AT&T problem
Address the outside threat to the internal operation of our industry
Amount of traffic related to reflect based DoS attackers. Illustrates activity on the internet not the attacks against the AT&T perimeter.
Hack-ma-geddon Columbia government Spam Hause Syria <- New York Times Target lost 40M credit/debit cards
Our TAP has evolved a lot over the last few year as we’ve moved into an Hadoop base architecture. I will briefly describe the roadmap.
Proprietary technology and lack of extensibility are killers
Past was SIEM dependent, based on large RDBMS and exclusively dependent on human detection and interpretation. Largely a data reduction system. Industry solution of yesterday.
The challenge is the cognitive intersection with automation.
An environment of innovation. Goal is to automate the security analysis process which are largely cognitive. Granted this is a different use of Hadoop rather than single use data. Its continual ingestion, NRT detections, alerting, etc. Not always a clear problem statement.
Spend some time developing the human dependency and cognitive processes
Takes a lot of data
Left to right, we move all the data through various processing platforms into an Hadoop base system for raw log management, data org, management, access, analysis and finally to visualization and reporting.
Near Real-Time Outlier Detection and Interpretation