Download to read offline
More Related Content
Similar to Big Data for Security - DNS Analytics
Big Data for Security - DNS Analytics
- 1. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Big Data for Security DNS Event Analysis Marco Casassa Mont (Principal Research Scientist) Bill Horne (Senior Project Manager) Security and Cloud Lab, HP Labs
- 2. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2 Motivation Identify Security Threats from Big Security Data Use Case: DNS Data • Big … • Gold mine for Security Information • Hard to collect and analyse
- 3. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3 DNS is an important vector of attack
- 4. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4 What is DNS? Local DNS Server Root DNS Server .com DNS Server company.com DNS Server Check for Zone Check Cache DNS Traffic Generated by: - Users (e.g. browsing web) - Applications, Servers, etc. Endpoints 8. REPLY: 58.25.88.90 5. REPLY: ask “company.com” 4. QUERY: service.company.com?1. QUERY: service.company.com?
- 5. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5 1 10 100 1000 10000 100000 1000000 Routers VPN McAfee ePO Active Directory Web Proxy DNS Eventspersecond(logarithmicscale) The Scale of DNS Data 0 20000 40000 60000 80000 100000 120000 140000 Routers VPN McAfee ePO Active Directory Web Proxy DNS Eventspersecond(linearscale) • HP IT is currently rolling out ArcSight internally • Once deployed it will be 25% larger than any other non- governmental installation by volume • 120,000 DNS events/second
- 6. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6 What is the Problem? Scale • DNS is one of the most voluminous sources of events in the enterprise Granularity of Information • Traditional DNS servers do not log enough detail to detect many kinds of attacks Analysis • Limited analysis tools on top of event collection for DNS
- 7. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7 What is the Problem? 1. Attacks on DNS servers Denial of Service Cache Poisoning Hijacking/Redirection Code Injection 2. Attacks that leverage DNS to attack third parties Footprinting Reflection & Amplification 3. Attacks that use DNS as infrastructure Communication to malicious servers Fast Flux Domain Name Generation DNS Tunneling Analysis
- 8. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8 End to end handling of DNS events DNS Event Processing • Bypass DNS logs completely. • Grab packets directly off the wire using custom hardware • Collect the information needed to detect attacks • Independent of DNS server vendor • Throw out known good data through data analysis • Reduce storage requirements by 99% • Events stored in ArcSight Logger • Real-time Analysis: - Correlations & Alerting - Utilizing ArcSight • Historical Analysis: - Analytics to detect wide variety of attacks - Anomaly Detection - Utilizing ArcSight, Vertica, Hadoop, Autonomy, etc. • Novel visualizations to help analysts deal with huge amount of data Real-Time Analysis Visualizatio n Storag eCollection Historical Analysis
- 9. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.9 HP HAVEn – Big Data platform HAVEn Social media IT/OT ImagesAudioVideo Transactional dataMobile Search engineEmail Texts Catalog massive volumes of distributed data Hadoop/ HDFS Process and index all information Autonomy IDOL Analyze at extreme scale in real-time Vertica Collect & unify machine data Enterprise Security Powering HP Software + your apps nApps Documents hp.com/haven
- 10. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10 Architecture DNS Event Processing DNS Server(s) HPL DNS Connector Whitelist network tap DNS queries and responses ArcSight Logger ArcSight ESM Blacklist Analytical and Visualization Solution (HP HAVEn* Inside …) Event Logging Correlation & Alerting Blacklist & Whitelist Manager Real-Time Processing Historical Analysis *HAVEn: Hadoop, Autonomy, Vertica, ESP, … DNS events: Queries & Replies
- 11. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11 DNS Packet Capturer Security Event Logs Security Analytical Workflows Vertica Anomaly Detectio n Threat Indicators HPL Threat Analytics+Visualizatio n Library HPL Security Analytical & Visualization Network Systems Visual Processin g ArcSight ESM Filtered DNS events Event Downloader & Processor ArcSight Logger Events ESM Alerts Whitelist/Blacklist Generator Real-time Analysis Historical Analysis Hadoop, Autonomy, R Web Server Threat Intelligenc e Anomalies, Threat Findings Anomalies, Threats, Graphs Syslog Server Alerts Introspection
- 12. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Demo …
- 13. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.13 13 Demonstrator GUI: Screenshot Demonstrator URL: http://15.25.28.107:8080/gui/insight.html
- 14. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14 14 Demonstrator GUI: Screenshot
- 15. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Thank you
- 16. © Copyright 2012Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16 Competitive Landscape Main HP Competitors in the Security Market already make claims about: • Network Packet Capture capabilities, inclusive of DNS traffic • Back-end advanced Analytics on large amounts of (security) events to complement real-time alert processing
Editor's Notes
- #7 HP IT is currently rolling out ArcSight internally Once deployed it will be 25% larger than any other installation by volume HTTP Proxy, Active Directory, VPN, McAfee ePO, router logs
- #10 What is HAVEn: HAVEn is the leading big platform for big data in the industry. HAVEn is not a single product. It is a platform that consists of multiple components. HAVEn data platform brings together Hadoop Autonomy Vertica Enterprise Security and any n number of applications As you see in the next slide we also have an HAVEn ecosystem around this platform HAVEn brings together everything you need to profit from big data; hardware, software and services. The 3 HAVEn platform components are connectors, applications, and engines. These are shipping already. We have 1000’s of customers using these components to build mission critical solutions. How does this all work together? As an example, one of the largest global banks does the following When you call them, 3 things happen in parallel – your call gets logged into Hadoop for compliance Your call gets analyzed through autonomy for sentiment – to determine if the customer is happy or unhappy and this info is inserted into Vertica for real time analytics Simultaneously, another thread gets other business info on this customer and merges it together to find if you are a profitable customer This information along with other information is analyzed in Vertica in real time to determine how to effectively handle the customer. Should be be offered any promotion or discounts. Details on connectors We have 400 connectors from Autonomy and 300 from Arcsight that help you bring all kinds of data. With these many connectors, it is highly likely that you will be able to have off-the-shelf connector to your data. In addition each of the engine components (Autonomy, Vertica and Arcsight) also provide additional data connector frameworks and tools to help you write custom connectors . Additionally the HAVEn platform supports popular frameworks like Hadoop flume and Chukwa. And it is open to all ETL frameworks. Details on engines (For more details refer to individual product pages) Many HP customers use Hadoop or experimenting with it. HP believes in a open Hadoop strategy. HP has been shipping preconfigured Hadoop appliances and/or reference architectures with all major Hadoop vendors – Cloudera, Horton works and MapR. . What we are seeing is that Hadoop is great as a data store to bring in all kinds of data and for ETL, but customers are telling us that they want better engines. As an example Novartis switched from using Hadoop to Vertica and the processing went down from several hours to several seconds using Vertica. That meant rapid drug discovery. The impact saving lives Autonomy has the leading algorithms protected by tens of patents for human information processing - video, audio, text –ex in London Olympics, camera images captured in London were matched in real time to terrorist database. The impact – saving lives. It is one of a kind technology. Vertica is designed ground up in the last ten years in MIT. It was designed for the peta byte wave for blazing fast real time analytics on peta byte size sets. It is designed as analytics platform that supports standard SQL/JDBC/ODBC and R natively. But most importantly because it is designed for large data analytics you can do it at a fraction of what legacy systems cost. Arcsight has been the leader Security and Events Information mgmt. system on Gartner MQ for years. It is used by some of the largest organizations in the world. It has been proven to scale at a million events a second range. Details on applications We have started modifying our existing application portfolio to use HAVEn. And we are building new applications that leverage power of HAVEn As an example, HP has launched a new application for operation analytics which leverages the power of multiple HAVEn components. Many customers are already building applications that use multiple HAVEn components together To help you get started we have lined up partners and SI’s that can help you build these solutions. Which brings us to the next point – the HAVEn ecosystem