Clairvoyant Squirrel: Large Scale Malicious Domain Classification
Upcoming SlideShare
Loading in...5
×
 

Clairvoyant Squirrel: Large Scale Malicious Domain Classification

on

  • 4,370 views

Clairvoyant Squirrel: Large Scale Malicious Domain Classification. Presentation from FloCon 2013

Clairvoyant Squirrel: Large Scale Malicious Domain Classification. Presentation from FloCon 2013

Statistics

Views

Total Views
4,370
Views on SlideShare
4,233
Embed Views
137

Actions

Likes
9
Downloads
57
Comments
1

5 Embeds 137

http://www.redditmedia.com 87
https://twitter.com 20
https://www.linkedin.com 15
http://www.linkedin.com 14
http://10.2.49.7 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Guys, good deck.
    Consider making aggregations on host, domain, site and ip, subnets. And create features based on this aggregations. Making those aggregations for different time intervals is another way to go. Then, this system will be even more real time.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Clairvoyant Squirrel: Large Scale Malicious Domain Classification Clairvoyant Squirrel: Large Scale Malicious Domain Classification Presentation Transcript

  • John Munro / jmunro@endgame.com Jason Trost / jtrost@endgame.comFlonCon 2013 | January 7–10 | Albuquerque, New Mexico
  • Introductions • John Munro (jmunro@endgame.com) – Network Security Researcher and Data Scientist • Jason Trost (jtrost@endgame.com) – Senior Software Engineer – Specializes in Hadoop/Storm/BigData
  • Agenda • The Problem • Our Approach • DGA Domain Classifier • String Statistics as Features • Malicious Domain Classifier • Demo • Real-time Streaming Platform View slide
  • The Problem txmxbo.info youtube.com yahoo.com Ct0u2xj5dbe4.wvw—game465.com p4.httzd5e2ufizo.3bawhfuec45dca65.401724.s1.v4.ipv6-exp.l.google.com abulqe.com za6.limfoklubs.com ns3.ohio.gov bibz01.apple.com docs.joomla.org Wmk41035u3751s0bgv4n91b0b7h74v.ipcheker.com View slide
  • The Problem txmxbo.info youtube.com yahoo.com Ct0u2xj5dbe4.wvw—game465.com p4.httzd5e2ufizo.3bawhfuec45dca65.401724.s1.v4.ipv6-exp.l.google.com abulqe.com za6.limfoklubs.com ns3.ohio.gov bibz01.apple.com docs.joomla.org Wmk41035u3751s0bgv4n91b0b7h74v.ipcheker.com
  • The Problem• Massive Volumes – Some of our partners deal with TBs per day of DNS PCAPs• Incredible Rates – One partner sees 13k requests/sec – Another closer to 100k/sec
  • Our Approach: Machine Learning! • Real-time streaming classification – In parallel across multiple servers • Markov Models – Random Domain Generation Traffic – Normal Benign Traffic • Random Forests – Benign vs Malicious • Periodically retrained – In order to maintain accuracy
  • Data Sources • Benign Domains – Millions of popular, real domains • Correlated with the Alexa top 10k domains • Malicious Domains – 800k domains gathered from an internal malware sandbox – Public blacklist domains from Conficker and Murofet Botnets
  • Markov Models
  • Markovian DGA Classifier • Domain Generation Algorithm (DGA) • Popular Domain Model – Trained: 258,039 domains from Day 1 of our Benign set – Tested: 331,359 domains from Day 2 of our Benign set – Accuracy: 99.40 % with 1,458 Unknown • Randomly Generated Domain Model – Trained: 90,884 domains from Conficker Botnet – Tested: 295,306 domains from Murofet Botnet – Accuracy: 99.34 % with 1,923 Unknown
  • String Statistics as Features
  • Feature Usefulness
  • Random Forests Algorithm FPO VIDEO TO COME
  • Random Forests • Pros: – Very high accuracy – Scalable across many nodes – Built-in protection from over fitting – Can handle very large data sets with many features – Robust with respect to goodness of features – Practical for real world use – Does not assume a distribution – Only two parameters to tune – Memory efficient • Cons: – Not the quickest classifier, but plenty fast in practice
  • Malicious Domain Classifier • Performance measured by 10 – fold Cross Validation • Training Set Cross Validation – 200k Benign 0.06 0.05 – 200k Malicious Out of Bag Error 0.04 0.03 0.02 0.01 0 10 20 30 40 50 60 70 80 90 100 Number of Trees K=3 K=5 K = 10
  • Results Bad Precision Good Precision 0.984 0.9795 0.983 0.979 0.982 0.9785 0.981 0.978 0.9775 PrecisionPrecision 0.98 0.977 0.979 0.9765 0.978 0.976 0.977 0.9755 0.976 0.975 0.975 0.9745 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 Number of Trees Number of Trees Bad Accuracy Good Accuracy 0.9805 0.9805 0.98 0.98 0.9795 0.9795 0.979 0.979Accuracy Accuracy 0.9785 0.9785 0.978 0.978 0.9775 0.9775 0.977 0.977 0.9765 0.9765 0.976 0.976 10 20 30 40 50 60 70 80 90 100 10 20 30 40 50 60 70 80 90 100 Number of Trees Number of Trees K=3 K=5 K = 10
  • Results Model Size 400 350 300 Model Size (MB) 250 200 K=3 150 K=5 100 K=10 50 0 10 20 30 40 50 60 70 80 90 100 Number of Trees Classification Throughput 60,000 50,000 Classifications/sec 40,000 K=3 30,000 K=5 K=10 20,000 10,000 0 10 20 30 40 50 60 70 80 90 100 Number of Trees
  • Results
  • Demo
  • Realtime Streaming Platform • Velocity is a platform for processing, analyzing, and visualizing large-scale event data in real- time • It was designed to be horizontally scalable and is built using Twitter’s Storm • It was built primarily for internal use with DNS events, IDS alerts, and netflow data, but it is in the process of being commercialized
  • Velocity Pipeline
  • Conclusion • Malicious domain classification • DGA domain identification using Markov Models • Summary Statistics based on domain string work well • Random Forests are very successful at classifying domains as Benign or Malicious • Real-time, distributed implementation
  • Future Work • Include more features: TTL, frequency seen, etc. • Correlation of bad domains based on ASN, Country, Organization, etc. • Identify subnets that are infected based on high traffic to bad domains • Identify Content Delivery Networks • Self Organizing Maps and other visualizations
  • Questions
  • Contact Information • John Munro • Email: jmunro@endgame.com • Jason Trost • Email: jtrost@endgame.com • Twitter: @jason_trost • Blog: www.covert.io