Building an IP Reputation Engine: Tracking the Miscreants

7,728 views
7,421 views

Published on

The AlienVault Open Threat Exchange™ (AV-OTX™) is a system for sharing threat intelligence among OSSIM users and AlienVault customers. Go behind the scenes and find out how it works!

Published in: Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,728
On SlideShare
0
From Embeds
0
Number of Embeds
24
Actions
Shares
0
Downloads
13
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

Building an IP Reputation Engine: Tracking the Miscreants

  1. 1. Building anIP Reputation engine Tracking the miscreants
  2. 2. Index1. What is IP Reputation2. Open Source IP Reputation Portal3. How is the engine4. Feeding the engine5. Current integrations
  3. 3. Index1. What is IP Reputation 1.1. The problem 1.2. What is IP Reputation? 1.3. What is an IP Reputation engine? 1.4. Features of an IP Reputation engine2. Open Source IP Reputation Portal3. How is the engine4. Feeding the engine5. Current integrations
  4. 4. The problem Security analyst: “How many of my network connections are going to bad sites?”
  5. 5. What is IP Reputation?IP Reputation is a summary of the past behavior activity detected on an IP An IP with reputation information add context when a network connection is observed
  6. 6. What is an IP Reputation engine?An IP Reputation engine is a system to classify and score large sets of IPs, in low or high reputation
  7. 7. Features of an IP Reputation engineUpdated informationAccurate values associated to every IPAssign activity classification to every IPRange of detection
  8. 8. Index1. What is IP Reputation2. Open Source IP Reputation Portal3. How is the engine4. Feeding the engine5. Current integrations
  9. 9. Open Source IP Reputation Portalhttp://labs.alienvault.com/labs/index.php/projects/open-source-ip-reputation-portal/
  10. 10. A register in the reputation.data file:<IP>#<RELIABILITY>#<RISK>#<ACTIVITY>#<COUNTRY>#<CITY>#<LAT>,<LON> 1...10 1...10 C&C Open Proxy Malicious Host Phishing Malware Domain Spamming Malware IP Scanning Host 64.44.240.225#4#3#Malicious Host#US#Chicago#41.9287986755,-87.6315002441 194.176.176.82#4#2#Spamming#RO#Bucharest#44.4333000183,26.1000003815 93.183.203.41#3#2#C&C;Malware Domain#UA#Kiev#50.4333000183,30.5167007446 64.141.101.204#1#2#Malware Domain#CA#Calgary#51.0833015442,-114.083297729 https://reputation.alienvault.com/reputation.data
  11. 11. Index1. What is IP Reputation2. Open Source IP Reputation Portal3. How is the engine 3.1. Architecture design 3.1.1. Server 3.1.2. Agent 3.1.3. URL system 3.2. Scoring system4. Feeding the engine
  12. 12. Architecture design Server Database PrefilterURL system Agent IPs/domains URLs Agent DATA IP reputation portal
  13. 13. Scoring system DNSBL + BULK DOMAINS + DYNAMIC IP DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
  14. 14. Scoring system DNSBL +$ host 6.6.6.6.zen.spamhaus.orgHost 6.6.6.6.zen.spamhaus.org not BULK DOMAINS +found: 3(NXDOMAIN) DYNAMIC IP$ host 2.0.0.127.zen.spamhaus.org2.0.0.127.zen.spamhaus.org has DYNAMIC DNS +address 127.0.0.102.0.0.127.zen.spamhaus.org hasaddress 127.0.0.2 GOOGLE SAFE BROWSING +2.0.0.127.zen.spamhaus.org hasaddress 127.0.0.4 FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
  15. 15. Scoring system DNSBL + *.co.be BULK DOMAINS + *.co.cc*.co.com.au DYNAMIC IP *.co.tv *.com.ua DYNAMIC DNS + *.cu.cc GOOGLE SAFE BROWSING + *.cw.cm *.cx.cc FILE-SHARING IP - *.cz.cc ALEXA TOP ONE MILLION - *.cz.tf HEURISTIC DOMAIN +
  16. 16. Scoring system DNSBL + BULK DOMAINS +$ host 87.216.x.x DYNAMIC IPx.x.216.87.in-addr.arpa domain namepointer x.x.216.87.dynamic.jazztel.es. DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
  17. 17. Scoring system DNSBL + BULK DOMAINS + *.ath.cx DYNAMIC IP*.dyndns.org DYNAMIC DNS + *.no-ip.biz *.no-ip.info GOOGLE SAFE BROWSING + *.no-ip.org FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
  18. 18. Scoring system DNSBL + BULK DOMAINS + DYNAMIC IP DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
  19. 19. Scoring system DNSBL + BULK DOMAINS + DYNAMIC IP DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
  20. 20. Scoring system DNSBL + BULK DOMAINS +1, google.com DYNAMIC IP2, facebook.com3, youtube.com4, yahoo.com DYNAMIC DNS +5, baidu.com6, wikipedia.org GOOGLE SAFE BROWSING +7, live.com8, blogspot.com9, amazon.com FILE-SHARING IP -10, twitter.com... ALEXA TOP ONE MILLION -999999, panciapiatta.net1000000, acsysun.co.jp HEURISTIC DOMAIN +
  21. 21. Scoring system DNSBL + BULK DOMAINS + ypyfp.com.tw jlmjalzjk.gs ewdkddr.me DYNAMIC IP xzasuf.com.pt nnis.co.uk DYNAMIC DNS + qzlx.co.za tuxs.com.ua GOOGLE SAFE BROWSING + upwcbab.tw hkwytkey.pe uzabfgqfk.my FILE-SHARING IP - http://labs.alienvault.com/labs/index.php/2012/detecting-malware- ALEXA TOP ONE MILLION - domains-by-syntax-heuristics/ HEURISTIC DOMAIN +
  22. 22. Index1. What is IP Reputation2. Open Source IP Reputation Portal3. How is the engine4. Feeding the engine 4.1. External sources 4.2. Our sandnet 4.3. AlienVault OTX5. Current integrations
  23. 23. Getting data from external sources { Malware Trackers Malicious Hosts lists Open Proxy lists Scanning Hosts lists SPAM Trackers and more...
  24. 24. Our sandnet Samples Queue Sandbox Sandnet web panelSandnet { }Database Traffic, rules trigger Traffic, no rules trigger No traffic!IP Reputation Database
  25. 25. AlienVault OTX is a system for sharing threatintelligence among OSSIM users and AlienVaultcustomers.http://www.alienvault.com/alienvault-labs/open-threat-exchange/
  26. 26. Index1. What is IP Reputation2. What is the Open Source IP Reputation Portal3. How is the engine4. Feeding the engine5. Current integrations 5.1. Integration in OSSIM 5.2. Other integrations
  27. 27. Integration in OSSIMOSSIM is an Open Source SIEM (Security Information Event Management). Acomprehensive compilation of tools that work together to provide a detailedview over each and every aspect of your networks, hosts, physical accessdevices, server, etc.http://communities.alienvault.com/communityA security event manager (SEM) (acronyms SIEM and SIM) is a computerizedtool used on enterprise data networks to centralize the storage andinterpretation of logs, or events, generated by other software running on thenetwork.http://en.wikipedia.org/wiki/Security_event_manager
  28. 28. { fprobe, nfSen (flow collector and analyzer) Snort (IDS) + EmergingThreats ruleset OSSEC (HIDS) Nagios (service and infrastructure monitoring) OpenVAS, Nessus (vulnerability assessment) p0f, PADS, arpwatch (passive network monitoring) nmap (network scanning) OCS Inventory NG (host-based inventory) Wireshark, tcpdump (full packet capture) and more...
  29. 29. { data collection with plugins: routers, firewalls, switches... load balancers, intrusion prevention systems honeypots, web proxies, web application firewalls ...
  30. 30. OSSIM architecture Find patterns Server Correlation engine Insert eventsNormalized data Sensors Database Detects new data DATA
  31. 31. Logic correlation if detected firewall or proxy event + and is an ACCEPT or HTTP code 200 OK event + and the destination IP has a low reputation = alarm<directive id="29001" name="Suspicious communication on SRC_IP" priority="5"> <rule type="detector" name="HTTP connection to low IP reputation destination" plugin_id="1503" plugin_sid="1" reliability="10" occurrence="1" from="HOME_NET" to="!HOME_NET" port_from="ANY" port_to="80,443" to_reputation="true" protocol="TCP"/></directive>
  32. 32. Logic correlation
  33. 33. Other integrations Snort reputation format Iptables format Squid format Unix (hosts.deny) formatMore to come: shellscripts, configuration guides, nfSen plugin...
  34. 34. Future of the IP reputationLive scoringAPIPredictive IP reputationExtent to domain blocklist
  35. 35. Conclusions1. Free to use IP Reputation database2. Detailed information about the activity and history of every IP through the web portal3. Continuously updated and maintained using different resources and improved with AlienVault OTX4. Fully integrated in OSSIM, ready to be easily integrated with another systems
  36. 36. http://labs.alienvault.com Alberto Ortega Guillermo Grande a0rtega Guillermo aortega@alienvault.com ggrande@alienvault.com

×