Your SlideShare is downloading. ×
How to Build the Collective Intelligence Framework - Hacker Hotshots 11/27/2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

How to Build the Collective Intelligence Framework - Hacker Hotshots 11/27/2013

2,842
views

Published on

This presentation is how to build and use the Collective Intelligence Framework to start to collect threat intelligence from open-sources and use that to protect your network. This presentation was …

This presentation is how to build and use the Collective Intelligence Framework to start to collect threat intelligence from open-sources and use that to protect your network. This presentation was given for Hacker Hotshots on 11/27/2013.

Published in: Technology

1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
2,842
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
29
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Hacker Hotshots – 11/27/2013 How to Build the Collective Intelligence Framework And Start to Protect Your Network John Bambenek Chief Forensic Examiner, Bambenek Consulting jcb@bambenekconsulting.com
  • 2. Problem  Lots of people product lots of data, blacklists, indicators of badness out there.  They all have their own formats and means of distribution.  How to take multiple datasets, normalize them and take action?  “Does anyone know anything about X?”
  • 3. Solution: Collective Intelligence Framework  Developed by REN-ISAC  http://code.google.com/p/collective-intelligence- framework/  Does not generate data, simply takes sources normalizes it and then outputs by given types  Not really a data sharing tool  Up to user to assess confidence in the data  Limited in the types of data it can handle
  • 4. Data Types URLs  Domains  IPs  MD5s   Certainly more to threat intel than this, but it’s a start
  • 5. CIF Architecture
  • 6. CIF Architecture  By default, everything lives in /opt/cif  Configs in /opt/cif/etc/*.cfg (CIF processes all files ending in .cfg)     cif_smrt – queries the feeds cif_feed – generates feeds by assessment cif – command-line client tool cif_crontool – used for querying all feeds automatically
  • 7. Requirements to Install  For a “real” instance, you would need some disk (250 GB – 500 GB) and RAM (16 GB)  Disk is driven by how long you want to keep old data  Memory is only needing while parsing data  CIF can be placed in a virtual infrastructure easily  Can install it on most everything, Debian/Ubuntu easiest mostly because the instructions are available and clear  Ubuntu 12 probably best, 13 has some undocumented changes that need to be made  Some kernel tweaking is needed
  • 8. CIF Queries  Generally an analyst investigating will use queries to see what is in the database.  cif –q <IP ADDRESS|DOMAIN NAME|MD5>  Will include search records in the response (unless suppressed)  Exact matching only (can’t search for part of a URL… yet)
  • 9. CIF Queries  CIF also ships a browser plugin which is a little easier for analysis  Use cif_apikeys –l to get your key, find your amazon IP and configure it now  Can query specific items or feeds
  • 10. CIF feeds  cif –q feed/assessment –p output type [-z 0]  (-z 0 will prevent truncating URLs)  cif –q infrastructure/scan  Try with a lower confidence level  cif –q url/phishing –c 45  Not all output plugins work for all feeds  Full list at: http://code.google.com/p/collective- intelligence-framework/wiki/API_FeedTypes_v1
  • 11. CIF output types bindzone Bind zone configuration bro bro (network monitor) csv comma separated value html Html-ized table iptables iptables drop rules json json pcapfilter pcap filter (i.e. tcpdump) snort snort alert rules table ascii table (default)
  • 12. CIF Output  There are dozens of sources (many don’t have configs in CIF), but you can integrate them all into CIF and/or a feed.  What to do with this now?      Snort Rules Feed to web proxy to block/alert Send to border device to blacklist IPs Set up a sinkhole You can also put your own data into CIF for later research
  • 13. Questions?  Thanks for tuning in!  See more courses and hacker hotshot sessions at concise-courses.com