0
Hacker Hotshots – 11/27/2013
How to Build the
Collective Intelligence Framework
And Start to Protect Your Network
John Bam...
Problem


Lots of people product lots of data,
blacklists, indicators of badness out there.



They all have their own f...
Solution: Collective Intelligence
Framework


Developed by REN-ISAC
 http://code.google.com/p/collective-intelligence-

...
Data Types
URLs
 Domains
 IPs
 MD5s




Certainly more to threat intel than
this, but it’s a start
CIF Architecture
CIF Architecture


By default, everything lives in /opt/cif



Configs in /opt/cif/etc/*.cfg (CIF processes
all files en...
Requirements to Install


For a “real” instance, you would need some disk
(250 GB – 500 GB) and RAM (16 GB)
 Disk is dri...
CIF Queries


Generally an analyst investigating will use
queries to see what is in the database.



cif –q <IP ADDRESS|...
CIF Queries


CIF also ships a browser plugin which is
a little easier for analysis



Use cif_apikeys –l to get your ke...
CIF feeds


cif –q feed/assessment –p output type [-z 0]
 (-z 0 will prevent truncating URLs)



cif –q infrastructure/...
CIF output types
bindzone

Bind zone configuration

bro

bro (network monitor)

csv

comma separated value

html

Html-ize...
CIF Output


There are dozens of sources (many don’t have
configs in CIF), but you can integrate them all
into CIF and/or...
Questions?


Thanks for tuning in!



See more courses and hacker hotshot
sessions at concise-courses.com
Upcoming SlideShare
Loading in...5
×

How to Build the Collective Intelligence Framework - Hacker Hotshots 11/27/2013

3,397

Published on

This presentation is how to build and use the Collective Intelligence Framework to start to collect threat intelligence from open-sources and use that to protect your network. This presentation was given for Hacker Hotshots on 11/27/2013.

Published in: Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total Views
3,397
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
42
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "How to Build the Collective Intelligence Framework - Hacker Hotshots 11/27/2013"

  1. 1. Hacker Hotshots – 11/27/2013 How to Build the Collective Intelligence Framework And Start to Protect Your Network John Bambenek Chief Forensic Examiner, Bambenek Consulting jcb@bambenekconsulting.com
  2. 2. Problem  Lots of people product lots of data, blacklists, indicators of badness out there.  They all have their own formats and means of distribution.  How to take multiple datasets, normalize them and take action?  “Does anyone know anything about X?”
  3. 3. Solution: Collective Intelligence Framework  Developed by REN-ISAC  http://code.google.com/p/collective-intelligence- framework/  Does not generate data, simply takes sources normalizes it and then outputs by given types  Not really a data sharing tool  Up to user to assess confidence in the data  Limited in the types of data it can handle
  4. 4. Data Types URLs  Domains  IPs  MD5s   Certainly more to threat intel than this, but it’s a start
  5. 5. CIF Architecture
  6. 6. CIF Architecture  By default, everything lives in /opt/cif  Configs in /opt/cif/etc/*.cfg (CIF processes all files ending in .cfg)     cif_smrt – queries the feeds cif_feed – generates feeds by assessment cif – command-line client tool cif_crontool – used for querying all feeds automatically
  7. 7. Requirements to Install  For a “real” instance, you would need some disk (250 GB – 500 GB) and RAM (16 GB)  Disk is driven by how long you want to keep old data  Memory is only needing while parsing data  CIF can be placed in a virtual infrastructure easily  Can install it on most everything, Debian/Ubuntu easiest mostly because the instructions are available and clear  Ubuntu 12 probably best, 13 has some undocumented changes that need to be made  Some kernel tweaking is needed
  8. 8. CIF Queries  Generally an analyst investigating will use queries to see what is in the database.  cif –q <IP ADDRESS|DOMAIN NAME|MD5>  Will include search records in the response (unless suppressed)  Exact matching only (can’t search for part of a URL… yet)
  9. 9. CIF Queries  CIF also ships a browser plugin which is a little easier for analysis  Use cif_apikeys –l to get your key, find your amazon IP and configure it now  Can query specific items or feeds
  10. 10. CIF feeds  cif –q feed/assessment –p output type [-z 0]  (-z 0 will prevent truncating URLs)  cif –q infrastructure/scan  Try with a lower confidence level  cif –q url/phishing –c 45  Not all output plugins work for all feeds  Full list at: http://code.google.com/p/collective- intelligence-framework/wiki/API_FeedTypes_v1
  11. 11. CIF output types bindzone Bind zone configuration bro bro (network monitor) csv comma separated value html Html-ized table iptables iptables drop rules json json pcapfilter pcap filter (i.e. tcpdump) snort snort alert rules table ascii table (default)
  12. 12. CIF Output  There are dozens of sources (many don’t have configs in CIF), but you can integrate them all into CIF and/or a feed.  What to do with this now?      Snort Rules Feed to web proxy to block/alert Send to border device to blacklist IPs Set up a sinkhole You can also put your own data into CIF for later research
  13. 13. Questions?  Thanks for tuning in!  See more courses and hacker hotshot sessions at concise-courses.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×