More Related Content


Improving Network Intrusion Detection with Traffic Denoise

  1. Improving Network IntrusionImproving Network Intrusion Detection with Traffic DenoiseDetection with Traffic Denoise Miroslav Stampar (Croatian Government CERT)Miroslav Stampar (Croatian Government CERT)
  2. IntroductionIntroduction  Network Intrusion-Detection Systems (NIDS) generate lots of false alerts, false alarms, false positives, etc.  Result of thinking that it is better to catch both suspicious and harmless events than to miss something  Lots of irrelevant and/or “frustrating” noise  Security managers/technicians are loosing interest very fast (serious problem)
  3. Detection/noise problemDetection/noise problem  Basis of a NIDS is “detection”  Like metal detector, NIDS doesn't stop network traffic  It highlights potential threats so that the security guards, like firewalls, can be more effective  Too much noise results with lessened usability and benefit  As a side effect, increased management cost  Most of all, lack of focus on real adversaries
  4. Unusable dashboardsUnusable dashboards
  5. Large quantity of eventsLarge quantity of events
  6. Real adversaries vs “junk”Real adversaries vs “junk”  What if we could distinguish regular noise from real adversaries?  Regular noise should be fairly common on Internet  Subjects targeting multiple ranges should not be considered as big threat as those targeting only us  Various subjects (i.e. “lurkers”) scan searching for easy to exploit systems  Unsophistication at its best  Free (skiddie level) vulnerability assessment
  7. How to capture noise?How to capture noise?  It should be visible by different Internet parties compared to adversaries targeting only our IP range(s)  If unwanted (inbound) traffic is visible from different IP ranges in predefined time-span (e.g. 1 day) we can presume that it is a noise  Two ways: 1) Utilizing multiple (public third-party) IP blacklists 2) Constructing own sensor network made of “silent” boxes
  8. IP blacklists (generally)IP blacklists (generally)  Multiple third-parties publicly share relatively up-to- date IP blacklists (i.e. IP addresses with “bad reputation”)  Inbound traffic coming from such IPs could be safely DROP-ed (in case of active boundary defense – e.g. IPS)  AlienVault, CINS, Deepviz, VoIPBL, Turris, etc.  IDSes, honeypots, server logs, “threat intel”, etc.  SSH, HTTP, Telnet, VoIP, NTP, DNS, etc.
  9. IP blacklists (examples)IP blacklists (examples)
  10. IP blacklists (distribution)IP blacklists (distribution)
  11. IPnoise (general)IPnoise (general)   Experimental run during the 2016  Setup of 4 sensors on different IP ranges: 37.228.134.XXX (gertrude), 78.47.177.XXX (hans), 5.135.71.XXX (pierre), 216.155.145.XXX (tommy)  Silent *nix boxes capturing (inbound) IP traffic  ICMP and TCP/UDP (ports 1-65535)  type, src_ip, dst_ip, dst_port, first_timestamp, last_timestamp, count
  12. IPnoise (sample raw data)IPnoise (sample raw data)
  13. IPnoise (statistics)IPnoise (statistics)
  14. Denoise (methodology)Denoise (methodology)  Suricata (ELK) IDS on /20 network (4096 IPs)  Fine tuned ruleset based on ETPro (removed all “paranoid” rules, rules based on “reputation”, INFO/POLICY/CORPORATE, etc.) with highly adapted thresholds  Experimental data based on May 2017 events  Denoise results were obtained by manual processing (for presentation purposes)  Presenting concept (no final product, yet :)
  15. Events (May 2017)Events (May 2017)
  16. Threats (May 2017)Threats (May 2017)
  17. ResultsResults  On average 35% less events  On average 37% less threats (i.e. (src_ip, event))
  18. ConclusionConclusion  Ignoring, DROPing or lowering the severity of events for noisy IPs should help to focus on real adversaries  Testing results look promising  Although currently only a concept, personal wish is that this presentation will give somebody an idea to further evolve it  If anything, collection of traffic on silent boxes around the could pinpoint the wandering (noisy) IP addresses (more than 1000 per day)
  19. Appendix (ipsum)Appendix (ipsum)   Collecting IP addresses from >30 different online blacklists  Creation of different (ip)sets based on minimum number of occurrences  For example, 3.txt contains all IPs found on at least 3 blacklists  Daily automatic updates
  20. Questions?Questions?