Cisco CSIRT Case Study: Forensic Investigations with NetFlow
•
1 like•5,428 views
Cisco CSIRT uses NetFlow to collect 16 billion flows from Cisco’s 175TB of traffic observed daily. The data is used to monitor, investigate, and contain incidents using 3 key playbook “plays” each day. Two leaders from Cisco's Computer Security Incident Response Team (CSIRT) will review a real cyber incident and the resulting investigation leveraging NetFlow collected via the StealthWatch System. Participants will learn how to use NetFlow and the StealthWatch System to: Investigate top use cases: C&C discovery, data loss and DOS attacks Gain contextual awareness of network activity Accelerate incident response Minimize costly outages and downtime from threats Protect the evolving network infrastructure Provide forensic evidence to prosecute adversaries
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Cisco CSIRT Case Study: Forensic Investigations with NetFlow
Similar to Cisco CSIRT Case Study: Forensic Investigations with NetFlow (20)
More from Lancope, Inc.
More from Lancope, Inc. (16)
Recently uploaded
Recently uploaded (20)
Cisco CSIRT Case Study: Forensic Investigations with NetFlow
- 1. © 2014 Lancope, Inc. All rights reserved. Cisco CSIRT: Security Analytics and Forensics with NetFlow Presented by: Michael Scheck, Information Security Manager, Cisco Paul Eckstein, CSIRT Engineering Manager, Cisco
- 2. © 2014 Lancope, Inc. All rights reserved. The Heartbleed Bug is a serious vulnerability in the popular OpenSSL cryptographic software library. This weakness allows stealing the information protected, under normal conditions, by SSL April 8, 2014: Heartbleed Vulnerability
- 3. © 2014 Lancope, Inc. All rights reserved. Cisco CSIRT Response to Heartbleed • Preparation • Scanned 1.2M vulnerable servers - 300 needed repair • Helped develop signatures for Sourcefire and Cisco IDS • Deployed signatures to IDS • Monitoring and response • Discovered 25 attacks: 21 benign, 4 malicious • Researched attack via NetFlow analysis to discern normal connections from those that were anomalous and malicious 3
- 4. © 2014 Lancope, Inc. All rights reserved. Heartbleed Benign Host 4
- 5. © 2014 Lancope, Inc. All rights reserved. Heartbleed Malicious Host 5
- 6. © 2014 Lancope, Inc. All rights reserved. NetFlow@Cisco History
- 7. © 2014 Lancope, Inc. All rights reserved. A B C C B A C A B NetFlow Basics 7
- 8. © 2014 Lancope, Inc. All rights reserved. NetFlow Collection and Analysis Solutions 8 OSU FlowTools nfdump Lancope StealthWatch License Open source from Ohio State Open source from SourceForge Commercial NetFlow versions V5 v5 and up v5 and up IPv6 ready? Yes Yes Yes Syntax Command-line, like ACLs Command-line, like tcpdump GUI Support Ad-hoc via Google Code Up-to-date Up-to-date
- 9. © 2014 Lancope, Inc. All rights reserved. NetFlow at Cisco Before StealthWatch • OSU FlowTools • 25+ systems running in parallel - Speeds up query time, but routers have to point at each collector •20+ Tb of physical storage -Files were stored in native nfdump/flowtools compressed format •No flow aggregation •Some connections passed through multiple devices, causing duplicate flows •Routers splitting up long running flows 9
- 10. © 2014 Lancope, Inc. All rights reserved. NetFlow Challenge:Support • Support of open source tools • OS support • Training staff • Feature requests • Protocol changes (NetFlow and IP) • Difficult to monitor for flow loss 10
- 11. © 2014 Lancope, Inc. All rights reserved. NetFlow Investigation with OSU FlowTools Query bot.acl file uses familiar ACL syntax. create a list named ‘bot’ [mynfchost]$ head bot.acl ip access-list standard bot permit host 69.50.180.3 ip access-list standard bot permit host 66.182.153.176 [mynfchost]$ flow-cat /var/local/flows/data/2007-02-12/ft* | flow-filter -Sbot -o -... Start End Sif SrcIPaddress SrcP DIf DstIPaddress DstP 0213.08:39:49.911 0213.08:40:34.519 58 10.10.71.100 8343 98 69.50.180.3 31337 0213.08:40:33.590 0213.08:40:42.294 98 69.50.180.3 31337 58 10.10.71.100 83 11
- 12. © 2014 Lancope, Inc. All rights reserved. NetFlow Investigation with OSU FlowTools Custom NetFlow Report Generator
- 13. © 2014 Lancope, Inc. All rights reserved. Our Installation
- 14. © 2014 Lancope, Inc. All rights reserved. Internet Data Center ISP Gateways NetFlow Collector DC Gateways Corporate Backbone NetFlow exported at network choke points NetFlow Export at Cisco Collect at chokepoints for egress detection 14
- 15. © 2014 Lancope, Inc. All rights reserved. NetFlow Collection at Cisco with StealthWatch 15
- 16. © 2014 Lancope, Inc. All rights reserved. Common collection infrastructure • Redundant forwarding • Regional storage • Global search • Applies to netflow, log collection 16
- 17. © 2014 Lancope, Inc. All rights reserved. Lancope Devices and Count StealthWatch Management Console FlowReplicator FlowSensor FlowCollector 2 2 10 13 17
- 18. © 2014 Lancope, Inc. All rights reserved. NetFlow Retention 18 SJC 4-18 months RCDN 10 months RTP 4 months LON 26 months BGL 5-9 months
- 19. © 2014 Lancope, Inc. All rights reserved. Problems Solved
- 20. © 2014 Lancope, Inc. All rights reserved. 30s 30s 30s NetFlow Challenge: Flow Timeouts One 90s flow creates 6 flows 30s timeout 90/30 = 3 x 2 collectors 30s 30s 30s NetFlow creates 3 flows NetFlow creates 3 flows Lab gateway ISP gateway 20
- 21. © 2014 Lancope, Inc. All rights reserved. Business Benefit #1 Storage Capacity 30s 30s 30s 30s 30s 30s NetFlow creates 3 flows NetFlow creates 3 flows Lab gateway ISP gateway 21
- 22. © 2014 Lancope, Inc. All rights reserved. Business Benefit #2 Ease of support • IPv4/IPv6 both supported • NetFlow v5/v9 both supported • All supported on the same system, on the same port! • No system administration required • Alarms built in for monitoring of lost flows 22
- 23. © 2014 Lancope, Inc. All rights reserved. Business Benefit #3 Ease of use 24
- 24. © 2014 Lancope, Inc. All rights reserved. • Other variables: host groups, time range, interfaces, ports • Defaults to 2000 flow records returned • Much simpler than syntax for CLI (example below) Flow Table Query 1. Create a file called‘flow.acl’with a named access list: linux-machine# cat ip access-list standard botnet permit ip 10.31.33.7 >flow.acl 2. Run a query for the time period you are interested in using the ACL linux-machine# flow-cat /var/local/flows/data/2006-12-01/ft* | flow-filter -f ~/flow.acl -Sbotnet -o -Dbotnet | flow- print -f5 25
- 25. © 2014 Lancope, Inc. All rights reserved. Flow Table Output 26
- 26. © 2014 Lancope, Inc. All rights reserved. FlowTable Results Server, DNS, and Country Traffic Type & Volume 27
- 27. © 2014 Lancope, Inc. All rights reserved. NetFlow Challenge: Limited Detection Capability • No concept of host groups for query • Effective for forensics • Can do basic DOS detection • Any other queries required writing algorithms 29
- 28. © 2014 Lancope, Inc. All rights reserved. Suspected Data Loss High File Sharing Index Max Flows Served Business Benefit #4: Analytics 30
- 29. © 2014 Lancope, Inc. All rights reserved. Use Cases
- 30. © 2014 Lancope, Inc. All rights reserved. NetFlow CNC discovery 32 2. Investigate other internal hosts communicating with the same CnC 1. Detect host communicating with external Command-and-Control 3. Uncover other malicious, external entities from the compromised hosts
- 31. Targeted Monitoring: DoS Detection 33
- 32. 34 Targeted Monitoring – Data Loss
- 33. 35 Targeted Monitoring: Data Loss
- 34. © 2014 Lancope, Inc. All rights reserved. StealthWatch Host Locking 36 Send syslog for any traffic seen between insides hosts and known C&C servers
- 35. © 2014 Lancope, Inc. All rights reserved. StealthWatch Host Locking 37 Modify known C&C server list via API
- 36. © 2014 Lancope, Inc. All rights reserved. CRiTs crits@mitre.org 38
- 37. © 2014 Lancope, Inc. All rights reserved. CRiTS Indicator Actions 39 Prevent DNS RPZ host IDS BGP Detect Syslog In Progress passive DNS Share Govt Current Future CSIRT Mandiant ESA HIPS LUPA/ PCAP WSA Partner CRITS MD5 IPV4 Regkey AV SBG CDSA Lancope
- 38. CRiTS NetFlow Alarms 40
- 39. © 2014 Lancope, Inc. All rights reserved. Splunk Integration – SMC Alarms Requirement: integrate flow events with other logs for a single investigation interface Solution: send relevant alarms as syslog messages to in-house Splunk™ architecture
- 40. © 2014 Lancope, Inc. All rights reserved. StealthWatch Splunk Alerts Link to StealthWatch host snapshot
- 41. © 2014 Lancope, Inc. All rights reserved. API Use Cases Requirement Problem API Script Solution Pull all flows for given time period SMC Flow Collector query limit Run consecutive, small queries then concatenate Keep SMC host groups up to date Manual configuration, old data Query internal source of truth, push subnet lists to host groups automatically Look up events for a particular IP for a specific timeframe No user attribution (yet) Find IP and lease time from internal source of truth, query StealthWatch for related events 43
- 42. © 2014 Lancope, Inc. All rights reserved. Network Subnets Mapped from IPAM
- 43. © 2014 Lancope, Inc. All rights reserved. Network Subnets Map to Lancope Zones 45
- 44. © 2014 Lancope, Inc. All rights reserved. Splunk integration: getFlows Find NetFlow events via Lancope API with the respective src/dst
- 45. © 2014 Lancope, Inc. All rights reserved. Splunk Integration - getFlows 47
- 46. © 2014 Lancope, Inc. All rights reserved. Next Steps How to get started: 1. Find a collection/query system for NetFlow 2. Export NetFlow from chokepoints 3. Map your network context from IPAM into zones for query 4. Configure alarms for specific zones 5. Setup performance monitoring to mitigate flow loss from exporters 6. Integrate with your portfolio via API 7. Train your users and administrators – attend Lancope webinars and training 48
- 47. © 2014 Lancope, Inc. All rights reserved. Contact information: Mike: mscheck@cisco.com Paul: peckstei@cisco.com