A an overview of network flow collection and an invitation to look at the fast_ip network flow platform.

1. There’s Plenty of Room at
the Bottom:
An Invitation to Explore with Network Flows
Benjamin Black
b@fastip.com

2. What are Flows
&
Why Should You Care?

3. You Should Care
Because Visibility Makes
Your Life Easier.

4. Network Flow Data
Means Great Visibility.

5. DDoS Detection
Capacity Planning
Traffic Management
Troubleshooting
Correlation
...

6. The Nature of Flows

7. [traffic]

8. [streams]

9. [packets]
Header Payload

10. [headers]
Protocol
Source IP Address
Destination IP Address
Source Port
Destination Port

11. [latency]

12. [jitter]

13. [packet loss]

14. The Structure of Flows

15. [flow keys]
Protocol Protocol
Source IP Address Source IP Address
Destination IP Address
Source Port
= Destination IP Address
Source Port
Destination Port Destination Port

16. [templates]
template_id 253
protocol
src IPv4 address
dest IPv4 address
src port
dst port
total octets
total packets
start time
end time

17. [flow records]
template_id 253
TCP
172.16.101.3
192.169.7.200
9801
80
27342 octets
24 packets
start 28349829023
end 28356729023

18. The Ecosystem of Flows

19. [metering process]
template_id 253 template_id 253 template_id 253 template_id 253
TCP TCP TCP TCP
172.16.101.3 172.16.101.3 172.16.101.3 172.16.101.3
192.169.7.200 192.169.7.200 192.169.7.200 192.169.7.200
9801 9801 9801 9801
80 80 80 80
27342 octets 27342 octets 27342 octets 27342 octets
24 packets 24 packets 24 packets 24 packets
start 28349829023 start 28349829023 start 28349829023 start 28349829023
end 28356729023 end 28356729023 end 28356729023 end 28356729023

20. [observation domain]
eth0
eth1
eth2

21. [collecting process]
template_id 253 template_id 253 template_id 253 template_id 253
TCP TCP TCP TCP
172.16.101.3 172.16.101.3 172.16.101.3 172.16.101.3
192.169.7.200 192.169.7.200 192.169.7.200 192.169.7.200
9801 9801 9801 9801
80 80 80 80
27342 octets 27342 octets 27342 octets 27342 octets
24 packets 24 packets 24 packets 24 packets
start 28349829023 start 28349829023 start 28349829023 start 28349829023
end 28356729023 end 28356729023 end 28356729023 end 28356729023
template_id 253 template_id 253 template_id 253 template_id 253
TCP TCP TCP TCP
172.16.101.3 172.16.101.3 172.16.101.3 172.16.101.3
192.169.7.200 192.169.7.200 192.169.7.200 192.169.7.200
9801 9801 9801 9801
80 80 80 80
27342 octets 27342 octets 27342 octets 27342 octets
24 packets 24 packets 24 packets 24 packets
start 28349829023 start 28349829023 start 28349829023 start 28349829023
end 28356729023 end 28356729023 end 28356729023 end 28356729023
template_id 253 template_id 253 template_id 253 template_id 253
TCP TCP TCP TCP
172.16.101.3 172.16.101.3 172.16.101.3 172.16.101.3
192.169.7.200 192.169.7.200 192.169.7.200 192.169.7.200
9801 9801 9801 9801
80 80 80 80
27342 octets 27342 octets 27342 octets 27342 octets
24 packets 24 packets 24 packets 24 packets
start 28349829023 start 28349829023 start 28349829023 start 28349829023
end 28356729023 end 28356729023 end 28356729023 end 28356729023

22. Storage and Analysis are
Left as an Exercise
for the Reader

23. Where Do Meters Run?

24. On Network Switches/Routers
[often sampled]

25. Dedicated Appliances
[expensive/limited storage]

26. On Hosts
[where does the data go?]

27. The Classical View

29. Where is this going?

30. Where is this going?
Where is this coming from?

32. The Flow View

35. TANSTAAFL

36. Flow Data Takes Up
LOTS of Space

37. [often >1% total traffic]

38. LOTS of Space Means Storage
Expense or Loss of Resolution or
Truncation

39. LOTS of (Multi-dimensional)
Data is
Hard to Analyze

40. Inflexible and Limited
or
Expensive and Complicated

42. [apologies]

43. [resources]
IPFIX WG
http://datatracker.ietf.org/wg/ipfix/charter/
nProbe
http://www.ntop.org/nProbe.html
Cisco NetFlow Collection Engine
http://www.cisco.com/en/US/products/sw/netmgtsw/ps1964/index.html
Arbor Networks
http://www.arbornetworks.com/
Dartware
http://www.intermapper.com/products/intermapper-flows

44. [finally...]

45. fast_ip is a platform for
flow analytics

46. Sign up for our beta
http://fastip.com