2. Agenda
• NetFlow
– What it is, and how it works
– Working with Flows
• Flow Exporting Protocols
Implementations
– Cisco NetFlow
– IPFIX
• Flows and their applications
– Uses for Flow
3. Network Flows
• “Packets or frames that have some
common properties” (Typically: IP
src/dst, Port src/dst, L4 Protocol, ToS,
VLANid)
• Creation and expiration policy – what
conditions start and stop a flow.
• By the way, it contains:
– Peers – flow source and destination
– Counters – packets, bytes, time
– Routing information – AS, network mask, interfaces
4. Network Flows
●
Unidirectional or bidirectional
• Bidirectional flows can contain other
information such as round trip time
(RTT), TCP behavior.
• Application flows look past the
headers to classify packets by their
contents
• Aggregated flows – flows of flows
5. Unidirectional Flow with
Source/Destination IP Key (L2)
% telnet 10.0.0.2 Port #23
Ethernet frame
login:
00-B0-D0-86-BB-F7 00-14-22-01-23-45
Ethernet frame
Active Flows
Flow Source MAC Destination MAC
1 00-B0-D0-86-BB-F7 00-14-22-01-23-45
2 00-14-22-01-23-45 00-B0-D0-86-BB-F7
6. Unidirectional Flow with
Source/Destination IP Key (L3)
% telnet 10.0.0.2 Port #23
IP packet
login:
10.0.0.1 IP packet 10.0.0.2
Active Flows
Flow Source IP Destination IP
1 10.0.0.1 10.0.0.2
2 10.0.0.2 10.0.0.1
7. Unidirectional Flow with
Source/Destination IP Key (L3)
% telnet 10.0.0.2 Port #23
% ping 10.0.0.2
ICMP request
login:
10.0.0.1 ICMP echo reply 10.0.0.2
ICMP reply
Active Flows
Flow Source IP Destination IP
1 10.0.0.1 10.0.0.2
2 10.0.0.2 10.0.0.1
8. Bidirectional Flow with IP, Port,
Protocol Key (L4)
% telnet 10.0.0.2
Port #32000 Port #23
% ping 10.0.0.2
ICMP request
login:
10.0.0.1 ICMP echo reply 10.0.0.2
ICMP reply
Active Flows
Flow Source IP Destination IP Prot srcPort dstPort
1 10.0.0.1 10.0.0.2 TCP 32000 23
2 10.0.0.1 10.0.0.2 ICMP 0 0
10. Application Flow (L7)
% firefox http://10.0.0.2:9090 Web server
Port #9090
HTTP request
10.0.0.1 Content-Type:
10.0.0.2
HTTP response
Active Flows
Flow Source IP Destination IP Application
1 10.0.0.1 10.0.0.2 HTTP
11. Aggregated Flow
Main Active flow table
Flow Source IP Destination IP prot srcPort dstPort
1 10.0.0.1 10.0.0.2 TCP 32000 23
2 10.0.0.2 10.0.0.1 TCP 23 32000
3 10.0.0.1 10.0.0.2 ICMP 0 0
4 10.0.0.2 10.0.0.1 ICMP 0 0
Source/Destination IP Aggregate
Flow Source IP Destination IP
1 10.0.0.1 10.0.0.2
2 10.0.0.2 10.0.0.1
13. NetFlow Architecture
• Observation point
– For example, an interface of a router
– Associated with an Observation Domain
• Observation domain
– Set of observation points in a domain
• NetFlow Exporter
– Device with the NetFlow services enabled
– Exports flows in the form of Flow Records
• NetFlow Collector
– Receives Flow Records from one or more exporters,
parsing and storing the real information
14. NetFlow Architecture
• Export Packet/Message
– Packet/Message with a Packet/Message Header
followed by one or more FlowSets , sent from the
NetFlow Exporter to the NetFlow Collector
• Observation domain
– Set of observation points in a domain
• NetFlow Exporter
– Device with the NetFlow services enabled
– Exports flows in the form of Flow Records
• NetFlow Collector
– Receives Flow Records from one or more exporters,
parsing and storing the information
16. NetFlow
Export Packet Format
• Common header among export
versions
• All except v1 have a sequence number
• Version specific data field where N
records of data type are exported
• N is determined by the size of the flow
definition. Packet size is kept under
~1480 bytes. No fragmentation on
Ethernet.
17. Working with Flows
• Generating Process
– Key Fields
– Non-Key Fields
• Exporting Process
– Types of flows
– Sampling rates
• Collecting Process
– Tools to Collect Flows (“Flow-tools”)
• Analyzing Process
– More tools available, can write your own (nProbe)
18. Working with Flows
Generating Process
• A Key with more elements will generate
more flows.
• Greater number of flows leads to more post
processing time to generate reports, more
memory and CPU requirements for device
generating flows.
• Depends on application. Traffic engineering
vs. intrusion detection.
19. Working with Flows
Generating Process
• Accounting information accumulated
with flows: Packets, Bytes, Start
Timestamp, End Timestamp.
• Network routing information – masks
and autonomous system (AS) number.
20. Working with Flows
Generating Process
• Passive monitor (transparent bridge)
– A passive monitor (usually a unix host)
receives all data, and generates flows.
– Resource intensive, newer investments
needed.
• Router or other existing network device
– Router or other existing devices like switch,
generate flows.
– Sampling is possible.
21. Working with Flows
Generating Process
Workstation A Workstation B
Flow probe connected
to switch port in Internet
“ traffic mirror” mode
22. Passive Monitor
• Directly connected to a LAN segment via a
switch port in “mirror” mode, optical splitter,
or repeated segment
• Generate flows for all local LAN traffic
• Must have an interface or monitor
deployed on each LAN segment
• More difficult to deploy, in general
• Support for more detailed flows –
bidirectional and application
23. Working with Flows
Generating Process
LAN LAN
LAN
LAN
Internet
Flow collector
stores exported flows from router.
24. Router Collection
• Router will generate flows for traffic
that is directed to the router
• In general, flows are not generated for
local LAN traffic
• Limited to “simple” flow criteria (packet
headers)
• Generally easier to deploy – no new
equipment
28. Working with Flows
Exporting Process
• Conditions to export:
– Inactive flows (no more packets received)
– Long flows (flow lifetime)
– Flow ends (for example, RST or FIN in TCP
communication)
– Internal contrains (for example, full cache or
low memory)
29. Working with Flows
Collecting Process
• Decoding using templates
– If the Template Records have not still been
received, the corresponding Flow Data
Records should store till to receive that
template
– Template Records life time is limited
32. Cisco NetFlow
• Most popular
• In the beginning, unidirectional flows
• IPv4 unicast and multicast
• Aggregated and unaggregated flows
• Flows exported via UDP Datagram
• Supported on IOS and CatOS platforms
• Catalyst NetFlow is different implementation
33. Cisco NetFlow Versions
• 4 Unaggregated types (1,5,6,7)
• 14 Aggregated types (8.x, 9)
• The version defines what type of data is in
the flow
• Each version has its own packet format:
Export packet
• Version 1 does not have sequence numbers
– no way to detect lost flows
• Some versions specific to Catalyst platform
35. NetFlow v5
• Key Fields: Src/Dst IP, Src/Dst Port,
IP Protocol, ToS, Input interface
• Accounting (Non-Key Fields):
Packets, Bytes, Start/End timestamp,
Output interface
• Other: Bitwise OR of TCP flags,
Source/Destination AS and IP Mask
• Packet format adds sequence
numbers for detecting lost exports
36. NetFlow v5
●
Packet format adds sequence
numbers for detecting lost exports:
Flow Sequence
●
Only support for Ipv4 and
unidirectional flows
• Not support for MPLS/VLAN
38. NetFlow v5
Export Packet (Header)
struct ftpdu_v5 {
/* 24 byte header */
u_int16 version; /* 5 */
u_int16 count; /* The number of records in the PDU */
u_int32 sysUpTime; /* Current time in millisecs since router booted */
u_int32 unix_secs; /* Current seconds since 0000 UTC 1970 */
u_int32 unix_nsecs; /* Residual nanoseconds since 0000 UTC 1970 */
u_int32 flow_sequence; /* Seq counter of total flows seen */
u_int8 engine_type; /* Type of flow switching engine (RP,VIP,etc.) */
u_int8 engine_id; /* Slot number of the flow switching engine */
u_int16 reserved;
39. NetFlow v5
Export Packet (Records)
/* 48 byte payload */
struct ftrec_v5 {
u_int32 srcaddr; /* Source IP Address */
u_int32 dstaddr; /* Destination IP Address */
u_int32 nexthop; /* Next hop router's IP Address */
u_int16 input; /* Input interface index */
u_int16 output; /* Output interface index */
u_int32 dPkts; /* Packets sent in Duration */
u_int32 dOctets; /* Octets sent in Duration. */
u_int32 First; /* SysUptime at start of flow */
u_int32 Last; /* and of last packet of flow */
u_int16 srcport; /* TCP/UDP source port number or equivalent */
u_int16 dstport; /* TCP/UDP destination port number or equiv */
u_int8 pad;
u_int8 tcp_flags; /* Cumulative OR of tcp flags */
u_int8 prot; /* IP protocol, e.g., 6=TCP, 17=UDP, ... */
u_int8 tos; /* IP Type-of-Service */
u_int16 src_as; /* originating AS of source address */
u_int16 dst_as; /* originating AS of destination address */
u_int8 src_mask; /* source address prefix mask bits */
u_int8 dst_mask; /* destination address prefix mask bits */
u_int16 drops;
} records[FT_PDU_V5_MAXFLOWS];
};
40. NetFlow v8
• Aggregated v5 flows
• Not all flow types available on all
equipments
• Much less data to post process, but
loses fine granularity of v5 – no IP
addresses
41. NetFlow v8
Export Packet Example
(AS Aggregation)
IP/UDP packet
NetFlow
v8 header
v8 record
…
…
v8 record
42. NetFlow v8 AS agg.
Export Packet
struct ftpdu_v8_1 {
/* 28 byte header */
u_int16 version; /* 8 */
u_int16 count; /* The number of records in the PDU */
u_int32 sysUpTime; /* Current time in millisecs since router booted */
u_int32 unix_secs; /* Current seconds since 0000 UTC 1970 */
u_int32 unix_nsecs; /* Residual nanoseconds since 0000 UTC 1970 */
u_int32 flow_sequence; /* Seq counter of total flows seen */
u_int8 engine_type; /* Type of flow switching engine (RP,VIP,etc.) */
u_int8 engine_id; /* Slot number of the flow switching engine */
u_int8 aggregation; /* Aggregation method being used */
u_int8 agg_version; /* Version of the aggregation export */
u_int32 reserved;
/* 28 byte payload */
struct ftrec_v8_1 {
u_int32 dFlows; /* Number of flows */
u_int32 dPkts; /* Packets sent in duration */
u_int32 dOctets; /* Octets sent in duration */
u_int32 First; /* SysUpTime at start of flow */
u_int32 Last; /* and of last packet of flow */
u_int16 src_as; /* originating AS of source address */
u_int16 dst_as; /* originating AS of destination address */
u_int16 input; /* input interface index */
u_int16 output; /* output interface index */
} records[FT_PDU_V8_1_MAXFLOWS];
};
43. NetFlow v9
• Flow format defined by user: Templates
– Template descriptions are communicated from the
router to the collector as well
• Flow records are sent from the router to the
NetFlow Collector, with minimal template
information, so that the NetFlow Collector can
relate the records to the appropriate template
• Version 9 is independent of the underlying
transport (UDP, TCP, SCTP, and so on)
46. NetFlow v9
Information Model
• FlowSet
– Collection of Flow Records with similar structure
– Identified by FlowS et Id (0-65535)
– Flow Set Id, with value 0, for Template FlowS et
– Flow Set Id, with value 1, for Options Template
FlowS et
– Flow Set Id, with values greater than 255, for Data
FlowS et
47. NetFlow v9
Information Model
• Template FlowSet
– One or more Template Records
• Options Template FlowSet
– One or more Options Template Records
• Data FlowSet
– One or more Flow Data Records or Options
Data Records
48. NetFlow v9
Information Model
• Template Record
– Defines the fields -type, length- in a Flow Data Record
• Flow Data Record
– Record which contains real information values
• Options Template Record
– Defines te fields -type, length, scope- in a Options Data
Record
• Options Data Record
– Record which contains configuration supply information -or
for any another specific process- values and scope
51. IPFIX
• ITEF Standard, using Cisco NetFlow v9 as
base
• Flow format defined by user: Templates
• No directly interoperable with NetFlow v9:
– FlowSet renamed to Set
– Count is now Length
– Sysuptime is removed, and exporttime is included
52. IPFIX
• Based on biflows:
– Data reduction (bandwidth-saving)
– Separation of “answered” traffic from
“unanswered”
– Full reconstruction of TCP sessions
53. IPFIX
Information Model
• Set
– Identified by S et Id (0-65535)
– Set Id, with values 0 and 1, not in us e
– Set Id, with value 2, for Te mplate S e t
– Set Id, with value 3, for Options Template S et
– Set Id, with values between 4 and 255, res erved
– Set Id, with values greater than 255, for Data S ets
58. Uses for Flow
• Identification/Solving
– Traffic classification
– Flow-based Detection
– DoS Traceback
• Traffic Analysis
– Inter-AS traffic analysis
– Reporting on application proxies
• Accounting
– Cross verification from other sources
– Can cross-check with SNMP data
59. Identification/Solving
Traffic Classification
• Using DPI, based on: Protocol, source
and destination ports
– Protocol identification (TCP, UDP, ICMP)
– Can define well known ports
– Can identify well known P2P ports
– Most common use
• Proxy measurement - http , ftp
• Rate limiting P2P traffic
60. Identification/Solving
Flow-based Detection
• Monitor flows (i.e., Network and Transport
Layer transactions) on the network and build
baselines for what normal behavior looks
like:
– Per interface
– Per prefix
– Per Transport Layer protocol & ports
– Build time-based buckets (e.g., 5 minutes, 30
minutes, 1 hours, 12 hours, day of week, day of
month, day of year)
61. Identification/Solving
Flow-based Detection
• Once baselines are built anomalous activity can be
detected
– Pure rate-based (pps or bps) anomalies may be
legitimate or malicious
– Many misuse attacks can be immediately recognized,
even without baselines (e.g., TCP SYN or RST floods)
– Signatures can also be defined to identify “interesting”
transactional data (e.g., proto udp and port 1434 and 404
octets (376 payload) == slammer!)
– Temporal compound signatures can be defined to detect
with higher precision
62. Identification/Solving
Traceback: Flow-based
• Trace attack by matching fingerprint/signature
at each interface via passive monitoring:
– Flow data (e.g., NetFlow, cflowd, sFlow, IPFIX)
– Span Data
– PSAMP (Packet Sampling, IETF PSAMP WG)
• Number of open source and commercial
products evolving in market
• Non-intrusive, widely supported
68. Traffic Analysis
• Can see traffic based on source and
destination AS
– Source and destination AS derived
through the routing table on the router
– Introduces the need to run full mesh BGP
at IXPs as well as transit and peering
– Source and destination prefix based flows
can be collected and plotted against
external prefix to ASN data
