The document describes an adaptive traffic sampling and management platform that aims to: 1) Provide an emulation platform for network-wide traffic sampling and monitoring that allows users to control experiments by tuning parameters like traffic sources, network topology, and sampling rates. 2) Enable experiments to be easily reproduced. 3) Allow developed solutions to be tested in real environments. 4) Be extensible and scale to large scenarios by changing sampling algorithms or adding advanced monitoring methods.

1. Adaptive Traffic Sampling and Management
Platform

2. Motivation && Objectives
 Motivation:
 The network research community is facing difficulties in both
getting widely collected traces of real traffic and having
control of the different real measurement points.
 Objectives: Providing an emulation platform for network wide traffic
sampling and monitoring:
 Users can tune and control the different parameters of their
experiments (traffic trace source, the network topology, and
the sampling rate in monitors, …).
 Users can easily reproduce their experiments.
 Developed solutions can be easily exported and tested in a real
environment.
 The platform is easily extensible and can scale for large
scenarios (change the sampling algorithm, add advanced
monitoring methods, …).

3. Traffic Emulation
Service
Traffic Monitoring
and Sampling
Service
Data Collection and
Analysis Service
Real TcpDump
Binary Trace Files
NetFlow
Reports
Control
Messages
- New sampling rate
- Monitor Address
Per-Monitor
TcpDump
Headers
Real
Node
Interface
Network Topology:
- List of ASs
- Transit Network
Topology(routers +
links)
- Selected Monitors,
etc
2
Our Platform
 Starting from a set of collected real traces (either from a single point during different
periods of time or from different points) and a given network topology, is it possible to
play real internet traffic (not synthetic) within the given topology while providing remote-
controllable traffic monitoring and sampling capabilities for each router of the described
topology ?
1
ME MLE

4. NetFlow Reports Details
 It is the Cisco Netflow(tm) version 5 packet format based on:
http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/nfc/nfc_3_0/nfc_ug/nfcform.htm
 NetFlow Reports packet format:
[NF5_HEADER, NF5_FLOW, NF5_FLOW, NF5_FLOW, …]
 NF5_HEADER resumes a description of the set of flows to report.
 struct NF5_HEADER {
u_int16_t version, flows *;
u_int32_t uptime_ms, time_sec, time_nanosec, flow_sequence;
u_int8_t engine_type, engine_id, reserved1, reserved2;
// Added to support the Traffic Emulation Service
u_int32_t emulator_monitor_adr *;
};

5. NetFlow Reports Details
 struct NF5_FLOW {
u_int32_t src_ip *, dest_ip *, nexthop_ip;
u_int16_t if_index_in, if_index_out;
u_int32_t flow_packets *, flow_octets *;
u_int32_t flow_start *, flow_finish *;
u_int16_t src_port *, dest_port *;
u_int8_t pad1;
u_int8_t tcp_flags *, protocol *, tos;
u_int32_t src_as *, dest_as *;
u_int8_t src_mask *, dst_mask *;
u_int16_t pad2;
};
 NF5_FLOW describes a flow to report.

6. AS - 3
AS - 2
AS - 1
 Input: TcpDump binary file + XML Configuration file.
 Link’ characteristics: Capacity, Delay, Cost
 AS’ characteristics: Weight, Edge router,
 Router’ characteristics: Interfaces, Interfaces’ monitoring status, Links, …
Monitor
Router
Autonomous
System
Transit Network
Link
Example of an Emulated Topology
 We provide as much details as it is needed to
re-produce the closest possible topology to the real one. An example of XML schema
is available at http://planete.inria.fr/GEANT.

7. Real TcpDump
Binary Trace Files
Xml File Describing
the Topology of the
Network
Input
Input
Per-Monitor
TcpDump
Headers
Traffic Loader
Module
Traffic Dispatching
Module
Data Forwarding
Module
Output
Emulation Module
Zoom on the Traffic Emulation Service
 Dispatching algorithm: weighted random based on one hand on the list of weights
that the user associates to the different ASes and in another hand on the prefix length
specified by the user.
 The four modules run in parallel so as to minimize the traffic emulation service’
memory footprint, enable it to scale to big topologies (example: GEANT’s backbone
topology) and to prevent the introduction of latency while scheduling packets.
 Routing Algorithm: the shortest path routing protocol based on the Dijkstra algorithm.
Routes are static and can be set up via the XML configuration file.

8. Pcap Library
Real Node
Interface
Per-Monitor
TcpDump
Headers
Input
1 2
Control
Messages
Input
Virtual Interface
Module
NetFlow Flow Constructer and Exporter
Module NetFlow
Reports
Output
Sampling
Algorithm
Sampling Module
Sampling Rate
Control Server
Zoom on the Traffic Monitoring and Sampling
Service
 Sampling algorithm: If a user chooses a sampling rate of A/B (A packets among B packets,
A < B, B > 0, A >= 0), then every B packets, the sampling module generates randomly a set S
of A numbers within the interval [1, B]. Packets with numbers outside the set S are rejected
and only the remaining packets are considered for 5-tuple flow construction.
 Sampling Rate Control server:
Enables to remotely control the
sampling rate.
 Flow Constructor and Exporter:
constructs 5-tuple flows starting from the
sampled packets, merges them within
NetFlow reports and sends them to the
Collector.

9. Remote Monitors Controller Module
Control
Messages
Input NetFlow Reports
Receiver Module
Flows’ Statistics
Generator and
Analyzer Module
Statistics
Trace File
Output
Output
NetFlow
Reports Decisions
Zoom on the Data Collection and Analysis Service
 NetFlow reports receiver module: receives and decode the reports sent by the
traffic monitoring and sampling service.
 Flows’ statistics generator and analyzer module: calculates statistics stating from
the collected flows (number of packets/flow, etc) and could takes the decision to
change the sampling rate in a given monitor in order optimize some measurements.
 Remote monitor controller module: sends control messages (new sampling rate,
monitor address) to the sampling rate control server within the traffic monitoring
and sampling service in order to change the monitor’ sampling rate.

10. Experimentation scenario
 Reproduced Topology: the GEANT’s backbone topology.
 Real Traffic Traces: the MAWI Working Group Traffic Archive.
 Dispatching Algorithm:
 ASes’ Weights: function of the capacities of the links connecting ASes to their edge
routers.

11. Validation Results
(the effectiveness of the traffic emulation service and especially of its
dispatching mechanism)
Does each AS generate as much traffic as the importance of the weight associated to it?
The resulting curves remain linear for different prefix lengths. So,
we conclude that our emulator dispatches the prefixes to ASes
without any bias.

12. Validation Results
Number of generated flows by different ASes
Having different weights
 The number of generated flows
scale with the AS weights but do
not fit a perfect line. Nevertheless,
the fitting improves when the prefix
granularity becomes finer.
 This indeed comes from the
presence of very large prefixes of
different volumes that, when
dispatched over the ASes, cause
such deviations in the traffic; the
coarser the prefix the more
important this phenomenon.

13. Validation Results
Number of ingoing/outgoing packets for ASes
Having different weights

14. Some Validation Results
 Clearly, there is a power-law behavior leading to very large prefixes compared
to the average prefix size (these are servers, heavy users, etc).