Self-Configuring Network Traffic Generation 03/18/2008 Linchew

Outline Abstract (2) Introduction (2) Architecture (6) Self-Configuration (2) Traffic Generation (3) Limitation (1) Conclusion (2)

Abstract (2)

Introduction (2)

Architecture (6)

Self-Configuration (2)

Traffic Generation (3)

Limitation (1)

Conclusion (2)

Abstract Generate repeatable, realistic network traffic is critical in both simulation and testbed environments. “ Harpoon” , a new application-independent tool for generating representative packet traffic at the IP flow level. Harpoon generates TCP and UDP packet flows that have the same byte, packet, temporal and spatial characteristics as measured at routers in live environment.

Generate repeatable, realistic network traffic is critical in both simulation and testbed environments.

“ Harpoon” , a new application-independent tool for generating representative packet traffic at the IP flow level.

Harpoon generates TCP and UDP packet flows that have the same byte, packet, temporal and spatial characteristics as measured at routers in live environment.

Abstract (cons.) Harpoon is distinguished from other tools that generate statistically representative traffic in that it can self-configure by automatically extracting parameters from standard Netflow logs or packet traces.

Harpoon is distinguished from other tools that generate statistically representative traffic in that it can self-configure by automatically extracting parameters from standard Netflow logs or packet traces.

Introduction To evaluate new algorithms, systems and protocols, using tools that (1) create a range of test conditions similar to those experienced in live deployment. (2) ensure reproducible results. So, Having appropriate tools for generating scalable, tunable and representative network traffic is therefore of fundamental importance.

To evaluate new algorithms, systems and protocols, using tools that

(1) create a range of test conditions similar to those experienced in live deployment.

(2) ensure reproducible results.

So, Having appropriate tools for generating scalable, tunable and representative network traffic is therefore of fundamental importance.

Introduction (cons.) Current best practices for traffic generation have focused on either simple packet streams or recreation of a single application-specific behavior . But, they lack nearly all of the richness and diversity of packet streams in the live Internet. Contribution of this paper is: Description and evaluation of a new network traffic generator capable of recreating IP traffic flows representative of those observed at routers in the Internet.

Current best practices for traffic generation have focused on either simple packet streams or recreation of a single application-specific behavior .

But, they lack nearly all of the richness and diversity of packet streams in the live Internet.

Contribution of this paper is:

Description and evaluation of a new network traffic generator capable of recreating IP traffic flows representative of those observed at routers in the Internet.

Architecture Design objectives of Harpoon are (1) to scalably generate application-independent network traffic at the IP flow level. (2) to be easily parameterized to create traffic that is statistically identical to traffic measured at a given vantage point in the Internet.

Design objectives of Harpoon are

(1) to scalably generate application-independent network traffic at the IP flow level.

(2) to be easily parameterized to create traffic that is statistically identical to traffic measured at a given vantage point in the Internet.

Architecture (cons.) Harpoon’s architecture begins with the notion of unicast file transfers using either TCP or UDP. And, Harpoon has two components: Client threads that make file transfer request Server threads that transfer the requested files using TCP or UDP. The result is byte/packet/flow traffic at the first hop router that is qualitatively the same as the traffic used to produce input parameters.

Harpoon’s architecture begins with the notion of unicast file transfers using either TCP or UDP.

And, Harpoon has two components:

Client threads that make file transfer request

Server threads that transfer the requested files using TCP or UDP.

The result is byte/packet/flow traffic at the first hop router that is qualitatively the same as the traffic used to produce input parameters.

Architecture (cons.) Modeling UDP, Harpoon contents 3 models: Constant bit rate Periodic ping-pong Exponentially ping-pong Modeling TCP, Harpoon contents 5 models: File size Inter-connection time Source IP ranges Destination IP ranges Number of active sessions

Modeling UDP, Harpoon contents 3 models:

Constant bit rate

Periodic ping-pong

Exponentially ping-pong

Modeling TCP, Harpoon contents 5 models:

File size

Inter-connection time

Source IP ranges

Destination IP ranges

Number of active sessions

Architecture (cons.) Harpoon flow model is a two level architecture. Lower: Connection level Size of the file transferred. Time interval between consecutive file transfer requests, the inter-connection time. Upper: Session level(TCP or UDP types) The number of active session. IP spatial distribution, source and destination IP ranges.

Harpoon flow model is a two level architecture.

Lower: Connection level

Size of the file transferred.

Time interval between consecutive file transfer requests, the inter-connection time.

Upper: Session level(TCP or UDP types)

The number of active session.

IP spatial distribution, source and destination IP ranges.

Architecture (cons.)

Architecture (cons.) Each of these distributions can be specified manually or extracted from packet traces or Netflow data collected at a live router. These models enable the workload generated by Harpoon to be application independent or to be tuned to a specific application .

Each of these distributions can be specified manually or extracted from packet traces or Netflow data collected at a live router.

These models enable the workload generated by Harpoon to be application independent or to be tuned to a specific application .

Self-Configuration A key feature of Harpoon is that it is self-configuring. Netflow logs or packet traces are used for parameterization without any intermediate modeling step. The self-configuration phase of Harpoon takes flow records as input and generate the necessary parameters for traffic generation.

A key feature of Harpoon is that it is self-configuring. Netflow logs or packet traces are used for parameterization without any intermediate modeling step.

The self-configuration phase of Harpoon takes flow records as input and generate the necessary parameters for traffic generation.

Self-Configuration (cons.) Key parameters are (1)file sizes (2)inter-connection time (3) source and destination IP addresses (4)number of active sessions. Each input flow records are divided into a series of intervals of equal duration to generate the number of active sessions in order to match average byte, packet, and flow volumes of the original flow or packet data over each interval.

Key parameters are

(1)file sizes (2)inter-connection time (3) source and destination IP addresses (4)number of active sessions.

Each input flow records are divided into a series of intervals of equal duration to generate the number of active sessions in order to match average byte, packet, and flow volumes of the original flow or packet data over each interval.

Traffic Generation Harpoon is implemented as a client-server application. Client process consists of threads that generate file requests. Server process consists of threads that service client requests. Also server controls the sizes of files transferred according to the input distribution. In each thread, connections are initiated according to the inter-connection time distribution which is dictated by the dynamics of the transport protocol and the underlying testbed configuration.

Harpoon is implemented as a client-server application.

Client process consists of threads that generate file requests.

Server process consists of threads that service client requests. Also server controls the sizes of files transferred according to the input distribution.

In each thread, connections are initiated according to the inter-connection time distribution which is dictated by the dynamics of the transport protocol and the underlying testbed configuration.

Traffic Generation (cons.) TCP file transfers are controlled by protocol dynamics. UDP dictates no control over packet emissions. For constant bit rates : roughly constant rates. For Periodic and exponential ping-pong : A TCP control connection with server. Sending information(such as port number, datagram size, rates, unique identifier, etc) to deliver files. A UDP connection for transfer requests and requested files.

TCP file transfers are controlled by protocol dynamics.

UDP dictates no control over packet emissions.

For constant bit rates : roughly constant rates.

For Periodic and exponential ping-pong :

A TCP control connection with server. Sending information(such as port number, datagram size, rates, unique identifier, etc) to deliver files.

A UDP connection for transfer requests and requested files.

Traffic Generation (cons.) Also, Harpoon with following functions Auto picking a IPv4 addresses from address pools. Modular fashion to enable relatively easy addition of new traffic models. Management of resources and tools with HTTP server along with an XML parser to enable remote management via XML RPC.

Also, Harpoon with following functions

Auto picking a IPv4 addresses from address pools.

Modular fashion to enable relatively easy addition of new traffic models.

Management of resources and tools with HTTP server along with an XML parser to enable remote management via XML RPC.

Limitation Two limitations to Harpoon’s traffic model are: It is designed to match byte, packet, and flow volumes over relatively coarse intervals and may not match over shorter intervals. Packet-level dynamics are not specified, traffic produced by Harpoon may not match other metrics of interest , such as scaling characteristics, queue length distribution for the first-hop router, packet loss process, and flow durations.

Two limitations to Harpoon’s traffic model are:

It is designed to match byte, packet, and flow volumes over relatively coarse intervals and may not match over shorter intervals.

Packet-level dynamics are not specified, traffic produced by Harpoon may not match other metrics of interest , such as scaling characteristics, queue length distribution for the first-hop router, packet loss process, and flow durations.

Conclusion Harpoon is a new tool for generating representative IP traffic based on eight distributional characteristics of TCP and UDP flows. Parameters for these distributions can be automatically extracted from NetFlow data collected form a live router. It is implemented as a client-server application that can be used in testbed environments.

Harpoon is a new tool for generating representative IP traffic based on eight distributional characteristics of TCP and UDP flows.

Parameters for these distributions can be automatically extracted from NetFlow data collected form a live router.

It is implemented as a client-server application that can be used in testbed environments.

Conclusion (cons.) The author parameterized Harpoon using data collected from a NetFlow trace and from a set of packet traces and verified in controlled laboratory tests that Harpoon generates traffic that is qualitatively the same as the input data.

The author parameterized Harpoon using data collected from a NetFlow trace and from a set of packet traces and verified in controlled laboratory tests that Harpoon generates traffic that is qualitatively the same as the input data.

Thanks for listening

Thanks for listening