This presentation features a walk through the Linux kernel networking stack for users and developers. It will cover insights into both, existing essential networking features and recent developments and will show how to use them properly. Our starting point is the network card driver as it feeds a packet into the stack. We will follow the packet as it traverses through various subsystems such as packet filtering, routing, protocol stacks, and the socket layer. We will pause here and there to look into concepts such as networking namespaces, segmentation offloading, TCP small queues, and low latency polling and will discuss how to configure them.
How does a packet get in and out of
the Network Stack?
Receive & Transmit Process
L2 & IP
NIC Network Stack
The 3 ways into the Network Stack
NAPI based Polling poll()
Ring Buffer Network
Busy Polling busy_poll()
RSS – Receive Side Scaling
● NIC distributes packets across multiple RX queues
allowing for parallel processing.
● Separate IRQ per RX queue, thus selects CPU to run
hardware interrupt handler on.
RPS – Receive Packet Steering
● Software filter to select CPU # for processing
● Use it to ...
... redo queue - CPU mapping ... distribute single queue to
● RX/TX Checksumming
● Perform CPU intensive checksumming in
● Virtual LAN filtering and tag stripping
● Strip 802.1Q header and store VLAN ID
in network packet meta data.
● Filter out unsubscribed VLANs.
● Segmentation Offload
Generic Receive Offload
(ethtool -K eth0 gro on)
NAPI based GRO
Up to 64K
It's more effective to process 1x64K bytes packet
instead of 40x1500 bytes packets.
(ethtool -K eth0 tso on)
(ethtool -K eth0 gso on)
Generic Segmentation Offload (GSO)
ethtool -K eth0 gso on
TCP Segmentation Offload (TSO)
ethtool -K eth0 tso on
Up to 64K
How does a packet get through the
(c) Karen Sagovac
Handler Route Lookup
TCP Fast Open
2x RTT2x RTT