Introduction to XDP and how to setup the virtual machine for testing

1. XDP: A High Performance
Network Data Path
SUSE Labs
Software Engineer
Gary Lin

2. eBPF / Networking

3. userspace
kernel
RX Packet Processing
Driver
Network
Stack
NIC
Network
Program

4. DDoS

5. userspace
kernel
Driver
Network
Stack
NIC
Network
Program

6. Firewall

7. userspace
kernel
iptables/netfilter
DriverNIC
Network
Program
Network Stack
netfilter
DROP

8. Traffic Control

9. userspace
kernel
cls_bpf (tc ingress)
Driver
Network
Stack
NIC
Network
Program
eBPF
DROP FORWARD

10. eXpress Data Path

11. userspace
kernel
XDP
Network
Stack
NIC
Network
Program
Driver
skb
alloceBPF
DROPTX

12. XDP
●
A high performance, programmable network
data path
●
No specialized hardware
●
No kernel bypass
●
Works with the existing network stack
●
Direct packet write

13. XDP Actions
●
XDP_ABORTED
eBPF program error, treat like DROP
●
XDP_DROP
Drop the packet
●
XDP_PASS
Pass the packet up to the stack
●
XDP_TX
Transmit packet out the same interface

14. #define KBUILD_MODNAME "foo" /*for some headers*/
#include <uapi/linux/bpf.h>
...
SEC("xdp_prog")
int xdp_prog(struct xdp_md *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
void *data = (void *)(long)ctx->data;
struct ethhdr *eth = data;
...
return XDP_DROP; /* the action */
}

15. XDP Restrictions
●
One packet buffer per 4K page
– Memory waste
– Needs the change of the memory model of the
driver
●
No per-RX-queue XDP instance
●
No transmit on other interface

16. Current Status (4.11)
●
XDP core: 4.8
●
Supported Drivers
– mlx4: 4.8
– mlx5: 4.9
– nfp, qed, virtio_net: 4.10
– bnxt_en: 4.11
●
Not upstream
– i40e, generic_xdp, ixgbe

17. XDP Benchmarks (mlx4)
●
Generated using pktgen
●
Single core
– ip routing drop: ~3.6 Mpps
– tc clsact using bpf: ~4.2 Mpps
– XDP drop: 20 Mpps (<10 % cpu util)
https://www.slideshare.net/IOVisor/express-data-path-linux-meetup-santa-clara-july-2016

18. XDP Benchmarks (virtio-net)
●
Generated using pktgen
●
Host: i7-4790 CPU @ 3.60GHz
●
Single core qemu guest
– iptables drop (raw preroute): 2.87 Mpps
– tc clsact using bpf: 3.05 Mpps
– XDP drop: 3.45 Mpps

19. virtio-net

20. static int virtnet_xdp_set(struct net_device *dev,
struct bpf_prog *prog)
{
...
if (dev->mtu > max_sz) {
netdev_warn(dev, "XDP requires MTU less than %lun",
max_sz);
return -EINVAL;
}
...
if (prog) {
prog = bpf_prog_add(prog, vi->max_queue_pairs - 1);
if (IS_ERR(prog))
return PTR_ERR(prog);
}
...
for (i = 0; i < vi->max_queue_pairs; i++) {
old_prog = rtnl_dereference(vi->rq[i].xdp_prog);
rcu_assign_pointer(vi->rq[i].xdp_prog, prog);
if (old_prog)
bpf_prog_put(old_prog);
}
return 0;
}

21. static struct sk_buff *receive_mergeable(struct net_device *dev,
...
unsigned int len)
{
...
rcu_read_lock();
xdp_prog = rcu_dereference(rq->xdp_prog);
if (xdp_prog) {
...
act = bpf_prog_run_xdp(xdp_prog, &xdp);
switch (act) {
case XDP_PASS:
...
case XDP_TX:
...
default:
...
case XDP_ABORTED:
...
case XDP_DROP:
...
}
}
rcu_read_unlock();
...
head_skb = page_to_skb(vi, rq, page, offset, len, truesize);

22. Test XDP with virtio-net
QEMU
tap0
192.168.100.1
pktgen
Host
virtio-net
192.168.100.2
XDP

23. Setup the host
●
Create the multiqueue tap
# ip tuntap add mode tap multi_queue
user <user_name> name tap0
●
Bring up tap0
# ip addr add 192.168.100.1 dev tap0
# ip link set tap0 up
●
Change the permission of /dev/vhost-net
# chmod 666 /dev/vhost-net

24. Setup the guest
●
Enable multiqueue for virtio-net (for 1 CPU)
-netdev tap,id=hn1,ifname=tap0,script=no,
downscript=no,vhost=on,queues=2
-device virtio-net-pci,netdev=hn1,mq=on,
vectors=6
●
Disable LRO support
-device guest_tso4=off,guest_tso6=off,
guest_ecn=off,guest_ufo=off

25. Start the guest
$ qemu-system-x86_64
-smp 1
...
-netdev tap,id=hn1,ifname=tap0,script=no,
downscript=no,vhost=on,queues=2
-device virtio-net-pci,netdev=hn1,mq=on,
vectors=6,guest_tso4=off,guest_tso6=off,
guest_ecn=off,guest_ufo=off

26. Test XDP
●
Install bcc (>= 0.3.0)
# zypper in python-bcc
●
Run xdp_drop_count.py*
# python xdp_drop_count.py <ifdev>
* https://github.com/iovisor/bcc/blob/master/examples/networking/xdp/xdp_drop_count.py

27. Some useful bpf map types

28. Program Array
PROG_ARRAY
0 fd0
2 fd2
3 fd3
XDP Program 2
XDP Program 3
XDP Program 0
XDP Program
Main
bpf_tail_call()

29. Program Array in BCC
array.c:
BPF_TABLE("prog", int, int, ptable, 4);
int func3(struct xdp_md *ctx) { ...
int my_main(struct xdp_md *ctx) {
...
ptable.call(ctx, 3);
...
array.py:
bpf = BPF(src_file="array.c")
func3 = bpf.load_func("func3", BPF.XDP)
ptable[c_int(3)] = c_int(func3.fd)

30. LRU Hash
ref
Active List
Inactive List
Free List
Pre-allocated Hash
bpf_lookup_elem()

31. LRU Hash in BCC
lru.c:
BPF_TABLE("lru_hash", uint_32, int, lru, 1024);
int my_main(struct xdp_md *ctx) {
...
lru.lookup(&key);
...
lru.py:
bpf = BPF(src_file="lru.c")
lru[c_uint(9527)] = c_int(1)

32. XDP
Monitor
XDP
DDoS
XDP
Load Balancer
blacklist
Analyzer update
packets
DROP
Possible Scenario
TX
PASS

33. 33
Questions?

34. 34
Thank You!

36. References
●
XDP project
https://www.iovisor.org/technology/xdp
●
eXpress Data Path: Getting Linux to 20Mpps
https://www.slideshare.net/IOVisor/express-data-path-linux-meetup-
santa-clara-july-2016?qid=247820dc-7ac0-494e-8e6c-3a21daeda22a&v=&
b=&from_search=5
●
XDP in practice: integrating XDP in our DDoS mitigation
pipeline
https://www.netdevconf.org/2.1/session.html?bertin
●
Droplet: DDoS countermeasures powered by BPF + XDP
https://www.netdevconf.org/2.1/session.html?zhou