Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
XDP: A High Performance
Network Data Path
SUSE Labs
Software Engineer
Gary Lin
eBPF / Networking
userspace
kernel
RX Packet Processing
Driver
Network
Stack
NIC
Network
Program
DDoS
userspace
kernel
Driver
Network
Stack
NIC
Network
Program
Firewall
userspace
kernel
iptables/netfilter
DriverNIC
Network
Program
Network Stack
netfilter
DROP
Traffic Control
userspace
kernel
cls_bpf (tc ingress)
Driver
Network
Stack
NIC
Network
Program
eBPF
DROP FORWARD
eXpress Data Path
userspace
kernel
XDP
Network
Stack
NIC
Network
Program
Driver
skb
alloceBPF
DROPTX
XDP
●
A high performance, programmable network
data path
●
No specialized hardware
●
No kernel bypass
●
Works with the exi...
XDP Actions
●
XDP_ABORTED
eBPF program error, treat like DROP
●
XDP_DROP
Drop the packet
●
XDP_PASS
Pass the packet up to ...
#define KBUILD_MODNAME "foo" /*for some headers*/
#include <uapi/linux/bpf.h>
...
SEC("xdp_prog")
int xdp_prog(struct xdp_...
XDP Restrictions
●
One packet buffer per 4K page
– Memory waste
– Needs the change of the memory model of the
driver
●
No ...
Current Status (4.11)
●
XDP core: 4.8
●
Supported Drivers
– mlx4: 4.8
– mlx5: 4.9
– nfp, qed, virtio_net: 4.10
– bnxt_en: ...
XDP Benchmarks (mlx4)
●
Generated using pktgen
●
Single core
– ip routing drop: ~3.6 Mpps
– tc clsact using bpf: ~4.2 Mpps...
XDP Benchmarks (virtio-net)
●
Generated using pktgen
●
Host: i7-4790 CPU @ 3.60GHz
●
Single core qemu guest
– iptables dro...
virtio-net
static int virtnet_xdp_set(struct net_device *dev,
struct bpf_prog *prog)
{
...
if (dev->mtu > max_sz) {
netdev_warn(dev, ...
static struct sk_buff *receive_mergeable(struct net_device *dev,
...
unsigned int len)
{
...
rcu_read_lock();
xdp_prog = r...
Test XDP with virtio-net
QEMU
tap0
192.168.100.1
pktgen
Host
virtio-net
192.168.100.2
XDP
Setup the host
●
Create the multiqueue tap
# ip tuntap add mode tap multi_queue 
user <user_name> name tap0
●
Bring up tap...
Setup the guest
●
Enable multiqueue for virtio-net (for 1 CPU)
-netdev tap,id=hn1,ifname=tap0,script=no,
downscript=no,vho...
Start the guest
$ qemu-system-x86_64 
-smp 1 
...
-netdev tap,id=hn1,ifname=tap0,script=no,
downscript=no,vhost=on,queues=...
Test XDP
●
Install bcc (>= 0.3.0)
# zypper in python-bcc
●
Run xdp_drop_count.py*
# python xdp_drop_count.py <ifdev>
* htt...
Some useful bpf map types
Program Array
PROG_ARRAY
0 fd0
2 fd2
3 fd3
XDP Program 2
XDP Program 3
XDP Program 0
XDP Program
Main
bpf_tail_call()
Program Array in BCC
array.c:
BPF_TABLE("prog", int, int, ptable, 4);
int func3(struct xdp_md *ctx) { ...
int my_main(stru...
LRU Hash
ref
Active List
Inactive List
Free List
Pre-allocated Hash
bpf_lookup_elem()
LRU Hash in BCC
lru.c:
BPF_TABLE("lru_hash", uint_32, int, lru, 1024);
int my_main(struct xdp_md *ctx) {
...
lru.lookup(&k...
XDP
Monitor
XDP
DDoS
XDP
Load Balancer
blacklist
Analyzer update
packets
DROP
Possible Scenario
TX
PASS
33
Questions?
34
Thank You!
References
●
XDP project
https://www.iovisor.org/technology/xdp
●
eXpress Data Path: Getting Linux to 20Mpps
https://www.s...
Xdp and ebpf_maps
Upcoming SlideShare
Loading in …5
×

Xdp and ebpf_maps

Introduction to XDP and how to setup the virtual machine for testing

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Xdp and ebpf_maps

  1. 1. XDP: A High Performance Network Data Path SUSE Labs Software Engineer Gary Lin
  2. 2. eBPF / Networking
  3. 3. userspace kernel RX Packet Processing Driver Network Stack NIC Network Program
  4. 4. DDoS
  5. 5. userspace kernel Driver Network Stack NIC Network Program
  6. 6. Firewall
  7. 7. userspace kernel iptables/netfilter DriverNIC Network Program Network Stack netfilter DROP
  8. 8. Traffic Control
  9. 9. userspace kernel cls_bpf (tc ingress) Driver Network Stack NIC Network Program eBPF DROP FORWARD
  10. 10. eXpress Data Path
  11. 11. userspace kernel XDP Network Stack NIC Network Program Driver skb alloceBPF DROPTX
  12. 12. XDP ● A high performance, programmable network data path ● No specialized hardware ● No kernel bypass ● Works with the existing network stack ● Direct packet write
  13. 13. XDP Actions ● XDP_ABORTED eBPF program error, treat like DROP ● XDP_DROP Drop the packet ● XDP_PASS Pass the packet up to the stack ● XDP_TX Transmit packet out the same interface
  14. 14. #define KBUILD_MODNAME "foo" /*for some headers*/ #include <uapi/linux/bpf.h> ... SEC("xdp_prog") int xdp_prog(struct xdp_md *ctx) { void *data_end = (void *)(long)ctx->data_end; void *data = (void *)(long)ctx->data; struct ethhdr *eth = data; ... return XDP_DROP; /* the action */ }
  15. 15. XDP Restrictions ● One packet buffer per 4K page – Memory waste – Needs the change of the memory model of the driver ● No per-RX-queue XDP instance ● No transmit on other interface
  16. 16. Current Status (4.11) ● XDP core: 4.8 ● Supported Drivers – mlx4: 4.8 – mlx5: 4.9 – nfp, qed, virtio_net: 4.10 – bnxt_en: 4.11 ● Not upstream – i40e, generic_xdp, ixgbe
  17. 17. XDP Benchmarks (mlx4) ● Generated using pktgen ● Single core – ip routing drop: ~3.6 Mpps – tc clsact using bpf: ~4.2 Mpps – XDP drop: 20 Mpps (<10 % cpu util) https://www.slideshare.net/IOVisor/express-data-path-linux-meetup-santa-clara-july-2016
  18. 18. XDP Benchmarks (virtio-net) ● Generated using pktgen ● Host: i7-4790 CPU @ 3.60GHz ● Single core qemu guest – iptables drop (raw preroute): 2.87 Mpps – tc clsact using bpf: 3.05 Mpps – XDP drop: 3.45 Mpps
  19. 19. virtio-net
  20. 20. static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog) { ... if (dev->mtu > max_sz) { netdev_warn(dev, "XDP requires MTU less than %lun", max_sz); return -EINVAL; } ... if (prog) { prog = bpf_prog_add(prog, vi->max_queue_pairs - 1); if (IS_ERR(prog)) return PTR_ERR(prog); } ... for (i = 0; i < vi->max_queue_pairs; i++) { old_prog = rtnl_dereference(vi->rq[i].xdp_prog); rcu_assign_pointer(vi->rq[i].xdp_prog, prog); if (old_prog) bpf_prog_put(old_prog); } return 0; }
  21. 21. static struct sk_buff *receive_mergeable(struct net_device *dev, ... unsigned int len) { ... rcu_read_lock(); xdp_prog = rcu_dereference(rq->xdp_prog); if (xdp_prog) { ... act = bpf_prog_run_xdp(xdp_prog, &xdp); switch (act) { case XDP_PASS: ... case XDP_TX: ... default: ... case XDP_ABORTED: ... case XDP_DROP: ... } } rcu_read_unlock(); ... head_skb = page_to_skb(vi, rq, page, offset, len, truesize);
  22. 22. Test XDP with virtio-net QEMU tap0 192.168.100.1 pktgen Host virtio-net 192.168.100.2 XDP
  23. 23. Setup the host ● Create the multiqueue tap # ip tuntap add mode tap multi_queue user <user_name> name tap0 ● Bring up tap0 # ip addr add 192.168.100.1 dev tap0 # ip link set tap0 up ● Change the permission of /dev/vhost-net # chmod 666 /dev/vhost-net
  24. 24. Setup the guest ● Enable multiqueue for virtio-net (for 1 CPU) -netdev tap,id=hn1,ifname=tap0,script=no, downscript=no,vhost=on,queues=2 -device virtio-net-pci,netdev=hn1,mq=on, vectors=6 ● Disable LRO support -device guest_tso4=off,guest_tso6=off, guest_ecn=off,guest_ufo=off
  25. 25. Start the guest $ qemu-system-x86_64 -smp 1 ... -netdev tap,id=hn1,ifname=tap0,script=no, downscript=no,vhost=on,queues=2 -device virtio-net-pci,netdev=hn1,mq=on, vectors=6,guest_tso4=off,guest_tso6=off, guest_ecn=off,guest_ufo=off
  26. 26. Test XDP ● Install bcc (>= 0.3.0) # zypper in python-bcc ● Run xdp_drop_count.py* # python xdp_drop_count.py <ifdev> * https://github.com/iovisor/bcc/blob/master/examples/networking/xdp/xdp_drop_count.py
  27. 27. Some useful bpf map types
  28. 28. Program Array PROG_ARRAY 0 fd0 2 fd2 3 fd3 XDP Program 2 XDP Program 3 XDP Program 0 XDP Program Main bpf_tail_call()
  29. 29. Program Array in BCC array.c: BPF_TABLE("prog", int, int, ptable, 4); int func3(struct xdp_md *ctx) { ... int my_main(struct xdp_md *ctx) { ... ptable.call(ctx, 3); ... array.py: bpf = BPF(src_file="array.c") func3 = bpf.load_func("func3", BPF.XDP) ptable[c_int(3)] = c_int(func3.fd)
  30. 30. LRU Hash ref Active List Inactive List Free List Pre-allocated Hash bpf_lookup_elem()
  31. 31. LRU Hash in BCC lru.c: BPF_TABLE("lru_hash", uint_32, int, lru, 1024); int my_main(struct xdp_md *ctx) { ... lru.lookup(&key); ... lru.py: bpf = BPF(src_file="lru.c") lru[c_uint(9527)] = c_int(1)
  32. 32. XDP Monitor XDP DDoS XDP Load Balancer blacklist Analyzer update packets DROP Possible Scenario TX PASS
  33. 33. 33 Questions?
  34. 34. 34 Thank You!
  35. 35. References ● XDP project https://www.iovisor.org/technology/xdp ● eXpress Data Path: Getting Linux to 20Mpps https://www.slideshare.net/IOVisor/express-data-path-linux-meetup- santa-clara-july-2016?qid=247820dc-7ac0-494e-8e6c-3a21daeda22a&v=& b=&from_search=5 ● XDP in practice: integrating XDP in our DDoS mitigation pipeline https://www.netdevconf.org/2.1/session.html?bertin ● Droplet: DDoS countermeasures powered by BPF + XDP https://www.netdevconf.org/2.1/session.html?zhou

×