High-Performance Networking Using eBPF, XDP, and io_uring

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Brought to you by
Bryan McCoid
Sr. Software Engineer at
High Performance Networking
Using eBPF, XDP, and io_uring

Bryan McCoid
Sr. Software Engineer
■ I work on the clustering/control plane component at Couchbase
■ Into all things Linux...

So who cares about networking anyways?!
Why is it that we even care about this? It all sounds very boring and I
just want ...

Some history...

So what are our options?
Most people deal with networking in linux at the socket level
■ Normal, blocking sockets
● 1 thre...

Enter eBPF...
“eBPF is a revolutionary technology with origins in the Linux kernel that
can run sandboxed programs in an o...

XDP/AF_XDP - Linux’s answer to
kernel bypass

AF_PACKET V4 -> AF_XDP
■ A new version of AF_PACKET was being developed around the
same time eBPF came into existence.
■ I...

XDP
XDP comes in two variants to the user, but these interfaces are
intertwined
■ XDP programs can be standalone, written ...

The linux kernel networking is great, but...
■ For many use-cases it’s not ideal
■ It’s relatively scalable but..
● At som...

My use for XDP
And how to use AF_XDP in general

How to use it? + Designing a runtime in Rust
■ Built for existing Rust asynchronous I/O library named Glommio
● Designed a...

Basic layout

Memory Region
(pre-allocated)
Basic layout

Memory Region
(pre-allocated)
Ring buffers
(queues)
Basic layout

Memory Region
(pre-allocated)
Ring buffers
(queues)
These are
single
producer,
single
consumer
Basic layout

Basic layout (including another cpu for driver / IRQ handling)

$“Default” eBPF code for AF_XDP socket #include <bpf/bpf_helpers.h> Struct bpf_map_def SEC(“maps”) xsks_map = { .type = BPF...$

$“Default” eBPF code for AF_XDP socket SEC(“xdp_sock”) int xdp_sock_prog(struct xdp_md *ctx) { int index = ctx->rx_queue_in...$

How about other designs?

Shared UMEM / multiple AF_XDP sockets

Many sockets, many UMEMs, 1 per core

Interfacing with kernel is easy
let count = _xsk_ring_cons__peek(
self.rx_queue.as_mut(),
filled_frames.len().unwrap_or(0)...

How this ﬁts into the larger framework
The framework I’m integrating to focuses on IO_URING so integrating can happen
a fe...

Other considerations
■ Is relying on the io_uring for needs_wakeup notiﬁcations of
readiness too slow?
● Does it add too m...

This is all a work in-progress...
Thanks for listening and we’d love for people to get interested and contribute!
Main rep...

Brought to you by
Bryan McCoid
bryan.mccoid@couchbase.com
@bryanmccoid

(BONUS) Popular projects
■ (DPDK) Seastar *cough* *cough* - asynchronous fast I/O
framework
■ (DPDK) mTCP - TCP stack
■ (X...