SlideShare a Scribd company logo
© 2018 NETRONOME SYSTEMS, INC.
David Beckett
Jakub Kicinski
eBPF/XDP
SIGCOMM 2018
© 2018 NETRONOME SYSTEMS, INC. 2
Introduction
Jakub Kicinski
Lead Software Engineer
eBPF Kernel Development
David Beckett
Software Engineer
eBPF Application Development
© 2018 NETRONOME SYSTEMS, INC. 3
Overview
●  What is eBPF/XDP?
●  Demos
●  SmartNIC eBPF offload
●  Host dataplane Acceleration
●  SmartNIC offload Demos
© 2018 NETRONOME SYSTEMS, INC. 4
eBPF System
RX Port
TCP Stack
Netfilter
TC
XDP
eBPF
Driver Space
Kernel Space
0
Key
...
Userspace
Maps
ABC
Value
...
© 2018 NETRONOME SYSTEMS, INC. 5
What is XDP?
XDP allows packets to be reflected, filtered or redirected
without traversing networking stack
▶  eBPF programs classify/modify traffic and return
XDP actions
Note: cls_bpf in TC works in same manner
▶  XDP Actions
•  XDP_PASS
•  XDP_DROP
•  XDP_TX
•  XDP_REDIRECT
•  XDP_ABORT - Something went wrong
▶  Currently hooks onto RX path only
•  Other hooks can also work on TX
RX port
TCP Stack
Netfilter (1 Mpps)
TC (5Mpps)
XDP (20Mpps)
eBPF
XDP
Actions
Redirect
socket
Intended
socket
Redirect
port
XDP_PASS
XDP_REDIRECT
XDP_DROP
XDP_TX
Return
XDP
action
Driver Space
Kernel Space
© 2018 NETRONOME SYSTEMS, INC. 6
What is the eBPF Architecture?
A kernel-based virtual machine to enable low-level packet processing
▶  Think Java VMs in the kernel
•  Networking focused ISA/bytecode
•  10 64-bit registers
-  32-bit subregisters
•  Small stack (512 bytes)
•  Infinite-size key value stores (maps)
▶  Write programs in C, P4, Go or Rust
•  C is LLVM compiled to BPF bytecode
•  Verifier checked
•  JIT converts to assembly
▶  Hooks into the kernel in many places
•  Final packet handling dependent on hook
LLVM/Clang
Prog.c
Verifier
JIT
(architecture
dependent)
CPU (ARM64/
X86/
PowerPC)
Prog.o (unverified)
Prog.o (verified)
Prog.asm
© 2018 NETRONOME SYSTEMS, INC. 7
Maps
Maps are key-value stores used to store state
▶  Up to 128 maps per program
▶  Infinite size
▶  Multiple different types-Non XDP
-  BPF_MAP_TYPE_HASH
-  BPF_MAP_TYPE_ARRAY
-  BPF_MAP_TYPE_PROG_ARRAY
-  BPF_MAP_TYPE_PERF_EVENT_ARRAY
-  BPF_MAP_TYPE_PERCPU_HASH
-  BPF_MAP_TYPE_PERCPU_ARRAY
-  BPF_MAP_TYPE_STACK_TRACE
-  BPF_MAP_TYPE_CGROUP_ARRAY
▶  Accessed via map helpers
-  BPF_MAP_TYPE_LRU_HASH
-  BPF_MAP_TYPE_LRU_PERCPU_HASH
-  BPF_MAP_TYPE_LPM_TRIE
-  BPF_MAP_TYPE_ARRAY_OF_MAPS
-  BPF_MAP_TYPE_HASH_OF_MAPS
-  BPF_MAP_TYPE_DEVMAP
-  BPF_MAP_TYPE_SOCKMAP
-  BPF_MAP_TYPE_CPUMAP
0
Key
19
4121
91
12111
...
10.0.0.1
Value
10.0.0.6
121.0.0.1
10.0.1.1
5.0.2.12
...
© 2018 NETRONOME SYSTEMS, INC. 8
Helpers
Helpers are used to add functionality that would otherwise be difficult
▶  Key XDP Map helpers
-  bpf_map_lookup_elem
-  bpf_map_update_elem
-  bpf_map_delete_elem
-  bpf_redirect_map
▶  Head Extend
-  bpf_xdp_adjust_head
-  bpf_xdp_adjust_meta
▶  Others
-  bpf_ktime_get_ns
-  bpf_trace_printk
-  bpf_tail_call
-  Bpf_redirect
https://github.com/torvalds/linux/blob/master/include/uapi/linux/bpf.h
© 2018 NETRONOME SYSTEMS, INC. 9
BPF Bytecode
64-bit, 2 operand BPF bytecode instructions are split as follows
op:8
BPF_JNE | BPF_K | BPF_JMP
dst_reg:4
0x1 src_reg:4
0x0
off:16
0x001
imm:32
0x00000800
operation:4
BPF_JNE
source:1
BPF_K
insn_class:3
BPF_JMP
mode:8
BPF_H
size:2
BPF_ABS
insn_class:3
BPF_LD
ALU/JMP LD/STO
© 2018 NETRONOME SYSTEMS, INC. 10
XDP Actions
Register 0 denotes the return value
Value Action Description
0 XDP_ABORTED Error, Block the packet
1 XDP_DROP Block the packet
2 XDP_PASS Allow packet to continue up to the kernel
3 XDP_TX Bounce the packet
© 2018 NETRONOME SYSTEMS, INC. 11
Code Snippet - XDP/eBPF Example
#include	<linux/bpf.h>	
#include	"bpf_api.h"	
#include	"bpf_helpers.h"	
	
SEC(“xdp_prog1”)	
int	xdp_prog1(struct	xdp_md	*xdp)	
{	
	unsigned	char	*data;	
	
	data	=	(void	*)(unsigned	long)xdp->data;		
	if	(data	+	14	>	(void	*)(long)xdp->data_end)	
	 	return	XDP_ABORTED;	
	
	if	(data[12]	!=	0x22	||	data[13]	!=	0x22)	
	 	return	XDP_DROP;	
	
	return	XDP_PASS;	
}
xdp_prog1:	
							0: 	b7	00	00	00	00	00	00	00	 	r0	=	0	
							1: 	61	12	04	00	00	00	00	00	 	r2	=	*(u32	*)(r1	+	4)	
							2: 	61	11	00	00	00	00	00	00	 	r1	=	*(u32	*)(r1	+	0)	
							3: 	bf	13	00	00	00	00	00	00	 	r3	=	r1	
							4: 	07	03	00	00	0e	00	00	00	 	r3	+=	14	
							5: 	2d	23	07	00	00	00	00	00	 	if	r3	>	r2	goto	7	
							6: 	b7	00	00	00	01	00	00	00	 	r0	=	1	
							7: 	71	12	0c	00	00	00	00	00	 	r2	=	*(u8	*)(r1	+	12)	
							8: 	55	02	04	00	22	00	00	00	 	if	r2	!=	34	goto	4	
							9: 	71	11	0d	00	00	00	00	00	 	r1	=	*(u8	*)(r1	+	13)	
						10: 	b7	00	00	00	02	00	00	00	 	r0	=	2	
						11: 	15	01	01	00	22	00	00	00	 	if	r1	==	34	goto	1	
						12: 	b7	00	00	00	01	00	00	00	 	r0	=	1	
	
LBB0_4:	
						13: 	95	00	00	00	00	00	00	00	 	exit	
Drop	packets	not	EtherType	0x2222	
Clang	Compiler
© 2018 NETRONOME SYSTEMS, INC. 12
Kernel Security and Stability
eBPF code injected into the kernel must be safe
▶  Potential risks
•  Infinite loops could crash the kernel
•  Buffer overflows
•  Uninitialized variables
•  Large programs may cause performance issues
•  Compiler errors
© 2018 NETRONOME SYSTEMS, INC. 13
eBPF Verifier
The verifier checks for the validity of programs
▶  Ensure that no back edges (loops) exist
•  Mitigated through the use #pragma unroll
▶  Ensure that the program has no more than 4,000 instructions
▶  There are also a number of other checks on the validity of register usage
•  These are done by traversing each path through the program
▶  If there are too many possible paths the program will also be rejected
•  1K branches
•  130K complexity of total instructions
© 2018 NETRONOME SYSTEMS, INC. 14
Verifier-Directed Acyclical Graph
The verifier checks for the DAG property
▶  Ensures that no back edges (loops) exist
▶  Backward jumps are allowed
•  Only if they do not cause loops
▶  Handled by check_cfg() in verifier.c
0
1
2
3
4
5
6
check_cfg()
Any program
with a loop is
rejected
© 2018 NETRONOME SYSTEMS, INC. 15
DAG Example
xdp_prog1:	
	 	r0	=	0	
	 	r2	=	*(u32	*)(r1	
+	4)	
	 	r1	=	*(u32	*)(r1	
+	0)	
	 	r3	=	r1	
	 	r3	+=	14	
	 	if	r3	>	r2	goto	7	
	 	r0	=	1	
	 	r2	=	*(u8	*)(r1	+	
12)	
	 	if	r2	!=	34	goto	
4	
	 	r1	=	*(u8	*)(r1	+	
13)	
	 	r0	=	2	
	 	if	r1	==	34	goto	
1	
	 	r0	=	1	
#include	<linux/bpf.h>	
#include	"bpf_api.h"	
#include	"bpf_helpers.h"	
	
SEC(“xdp_prog1”)	
int	xdp_prog1(struct	xdp_md	*xdp)	
{	
	unsigned	char	*data;	
	
	data	=	(void	*)(unsigned	long)xdp->data;		
	if	(data	+	14	>	(void	*)(long)xdp->data_end)	
	 	return	XDP_ABORTED;	
	
	if	(data[12]	!=	0x22	||	data[13]	!=	0x22)	
	 	return	XDP_DROP;	
	
	return	XDP_PASS;	
}
DAG	shown	with	bpftool	and	dot	graph	generator
		#	bpftool	prog	dump	xlated	id	13	visual	>	cfg.txt	
		#	dot	-Tps	cfg.txt	-o	cfg.ps
© 2018 NETRONOME SYSTEMS, INC. 16
x86 JIT Code - XDP/eBPF Example
JITed	for	
x86	CPU	
			0: 	push			%rbp	
			1: 	mov				%rsp,%rbp	
			4: 	sub				$0x28,%rsp	
			b: 	sub				$0x28,%rbp	
			f: 	mov				%rbx,0x0(%rbp)	
		13: 	mov				%r13,0x8(%rbp)	
		17: 	mov				%r14,0x10(%rbp)	
		1b: 	mov				%r15,0x18(%rbp)	
		1f: 	xor				%eax,%eax	
		21: 	mov				%rax,0x20(%rbp)	
		25: 	xor				%eax,%eax	
		27: 	mov				0x8(%rdi),%rsi	
		2b: 	mov				0x0(%rdi),%rdi	
		2f: 	mov				%rdi,%rdx	
		32: 	add				$0xe,%rdx	
		36: 	cmp				%rsi,%rdx	
		39: 	ja					
0x0000000000000060	
		3b: 	mov				$0x1,%eax	
		40: 	movzbq	0xc(%rdi),%rsi	
		45: 	cmp				$0x22,%rsi	
		49: 	jne				
0x0000000000000060	
		4b: 	movzbq	0xd(%rdi),%rdi	
		50: 	mov				$0x2,%eax	
		55: 	cmp				$0x22,%rdi	
		59: 	je					
0x0000000000000060	
		5b: 	mov				$0x1,%eax	
		60: 	mov				0x0(%rbp),%rbx	
		64: 	mov				0x8(%rbp),%r13	
		68: 	mov				0x10(%rbp),%r14	
		6c: 	mov				0x18(%rbp),%r15	
		70: 	add				$0x28,%rbp	
		74: 	leaveq		
		75: 	retq				
Verifier	
xdp_prog1:	
	 	r0	=	0	
	 	r2	=	*(u32	*)(r1	
+	4)	
	 	r1	=	*(u32	*)(r1	
+	0)	
	 	r3	=	r1	
	 	r3	+=	14	
	 	if	r3	>	r2	goto	7	
	 	r0	=	1	
	 	r2	=	*(u8	*)(r1	+	
12)	
	 	if	r2	!=	34	goto	
4	
	 	r1	=	*(u8	*)(r1	+	
13)	
	 	r0	=	2	
	 	if	r1	==	34	goto	
1	
	 	r0	=	1
© 2018 NETRONOME SYSTEMS, INC. 17
Open Source Tools
Bpftool
▶  Lists active bpf programs and maps
▶  Interactions with eBPF maps (lookups or updates)
▶  Dump assembly code (JIT and Pre-JIT)
Iproute2
▶  Can load and attach eBPF programs to TC, XDP or XDP offload (SmartNIC)
Libbpf
▶  BPF library allowing for user space program access to eBPF api
© 2018 NETRONOME SYSTEMS, INC. 18
Public XDP Use Cases
Current use cases focus on load balancers, DDoS mitigation and simple monitoring
▶  Load balancer
•  Used by FB Katran to replace IPVS - 2X performance per core
▶  DDoS mitigation
•  Cloudflare starting the transition to eBPF
▶  Distributed Firewall
•  Flexible, high-performance blacklisting
FB Load Balancer throughput: XDP vs IPVS
© 2018 NETRONOME SYSTEMS, INC. 19
Use Cases
Suricata Intrusion Detection System (IDS)
▶  Whitelist large flows (e.g. Netflix stream)
“Suricata Performance with a S like Security” É. Leblond
WAN LAN
IDS
© 2018 NETRONOME SYSTEMS, INC. 20
Summary: Driver XDP
Advantages
▶  Increased performance - 4X
▶  Reuses kernel infrastructure
▶  Upstream-boot Linux and you are good to go
▶  Allows updates of low-level functionality without kernel reboot
•  This should not be underestimated
•  A particular DC provider spent 3 months rebooting servers when a bug was found
Disadvantages
▶  CPU still limits the use-cases at high data rates
© 2018 NETRONOME SYSTEMS, INC. 21
Demo 1 - XDP Actions and Packet Modification
xdp_drop
#include	<linux/bpf.h>	
	
int	main()	
{	
				return	XDP_DROP;	
} RX Port
TCP Stack
Netfilter
TC
XDP
eBPF
Userspace
C Program
BPF Program
clang
iproute
Program Loaded
© 2018 NETRONOME SYSTEMS, INC. 22
Demo 2 - Maps
xdp_actions based on eBPF map
RX Port
TCP Stack
Netfilter
TC
XDP
eBPF
Driver Space
Kernel Space
Userspace
Map
0
Key
XDP_TX
Value
Value Action
0 XDP_ABORTED
1 XDP_DROP
2 XDP_PASS
3 XDP_TX
© 2018 NETRONOME SYSTEMS, INC. 23
Demo 3 - Load Balancer
Demo Source: https://github.com/Netronome/bpf-samples/tree/master/l4lb
1.1.1.1 2.2.2.2
TCP
1292 80
4 Tuple Hash
0
Hash Key
1
2
10.0.0.1
Server
10.0.0.6
10.0.0.9
2.2.2.2 10.0.0.9
1.1.1.1 2.2.2.2
TCP
1292 80
© 2018 NETRONOME SYSTEMS, INC. 24
DAG Example - Load Balancer Demo
https://github.com/Netronome/bpf-samples/tree/master/l4lb
© 2018 NETRONOME SYSTEMS, INC. 25
XDP Offload
Core 1 Core 2
Core 3 Core 4
Network packets
eBPF running
on Driver (XDP)
Linux Kernel
User Space
© 2018 NETRONOME SYSTEMS, INC. 26
BPF for Host Datapath Acceleration
▶  BPF VM provides a simple and well understood execution environment
▶  Most RISC cores should be able to execute JITed BPF code
▶  Kernel infrastructure improves, including verifier/analyzer, JIT compilers for all common host
architectures and some common embedded architectures like ARM
or x86
▶  Unlike higher level languages BPF is a intermediate representation (IR) which provides binary
compatibility
▶  Advanced networking devices are capable of creating appropriate sandboxes
▶  Android APF targets smaller processors in mobile handsets for filtering wake ups from remote
processors (most likely network interfaces) to improve battery life
▶  Linux kernel community is very active in extending performance and improving BPF feature set,
with AF_XDP being a most recent example
▶  BPF is extensible through helpers and maps allowing us to make use of special HW features
(when gain justifies the effort)
© 2018 NETRONOME SYSTEMS, INC. 27
Kernel Offload - BPF Offload Memory Mapping
NIC
Chip
Island (x6 per Chip)
CTM (256 KB)
IMEM(4 MB)
DRAM
(2+GB)
CLS
(64 KB)
Thread (x4 per Core)
800Mhz Core
LMEM
(1 KB)
GPRs
10 Registers
(64-bit, 32-bit
subregisters)
512 byte
stack
Maps, varying
sizes
Driver
x50 BPF
workers
© 2018 NETRONOME SYSTEMS, INC. 28
Kernel Offload - Programming Model
▶  LLVM compilation as normal
▶  iproute/tc/libbpf loads the program as
normal but specifying “offload enable” flag
▶  maps are created on the device
▶  kernel directs the eBPF program to nfp/src/
bpf/jit.c to converts to NFP machine code
▶  translation reuses the kernel verifier
infrastructure for analysis
▶  full ABI compatibility with the in-kernel
BPF
© 2018 NETRONOME SYSTEMS, INC. 29
NFP JIT
▶  LLVM optimizations can tune the code for
BPF or even NFP BPF
▶  JIT steps:
•  preparation - build data structures
•  analysis - uses kernel verifier infrastructure
•  code generation
•  loading/relocation
▶  two pass translator:
•  convert memory accesses
•  inline helpers
Linux kernel: driver/net/ethernet/netronome/nfp/
bpf/jit.c
GitHub:
Netronome/nfp-drv-kmods/blob/master/src/bpf/jit.c
© 2018 NETRONOME SYSTEMS, INC. 30
NFP JIT Example
Bpftool	prog	dump	jited	id	1	
			0: 		.0		immed[gprB_6,	0x3fff]	
			8: 		.1		alu[gprB_6,	gprB_6,	AND,	*l$index1]	
		10: 		.2		immed[gprA_0,	0x0],	gpr_wrboth	
		18: 		.3		immed[gprA_1,	0x0],	gpr_wrboth	
		20: 		.4		alu[gprA_4,	gprB_6,	+,	*l$index1[2]],	gpr_wrboth	
		28: 		.5		immed[gprA_5,	0x0],	gpr_wrboth	
		30: 		.6		alu[gprA_2,	--,	B,	*l$index1[2]],	gpr_wrboth	
		38: 		.7		immed[gprA_3,	0x0],	gpr_wrboth	
		40: 		.8		alu[gprA_6,	--,	B,	gprB_2],	gpr_wrboth	
		48: 		.9		alu[gprA_7,	--,	B,	gprB_3],	gpr_wrboth	
		50: 	.10		alu[gprA_6,	gprA_6,	+,	0xe],	gpr_wrboth	
		58: 	.11		alu[gprA_7,	gprA_7,	+carry,	0x0],	gpr_wrboth	
		60: 	.12		alu[--,	gprA_4,	-,	gprB_6]	
		68: 	.13		alu[--,	gprA_5,	-carry,	gprB_7]	
		70: 	.14		bcc[.33]	
		78: 	.15		immed[gprA_0,	0x1],	gpr_wrboth	
		80: 	.16		immed[gprA_1,	0x0],	gpr_wrboth	
		88: 	.17		mem[read32_swap,	$xfer_0,	gprA_2,	0xc,	1],	
ctx_swap[sig1]	
		90: 	.18		ld_field_w_clr[gprA_4,	0001,	$xfer_0],	gpr_wrboth	
		98: 	.19		immed[gprA_5,	0x0],	gpr_wrboth	
		a0: 	.20		alu[--,	gprA_4,	XOR,	0x22]	
		a8: 	.21		bne[.33]	
		b0: 	.22		alu[--,	gprA_5,	XOR,	0x0]	
		b8: 	.23		bne[.33]	
		c0: 	.24		ld_field_w_clr[gprA_2,	0001,	$xfer_0,	>>8],	
gpr_wrboth	
		c8: 	.25		immed[gprA_3,	0x0],	gpr_wrboth	
		d0: 	.26		immed[gprA_0,	0x2],	gpr_wrboth	
		d8: 	.27		immed[gprA_1,	0x0],	gpr_wrboth	
																					…	
JITed	into	
NFP	Microcode	
All upstream bpftool with libbfd support;
no vendor tools needed
xdp_prog1:	
	 	r0	=	0	
	 	r2	=	*(u32	*)(r1	
+	4)	
	 	r1	=	*(u32	*)(r1	
+	0)	
	 	r3	=	r1	
	 	r3	+=	14	
	 	if	r3	>	r2	goto	7	
	 	r0	=	1	
	 	r2	=	*(u8	*)(r1	+	
12)	
	 	if	r2	!=	34	goto	
4	
	 	r1	=	*(u8	*)(r1	+	
13)	
	 	r0	=	2	
	 	if	r1	==	34	goto	
1	
	 	r0	=	1
© 2018 NETRONOME SYSTEMS, INC. 31
JIT Optimizations
We can identify from assembly code certain sequences that can be replaced with
fewer/faster NFP instructions, e.g.:
▶  memcpy(new_eth, old_eth, sizeof(*old_eth))
▶  Rotation
▶  ALU operation + register move
▶  bit operations
▶  compare and jump
32-bit subregister use; batching atomic
operations; optimizing out helpers, e.g.:
▶  packet extend
▶  memory lookups
Creating read-only maps on the device
© 2018 NETRONOME SYSTEMS, INC. 32
Demo 4 - Load Balancer on Offload
Demo Source: https://github.com/Netronome/bpf-samples/tree/master/l4lb
© 2018 NETRONOME SYSTEMS, INC. 33
Kernel Offload - Multi-Stage Processing
▶  Use of offloads does not preclude standard in-driver XDP use
▶  Offload some programs, leave some running on the host
▶  Maximize efficiency by playing to NFPs and host’s strengths
▶  Communication between programs via XDP/SKB metadata
© 2018 NETRONOME SYSTEMS, INC. 34
Redefining NIC Behavior
BPF offload allows users to change standard NIC features, e.g.:
▶  RSS
•  Users can create their own RSS schemes and parse arbitrary protocols
•  On standard NIC all packets go to queue 0 if protocols can’t be parsed
•  More examples schemes in presentation about demos
▶  Flow affinity - similarly to RSS any flow affinity to RX queues can be defined
▶  SR-IOV forwarding (future)
•  With upcoming kernel extensions users will be able to define SR-IOV datapath in BPF
•  BPF-defined filtering and forwarding in HW
•  Any custom encapsulation/overlay supported
© 2018 NETRONOME SYSTEMS, INC. 35CONFIDENTIAL
Switching with eBPF (incl. SR-IOV)
●  full switchdev mode
○  Linux term for representing all ports as interfaces
●  XDP ingress on all reprs (just link TC forwarding)
●  XDP_REDIRECT support for forwarding decisions
●  fallback path driver XDP? AF_XDP? up to users
●  per-ASIC program and map sharing
●  ingress device from xdp_rxq_info
●  dealing with mcast/bcast requires a new BPF helper
Port1Port0
PF VFs
BPF
© 2018 NETRONOME SYSTEMS, INC. 36
PCIe Rings
The queue is chosen using a hash on
the header values, such as:
▶  IP Addresses
▶  UDP/TCP port numbers
Core 1 Core 2
Core 3 Core 4
© 2018 NETRONOME SYSTEMS, INC. 37
Programmable RSS
User programmable RSS
▶  Hash on payload headers
▶  Hash on inner IP headers
Core 1 Core 2
Core 3 Core 4
© 2018 NETRONOME SYSTEMS, INC. 38
Demo 5 - Programmable RSS
https://github.com/Netronome/bpf-samples/tree/master/programmable_rss
© 2018 NETRONOME SYSTEMS, INC. 39
Offload Support
Category Functionality Kernel
4.16
Kernel
4.17
Kernel
4.18
Near
Future
eBPF offload
program features
XDP_DROP
XDP_PASS
XDP_TX
XDP_ABORTED
Packet read access
Conditional statements
xdp_adjust_head()
bpf_get_prandom_u32()
perf_event_output()
RSS rx_queue_index selection
bpf_tail_call()
bpf_adjust_tail()
eBPF offload
map features
Hash maps
Array maps
bpf_map_lookup_elem()
bpf_map_delete_elem()
Atomic write (sync_fetch_and_add)
eBPF offload
performance
optimizations
Localized packet cache
32-bit BPF support
© 2018 NETRONOME SYSTEMS, INC. 40
How to Participate with eBPF?
Netronome Guides and Firmware
▶  https://help.netronome.com/support/solutions/folders/36000172266
Demo Applications
▶  https://github.com/Netronome/bpf-samples
© 2018 NETRONOME SYSTEMS, INC.
Thank You

More Related Content

What's hot

BPF: Tracing and more
BPF: Tracing and moreBPF: Tracing and more
BPF: Tracing and more
Brendan Gregg
 
EBPF and Linux Networking
EBPF and Linux NetworkingEBPF and Linux Networking
EBPF and Linux Networking
PLUMgrid
 
Building Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCCBuilding Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCC
Kernel TLV
 
Introduction to eBPF and XDP
Introduction to eBPF and XDPIntroduction to eBPF and XDP
Introduction to eBPF and XDP
lcplcp1
 
Dpdk applications
Dpdk applicationsDpdk applications
Dpdk applications
Vipin Varghese
 
DPDK: Multi Architecture High Performance Packet Processing
DPDK: Multi Architecture High Performance Packet ProcessingDPDK: Multi Architecture High Performance Packet Processing
DPDK: Multi Architecture High Performance Packet Processing
Michelle Holley
 
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)
Brendan Gregg
 
Understanding DPDK
Understanding DPDKUnderstanding DPDK
Understanding DPDK
Denys Haryachyy
 
DPDK In Depth
DPDK In DepthDPDK In Depth
DPDK In Depth
Kernel TLV
 
Linux Network Stack
Linux Network StackLinux Network Stack
Linux Network Stack
Adrien Mahieux
 
eBPF Workshop
eBPF WorkshopeBPF Workshop
eBPF Workshop
Michael Kehoe
 
introduction to linux kernel tcp/ip ptocotol stack
introduction to linux kernel tcp/ip ptocotol stack introduction to linux kernel tcp/ip ptocotol stack
introduction to linux kernel tcp/ip ptocotol stack
monad bobo
 
Using GTP on Linux with libgtpnl
Using GTP on Linux with libgtpnlUsing GTP on Linux with libgtpnl
Using GTP on Linux with libgtpnl
Kentaro Ebisawa
 
Fun with Network Interfaces
Fun with Network InterfacesFun with Network Interfaces
Fun with Network Interfaces
Kernel TLV
 
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPDockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
Thomas Graf
 
BPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLabBPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLab
Taeung Song
 
Linux BPF Superpowers
Linux BPF SuperpowersLinux BPF Superpowers
Linux BPF Superpowers
Brendan Gregg
 
Linux Networking Explained
Linux Networking ExplainedLinux Networking Explained
Linux Networking Explained
Thomas Graf
 
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
Cilium - Bringing the BPF Revolution to Kubernetes Networking and SecurityCilium - Bringing the BPF Revolution to Kubernetes Networking and Security
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
Thomas Graf
 
SR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and ImprovementSR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and Improvement
LF Events
 

What's hot (20)

BPF: Tracing and more
BPF: Tracing and moreBPF: Tracing and more
BPF: Tracing and more
 
EBPF and Linux Networking
EBPF and Linux NetworkingEBPF and Linux Networking
EBPF and Linux Networking
 
Building Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCCBuilding Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCC
 
Introduction to eBPF and XDP
Introduction to eBPF and XDPIntroduction to eBPF and XDP
Introduction to eBPF and XDP
 
Dpdk applications
Dpdk applicationsDpdk applications
Dpdk applications
 
DPDK: Multi Architecture High Performance Packet Processing
DPDK: Multi Architecture High Performance Packet ProcessingDPDK: Multi Architecture High Performance Packet Processing
DPDK: Multi Architecture High Performance Packet Processing
 
Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)Performance Wins with eBPF: Getting Started (2021)
Performance Wins with eBPF: Getting Started (2021)
 
Understanding DPDK
Understanding DPDKUnderstanding DPDK
Understanding DPDK
 
DPDK In Depth
DPDK In DepthDPDK In Depth
DPDK In Depth
 
Linux Network Stack
Linux Network StackLinux Network Stack
Linux Network Stack
 
eBPF Workshop
eBPF WorkshopeBPF Workshop
eBPF Workshop
 
introduction to linux kernel tcp/ip ptocotol stack
introduction to linux kernel tcp/ip ptocotol stack introduction to linux kernel tcp/ip ptocotol stack
introduction to linux kernel tcp/ip ptocotol stack
 
Using GTP on Linux with libgtpnl
Using GTP on Linux with libgtpnlUsing GTP on Linux with libgtpnl
Using GTP on Linux with libgtpnl
 
Fun with Network Interfaces
Fun with Network InterfacesFun with Network Interfaces
Fun with Network Interfaces
 
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPDockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
 
BPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLabBPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLab
 
Linux BPF Superpowers
Linux BPF SuperpowersLinux BPF Superpowers
Linux BPF Superpowers
 
Linux Networking Explained
Linux Networking ExplainedLinux Networking Explained
Linux Networking Explained
 
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
Cilium - Bringing the BPF Revolution to Kubernetes Networking and SecurityCilium - Bringing the BPF Revolution to Kubernetes Networking and Security
Cilium - Bringing the BPF Revolution to Kubernetes Networking and Security
 
SR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and ImprovementSR-IOV ixgbe Driver Limitations and Improvement
SR-IOV ixgbe Driver Limitations and Improvement
 

Similar to eBPF/XDP

P4 Introduction
P4 Introduction P4 Introduction
P4 Introduction
Netronome
 
Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...
Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...
Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...
Anne Nicolas
 
Linkmeup v076(2019-06).2
Linkmeup v076(2019-06).2Linkmeup v076(2019-06).2
Linkmeup v076(2019-06).2
eucariot
 
BPF Hardware Offload Deep Dive
BPF Hardware Offload Deep DiveBPF Hardware Offload Deep Dive
BPF Hardware Offload Deep Dive
Netronome
 
Host Data Plane Acceleration: SmartNIC Deployment Models
Host Data Plane Acceleration: SmartNIC Deployment ModelsHost Data Plane Acceleration: SmartNIC Deployment Models
Host Data Plane Acceleration: SmartNIC Deployment Models
Netronome
 
P4_tutorial.pdf
P4_tutorial.pdfP4_tutorial.pdf
P4_tutorial.pdf
PramodhN3
 
Multipath TCP Upstreaming
Multipath TCP UpstreamingMultipath TCP Upstreaming
Multipath TCP Upstreaming
Graham G. Turnbull
 
HTTP Analytics for 6M requests per second using ClickHouse
HTTP Analytics for 6M requests per second using ClickHouseHTTP Analytics for 6M requests per second using ClickHouse
HTTP Analytics for 6M requests per second using ClickHouse
Alexander Bocharov
 
GPU profiling for computer vision applications
GPU profiling for computer vision applicationsGPU profiling for computer vision applications
GPU profiling for computer vision applications
Mai Nishimura
 
"Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr...
"Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr..."Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr...
"Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr...
Edge AI and Vision Alliance
 
Always on high availability best practices for informix
Always on high availability best practices for informixAlways on high availability best practices for informix
Always on high availability best practices for informix
IBM_Info_Management
 
Informix HA Best Practices
Informix HA Best Practices Informix HA Best Practices
Informix HA Best Practices
Scott Lashley
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
Altinity Ltd
 
Movable Type 5.2 Overview at MTDDC 2012
Movable Type 5.2 Overview at MTDDC 2012Movable Type 5.2 Overview at MTDDC 2012
Movable Type 5.2 Overview at MTDDC 2012
Yuji Takayama
 
eBPF in the view of a storage developer
eBPF in the view of a storage developereBPF in the view of a storage developer
eBPF in the view of a storage developer
Richárd Kovács
 
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward
 
Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019
Thomas Weise
 
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward
 
CAPI and OpenCAPI Hardware acceleration enablement
CAPI and OpenCAPI Hardware acceleration enablementCAPI and OpenCAPI Hardware acceleration enablement
CAPI and OpenCAPI Hardware acceleration enablement
Ganesan Narayanasamy
 
FIWARE Tech Summit - Stream Processing with Kurento Media Server
FIWARE Tech Summit - Stream Processing with Kurento Media ServerFIWARE Tech Summit - Stream Processing with Kurento Media Server
FIWARE Tech Summit - Stream Processing with Kurento Media Server
FIWARE
 

Similar to eBPF/XDP (20)

P4 Introduction
P4 Introduction P4 Introduction
P4 Introduction
 
Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...
Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...
Kernel Recipes 2018 - XDP: a new fast and programmable network layer - Jesper...
 
Linkmeup v076(2019-06).2
Linkmeup v076(2019-06).2Linkmeup v076(2019-06).2
Linkmeup v076(2019-06).2
 
BPF Hardware Offload Deep Dive
BPF Hardware Offload Deep DiveBPF Hardware Offload Deep Dive
BPF Hardware Offload Deep Dive
 
Host Data Plane Acceleration: SmartNIC Deployment Models
Host Data Plane Acceleration: SmartNIC Deployment ModelsHost Data Plane Acceleration: SmartNIC Deployment Models
Host Data Plane Acceleration: SmartNIC Deployment Models
 
P4_tutorial.pdf
P4_tutorial.pdfP4_tutorial.pdf
P4_tutorial.pdf
 
Multipath TCP Upstreaming
Multipath TCP UpstreamingMultipath TCP Upstreaming
Multipath TCP Upstreaming
 
HTTP Analytics for 6M requests per second using ClickHouse
HTTP Analytics for 6M requests per second using ClickHouseHTTP Analytics for 6M requests per second using ClickHouse
HTTP Analytics for 6M requests per second using ClickHouse
 
GPU profiling for computer vision applications
GPU profiling for computer vision applicationsGPU profiling for computer vision applications
GPU profiling for computer vision applications
 
"Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr...
"Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr..."Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr...
"Dynamically Reconfigurable Processor Technology for Vision Processing," a Pr...
 
Always on high availability best practices for informix
Always on high availability best practices for informixAlways on high availability best practices for informix
Always on high availability best practices for informix
 
Informix HA Best Practices
Informix HA Best Practices Informix HA Best Practices
Informix HA Best Practices
 
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...HTTP Analytics for 6M requests per second using ClickHouse, by  Alexander Boc...
HTTP Analytics for 6M requests per second using ClickHouse, by Alexander Boc...
 
Movable Type 5.2 Overview at MTDDC 2012
Movable Type 5.2 Overview at MTDDC 2012Movable Type 5.2 Overview at MTDDC 2012
Movable Type 5.2 Overview at MTDDC 2012
 
eBPF in the view of a storage developer
eBPF in the view of a storage developereBPF in the view of a storage developer
eBPF in the view of a storage developer
 
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
 
Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019Streaming your Lyft Ride Prices - Flink Forward SF 2019
Streaming your Lyft Ride Prices - Flink Forward SF 2019
 
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
Flink Forward San Francisco 2019: Streaming your Lyft Ride Prices - Thomas We...
 
CAPI and OpenCAPI Hardware acceleration enablement
CAPI and OpenCAPI Hardware acceleration enablementCAPI and OpenCAPI Hardware acceleration enablement
CAPI and OpenCAPI Hardware acceleration enablement
 
FIWARE Tech Summit - Stream Processing with Kurento Media Server
FIWARE Tech Summit - Stream Processing with Kurento Media ServerFIWARE Tech Summit - Stream Processing with Kurento Media Server
FIWARE Tech Summit - Stream Processing with Kurento Media Server
 

More from Netronome

Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...
Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...
Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...
Netronome
 
LFSMM AF XDP Queue I-DS
LFSMM AF XDP Queue I-DSLFSMM AF XDP Queue I-DS
LFSMM AF XDP Queue I-DS
Netronome
 
LFSMM Verifier Optimizations and 1 M Instructions
LFSMM Verifier Optimizations and 1 M InstructionsLFSMM Verifier Optimizations and 1 M Instructions
LFSMM Verifier Optimizations and 1 M Instructions
Netronome
 
Using Network Acceleration for an Optimized Edge Cloud Server Architecture
Using Network Acceleration for an Optimized Edge Cloud Server ArchitectureUsing Network Acceleration for an Optimized Edge Cloud Server Architecture
Using Network Acceleration for an Optimized Edge Cloud Server Architecture
Netronome
 
Offloading TC Rules on OVS Internal Ports
Offloading TC Rules on OVS Internal Ports Offloading TC Rules on OVS Internal Ports
Offloading TC Rules on OVS Internal Ports
Netronome
 
Quality of Service Ingress Rate Limiting and OVS Hardware Offloads
Quality of Service Ingress Rate Limiting and OVS Hardware OffloadsQuality of Service Ingress Rate Limiting and OVS Hardware Offloads
Quality of Service Ingress Rate Limiting and OVS Hardware Offloads
Netronome
 
ODSA Sub-Project Launch
 ODSA Sub-Project Launch ODSA Sub-Project Launch
ODSA Sub-Project Launch
Netronome
 
Flexible and Scalable Domain-Specific Architectures
Flexible and Scalable Domain-Specific ArchitecturesFlexible and Scalable Domain-Specific Architectures
Flexible and Scalable Domain-Specific Architectures
Netronome
 
Unifying Network Filtering Rules for the Linux Kernel with eBPF
Unifying Network Filtering Rules for the Linux Kernel with eBPFUnifying Network Filtering Rules for the Linux Kernel with eBPF
Unifying Network Filtering Rules for the Linux Kernel with eBPF
Netronome
 
Massively Parallel RISC-V Processing with Transactional Memory
Massively Parallel RISC-V Processing with Transactional MemoryMassively Parallel RISC-V Processing with Transactional Memory
Massively Parallel RISC-V Processing with Transactional Memory
Netronome
 
Offloading Linux LAG Devices Via Open vSwitch and TC
Offloading Linux LAG Devices Via Open vSwitch and TCOffloading Linux LAG Devices Via Open vSwitch and TC
Offloading Linux LAG Devices Via Open vSwitch and TC
Netronome
 
eBPF Debugging Infrastructure - Current Techniques
eBPF Debugging Infrastructure - Current TechniqueseBPF Debugging Infrastructure - Current Techniques
eBPF Debugging Infrastructure - Current Techniques
Netronome
 
Efficient JIT to 32-bit Arches
Efficient JIT to 32-bit ArchesEfficient JIT to 32-bit Arches
Efficient JIT to 32-bit Arches
Netronome
 
eBPF & Switch Abstractions
eBPF & Switch AbstractionseBPF & Switch Abstractions
eBPF & Switch Abstractions
Netronome
 
eBPF Tooling and Debugging Infrastructure
eBPF Tooling and Debugging InfrastructureeBPF Tooling and Debugging Infrastructure
eBPF Tooling and Debugging Infrastructure
Netronome
 
Demystify eBPF JIT Compiler
Demystify eBPF JIT CompilerDemystify eBPF JIT Compiler
Demystify eBPF JIT Compiler
Netronome
 
The Power of SmartNICs
The Power of SmartNICsThe Power of SmartNICs
The Power of SmartNICs
Netronome
 
DPDK Support for New HW Offloads
DPDK Support for New HW OffloadsDPDK Support for New HW Offloads
DPDK Support for New HW Offloads
Netronome
 
Comprehensive XDP Off‌load-handling the Edge Cases
Comprehensive XDP Off‌load-handling the Edge CasesComprehensive XDP Off‌load-handling the Edge Cases
Comprehensive XDP Off‌load-handling the Edge Cases
Netronome
 
Open vSwitch Offload: Conntrack and the Upstream Kernel
Open vSwitch Offload: Conntrack and the Upstream KernelOpen vSwitch Offload: Conntrack and the Upstream Kernel
Open vSwitch Offload: Conntrack and the Upstream Kernel
Netronome
 

More from Netronome (20)

Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...
Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...
Disaggregation a Primer: Optimizing design for Edge Cloud & Bare Metal applic...
 
LFSMM AF XDP Queue I-DS
LFSMM AF XDP Queue I-DSLFSMM AF XDP Queue I-DS
LFSMM AF XDP Queue I-DS
 
LFSMM Verifier Optimizations and 1 M Instructions
LFSMM Verifier Optimizations and 1 M InstructionsLFSMM Verifier Optimizations and 1 M Instructions
LFSMM Verifier Optimizations and 1 M Instructions
 
Using Network Acceleration for an Optimized Edge Cloud Server Architecture
Using Network Acceleration for an Optimized Edge Cloud Server ArchitectureUsing Network Acceleration for an Optimized Edge Cloud Server Architecture
Using Network Acceleration for an Optimized Edge Cloud Server Architecture
 
Offloading TC Rules on OVS Internal Ports
Offloading TC Rules on OVS Internal Ports Offloading TC Rules on OVS Internal Ports
Offloading TC Rules on OVS Internal Ports
 
Quality of Service Ingress Rate Limiting and OVS Hardware Offloads
Quality of Service Ingress Rate Limiting and OVS Hardware OffloadsQuality of Service Ingress Rate Limiting and OVS Hardware Offloads
Quality of Service Ingress Rate Limiting and OVS Hardware Offloads
 
ODSA Sub-Project Launch
 ODSA Sub-Project Launch ODSA Sub-Project Launch
ODSA Sub-Project Launch
 
Flexible and Scalable Domain-Specific Architectures
Flexible and Scalable Domain-Specific ArchitecturesFlexible and Scalable Domain-Specific Architectures
Flexible and Scalable Domain-Specific Architectures
 
Unifying Network Filtering Rules for the Linux Kernel with eBPF
Unifying Network Filtering Rules for the Linux Kernel with eBPFUnifying Network Filtering Rules for the Linux Kernel with eBPF
Unifying Network Filtering Rules for the Linux Kernel with eBPF
 
Massively Parallel RISC-V Processing with Transactional Memory
Massively Parallel RISC-V Processing with Transactional MemoryMassively Parallel RISC-V Processing with Transactional Memory
Massively Parallel RISC-V Processing with Transactional Memory
 
Offloading Linux LAG Devices Via Open vSwitch and TC
Offloading Linux LAG Devices Via Open vSwitch and TCOffloading Linux LAG Devices Via Open vSwitch and TC
Offloading Linux LAG Devices Via Open vSwitch and TC
 
eBPF Debugging Infrastructure - Current Techniques
eBPF Debugging Infrastructure - Current TechniqueseBPF Debugging Infrastructure - Current Techniques
eBPF Debugging Infrastructure - Current Techniques
 
Efficient JIT to 32-bit Arches
Efficient JIT to 32-bit ArchesEfficient JIT to 32-bit Arches
Efficient JIT to 32-bit Arches
 
eBPF & Switch Abstractions
eBPF & Switch AbstractionseBPF & Switch Abstractions
eBPF & Switch Abstractions
 
eBPF Tooling and Debugging Infrastructure
eBPF Tooling and Debugging InfrastructureeBPF Tooling and Debugging Infrastructure
eBPF Tooling and Debugging Infrastructure
 
Demystify eBPF JIT Compiler
Demystify eBPF JIT CompilerDemystify eBPF JIT Compiler
Demystify eBPF JIT Compiler
 
The Power of SmartNICs
The Power of SmartNICsThe Power of SmartNICs
The Power of SmartNICs
 
DPDK Support for New HW Offloads
DPDK Support for New HW OffloadsDPDK Support for New HW Offloads
DPDK Support for New HW Offloads
 
Comprehensive XDP Off‌load-handling the Edge Cases
Comprehensive XDP Off‌load-handling the Edge CasesComprehensive XDP Off‌load-handling the Edge Cases
Comprehensive XDP Off‌load-handling the Edge Cases
 
Open vSwitch Offload: Conntrack and the Upstream Kernel
Open vSwitch Offload: Conntrack and the Upstream KernelOpen vSwitch Offload: Conntrack and the Upstream Kernel
Open vSwitch Offload: Conntrack and the Upstream Kernel
 

Recently uploaded

Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
ldtexsolbl
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
Shiv Technolabs
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Kunal Gupta
 
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSECHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
kumarjarun2010
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc
 
Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...
chetankumar9855
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
aslasdfmkhan4750
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
aakash malhotra
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Zilliz
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
huseindihon
 
Figma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdfFigma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdf
Management Institute of Skills Development
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
HackersList
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
HackersList
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
Priyanka Aash
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
digitalxplive
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 

Recently uploaded (20)

Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
 
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptxDublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
Dublin_mulesoft_meetup_Mulesoft_Salesforce_Integration (1).pptx
 
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSECHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
CHAPTER-8 COMPONENTS OF COMPUTER SYSTEM CLASS 9 CBSE
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
 
Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
 
Figma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdfFigma AI Design Generator_ In-Depth Review.pdf
Figma AI Design Generator_ In-Depth Review.pdf
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 

eBPF/XDP

  • 1. © 2018 NETRONOME SYSTEMS, INC. David Beckett Jakub Kicinski eBPF/XDP SIGCOMM 2018
  • 2. © 2018 NETRONOME SYSTEMS, INC. 2 Introduction Jakub Kicinski Lead Software Engineer eBPF Kernel Development David Beckett Software Engineer eBPF Application Development
  • 3. © 2018 NETRONOME SYSTEMS, INC. 3 Overview ●  What is eBPF/XDP? ●  Demos ●  SmartNIC eBPF offload ●  Host dataplane Acceleration ●  SmartNIC offload Demos
  • 4. © 2018 NETRONOME SYSTEMS, INC. 4 eBPF System RX Port TCP Stack Netfilter TC XDP eBPF Driver Space Kernel Space 0 Key ... Userspace Maps ABC Value ...
  • 5. © 2018 NETRONOME SYSTEMS, INC. 5 What is XDP? XDP allows packets to be reflected, filtered or redirected without traversing networking stack ▶  eBPF programs classify/modify traffic and return XDP actions Note: cls_bpf in TC works in same manner ▶  XDP Actions •  XDP_PASS •  XDP_DROP •  XDP_TX •  XDP_REDIRECT •  XDP_ABORT - Something went wrong ▶  Currently hooks onto RX path only •  Other hooks can also work on TX RX port TCP Stack Netfilter (1 Mpps) TC (5Mpps) XDP (20Mpps) eBPF XDP Actions Redirect socket Intended socket Redirect port XDP_PASS XDP_REDIRECT XDP_DROP XDP_TX Return XDP action Driver Space Kernel Space
  • 6. © 2018 NETRONOME SYSTEMS, INC. 6 What is the eBPF Architecture? A kernel-based virtual machine to enable low-level packet processing ▶  Think Java VMs in the kernel •  Networking focused ISA/bytecode •  10 64-bit registers -  32-bit subregisters •  Small stack (512 bytes) •  Infinite-size key value stores (maps) ▶  Write programs in C, P4, Go or Rust •  C is LLVM compiled to BPF bytecode •  Verifier checked •  JIT converts to assembly ▶  Hooks into the kernel in many places •  Final packet handling dependent on hook LLVM/Clang Prog.c Verifier JIT (architecture dependent) CPU (ARM64/ X86/ PowerPC) Prog.o (unverified) Prog.o (verified) Prog.asm
  • 7. © 2018 NETRONOME SYSTEMS, INC. 7 Maps Maps are key-value stores used to store state ▶  Up to 128 maps per program ▶  Infinite size ▶  Multiple different types-Non XDP -  BPF_MAP_TYPE_HASH -  BPF_MAP_TYPE_ARRAY -  BPF_MAP_TYPE_PROG_ARRAY -  BPF_MAP_TYPE_PERF_EVENT_ARRAY -  BPF_MAP_TYPE_PERCPU_HASH -  BPF_MAP_TYPE_PERCPU_ARRAY -  BPF_MAP_TYPE_STACK_TRACE -  BPF_MAP_TYPE_CGROUP_ARRAY ▶  Accessed via map helpers -  BPF_MAP_TYPE_LRU_HASH -  BPF_MAP_TYPE_LRU_PERCPU_HASH -  BPF_MAP_TYPE_LPM_TRIE -  BPF_MAP_TYPE_ARRAY_OF_MAPS -  BPF_MAP_TYPE_HASH_OF_MAPS -  BPF_MAP_TYPE_DEVMAP -  BPF_MAP_TYPE_SOCKMAP -  BPF_MAP_TYPE_CPUMAP 0 Key 19 4121 91 12111 ... 10.0.0.1 Value 10.0.0.6 121.0.0.1 10.0.1.1 5.0.2.12 ...
  • 8. © 2018 NETRONOME SYSTEMS, INC. 8 Helpers Helpers are used to add functionality that would otherwise be difficult ▶  Key XDP Map helpers -  bpf_map_lookup_elem -  bpf_map_update_elem -  bpf_map_delete_elem -  bpf_redirect_map ▶  Head Extend -  bpf_xdp_adjust_head -  bpf_xdp_adjust_meta ▶  Others -  bpf_ktime_get_ns -  bpf_trace_printk -  bpf_tail_call -  Bpf_redirect https://github.com/torvalds/linux/blob/master/include/uapi/linux/bpf.h
  • 9. © 2018 NETRONOME SYSTEMS, INC. 9 BPF Bytecode 64-bit, 2 operand BPF bytecode instructions are split as follows op:8 BPF_JNE | BPF_K | BPF_JMP dst_reg:4 0x1 src_reg:4 0x0 off:16 0x001 imm:32 0x00000800 operation:4 BPF_JNE source:1 BPF_K insn_class:3 BPF_JMP mode:8 BPF_H size:2 BPF_ABS insn_class:3 BPF_LD ALU/JMP LD/STO
  • 10. © 2018 NETRONOME SYSTEMS, INC. 10 XDP Actions Register 0 denotes the return value Value Action Description 0 XDP_ABORTED Error, Block the packet 1 XDP_DROP Block the packet 2 XDP_PASS Allow packet to continue up to the kernel 3 XDP_TX Bounce the packet
  • 11. © 2018 NETRONOME SYSTEMS, INC. 11 Code Snippet - XDP/eBPF Example #include <linux/bpf.h> #include "bpf_api.h" #include "bpf_helpers.h" SEC(“xdp_prog1”) int xdp_prog1(struct xdp_md *xdp) { unsigned char *data; data = (void *)(unsigned long)xdp->data; if (data + 14 > (void *)(long)xdp->data_end) return XDP_ABORTED; if (data[12] != 0x22 || data[13] != 0x22) return XDP_DROP; return XDP_PASS; } xdp_prog1: 0: b7 00 00 00 00 00 00 00 r0 = 0 1: 61 12 04 00 00 00 00 00 r2 = *(u32 *)(r1 + 4) 2: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0) 3: bf 13 00 00 00 00 00 00 r3 = r1 4: 07 03 00 00 0e 00 00 00 r3 += 14 5: 2d 23 07 00 00 00 00 00 if r3 > r2 goto 7 6: b7 00 00 00 01 00 00 00 r0 = 1 7: 71 12 0c 00 00 00 00 00 r2 = *(u8 *)(r1 + 12) 8: 55 02 04 00 22 00 00 00 if r2 != 34 goto 4 9: 71 11 0d 00 00 00 00 00 r1 = *(u8 *)(r1 + 13) 10: b7 00 00 00 02 00 00 00 r0 = 2 11: 15 01 01 00 22 00 00 00 if r1 == 34 goto 1 12: b7 00 00 00 01 00 00 00 r0 = 1 LBB0_4: 13: 95 00 00 00 00 00 00 00 exit Drop packets not EtherType 0x2222 Clang Compiler
  • 12. © 2018 NETRONOME SYSTEMS, INC. 12 Kernel Security and Stability eBPF code injected into the kernel must be safe ▶  Potential risks •  Infinite loops could crash the kernel •  Buffer overflows •  Uninitialized variables •  Large programs may cause performance issues •  Compiler errors
  • 13. © 2018 NETRONOME SYSTEMS, INC. 13 eBPF Verifier The verifier checks for the validity of programs ▶  Ensure that no back edges (loops) exist •  Mitigated through the use #pragma unroll ▶  Ensure that the program has no more than 4,000 instructions ▶  There are also a number of other checks on the validity of register usage •  These are done by traversing each path through the program ▶  If there are too many possible paths the program will also be rejected •  1K branches •  130K complexity of total instructions
  • 14. © 2018 NETRONOME SYSTEMS, INC. 14 Verifier-Directed Acyclical Graph The verifier checks for the DAG property ▶  Ensures that no back edges (loops) exist ▶  Backward jumps are allowed •  Only if they do not cause loops ▶  Handled by check_cfg() in verifier.c 0 1 2 3 4 5 6 check_cfg() Any program with a loop is rejected
  • 15. © 2018 NETRONOME SYSTEMS, INC. 15 DAG Example xdp_prog1: r0 = 0 r2 = *(u32 *)(r1 + 4) r1 = *(u32 *)(r1 + 0) r3 = r1 r3 += 14 if r3 > r2 goto 7 r0 = 1 r2 = *(u8 *)(r1 + 12) if r2 != 34 goto 4 r1 = *(u8 *)(r1 + 13) r0 = 2 if r1 == 34 goto 1 r0 = 1 #include <linux/bpf.h> #include "bpf_api.h" #include "bpf_helpers.h" SEC(“xdp_prog1”) int xdp_prog1(struct xdp_md *xdp) { unsigned char *data; data = (void *)(unsigned long)xdp->data; if (data + 14 > (void *)(long)xdp->data_end) return XDP_ABORTED; if (data[12] != 0x22 || data[13] != 0x22) return XDP_DROP; return XDP_PASS; } DAG shown with bpftool and dot graph generator # bpftool prog dump xlated id 13 visual > cfg.txt # dot -Tps cfg.txt -o cfg.ps
  • 16. © 2018 NETRONOME SYSTEMS, INC. 16 x86 JIT Code - XDP/eBPF Example JITed for x86 CPU 0: push %rbp 1: mov %rsp,%rbp 4: sub $0x28,%rsp b: sub $0x28,%rbp f: mov %rbx,0x0(%rbp) 13: mov %r13,0x8(%rbp) 17: mov %r14,0x10(%rbp) 1b: mov %r15,0x18(%rbp) 1f: xor %eax,%eax 21: mov %rax,0x20(%rbp) 25: xor %eax,%eax 27: mov 0x8(%rdi),%rsi 2b: mov 0x0(%rdi),%rdi 2f: mov %rdi,%rdx 32: add $0xe,%rdx 36: cmp %rsi,%rdx 39: ja 0x0000000000000060 3b: mov $0x1,%eax 40: movzbq 0xc(%rdi),%rsi 45: cmp $0x22,%rsi 49: jne 0x0000000000000060 4b: movzbq 0xd(%rdi),%rdi 50: mov $0x2,%eax 55: cmp $0x22,%rdi 59: je 0x0000000000000060 5b: mov $0x1,%eax 60: mov 0x0(%rbp),%rbx 64: mov 0x8(%rbp),%r13 68: mov 0x10(%rbp),%r14 6c: mov 0x18(%rbp),%r15 70: add $0x28,%rbp 74: leaveq 75: retq Verifier xdp_prog1: r0 = 0 r2 = *(u32 *)(r1 + 4) r1 = *(u32 *)(r1 + 0) r3 = r1 r3 += 14 if r3 > r2 goto 7 r0 = 1 r2 = *(u8 *)(r1 + 12) if r2 != 34 goto 4 r1 = *(u8 *)(r1 + 13) r0 = 2 if r1 == 34 goto 1 r0 = 1
  • 17. © 2018 NETRONOME SYSTEMS, INC. 17 Open Source Tools Bpftool ▶  Lists active bpf programs and maps ▶  Interactions with eBPF maps (lookups or updates) ▶  Dump assembly code (JIT and Pre-JIT) Iproute2 ▶  Can load and attach eBPF programs to TC, XDP or XDP offload (SmartNIC) Libbpf ▶  BPF library allowing for user space program access to eBPF api
  • 18. © 2018 NETRONOME SYSTEMS, INC. 18 Public XDP Use Cases Current use cases focus on load balancers, DDoS mitigation and simple monitoring ▶  Load balancer •  Used by FB Katran to replace IPVS - 2X performance per core ▶  DDoS mitigation •  Cloudflare starting the transition to eBPF ▶  Distributed Firewall •  Flexible, high-performance blacklisting FB Load Balancer throughput: XDP vs IPVS
  • 19. © 2018 NETRONOME SYSTEMS, INC. 19 Use Cases Suricata Intrusion Detection System (IDS) ▶  Whitelist large flows (e.g. Netflix stream) “Suricata Performance with a S like Security” É. Leblond WAN LAN IDS
  • 20. © 2018 NETRONOME SYSTEMS, INC. 20 Summary: Driver XDP Advantages ▶  Increased performance - 4X ▶  Reuses kernel infrastructure ▶  Upstream-boot Linux and you are good to go ▶  Allows updates of low-level functionality without kernel reboot •  This should not be underestimated •  A particular DC provider spent 3 months rebooting servers when a bug was found Disadvantages ▶  CPU still limits the use-cases at high data rates
  • 21. © 2018 NETRONOME SYSTEMS, INC. 21 Demo 1 - XDP Actions and Packet Modification xdp_drop #include <linux/bpf.h> int main() { return XDP_DROP; } RX Port TCP Stack Netfilter TC XDP eBPF Userspace C Program BPF Program clang iproute Program Loaded
  • 22. © 2018 NETRONOME SYSTEMS, INC. 22 Demo 2 - Maps xdp_actions based on eBPF map RX Port TCP Stack Netfilter TC XDP eBPF Driver Space Kernel Space Userspace Map 0 Key XDP_TX Value Value Action 0 XDP_ABORTED 1 XDP_DROP 2 XDP_PASS 3 XDP_TX
  • 23. © 2018 NETRONOME SYSTEMS, INC. 23 Demo 3 - Load Balancer Demo Source: https://github.com/Netronome/bpf-samples/tree/master/l4lb 1.1.1.1 2.2.2.2 TCP 1292 80 4 Tuple Hash 0 Hash Key 1 2 10.0.0.1 Server 10.0.0.6 10.0.0.9 2.2.2.2 10.0.0.9 1.1.1.1 2.2.2.2 TCP 1292 80
  • 24. © 2018 NETRONOME SYSTEMS, INC. 24 DAG Example - Load Balancer Demo https://github.com/Netronome/bpf-samples/tree/master/l4lb
  • 25. © 2018 NETRONOME SYSTEMS, INC. 25 XDP Offload Core 1 Core 2 Core 3 Core 4 Network packets eBPF running on Driver (XDP) Linux Kernel User Space
  • 26. © 2018 NETRONOME SYSTEMS, INC. 26 BPF for Host Datapath Acceleration ▶  BPF VM provides a simple and well understood execution environment ▶  Most RISC cores should be able to execute JITed BPF code ▶  Kernel infrastructure improves, including verifier/analyzer, JIT compilers for all common host architectures and some common embedded architectures like ARM or x86 ▶  Unlike higher level languages BPF is a intermediate representation (IR) which provides binary compatibility ▶  Advanced networking devices are capable of creating appropriate sandboxes ▶  Android APF targets smaller processors in mobile handsets for filtering wake ups from remote processors (most likely network interfaces) to improve battery life ▶  Linux kernel community is very active in extending performance and improving BPF feature set, with AF_XDP being a most recent example ▶  BPF is extensible through helpers and maps allowing us to make use of special HW features (when gain justifies the effort)
  • 27. © 2018 NETRONOME SYSTEMS, INC. 27 Kernel Offload - BPF Offload Memory Mapping NIC Chip Island (x6 per Chip) CTM (256 KB) IMEM(4 MB) DRAM (2+GB) CLS (64 KB) Thread (x4 per Core) 800Mhz Core LMEM (1 KB) GPRs 10 Registers (64-bit, 32-bit subregisters) 512 byte stack Maps, varying sizes Driver x50 BPF workers
  • 28. © 2018 NETRONOME SYSTEMS, INC. 28 Kernel Offload - Programming Model ▶  LLVM compilation as normal ▶  iproute/tc/libbpf loads the program as normal but specifying “offload enable” flag ▶  maps are created on the device ▶  kernel directs the eBPF program to nfp/src/ bpf/jit.c to converts to NFP machine code ▶  translation reuses the kernel verifier infrastructure for analysis ▶  full ABI compatibility with the in-kernel BPF
  • 29. © 2018 NETRONOME SYSTEMS, INC. 29 NFP JIT ▶  LLVM optimizations can tune the code for BPF or even NFP BPF ▶  JIT steps: •  preparation - build data structures •  analysis - uses kernel verifier infrastructure •  code generation •  loading/relocation ▶  two pass translator: •  convert memory accesses •  inline helpers Linux kernel: driver/net/ethernet/netronome/nfp/ bpf/jit.c GitHub: Netronome/nfp-drv-kmods/blob/master/src/bpf/jit.c
  • 30. © 2018 NETRONOME SYSTEMS, INC. 30 NFP JIT Example Bpftool prog dump jited id 1 0: .0 immed[gprB_6, 0x3fff] 8: .1 alu[gprB_6, gprB_6, AND, *l$index1] 10: .2 immed[gprA_0, 0x0], gpr_wrboth 18: .3 immed[gprA_1, 0x0], gpr_wrboth 20: .4 alu[gprA_4, gprB_6, +, *l$index1[2]], gpr_wrboth 28: .5 immed[gprA_5, 0x0], gpr_wrboth 30: .6 alu[gprA_2, --, B, *l$index1[2]], gpr_wrboth 38: .7 immed[gprA_3, 0x0], gpr_wrboth 40: .8 alu[gprA_6, --, B, gprB_2], gpr_wrboth 48: .9 alu[gprA_7, --, B, gprB_3], gpr_wrboth 50: .10 alu[gprA_6, gprA_6, +, 0xe], gpr_wrboth 58: .11 alu[gprA_7, gprA_7, +carry, 0x0], gpr_wrboth 60: .12 alu[--, gprA_4, -, gprB_6] 68: .13 alu[--, gprA_5, -carry, gprB_7] 70: .14 bcc[.33] 78: .15 immed[gprA_0, 0x1], gpr_wrboth 80: .16 immed[gprA_1, 0x0], gpr_wrboth 88: .17 mem[read32_swap, $xfer_0, gprA_2, 0xc, 1], ctx_swap[sig1] 90: .18 ld_field_w_clr[gprA_4, 0001, $xfer_0], gpr_wrboth 98: .19 immed[gprA_5, 0x0], gpr_wrboth a0: .20 alu[--, gprA_4, XOR, 0x22] a8: .21 bne[.33] b0: .22 alu[--, gprA_5, XOR, 0x0] b8: .23 bne[.33] c0: .24 ld_field_w_clr[gprA_2, 0001, $xfer_0, >>8], gpr_wrboth c8: .25 immed[gprA_3, 0x0], gpr_wrboth d0: .26 immed[gprA_0, 0x2], gpr_wrboth d8: .27 immed[gprA_1, 0x0], gpr_wrboth … JITed into NFP Microcode All upstream bpftool with libbfd support; no vendor tools needed xdp_prog1: r0 = 0 r2 = *(u32 *)(r1 + 4) r1 = *(u32 *)(r1 + 0) r3 = r1 r3 += 14 if r3 > r2 goto 7 r0 = 1 r2 = *(u8 *)(r1 + 12) if r2 != 34 goto 4 r1 = *(u8 *)(r1 + 13) r0 = 2 if r1 == 34 goto 1 r0 = 1
  • 31. © 2018 NETRONOME SYSTEMS, INC. 31 JIT Optimizations We can identify from assembly code certain sequences that can be replaced with fewer/faster NFP instructions, e.g.: ▶  memcpy(new_eth, old_eth, sizeof(*old_eth)) ▶  Rotation ▶  ALU operation + register move ▶  bit operations ▶  compare and jump 32-bit subregister use; batching atomic operations; optimizing out helpers, e.g.: ▶  packet extend ▶  memory lookups Creating read-only maps on the device
  • 32. © 2018 NETRONOME SYSTEMS, INC. 32 Demo 4 - Load Balancer on Offload Demo Source: https://github.com/Netronome/bpf-samples/tree/master/l4lb
  • 33. © 2018 NETRONOME SYSTEMS, INC. 33 Kernel Offload - Multi-Stage Processing ▶  Use of offloads does not preclude standard in-driver XDP use ▶  Offload some programs, leave some running on the host ▶  Maximize efficiency by playing to NFPs and host’s strengths ▶  Communication between programs via XDP/SKB metadata
  • 34. © 2018 NETRONOME SYSTEMS, INC. 34 Redefining NIC Behavior BPF offload allows users to change standard NIC features, e.g.: ▶  RSS •  Users can create their own RSS schemes and parse arbitrary protocols •  On standard NIC all packets go to queue 0 if protocols can’t be parsed •  More examples schemes in presentation about demos ▶  Flow affinity - similarly to RSS any flow affinity to RX queues can be defined ▶  SR-IOV forwarding (future) •  With upcoming kernel extensions users will be able to define SR-IOV datapath in BPF •  BPF-defined filtering and forwarding in HW •  Any custom encapsulation/overlay supported
  • 35. © 2018 NETRONOME SYSTEMS, INC. 35CONFIDENTIAL Switching with eBPF (incl. SR-IOV) ●  full switchdev mode ○  Linux term for representing all ports as interfaces ●  XDP ingress on all reprs (just link TC forwarding) ●  XDP_REDIRECT support for forwarding decisions ●  fallback path driver XDP? AF_XDP? up to users ●  per-ASIC program and map sharing ●  ingress device from xdp_rxq_info ●  dealing with mcast/bcast requires a new BPF helper Port1Port0 PF VFs BPF
  • 36. © 2018 NETRONOME SYSTEMS, INC. 36 PCIe Rings The queue is chosen using a hash on the header values, such as: ▶  IP Addresses ▶  UDP/TCP port numbers Core 1 Core 2 Core 3 Core 4
  • 37. © 2018 NETRONOME SYSTEMS, INC. 37 Programmable RSS User programmable RSS ▶  Hash on payload headers ▶  Hash on inner IP headers Core 1 Core 2 Core 3 Core 4
  • 38. © 2018 NETRONOME SYSTEMS, INC. 38 Demo 5 - Programmable RSS https://github.com/Netronome/bpf-samples/tree/master/programmable_rss
  • 39. © 2018 NETRONOME SYSTEMS, INC. 39 Offload Support Category Functionality Kernel 4.16 Kernel 4.17 Kernel 4.18 Near Future eBPF offload program features XDP_DROP XDP_PASS XDP_TX XDP_ABORTED Packet read access Conditional statements xdp_adjust_head() bpf_get_prandom_u32() perf_event_output() RSS rx_queue_index selection bpf_tail_call() bpf_adjust_tail() eBPF offload map features Hash maps Array maps bpf_map_lookup_elem() bpf_map_delete_elem() Atomic write (sync_fetch_and_add) eBPF offload performance optimizations Localized packet cache 32-bit BPF support
  • 40. © 2018 NETRONOME SYSTEMS, INC. 40 How to Participate with eBPF? Netronome Guides and Firmware ▶  https://help.netronome.com/support/solutions/folders/36000172266 Demo Applications ▶  https://github.com/Netronome/bpf-samples
  • 41. © 2018 NETRONOME SYSTEMS, INC. Thank You