SlideShare a Scribd company logo
1©2016 Open-NFP
Open-NFP Summer Webinar Series:
Session 4: P4, EBPF And Linux TC Offload
Dinan Gunawardena & Jakub Kicinski -
Netronome
August 24, 2016
2©2016 Open-NFP
Open-NFP www.open-nfp.org
Support and grow reusable research in accelerating dataplane network functions processing
Reduce/eliminate the cost and technology barriers to research in this space
•  Technologies:
–  P4, eBPF, SDN, OpenFlow, Open vSwitch (OVS) offload
•  Tools:
–  Discounted hardware, development tools, software, cloud access
•  Community:
–  Website (www.open-nfp.org): learning & training materials, active Google group
https://groups.google.com/d/forum/open-nfp, open project descriptions, code repository
•  Learning/Education/Research support:
–  Summer seminar series, P4DevCon conference, Tutorials (P4 Developer Day), research proposal support
for proposals to the NSF, state agencies
Summer seminar series to further progress to our objective. Present applied reusable research.
3©2016 Open-NFP
Universities Companies
P4DevCon Attendees/Open-NFP Projects*
*This	does	not	imply	that	these	organiza4ons	endorse	Open-NFP	or	Netronome
4©2016 Open-NFP
Session Agenda
1. Introduction
–  Objectives
2. Overview: The high level model for offload
-  What we are offloading – P4 / eBPF
-  Overall programmer model for transparent
offload
3. Linux Kernel Infrastructure
–  The Traffic Classifier (TC)
–  eXpress Data Path (XDP)
–  Current eBPF translation on X86/ARM64/PPC64
4. Hardware Intro to NFP (Network Flow
Processor) architecture
–  SmartNICs-Multi Core, Many Core
–  NUMA, Memory Hierarchy
5. Accelerating P4/eBPF in NFP :
Implementation
–  Kernel core infrastructure
–  Map handling in the kernel
–  Translating instructions
–  Basic Map Support
–  Optimizations
6. & 7. Demo; Summary
5©2016 Open-NFP
Session Objectives
6©2016 Open-NFP
Introduction: Objectives
Understanding how eBPF is relevant to P4
Understanding the support for offload and state of art in the Linux Kernel
The Code
– Understand the structure of a eBPF program
– Gain an insight into how this is translated and executed in hardware
Understanding how the NFP architecture on the Agilio CX enables high
performing, fully programmable network offload
– The Many Core architecture and its advantages
7©2016 Open-NFP
Overview: High level model for offload
•  What we are offloading – P4 / eBPF
•  Overall programmer model for
transparent offload
8©2016 Open-NFP
P4 and eBPF
What are P4 and eBPF?
Domain specific languages for specifying forwarding
behaviour of the data plane of network components
P4 - Programming Protocol-Independent Packet
Processors
•  Header format description
•  Parse Graphs (FSM)
•  Tables (<keys,actions>)
•  Actions manipulate packet header/metadata
•  Control flow – an imperative program, describing
sequence of data dependent match/actions
eBPF – Extended Berkley Packet Filters
•  Low level (machine code like) language
•  Code executed by a VM (restricted memory, no
sleeps/locks, limited API to kernel)in the Kernel (TC)
•  Code injected into netfilter hook points in kernel data
plane
•  Maps (<key, value> stores)
•  Chained filter functions
•  Match/action
•  Static verification of safety, guaranteed to terminate
9©2016 Open-NFP
Translating P4->eBPF
John Fastabend P4 to eBPF compiler
Why translate P4 to eBPF?
Table	and	diagrams	©	Mihai	Budiu
10©2016 Open-NFP
Model for Transparent Offload
Programmer / user is “unaware” that eBPF code is “offloaded”
Requirements
• Partial pipeline offload
• Fallback to software for any eBPF code block
• Transparent user mode / kernel mode access to tables
eBPF	
Program	2	
(HW	offload)	
eBPF	
Program	3	
Packet	
In	
eBPF	
Program	1	
(HW	offload)	
PCI-E	crossing
11©2016 Open-NFP
Linux Kernel Infrastucture
•  The Traffic Classifier (TC)
•  eXpress Data Path (XDP)
12©2016 Open-NFP
Linux Traffic Classifier (TC)
Component	 Linux	Component	
Shaping	 The	class	offers	shaping	capabili4es	
Scheduling	 A	qdisc	acts	as	a	scheduler	e.g.	FIFO	
Classifying	 The	filter	performs	classifica4on	through	a	classifier	
object.	
Policing	 A	policer	performs	policing	within	a	filter	
Dropping	 The	“drop”	ac4on	occurs	with	a	filter+policer	
Marking	 The	dsmark	qdisc	is	used	for	marking	
eBPF	
program	
TC	diagram	and	example	program	©	Jamal	Hadi	Salim
13©2016 Open-NFP
eXpress Data Path (XDP)
What
•  High performance,
programmable network
data path
Utility
•  Useful for packet processing
•  forwarding
•  load balancing
•  DOS mitigation
•  firewalls, etc.
XDP	architecture	diagram	©	Tom	Herbert
14©2016 Open-NFP
Hardware Intro to NFP (Network Flow
Processor) architecture
•  NUMA, Memory Hierarchy
•  Current eBPF translation on X86/ARM64/
PPC64
15©2016 Open-NFP
NUMA, Memory Hierarchy
Architecural Philosopies:
•  Bring the data close to where it needs to be processed
•  Facilitate highly concurrent access to memories
•  Mitigate branch costs and hide I/O & memory access
latencies
16©2016 Open-NFP
Current eBPF translation on X86/ARM64/PPC64
1) Write eBPF program as a simple
C Program
2) Compiled to eBPF byte code
3) Loaded into the Linux TC
4) Run through verifier
5) Cross compiled to X86/ARM64/PPC64
… Or now NFP Byte Code!
NFP	hardware	
offloaded	eBPF	
TC
17©2016 Open-NFP
Dataflow
User	Space	
Traffic	Classifier	(TC)	
Driver	(XDP)	
Firmware	&	Hardware	
PCI-E	
Network	
Kernel	
User	
NFP	
TCP	Stack	
Packet	+	Descriptor,	
Meta	data	
Packet	+	Sk_Buf
18©2016 Open-NFP
Supported Actions
User	Space	
Traffic	Classifier	(TC)	
Driver	(XDP)	
Firmware	&	Hardware	
PCI-E	
Network	
Kernel	
User	
NFP	
TCP	Stack	
Packet	+	Descriptor,	
Meta	data	
Packet	+	Sk_Buf	
Supported	Ac2ons	
•  Drop	
•  Mark	
•  Redirect	
Redirect	Drop	
Mark
19©2016 Open-NFP
Accelerating P4/eBPF in NFP :
Implementation
•  Kernel core infrastructure
•  Map handling in the kernel
•  eBPF to NFP
•  Map Support
•  Optimizations
20©2016 Open-NFP
Kernel core infrastructure
User	Space	
Traffic	Classifier	(TC)	
Driver	(XDP)	
NFP	Offload	
Control	
eBPF	Program	
+	Flags	
Firmware	&	Hardware	
PCI-E	
Network	
Kernel	
User	
NFP	
TCP	Stack	
skip_sw	|	skip_hw	
Offload	obj	
Stats
21©2016 Open-NFP
eBPF to NFP
eBPF	Registers	
10	x	
64bits	
MAP	
DRAM	
X	 X	
X	 X	
A	
B	
16	x		
	2	x	
32bits	
ME	Registers	
Translate	
Communica4on	
Registers	
NFP	
•  Non-linear mapping
•  32 bit translation
22©2016 Open-NFP
•  Write interception
•  Ownership
•  Associating with a device
•  Read-Only single-user maps
•  Read-Only multi-user maps
•  Write-Enabled single-user maps
•  Write-Enabled multi-user maps
Map Handling in Kernel & Map Write Reflection
User	
Space	
Network	
MAP	 Kernel	
Space	
NFP	MAP	
Write	Reflec4on	
PCI-E	
Control	
Applica4on
23©2016 Open-NFP
Optimizations
•  Dealing with different memory types
•  Peephole Optimizer
•  Dropping unnecessary jumps
•  Optimizing instructions with immediates
•  Fusing instructions
•  Full A/B register allocations
•  Liveness analysis with real state size tracking
24©2016 Open-NFP
Demo
25©2016 Open-NFP
Summary
•  Learnt the relationship between P4 and eBPF
•  Discovered the infrastructure in the Linux Kernel for eBPF
offload
•  Learnt about how the Netronome Smart NIC architecture is
optimised for network flow processing
•  Explored how we implemented the eBPF offload in hardware
26©2016 Open-NFP
QUESTIONS?
Dinan Gunawardena
dinan.gunawardena@netronome.com
Jakub Kicinski
Jakub.Kicinski@netronome.com
27©2016 Open-NFP
THANK YOU

More Related Content

What's hot

Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!
Ray Jenkins
 
BPF Internals (eBPF)
BPF Internals (eBPF)BPF Internals (eBPF)
BPF Internals (eBPF)
Brendan Gregg
 
eBPF maps 101
eBPF maps 101eBPF maps 101
eBPF maps 101
SUSE Labs Taipei
 
Neutron packet logging framework
Neutron packet logging frameworkNeutron packet logging framework
Neutron packet logging framework
Vietnam Open Infrastructure User Group
 
Using eBPF for High-Performance Networking in Cilium
Using eBPF for High-Performance Networking in CiliumUsing eBPF for High-Performance Networking in Cilium
Using eBPF for High-Performance Networking in Cilium
ScyllaDB
 
OFI libfabric Tutorial
OFI libfabric TutorialOFI libfabric Tutorial
OFI libfabric Tutorial
dgoodell
 
NETCONFとYANGの話
NETCONFとYANGの話NETCONFとYANGの話
NETCONFとYANGの話
Masakazu Asama
 
CETH for XDP [Linux Meetup Santa Clara | July 2016]
CETH for XDP [Linux Meetup Santa Clara | July 2016] CETH for XDP [Linux Meetup Santa Clara | July 2016]
CETH for XDP [Linux Meetup Santa Clara | July 2016]
IO Visor Project
 
Faster packet processing in Linux: XDP
Faster packet processing in Linux: XDPFaster packet processing in Linux: XDP
Faster packet processing in Linux: XDP
Daniel T. Lee
 
GTPing, How To
GTPing, How ToGTPing, How To
GTPing, How To
Kentaro Ebisawa
 
netfilter and iptables
netfilter and iptablesnetfilter and iptables
netfilter and iptables
Kernel TLV
 
eBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to UserspaceeBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to Userspace
SUSE Labs Taipei
 
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPDockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
Thomas Graf
 
FD.io VPP事始め
FD.io VPP事始めFD.io VPP事始め
FD.io VPP事始め
tetsusat
 
eBPF - Rethinking the Linux Kernel
eBPF - Rethinking the Linux KerneleBPF - Rethinking the Linux Kernel
eBPF - Rethinking the Linux Kernel
Thomas Graf
 
プログラマ目線から見たRDMAのメリットと その応用例について
プログラマ目線から見たRDMAのメリットとその応用例についてプログラマ目線から見たRDMAのメリットとその応用例について
プログラマ目線から見たRDMAのメリットと その応用例について
Masanori Itoh
 
Packet Walk(s) In Kubernetes
Packet Walk(s) In KubernetesPacket Walk(s) In Kubernetes
Packet Walk(s) In Kubernetes
Don Jayakody
 
Understanding DPDK algorithmics
Understanding DPDK algorithmicsUnderstanding DPDK algorithmics
Understanding DPDK algorithmics
Denys Haryachyy
 
UM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of SoftwareUM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of Software
Brendan Gregg
 
大規模サービスを支えるネットワークインフラの全貌
大規模サービスを支えるネットワークインフラの全貌大規模サービスを支えるネットワークインフラの全貌
大規模サービスを支えるネットワークインフラの全貌
LINE Corporation
 

What's hot (20)

Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!
 
BPF Internals (eBPF)
BPF Internals (eBPF)BPF Internals (eBPF)
BPF Internals (eBPF)
 
eBPF maps 101
eBPF maps 101eBPF maps 101
eBPF maps 101
 
Neutron packet logging framework
Neutron packet logging frameworkNeutron packet logging framework
Neutron packet logging framework
 
Using eBPF for High-Performance Networking in Cilium
Using eBPF for High-Performance Networking in CiliumUsing eBPF for High-Performance Networking in Cilium
Using eBPF for High-Performance Networking in Cilium
 
OFI libfabric Tutorial
OFI libfabric TutorialOFI libfabric Tutorial
OFI libfabric Tutorial
 
NETCONFとYANGの話
NETCONFとYANGの話NETCONFとYANGの話
NETCONFとYANGの話
 
CETH for XDP [Linux Meetup Santa Clara | July 2016]
CETH for XDP [Linux Meetup Santa Clara | July 2016] CETH for XDP [Linux Meetup Santa Clara | July 2016]
CETH for XDP [Linux Meetup Santa Clara | July 2016]
 
Faster packet processing in Linux: XDP
Faster packet processing in Linux: XDPFaster packet processing in Linux: XDP
Faster packet processing in Linux: XDP
 
GTPing, How To
GTPing, How ToGTPing, How To
GTPing, How To
 
netfilter and iptables
netfilter and iptablesnetfilter and iptables
netfilter and iptables
 
eBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to UserspaceeBPF Trace from Kernel to Userspace
eBPF Trace from Kernel to Userspace
 
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDPDockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
DockerCon 2017 - Cilium - Network and Application Security with BPF and XDP
 
FD.io VPP事始め
FD.io VPP事始めFD.io VPP事始め
FD.io VPP事始め
 
eBPF - Rethinking the Linux Kernel
eBPF - Rethinking the Linux KerneleBPF - Rethinking the Linux Kernel
eBPF - Rethinking the Linux Kernel
 
プログラマ目線から見たRDMAのメリットと その応用例について
プログラマ目線から見たRDMAのメリットとその応用例についてプログラマ目線から見たRDMAのメリットとその応用例について
プログラマ目線から見たRDMAのメリットと その応用例について
 
Packet Walk(s) In Kubernetes
Packet Walk(s) In KubernetesPacket Walk(s) In Kubernetes
Packet Walk(s) In Kubernetes
 
Understanding DPDK algorithmics
Understanding DPDK algorithmicsUnderstanding DPDK algorithmics
Understanding DPDK algorithmics
 
UM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of SoftwareUM2019 Extended BPF: A New Type of Software
UM2019 Extended BPF: A New Type of Software
 
大規模サービスを支えるネットワークインフラの全貌
大規模サービスを支えるネットワークインフラの全貌大規模サービスを支えるネットワークインフラの全貌
大規模サービスを支えるネットワークインフラの全貌
 

Similar to P4, EPBF, and Linux TC Offload

P4 for Custom Identification, Flow Tagging, Monitoring and Control
P4 for Custom Identification, Flow Tagging, Monitoring and ControlP4 for Custom Identification, Flow Tagging, Monitoring and Control
P4 for Custom Identification, Flow Tagging, Monitoring and Control
Open-NFP
 
High Performance Flow Matching Architecture for Openflow Data Plane
High Performance Flow Matching Architecture for Openflow Data PlaneHigh Performance Flow Matching Architecture for Openflow Data Plane
High Performance Flow Matching Architecture for Openflow Data Plane
Mahesh Dananjaya
 
1. OPNFV Updates @ Tokyo Meetup#2
1. OPNFV Updates @ Tokyo Meetup#21. OPNFV Updates @ Tokyo Meetup#2
1. OPNFV Updates @ Tokyo Meetup#2
Mibu Ryota
 
eBPF Basics
eBPF BasicseBPF Basics
eBPF Basics
Michael Kehoe
 
Programming the Network Data Plane
Programming the Network Data PlaneProgramming the Network Data Plane
Programming the Network Data Plane
C4Media
 
High-Performance and Scalable Designs of Programming Models for Exascale Systems
High-Performance and Scalable Designs of Programming Models for Exascale SystemsHigh-Performance and Scalable Designs of Programming Models for Exascale Systems
High-Performance and Scalable Designs of Programming Models for Exascale Systems
inside-BigData.com
 
P4+ONOS SRv6 tutorial.pptx
P4+ONOS SRv6 tutorial.pptxP4+ONOS SRv6 tutorial.pptx
P4+ONOS SRv6 tutorial.pptx
tampham61268
 
Stacks and Layers: Integrating P4, C, OVS and OpenStack
Stacks and Layers: Integrating P4, C, OVS and OpenStackStacks and Layers: Integrating P4, C, OVS and OpenStack
Stacks and Layers: Integrating P4, C, OVS and OpenStack
Open-NFP
 
Data Plane and VNF Acceleration Mini Summit
Data Plane and VNF Acceleration Mini Summit Data Plane and VNF Acceleration Mini Summit
Data Plane and VNF Acceleration Mini Summit
Open-NFP
 
Scalable and Distributed DNN Training on Modern HPC Systems
Scalable and Distributed DNN Training on Modern HPC SystemsScalable and Distributed DNN Training on Modern HPC Systems
Scalable and Distributed DNN Training on Modern HPC Systems
inside-BigData.com
 
Communication Frameworks for HPC and Big Data
Communication Frameworks for HPC and Big DataCommunication Frameworks for HPC and Big Data
Communication Frameworks for HPC and Big Data
inside-BigData.com
 
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
inside-BigData.com
 
Transparent eBPF Offload: Playing Nice with the Linux Kernel
Transparent eBPF Offload: Playing Nice with the Linux KernelTransparent eBPF Offload: Playing Nice with the Linux Kernel
Transparent eBPF Offload: Playing Nice with the Linux Kernel
Open-NFP
 
Making our networking stack truly extensible
Making our networking stack truly extensible Making our networking stack truly extensible
Making our networking stack truly extensible
Olivier Bonaventure
 
Japan's post K Computer
Japan's post K ComputerJapan's post K Computer
Japan's post K Computer
inside-BigData.com
 
P4_tutorial.pdf
P4_tutorial.pdfP4_tutorial.pdf
P4_tutorial.pdf
PramodhN3
 
OpenPOWER Acceleration of HPCC Systems
OpenPOWER Acceleration of HPCC SystemsOpenPOWER Acceleration of HPCC Systems
OpenPOWER Acceleration of HPCC Systems
HPCC Systems
 
Designing Scalable HPC, Deep Learning and Cloud Middleware for Exascale Systems
Designing Scalable HPC, Deep Learning and Cloud Middleware for Exascale SystemsDesigning Scalable HPC, Deep Learning and Cloud Middleware for Exascale Systems
Designing Scalable HPC, Deep Learning and Cloud Middleware for Exascale Systems
inside-BigData.com
 
Panda scalable hpc_bestpractices_tue100418
Panda scalable hpc_bestpractices_tue100418Panda scalable hpc_bestpractices_tue100418
Panda scalable hpc_bestpractices_tue100418
inside-BigData.com
 
2017 dagstuhl-nfv-rothenberg
2017 dagstuhl-nfv-rothenberg2017 dagstuhl-nfv-rothenberg
2017 dagstuhl-nfv-rothenberg
Christian Esteve Rothenberg
 

Similar to P4, EPBF, and Linux TC Offload (20)

P4 for Custom Identification, Flow Tagging, Monitoring and Control
P4 for Custom Identification, Flow Tagging, Monitoring and ControlP4 for Custom Identification, Flow Tagging, Monitoring and Control
P4 for Custom Identification, Flow Tagging, Monitoring and Control
 
High Performance Flow Matching Architecture for Openflow Data Plane
High Performance Flow Matching Architecture for Openflow Data PlaneHigh Performance Flow Matching Architecture for Openflow Data Plane
High Performance Flow Matching Architecture for Openflow Data Plane
 
1. OPNFV Updates @ Tokyo Meetup#2
1. OPNFV Updates @ Tokyo Meetup#21. OPNFV Updates @ Tokyo Meetup#2
1. OPNFV Updates @ Tokyo Meetup#2
 
eBPF Basics
eBPF BasicseBPF Basics
eBPF Basics
 
Programming the Network Data Plane
Programming the Network Data PlaneProgramming the Network Data Plane
Programming the Network Data Plane
 
High-Performance and Scalable Designs of Programming Models for Exascale Systems
High-Performance and Scalable Designs of Programming Models for Exascale SystemsHigh-Performance and Scalable Designs of Programming Models for Exascale Systems
High-Performance and Scalable Designs of Programming Models for Exascale Systems
 
P4+ONOS SRv6 tutorial.pptx
P4+ONOS SRv6 tutorial.pptxP4+ONOS SRv6 tutorial.pptx
P4+ONOS SRv6 tutorial.pptx
 
Stacks and Layers: Integrating P4, C, OVS and OpenStack
Stacks and Layers: Integrating P4, C, OVS and OpenStackStacks and Layers: Integrating P4, C, OVS and OpenStack
Stacks and Layers: Integrating P4, C, OVS and OpenStack
 
Data Plane and VNF Acceleration Mini Summit
Data Plane and VNF Acceleration Mini Summit Data Plane and VNF Acceleration Mini Summit
Data Plane and VNF Acceleration Mini Summit
 
Scalable and Distributed DNN Training on Modern HPC Systems
Scalable and Distributed DNN Training on Modern HPC SystemsScalable and Distributed DNN Training on Modern HPC Systems
Scalable and Distributed DNN Training on Modern HPC Systems
 
Communication Frameworks for HPC and Big Data
Communication Frameworks for HPC and Big DataCommunication Frameworks for HPC and Big Data
Communication Frameworks for HPC and Big Data
 
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
 
Transparent eBPF Offload: Playing Nice with the Linux Kernel
Transparent eBPF Offload: Playing Nice with the Linux KernelTransparent eBPF Offload: Playing Nice with the Linux Kernel
Transparent eBPF Offload: Playing Nice with the Linux Kernel
 
Making our networking stack truly extensible
Making our networking stack truly extensible Making our networking stack truly extensible
Making our networking stack truly extensible
 
Japan's post K Computer
Japan's post K ComputerJapan's post K Computer
Japan's post K Computer
 
P4_tutorial.pdf
P4_tutorial.pdfP4_tutorial.pdf
P4_tutorial.pdf
 
OpenPOWER Acceleration of HPCC Systems
OpenPOWER Acceleration of HPCC SystemsOpenPOWER Acceleration of HPCC Systems
OpenPOWER Acceleration of HPCC Systems
 
Designing Scalable HPC, Deep Learning and Cloud Middleware for Exascale Systems
Designing Scalable HPC, Deep Learning and Cloud Middleware for Exascale SystemsDesigning Scalable HPC, Deep Learning and Cloud Middleware for Exascale Systems
Designing Scalable HPC, Deep Learning and Cloud Middleware for Exascale Systems
 
Panda scalable hpc_bestpractices_tue100418
Panda scalable hpc_bestpractices_tue100418Panda scalable hpc_bestpractices_tue100418
Panda scalable hpc_bestpractices_tue100418
 
2017 dagstuhl-nfv-rothenberg
2017 dagstuhl-nfv-rothenberg2017 dagstuhl-nfv-rothenberg
2017 dagstuhl-nfv-rothenberg
 

More from Open-NFP

Accelerating Networked Applications with Flexible Packet Processing
Accelerating Networked Applications with Flexible Packet ProcessingAccelerating Networked Applications with Flexible Packet Processing
Accelerating Networked Applications with Flexible Packet Processing
Open-NFP
 
Network Measurement with P4 and C on Netronome Agilio
Network Measurement with P4 and C on Netronome AgilioNetwork Measurement with P4 and C on Netronome Agilio
Network Measurement with P4 and C on Netronome Agilio
Open-NFP
 
Measuring a 25 and 40Gb/s Data Plane
Measuring a 25 and 40Gb/s Data PlaneMeasuring a 25 and 40Gb/s Data Plane
Measuring a 25 and 40Gb/s Data Plane
Open-NFP
 
OpenContrail, Real Speed: Offloading vRouter
OpenContrail, Real Speed: Offloading vRouterOpenContrail, Real Speed: Offloading vRouter
OpenContrail, Real Speed: Offloading vRouter
Open-NFP
 
Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...
Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...
Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...
Open-NFP
 
P4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server Adapters
P4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server AdaptersP4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server Adapters
P4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server Adapters
Open-NFP
 
Consensus as a Network Service
Consensus as a Network ServiceConsensus as a Network Service
Consensus as a Network Service
Open-NFP
 

More from Open-NFP (7)

Accelerating Networked Applications with Flexible Packet Processing
Accelerating Networked Applications with Flexible Packet ProcessingAccelerating Networked Applications with Flexible Packet Processing
Accelerating Networked Applications with Flexible Packet Processing
 
Network Measurement with P4 and C on Netronome Agilio
Network Measurement with P4 and C on Netronome AgilioNetwork Measurement with P4 and C on Netronome Agilio
Network Measurement with P4 and C on Netronome Agilio
 
Measuring a 25 and 40Gb/s Data Plane
Measuring a 25 and 40Gb/s Data PlaneMeasuring a 25 and 40Gb/s Data Plane
Measuring a 25 and 40Gb/s Data Plane
 
OpenContrail, Real Speed: Offloading vRouter
OpenContrail, Real Speed: Offloading vRouterOpenContrail, Real Speed: Offloading vRouter
OpenContrail, Real Speed: Offloading vRouter
 
Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...
Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...
Protecting the Privacy of the Network – Using P4 to Prototype and Extend Netw...
 
P4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server Adapters
P4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server AdaptersP4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server Adapters
P4-based VNF and Micro-VNF Chaining for Servers With Intelligent Server Adapters
 
Consensus as a Network Service
Consensus as a Network ServiceConsensus as a Network Service
Consensus as a Network Service
 

Recently uploaded

What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Data Hops
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 

Recently uploaded (20)

What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 

P4, EPBF, and Linux TC Offload

  • 1. 1©2016 Open-NFP Open-NFP Summer Webinar Series: Session 4: P4, EBPF And Linux TC Offload Dinan Gunawardena & Jakub Kicinski - Netronome August 24, 2016
  • 2. 2©2016 Open-NFP Open-NFP www.open-nfp.org Support and grow reusable research in accelerating dataplane network functions processing Reduce/eliminate the cost and technology barriers to research in this space •  Technologies: –  P4, eBPF, SDN, OpenFlow, Open vSwitch (OVS) offload •  Tools: –  Discounted hardware, development tools, software, cloud access •  Community: –  Website (www.open-nfp.org): learning & training materials, active Google group https://groups.google.com/d/forum/open-nfp, open project descriptions, code repository •  Learning/Education/Research support: –  Summer seminar series, P4DevCon conference, Tutorials (P4 Developer Day), research proposal support for proposals to the NSF, state agencies Summer seminar series to further progress to our objective. Present applied reusable research.
  • 3. 3©2016 Open-NFP Universities Companies P4DevCon Attendees/Open-NFP Projects* *This does not imply that these organiza4ons endorse Open-NFP or Netronome
  • 4. 4©2016 Open-NFP Session Agenda 1. Introduction –  Objectives 2. Overview: The high level model for offload -  What we are offloading – P4 / eBPF -  Overall programmer model for transparent offload 3. Linux Kernel Infrastructure –  The Traffic Classifier (TC) –  eXpress Data Path (XDP) –  Current eBPF translation on X86/ARM64/PPC64 4. Hardware Intro to NFP (Network Flow Processor) architecture –  SmartNICs-Multi Core, Many Core –  NUMA, Memory Hierarchy 5. Accelerating P4/eBPF in NFP : Implementation –  Kernel core infrastructure –  Map handling in the kernel –  Translating instructions –  Basic Map Support –  Optimizations 6. & 7. Demo; Summary
  • 6. 6©2016 Open-NFP Introduction: Objectives Understanding how eBPF is relevant to P4 Understanding the support for offload and state of art in the Linux Kernel The Code – Understand the structure of a eBPF program – Gain an insight into how this is translated and executed in hardware Understanding how the NFP architecture on the Agilio CX enables high performing, fully programmable network offload – The Many Core architecture and its advantages
  • 7. 7©2016 Open-NFP Overview: High level model for offload •  What we are offloading – P4 / eBPF •  Overall programmer model for transparent offload
  • 8. 8©2016 Open-NFP P4 and eBPF What are P4 and eBPF? Domain specific languages for specifying forwarding behaviour of the data plane of network components P4 - Programming Protocol-Independent Packet Processors •  Header format description •  Parse Graphs (FSM) •  Tables (<keys,actions>) •  Actions manipulate packet header/metadata •  Control flow – an imperative program, describing sequence of data dependent match/actions eBPF – Extended Berkley Packet Filters •  Low level (machine code like) language •  Code executed by a VM (restricted memory, no sleeps/locks, limited API to kernel)in the Kernel (TC) •  Code injected into netfilter hook points in kernel data plane •  Maps (<key, value> stores) •  Chained filter functions •  Match/action •  Static verification of safety, guaranteed to terminate
  • 9. 9©2016 Open-NFP Translating P4->eBPF John Fastabend P4 to eBPF compiler Why translate P4 to eBPF? Table and diagrams © Mihai Budiu
  • 10. 10©2016 Open-NFP Model for Transparent Offload Programmer / user is “unaware” that eBPF code is “offloaded” Requirements • Partial pipeline offload • Fallback to software for any eBPF code block • Transparent user mode / kernel mode access to tables eBPF Program 2 (HW offload) eBPF Program 3 Packet In eBPF Program 1 (HW offload) PCI-E crossing
  • 11. 11©2016 Open-NFP Linux Kernel Infrastucture •  The Traffic Classifier (TC) •  eXpress Data Path (XDP)
  • 12. 12©2016 Open-NFP Linux Traffic Classifier (TC) Component Linux Component Shaping The class offers shaping capabili4es Scheduling A qdisc acts as a scheduler e.g. FIFO Classifying The filter performs classifica4on through a classifier object. Policing A policer performs policing within a filter Dropping The “drop” ac4on occurs with a filter+policer Marking The dsmark qdisc is used for marking eBPF program TC diagram and example program © Jamal Hadi Salim
  • 13. 13©2016 Open-NFP eXpress Data Path (XDP) What •  High performance, programmable network data path Utility •  Useful for packet processing •  forwarding •  load balancing •  DOS mitigation •  firewalls, etc. XDP architecture diagram © Tom Herbert
  • 14. 14©2016 Open-NFP Hardware Intro to NFP (Network Flow Processor) architecture •  NUMA, Memory Hierarchy •  Current eBPF translation on X86/ARM64/ PPC64
  • 15. 15©2016 Open-NFP NUMA, Memory Hierarchy Architecural Philosopies: •  Bring the data close to where it needs to be processed •  Facilitate highly concurrent access to memories •  Mitigate branch costs and hide I/O & memory access latencies
  • 16. 16©2016 Open-NFP Current eBPF translation on X86/ARM64/PPC64 1) Write eBPF program as a simple C Program 2) Compiled to eBPF byte code 3) Loaded into the Linux TC 4) Run through verifier 5) Cross compiled to X86/ARM64/PPC64 … Or now NFP Byte Code! NFP hardware offloaded eBPF TC
  • 19. 19©2016 Open-NFP Accelerating P4/eBPF in NFP : Implementation •  Kernel core infrastructure •  Map handling in the kernel •  eBPF to NFP •  Map Support •  Optimizations
  • 20. 20©2016 Open-NFP Kernel core infrastructure User Space Traffic Classifier (TC) Driver (XDP) NFP Offload Control eBPF Program + Flags Firmware & Hardware PCI-E Network Kernel User NFP TCP Stack skip_sw | skip_hw Offload obj Stats
  • 21. 21©2016 Open-NFP eBPF to NFP eBPF Registers 10 x 64bits MAP DRAM X X X X A B 16 x 2 x 32bits ME Registers Translate Communica4on Registers NFP •  Non-linear mapping •  32 bit translation
  • 22. 22©2016 Open-NFP •  Write interception •  Ownership •  Associating with a device •  Read-Only single-user maps •  Read-Only multi-user maps •  Write-Enabled single-user maps •  Write-Enabled multi-user maps Map Handling in Kernel & Map Write Reflection User Space Network MAP Kernel Space NFP MAP Write Reflec4on PCI-E Control Applica4on
  • 23. 23©2016 Open-NFP Optimizations •  Dealing with different memory types •  Peephole Optimizer •  Dropping unnecessary jumps •  Optimizing instructions with immediates •  Fusing instructions •  Full A/B register allocations •  Liveness analysis with real state size tracking
  • 25. 25©2016 Open-NFP Summary •  Learnt the relationship between P4 and eBPF •  Discovered the infrastructure in the Linux Kernel for eBPF offload •  Learnt about how the Netronome Smart NIC architecture is optimised for network flow processing •  Explored how we implemented the eBPF offload in hardware