Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1©2016 Open-NFP
Open-NFP Summer Webinar Series:
Session 4: P4, EBPF And Linux TC Offload
Dinan Gunawardena & Jakub Kicinsk...
2©2016 Open-NFP
Open-NFP www.open-nfp.org
Support and grow reusable research in accelerating dataplane network functions p...
3©2016 Open-NFP
Universities Companies
P4DevCon Attendees/Open-NFP Projects*
*This	does	not	imply	that	these	organiza4ons	...
4©2016 Open-NFP
Session Agenda
1. Introduction
–  Objectives
2. Overview: The high level model for offload
-  What we are ...
5©2016 Open-NFP
Session Objectives
6©2016 Open-NFP
Introduction: Objectives
Understanding how eBPF is relevant to P4
Understanding the support for offload an...
7©2016 Open-NFP
Overview: High level model for offload
•  What we are offloading – P4 / eBPF
•  Overall programmer model f...
8©2016 Open-NFP
P4 and eBPF
What are P4 and eBPF?
Domain specific languages for specifying forwarding
behaviour of the dat...
9©2016 Open-NFP
Translating P4->eBPF
John Fastabend P4 to eBPF compiler
Why translate P4 to eBPF?
Table	and	diagrams	©	Mih...
10©2016 Open-NFP
Model for Transparent Offload
Programmer / user is “unaware” that eBPF code is “offloaded”
Requirements
•...
11©2016 Open-NFP
Linux Kernel Infrastucture
•  The Traffic Classifier (TC)
•  eXpress Data Path (XDP)
12©2016 Open-NFP
Linux Traffic Classifier (TC)
Component	 Linux	Component	
Shaping	 The	class	offers	shaping	capabili4es	
S...
13©2016 Open-NFP
eXpress Data Path (XDP)
What
•  High performance,
programmable network
data path
Utility
•  Useful for pa...
14©2016 Open-NFP
Hardware Intro to NFP (Network Flow
Processor) architecture
•  NUMA, Memory Hierarchy
•  Current eBPF tra...
15©2016 Open-NFP
NUMA, Memory Hierarchy
Architecural Philosopies:
•  Bring the data close to where it needs to be processe...
16©2016 Open-NFP
Current eBPF translation on X86/ARM64/PPC64
1) Write eBPF program as a simple
C Program
2) Compiled to eB...
17©2016 Open-NFP
Dataflow
User	Space	
Traffic	Classifier	(TC)	
Driver	(XDP)	
Firmware	&	Hardware	
PCI-E	
Network	
Kernel	
Use...
18©2016 Open-NFP
Supported Actions
User	Space	
Traffic	Classifier	(TC)	
Driver	(XDP)	
Firmware	&	Hardware	
PCI-E	
Network	
Ke...
19©2016 Open-NFP
Accelerating P4/eBPF in NFP :
Implementation
•  Kernel core infrastructure
•  Map handling in the kernel
...
20©2016 Open-NFP
Kernel core infrastructure
User	Space	
Traffic	Classifier	(TC)	
Driver	(XDP)	
NFP	Offload	
Control	
eBPF	Progr...
21©2016 Open-NFP
eBPF to NFP
eBPF	Registers	
10	x	
64bits	
MAP	
DRAM	
X	 X	
X	 X	
A	
B	
16	x		
	2	x	
32bits	
ME	Registers	...
22©2016 Open-NFP
•  Write interception
•  Ownership
•  Associating with a device
•  Read-Only single-user maps
•  Read-Onl...
23©2016 Open-NFP
Optimizations
•  Dealing with different memory types
•  Peephole Optimizer
•  Dropping unnecessary jumps
...
24©2016 Open-NFP
Demo
25©2016 Open-NFP
Summary
•  Learnt the relationship between P4 and eBPF
•  Discovered the infrastructure in the Linux Kern...
26©2016 Open-NFP
QUESTIONS?
Dinan Gunawardena
dinan.gunawardena@netronome.com
Jakub Kicinski
Jakub.Kicinski@netronome.com
27©2016 Open-NFP
THANK YOU
Upcoming SlideShare
Loading in …5
×

P4, EPBF, and Linux TC Offload

3,314 views

Published on

In this talk we discuss the mechanisms of utilizing the eBPF language to perform hardware accelerated network packet manipulation and filtering. P4 programs can be compiled into eBPF scripts for offload in the Linux kernel using the Traffic Classifier (TC) subsystem. We demonstrate how, using eBPF as an intermediate language, it has been possible to extend the TC to either Just In Time (JIT) compile eBPF code to x86 assembler for software offload or to IXP byte code for execution in a trusted hardware environment within the Netronome Agilio intelligent server adapter. We finish by encouraging the audience to experiment with their own eBPF applications within the TC hardware accelerated system. The TC kernel patches are available on the Linux Kernel Networking mailing list as a Request For Comment (RFC) contribution.

Dinan Gunawardena, Director, Software Engineering, Netronome

Dinan Gunawardena is a Software Director focusing on running the driver team at Netronome. Previously, Dinan founded a software startup and was a Senior Research Engineer within the Operating Systems and Networking Group at Microsoft Research for 12 years, shipping technology in several versions of Microsoft Windows and the Bing Search Engine. Dinan has received over 20 patents and is a Chartered Software Engineer. Dinan has a Masters in Computer Science from University of Cambridge and a M.B.A. from WBS.

Jakub Kicinski, Software Engineering, Netronome

Jakub Kicinski is a Software Engineer specializing in the Linux Kernel drivers for Netronome SmartNICs. Jakub has previously worked as an intern for Intel Corporation. Jakub is also a researcher with expertise in Linux kernel. Experience in application development on complex multi-CPU and FPGA platforms. He is interested in high-performance software exploiting hardware capabilities and is passionate about networking. Jakub has a Masters in Computer Science from Gdansk University of Technology.

Published in: Technology
  • Be the first to comment

P4, EPBF, and Linux TC Offload

  1. 1. 1©2016 Open-NFP Open-NFP Summer Webinar Series: Session 4: P4, EBPF And Linux TC Offload Dinan Gunawardena & Jakub Kicinski - Netronome August 24, 2016
  2. 2. 2©2016 Open-NFP Open-NFP www.open-nfp.org Support and grow reusable research in accelerating dataplane network functions processing Reduce/eliminate the cost and technology barriers to research in this space •  Technologies: –  P4, eBPF, SDN, OpenFlow, Open vSwitch (OVS) offload •  Tools: –  Discounted hardware, development tools, software, cloud access •  Community: –  Website (www.open-nfp.org): learning & training materials, active Google group https://groups.google.com/d/forum/open-nfp, open project descriptions, code repository •  Learning/Education/Research support: –  Summer seminar series, P4DevCon conference, Tutorials (P4 Developer Day), research proposal support for proposals to the NSF, state agencies Summer seminar series to further progress to our objective. Present applied reusable research.
  3. 3. 3©2016 Open-NFP Universities Companies P4DevCon Attendees/Open-NFP Projects* *This does not imply that these organiza4ons endorse Open-NFP or Netronome
  4. 4. 4©2016 Open-NFP Session Agenda 1. Introduction –  Objectives 2. Overview: The high level model for offload -  What we are offloading – P4 / eBPF -  Overall programmer model for transparent offload 3. Linux Kernel Infrastructure –  The Traffic Classifier (TC) –  eXpress Data Path (XDP) –  Current eBPF translation on X86/ARM64/PPC64 4. Hardware Intro to NFP (Network Flow Processor) architecture –  SmartNICs-Multi Core, Many Core –  NUMA, Memory Hierarchy 5. Accelerating P4/eBPF in NFP : Implementation –  Kernel core infrastructure –  Map handling in the kernel –  Translating instructions –  Basic Map Support –  Optimizations 6. & 7. Demo; Summary
  5. 5. 5©2016 Open-NFP Session Objectives
  6. 6. 6©2016 Open-NFP Introduction: Objectives Understanding how eBPF is relevant to P4 Understanding the support for offload and state of art in the Linux Kernel The Code – Understand the structure of a eBPF program – Gain an insight into how this is translated and executed in hardware Understanding how the NFP architecture on the Agilio CX enables high performing, fully programmable network offload – The Many Core architecture and its advantages
  7. 7. 7©2016 Open-NFP Overview: High level model for offload •  What we are offloading – P4 / eBPF •  Overall programmer model for transparent offload
  8. 8. 8©2016 Open-NFP P4 and eBPF What are P4 and eBPF? Domain specific languages for specifying forwarding behaviour of the data plane of network components P4 - Programming Protocol-Independent Packet Processors •  Header format description •  Parse Graphs (FSM) •  Tables (<keys,actions>) •  Actions manipulate packet header/metadata •  Control flow – an imperative program, describing sequence of data dependent match/actions eBPF – Extended Berkley Packet Filters •  Low level (machine code like) language •  Code executed by a VM (restricted memory, no sleeps/locks, limited API to kernel)in the Kernel (TC) •  Code injected into netfilter hook points in kernel data plane •  Maps (<key, value> stores) •  Chained filter functions •  Match/action •  Static verification of safety, guaranteed to terminate
  9. 9. 9©2016 Open-NFP Translating P4->eBPF John Fastabend P4 to eBPF compiler Why translate P4 to eBPF? Table and diagrams © Mihai Budiu
  10. 10. 10©2016 Open-NFP Model for Transparent Offload Programmer / user is “unaware” that eBPF code is “offloaded” Requirements • Partial pipeline offload • Fallback to software for any eBPF code block • Transparent user mode / kernel mode access to tables eBPF Program 2 (HW offload) eBPF Program 3 Packet In eBPF Program 1 (HW offload) PCI-E crossing
  11. 11. 11©2016 Open-NFP Linux Kernel Infrastucture •  The Traffic Classifier (TC) •  eXpress Data Path (XDP)
  12. 12. 12©2016 Open-NFP Linux Traffic Classifier (TC) Component Linux Component Shaping The class offers shaping capabili4es Scheduling A qdisc acts as a scheduler e.g. FIFO Classifying The filter performs classifica4on through a classifier object. Policing A policer performs policing within a filter Dropping The “drop” ac4on occurs with a filter+policer Marking The dsmark qdisc is used for marking eBPF program TC diagram and example program © Jamal Hadi Salim
  13. 13. 13©2016 Open-NFP eXpress Data Path (XDP) What •  High performance, programmable network data path Utility •  Useful for packet processing •  forwarding •  load balancing •  DOS mitigation •  firewalls, etc. XDP architecture diagram © Tom Herbert
  14. 14. 14©2016 Open-NFP Hardware Intro to NFP (Network Flow Processor) architecture •  NUMA, Memory Hierarchy •  Current eBPF translation on X86/ARM64/ PPC64
  15. 15. 15©2016 Open-NFP NUMA, Memory Hierarchy Architecural Philosopies: •  Bring the data close to where it needs to be processed •  Facilitate highly concurrent access to memories •  Mitigate branch costs and hide I/O & memory access latencies
  16. 16. 16©2016 Open-NFP Current eBPF translation on X86/ARM64/PPC64 1) Write eBPF program as a simple C Program 2) Compiled to eBPF byte code 3) Loaded into the Linux TC 4) Run through verifier 5) Cross compiled to X86/ARM64/PPC64 … Or now NFP Byte Code! NFP hardware offloaded eBPF TC
  17. 17. 17©2016 Open-NFP Dataflow User Space Traffic Classifier (TC) Driver (XDP) Firmware & Hardware PCI-E Network Kernel User NFP TCP Stack Packet + Descriptor, Meta data Packet + Sk_Buf
  18. 18. 18©2016 Open-NFP Supported Actions User Space Traffic Classifier (TC) Driver (XDP) Firmware & Hardware PCI-E Network Kernel User NFP TCP Stack Packet + Descriptor, Meta data Packet + Sk_Buf Supported Ac2ons •  Drop •  Mark •  Redirect Redirect Drop Mark
  19. 19. 19©2016 Open-NFP Accelerating P4/eBPF in NFP : Implementation •  Kernel core infrastructure •  Map handling in the kernel •  eBPF to NFP •  Map Support •  Optimizations
  20. 20. 20©2016 Open-NFP Kernel core infrastructure User Space Traffic Classifier (TC) Driver (XDP) NFP Offload Control eBPF Program + Flags Firmware & Hardware PCI-E Network Kernel User NFP TCP Stack skip_sw | skip_hw Offload obj Stats
  21. 21. 21©2016 Open-NFP eBPF to NFP eBPF Registers 10 x 64bits MAP DRAM X X X X A B 16 x 2 x 32bits ME Registers Translate Communica4on Registers NFP •  Non-linear mapping •  32 bit translation
  22. 22. 22©2016 Open-NFP •  Write interception •  Ownership •  Associating with a device •  Read-Only single-user maps •  Read-Only multi-user maps •  Write-Enabled single-user maps •  Write-Enabled multi-user maps Map Handling in Kernel & Map Write Reflection User Space Network MAP Kernel Space NFP MAP Write Reflec4on PCI-E Control Applica4on
  23. 23. 23©2016 Open-NFP Optimizations •  Dealing with different memory types •  Peephole Optimizer •  Dropping unnecessary jumps •  Optimizing instructions with immediates •  Fusing instructions •  Full A/B register allocations •  Liveness analysis with real state size tracking
  24. 24. 24©2016 Open-NFP Demo
  25. 25. 25©2016 Open-NFP Summary •  Learnt the relationship between P4 and eBPF •  Discovered the infrastructure in the Linux Kernel for eBPF offload •  Learnt about how the Netronome Smart NIC architecture is optimised for network flow processing •  Explored how we implemented the eBPF offload in hardware
  26. 26. 26©2016 Open-NFP QUESTIONS? Dinan Gunawardena dinan.gunawardena@netronome.com Jakub Kicinski Jakub.Kicinski@netronome.com
  27. 27. 27©2016 Open-NFP THANK YOU

×