This document discusses programmable networking and the future of networking. It provides an overview of programmable switches and how they differ from traditional fixed-function switches. Programmable switches use a domain-specific compiler called P4 to program the forwarding plane and define protocols, packet parsing, and processing pipelines. This allows the behavior of the switch to be defined through software rather than being fixed in hardware. The document demonstrates how P4 programs map to the Programmable Switch Architecture (PISA) and provides examples of simple and complex data plane programs. It also previews some demonstrations of in-band network telemetry and traffic monitoring capabilities enabled by programmable switches.

1. Programmable Forwarding Planes
and the
Future of Networking
Yandex Nexthop 2018
June 6, 2018
Copyright © 2018 - Barefoot Networks, Inc.

2. Most network systems are built “bottom-up”
Switch OS
Fixed-function switch
Driver
Datasheet
ASIC
Copyright © 2018 - Barefoot Networks, Inc.

3. Network systems will be programmed “top-down”
Programmable Switch
Driver
Switch OS“Precise behavior specified in P4”
Copyright © 2018 - Barefoot Networks, Inc.

4. Fixed Function Switches
Switch OS
Static Run-time API
Driver
Fixed Meta-Data
Traffic
Manager
Fixed
Parser
Fixed
Look
ups
Fixed
Memories
Fixed
Actio
ns
Fixed
Looku
ps
Fixed
Memories
Fixed
Action
s
Fixed
Looku
ps
Fixed
Memories
Fixed
Action
s
Fixed
Looku
ps
Fixed
Memories
Fixed
Action
s
Fixed
Packet
Mods
Fixed Function ASIC
Copyright © 2018 - Barefoot Networks, Inc.

5. Programmable Switch Approach
Fixed Meta-Data
Traffic
Manage
r
Fixed
Parser
Fixe
d
Look
ups
Fixed
Memories
Fixe
d
Look
ups
Fixe
d
Look
ups
Fixed
Memories
Fixe
d
Look
ups
Fixe
d
Look
ups
Fixed
Memories
Fixe
d
Look
ups
Fixe
d
Look
ups
Fixed
Memories
Fixe
d
Look
ups
Fixed
Packe
t Mods
Programmable Meta-Data
Traffic
Manager
Flexible
Parser
Flexibl
e
Looku
ps
Shared
Memories
Custo
m
Action
s
Flexibl
e
Looku
ps
Shared
Memories
Custo
m
Action
s
Flexibl
e
Looku
ps
Shared
Memories
Custo
m
Action
s
Flexibl
e
Looku
ps
Shared
Memories
Custo
m
Action
s
DeParser
Programmable ASIC
switch.p4
Protocol
Authoring
1
Compile
2
Configure
3
Auto Generated
Run-time API
Switch OS
Driver
Copyright © 2018 - Barefoot Networks, Inc.

6. P4 - Open and Community Owned
Original P4 Paper Authors:

7. P4 – Programming Language for Networking
Transport - FPGA
Routers - NPU
Server Connectivity - NIC
Leaf & Spine - Switch ASIC & ASSP
OS - Kernel Networking
Hypervisor - vSwitch
Copyright © 2018 - Barefoot Networks, Inc.

8. Domain Specific Processors
CPU
Computers
Java
Compiler
GPU
Graphics
OpenCL
Compiler
DSP
Signal
Processing
Matlab
Compiler
Machine
Learning
?
TPU
TensorFlow
Compiler
Networking
?
??
Compiler
>>>
Copyright © 2018 - Barefoot Networks, Inc.

9. Programmability Comes to Network Forwarding Plane
CPU
Computers
Java
Compiler
GPU
Graphics
OpenCL
Compiler
DSP
Signal
Processing
Matlab
Compiler
Machine
Learning
?
TPU
TensorFlow
Compiler
Tofino
(PISA)
Networking
P4
Compiler
>>>
Copyright © 2018 - Barefoot Networks, Inc.

10. Copyright © 2018 - Barefoot Networks, Inc.
Barefoot Tofino Summary
• State of the art design
◦ Single Shared Packet Buffer
◦ TSMC 16nm FinFET+
• Four Match+Action Pipelines
◦ Fully programmable PISA Embodiment
◦ All compiled programs run at line-rate.
◦ Programmable table scale
• Port Configurations
◦ 65 x 100GE/40GE
◦ 130 x 50GE
◦ 260 x 25GE/10GE
• CPU Interfaces
◦ PCIe: Gen3 x4/x2/x1
◦ Dedicated 100GE port
Match+Action
Pipeline 2
Match+Action
Pipeline 3
Match+Action
Pipeline 1
Match+Action
Pipeline 0
Shared
Packet
Buffer
&
TM
MAC + Serial I/O

11. Copyright © 2018 - Barefoot Networks, Inc.
Tofino Block Diagram
pipe 0
Rx MACs
10/25/40/50/100
Ingress
Pipeline
Tx MAC
10/25/40/50/100
Control & configuration
Reset /
Clocks
PCIe CPU MAC
DMA
engines
pipe 1
Rx MACs
10/25/40/50/100
Ingress
Pipeline
Tx MAC
10/25/40/50/100
pipe 2
Rx MACs
10/25/40/50/100
Ingress
Pipeline
Tx MAC
10/25/40/50/100
pipe 3
Rx MACs
10/25/40/50/100
Ingress
Pipeline
Tx MAC
10/25/40/50/100
Traffic
Manager
Egress
Pipeline
Egress
Pipeline
Egress
Pipeline
Egress
Pipeline
Pkt
Gen
Pkt
Gen
Pkt
Gen
Pkt
Gen

12. Copyright © 2018 - Barefoot Networks, Inc.
PISA: Protocol-Independent Switch Architecture
Match+Action
Stage (Unit)
Memory ALU
Programmable
Parser
Programmable
Deparser
Programmable Match-Action Pipeline
Programmer declares the
headers that should be
recognized and their order in
the packet
Programmer defines the
tables and the exact
processing algorithm
Programmer declares
how the output packet
will look on the wire

13. P4 Program
Memory ALU
Programmable
Parser Programmable Match-Action Pipeline
header_type ethernet_t { … }
header_type l2_metadata_t { … }
header ethernet_t ethernet;
header vlan_tag_t
vlan_tag[2];
metadata l2_metadata_t l2_meta;
Header and Data DeclarationsParser Program
parser parse_ethernet {
extract(ethernet);
return switch(ethernet.ethertype) {
0x8100 : parse_vlan_tag;
0x0800 : parse_ipv4;
0x8847 : parse_mpls;
default: ingress;
}
Tables and Control Flow
table port_table { … }
control ingress {
apply(port_table);
if (l2_meta.vlan_tags == 0) {
process_assign_vlan();
}
}
Copyright © 2018 - Barefoot Networks, Inc.

14. Copyright © 2018 - Barefoot Networks, Inc.
PISA in Action
• Packet is parsed into individual headers (parsed representation)
• Headers and intermediate results can be used for matching and actions
• Headers can be modified, added or removed
• Packet is deparsed (serialized)
Match+Action
Stage (Unit)
Programmable
Parser
Programmable
Deparser
Programmable Match-Action Pipeline

15. Copyright © 2018 - Barefoot Networks, Inc.
Mapping a Simple L3 Data Plane Program on PISA
Match+Action
Stage (Unit)
Programmable
Parser
Programmable
Deparser
Programmable Match-Action Pipeline
Ethernet
MAC
Address
Table
ACL
Rules
IPv4
Address
Table
L2 ACL
IPv4
IPv6
Address
Table
IPv6

16. Copyright © 2018 - Barefoot Networks, Inc.
Mapping a More Complex Data Plane Program on PISA
Match+Action
Stage (Unit)
Programmable
Parser
Programmable
Deparser
Programmable Match-Action Pipeline
Ethernet
MAC
Address
Table
ACL
Rules
IPv4
Address
Table
L2 ACL
IPv4
IPv6
Address
Table
IPv4
Address
Table
MPLS
Table
MPLS
IPv6

17. Copyright © 2018 - Barefoot Networks, Inc.
Demos

18. Copyright © 2018 - Barefoot Networks, Inc.
Packet & Flow Monitor
❏ Monitor every packet and produce traffic summaries
❏ Have “n” summaries in the data plane
❏ No control-plane involvement during monitoring
❏ Fetch summaries through CPU only when needed
❏ Summaries kept in a circular buffer in the data plane
❏ A summary consists:
❏ Packet size distribution
❏ Flow size distribution
❏ Number of flows

19. Copyright © 2018 - Barefoot Networks, Inc.
Example Implementation
compile
Control plane (Python)
Driver
configure / fetch summary (Thrift)
PD API
binary
api
program

20. Copyright © 2018 - Barefoot Networks, Inc.
Demo Setup
❑ Snake routing
▪ 40G server port to switch 1/0
▪ Internal loopback on all other ports
❑ Monitor traffic on port 1/0

21. Copyright © 2018 - Barefoot Networks, Inc.

22. In-band Network Telemetry
“Which path did my packet take?”1
“I visited Switch 1 @780ns,
Switch 9 @1.3µs, Switch 12 @2.4µs”
“Which rules did my packet follow?”2
“In Switch 1, I followed rules 75 and
250. In Switch 9, I followed rules 3 and
80. ”
# Rule
1
2
3
…
75 192.168.0/24
…
Copyright © 2018 - Barefoot Networks, Inc.

23. In-band Network Telemetry
“How long did my packet queue at each switch?”3 “Delay: 100ns, 200ns, 19740ns”
Time
Queue
“Who did my packet share the queue with?”4
Copyright © 2018 - Barefoot Networks, Inc.

24. Log, Analyze
Replay
How it works and how we use the data
Add: SwitchID, Arrival Time,
Queue Delay, Matched Rules, …
Original Packet
Visualize
Copyright © 2018 - Barefoot Networks, Inc.

25. Log, Analyze
Replay
Visualize
/* INT: add switch id */
action int_set_header_0() {
add_header(int_switch_id_header);
modify_field(int_switch_id_header.switch_id,
global_config_metadata.switch_id);
}
/* INT: add ingress timestamp */
action int_set_header_1() {
add_header(int_ingress_tstamp_header);
modify_field(int_ingress_tstamp_header.ingress_tstamp, i2e_metadata.ingress_tstamp);
}
/* INT: add egress timestamp */
action int_set_header_2() {
add_header(int_egress_tstamp_header);
modify_field(int_egress_tstamp_header.egress_tstamp,
eg_intr_md_from_parser_aux.egress_global_tstamp);
}
P4 code snippet: Insert switch ID, ingress and egress timestamp
25Copyright © 2018 - Barefoot Networks, Inc.

26. Copyright © 2018 - Barefoot Networks, Inc.

27. Copyright © 2018 - Barefoot Networks, Inc.

28. Copyright © 2018 - Barefoot Networks, Inc.

29. Programmability & Performance
P4-programmable Ethernet switch ASIC family.
6.5 Tb/s
3.3 Tb/s 2.4 Tb/s 1.9 Tb/s
Open Source P4
Programming Language
Open PISA Target
Architecture
Copyright © 2018 - Barefoot Networks, Inc.

30. Thank you!