The Open vSwitch kernel datapath may have flows offloaded to hardware using the TC Flower classifier and related actions. This is a powerful mechanism to both increase throughput and reduce CPU utilisation. This presentation will give an overview of the evolution of this offload mechanism: features available in OvS v2.8, those targeted at v2.9 and possible future directions.

1. © 2017 NETRONOME SYSTEMS, INC.
Simon Horman
Open vSwitch 2017 Fall Conference
San Jose
OVS Hardware Offload with
TC Flower

2. © 2017 NETRONOME SYSTEMS, INC. 2
Overview
● OVS Kernel Datapath Offload Models
● Overview of TC Flower
● TC Flower-Based Offload

3. 3© 2017 NETRONOME SYSTEMS, INC.
Motivation for
Hardware Offload

4. © 2017 NETRONOME SYSTEMS, INC. 4
Motivation for Hardware Offload
● Provide greater throughput
● Increase CPU core efficiency and scalability

5. © 2017 NETRONOME SYSTEMS, INC. 5
Framerate


6. © 2017 NETRONOME SYSTEMS, INC. 6
Throughput

7. 7© 2017 NETRONOME SYSTEMS, INC.
OVS Kernel Datapath
Offload Models

8. © 2017 NETRONOME SYSTEMS, INC. 8
Kernel Datapath
ovs-vswitchd
User-Space
Kernel
OVS
Datapath
misses, flows, stats
packet packet

9. © 2017 NETRONOME SYSTEMS, INC. 9
OVS Datapath Hooks
User-Space
Kernel
flows, stats, misses
SmartNIC
flows, stats, misses
ovs-vswitchd
OVS
Datapath
packet packet
SmartNIC
Datapath

10. © 2017 NETRONOME SYSTEMS, INC. 10
OVS-TC
User-Space
Kernel
flows, stats, misses
SmartNIC
packet
flows, stats, misses
TC
Datapath misses
flows, stats
ovs-vswitchd
OVS
Datapath
SmartNIC
Datapath packet

11. 11© 2017 NETRONOME SYSTEMS, INC.
Overview of
TC Flower

12. © 2017 NETRONOME SYSTEMS, INC. 12
Overview of TC Flower
● Packet classifier for Linux kernel traffic classification (TC) subsystem
● TC Flower classifier allows matching packets against pre-defined flow key
fields:
○ Packet headers: f.e. IPv6 source address
○ Tunnel metadata: f.e. Tunnel Key ID
○ Metadata: Input port
● TC actions allow packet to be modified, forwarded, dropped, etc…
○ pedit: modify packet data
○ mirred: output packet
○ vlan: push, pop or modify VLAN
○ ...

13. © 2017 NETRONOME SYSTEMS, INC. 13
Example of TC Flower
● Filter packets received on eth0
● Drop TCP packets with destination port 80
# tc qdisc add dev eth0 ingress
# tc filter add dev eth0 protocol ip parent ffff:
flower ip_proto tcp dst_port 80
action drop

14. © 2017 NETRONOME SYSTEMS, INC. 14
Hardware Offload Policy
● per-netdev configuration
○ Allow disabling/enabling adding flows to hardware
# ethtool -K eth0 hw-tc-offload on
# ethtool -K eth0 hw-tc-offload off
● skip_hw and skip_sw flags
○ Allow users to influence placement of flows by kernel
○ Default is to add to hardware and try to add to software
● in_hw and not_in_hw flags
○ Allow kernel to report presence of flow in hardware

15. © 2017 NETRONOME SYSTEMS, INC. 15
Example of Setting Hardware Policy
● Add flow only to hardware
# tc qdisc add dev eth0 ingress
# tc filter add dev eth0 protocol ip parent ffff:
flower skip_sw ip_proto sctp dst_port 80
action drop

16. © 2017 NETRONOME SYSTEMS, INC. 16
Example of Viewing Rule in Hardware
● Policy was to only add rule to hardware (skip_sw)
● Rule is present in hardware (in_hw)
# tc filter show dev eth0 ingress
filter parent ffff: protocol ip
pref 49152 flower chain 0
handle 0x1
eth_type ipv4
ip_proto sctp
dst_port 80
skip_sw
in_hw
...

17. 17© 2017 NETRONOME SYSTEMS, INC.
TC Flower-Based
Offload

18. © 2017 NETRONOME SYSTEMS, INC. 18
Tables and Flows
● OVS Datapath
○ Single table
○ Match on in_port
○ Flows have a wide key and are disjoint
○ And therefore can be partitioned into slices
○ Megaflows are priority independent
● TC Flower
○ Multi-table (chain) support
○ Attached to in_port
○ Flows have a wide key
○ Only one mask per priority

19. © 2017 NETRONOME SYSTEMS, INC. 19
Offload Integration in OVS
● New netdev ops called by DPIF layer
● Try to offload each flow
○ f.e. By adding to TC Flower
● If unsuccessful then add to software datapath
○ f.e. kernel datapath

20. © 2017 NETRONOME SYSTEMS, INC. 20
Configuration
● Disabled by default
● Enabled/disabled globally
# ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
● TC Policy controls placement of flows
○ none (default): Try to add to TC software datapath and hardware if present
○ skip_sw: Try to add to TC software datapath
○ skip_hw: Try to add to hardware
● Also set globally
# ovs-vsctl set Open_vSwitch . other_config:tc-policy=none

21. © 2017 NETRONOME SYSTEMS, INC. 21
Viewing Flows
● Dump all datapath flows (default)
# ovs-dpctl dump-flows
● Dump only flows that in kernel datapath
# ovs-dpctl dump-flows type=ovs
● Dump only flows that are offloaded
# ovs-dpctl dump-flows type=offloaded

22. © 2017 NETRONOME SYSTEMS, INC. 22
Current Features
● Matches
○ L2 ~ L4 and Tunnel metadata matches
○ L2: type, addresses, VLANs
○ MPLS: LSE fields
○ L3: Addresses, protocol, TTL, ...
○ L4: UDP/TCP/SCTP ports
○ Tunnel Metadata: Tunnel ID
● Actions:
○ Drop, output, VLAN push/pop

23. © 2017 NETRONOME SYSTEMS, INC. 23
Status
● Offload Integration in OVS
○ Included in OVS v2.8
● TC Flower
○ Initially added in Linux kernel v4.2
● NFP Driver
○ Basic offload support present since Linux kernel v4.13

24. 24© 2017 NETRONOME SYSTEMS, INC.
Future Work

25. © 2017 NETRONOME SYSTEMS, INC. 25
Stateless Match/Action Enhancements
● Set Action
○ Patches available
● IPv6 label and neighbour discovery
● Maskable match of MPLS LSE fields
● GENEVE options

26. © 2017 NETRONOME SYSTEMS, INC. 26
Packet Scheduling
● Metering
● QoS

27. © 2017 NETRONOME SYSTEMS, INC. 27
Conntrack
● Aim to allow enhanced rules to be written
○ By taking into account Conntrack state
● Proposal is to follow implemented by Open vSwitch kernel datapath:
○ Conntrack action passes packet to conntrack subsystem
○ Packet is then classified for a second time;
conntrack state may form part of flow key
Match Action Match Action

28. © 2017 NETRONOME SYSTEMS, INC.
Thank You