© 2017 NETRONOME SYSTEMS, INC.
Simon Horman
Netdev 2.2, Seoul
TC Flower Offload
© 2017 NETRONOME SYSTEMS, INC. 2
Match-Action Datapath
● Programmable datapaths are new and exciting
● Match-action tables are more static but widely used
● Packets traverse one or more table of match-action rules
○ Classifier seeds flow key
○ Flow key is optionally masked
○ Flow key is used to lookup rules of table
○ If a match is found the actions of the rule are executed
○ Otherwise some fall-through occurs
Match Action
© 2017 NETRONOME SYSTEMS, INC. 3
Overview of TC Flower
● TC Flower classifier allows matching packets against pre-defined flow key
fields:
○ Packet headers: f.e. IPv6 source address
○ Tunnel metadata: f.e. Tunnel Key ID
○ Metadata: Input port
● TC actions allow packet to be modified, forwarded, dropped, etc…
○ pedit: modify packet data
○ mirred: output packet
○ vlan: push, pop or modify VLAN
○ ...
© 2017 NETRONOME SYSTEMS, INC. 4
Example of TC Flower
● Filter packets recieved on eth0
● Drop SCTP packets with destination port 80
# tc qdisc add dev eth0 ingress
# tc filter add dev eth0 protocol ip parent ffff: 
flower ip_proto sctp dst_port 80 
action drop
© 2017 NETRONOME SYSTEMS, INC. 5
Implementation
● TC Flower makes use of the flow dissector to perform classification
● Resulting flow key is masked
○ Only one mask allowed per priority
● Used to lookup flows
● If a match is found then TC actions of rule are executed
● If not packet processing proceeds to next classifier
○ Standard TC behaviour
© 2017 NETRONOME SYSTEMS, INC. 6
Motivation for Hardware Offload
● Provide greater throughput
● Increase CPU core efficiency and scalability
© 2017 NETRONOME SYSTEMS, INC. 7
History of TC Hardware Offload
● mqprio offload introduced in v2.6.39
● Introduced ndo_setup_tc
● u32 Classifier offload introduced in v4.6
● As of v4.15-rc4 BPF, flower and matchall classifier offload also supported
● Netronome NFP driver implements BPF and flower classifier offload
● Actions may be offloaded at discretion of driver
© 2017 NETRONOME SYSTEMS, INC. 8
Hardware Offload Policy
● per-netdev configuration
○ Allow disabling/enabling adding flows to hardware
# ethtool -K eth0 hw-tc-offload on
# ethtool -K eth0 hw-tc-offload off
● skip_hw and skip_sw flags
○ Allow users to influence placement of flows by kernel
○ Default is to add to hardware and try to add to software
● in_hw and not_in_hw flags
○ Allow kernel to report presence of flow in hardware
© 2017 NETRONOME SYSTEMS, INC. 9
Example of Setting Hardware Policy
● Add flow only to hardware
# tc qdisc add dev eth0 ingress
# tc filter add dev eth0 protocol ip parent ffff: 
flower skip_sw ip_proto sctp dst_port 80 
action drop
© 2017 NETRONOME SYSTEMS, INC. 10
Example of Viewing Rule in Hardware
● Policy was to only add rule to hardware (skip_sw)
● Rule is present in hardware (in_hw)
# tc filter show dev eth0 ingress
filter parent ffff: protocol ip
pref 49152 flower chain 0
handle 0x1
eth_type ipv4
ip_proto sctp
dst_port 80
skip_sw
in_hw
...
© 2017 NETRONOME SYSTEMS, INC. 11
Match Enhancements
● IPv6 label and neighbour discovery
● Maskable match of MPLS LSE fields
● GENEVE options
© 2017 NETRONOME SYSTEMS, INC. 12
Conntrack
● Aim would be to allow enhanced rules to be written
○ By taking into account Conntrack state
● Scheme implemented by Open vSwitch datapath is:
○ Conntrack action passes packet to conntrack subsystem
○ Packet is then classified for a second time;
conntrack state may form part of flow key
Match Action Match Action
© 2017 NETRONOME SYSTEMS, INC.
Thank You

TC Flower Offload

  • 1.
    © 2017 NETRONOMESYSTEMS, INC. Simon Horman Netdev 2.2, Seoul TC Flower Offload
  • 2.
    © 2017 NETRONOMESYSTEMS, INC. 2 Match-Action Datapath ● Programmable datapaths are new and exciting ● Match-action tables are more static but widely used ● Packets traverse one or more table of match-action rules ○ Classifier seeds flow key ○ Flow key is optionally masked ○ Flow key is used to lookup rules of table ○ If a match is found the actions of the rule are executed ○ Otherwise some fall-through occurs Match Action
  • 3.
    © 2017 NETRONOMESYSTEMS, INC. 3 Overview of TC Flower ● TC Flower classifier allows matching packets against pre-defined flow key fields: ○ Packet headers: f.e. IPv6 source address ○ Tunnel metadata: f.e. Tunnel Key ID ○ Metadata: Input port ● TC actions allow packet to be modified, forwarded, dropped, etc… ○ pedit: modify packet data ○ mirred: output packet ○ vlan: push, pop or modify VLAN ○ ...
  • 4.
    © 2017 NETRONOMESYSTEMS, INC. 4 Example of TC Flower ● Filter packets recieved on eth0 ● Drop SCTP packets with destination port 80 # tc qdisc add dev eth0 ingress # tc filter add dev eth0 protocol ip parent ffff: flower ip_proto sctp dst_port 80 action drop
  • 5.
    © 2017 NETRONOMESYSTEMS, INC. 5 Implementation ● TC Flower makes use of the flow dissector to perform classification ● Resulting flow key is masked ○ Only one mask allowed per priority ● Used to lookup flows ● If a match is found then TC actions of rule are executed ● If not packet processing proceeds to next classifier ○ Standard TC behaviour
  • 6.
    © 2017 NETRONOMESYSTEMS, INC. 6 Motivation for Hardware Offload ● Provide greater throughput ● Increase CPU core efficiency and scalability
  • 7.
    © 2017 NETRONOMESYSTEMS, INC. 7 History of TC Hardware Offload ● mqprio offload introduced in v2.6.39 ● Introduced ndo_setup_tc ● u32 Classifier offload introduced in v4.6 ● As of v4.15-rc4 BPF, flower and matchall classifier offload also supported ● Netronome NFP driver implements BPF and flower classifier offload ● Actions may be offloaded at discretion of driver
  • 8.
    © 2017 NETRONOMESYSTEMS, INC. 8 Hardware Offload Policy ● per-netdev configuration ○ Allow disabling/enabling adding flows to hardware # ethtool -K eth0 hw-tc-offload on # ethtool -K eth0 hw-tc-offload off ● skip_hw and skip_sw flags ○ Allow users to influence placement of flows by kernel ○ Default is to add to hardware and try to add to software ● in_hw and not_in_hw flags ○ Allow kernel to report presence of flow in hardware
  • 9.
    © 2017 NETRONOMESYSTEMS, INC. 9 Example of Setting Hardware Policy ● Add flow only to hardware # tc qdisc add dev eth0 ingress # tc filter add dev eth0 protocol ip parent ffff: flower skip_sw ip_proto sctp dst_port 80 action drop
  • 10.
    © 2017 NETRONOMESYSTEMS, INC. 10 Example of Viewing Rule in Hardware ● Policy was to only add rule to hardware (skip_sw) ● Rule is present in hardware (in_hw) # tc filter show dev eth0 ingress filter parent ffff: protocol ip pref 49152 flower chain 0 handle 0x1 eth_type ipv4 ip_proto sctp dst_port 80 skip_sw in_hw ...
  • 11.
    © 2017 NETRONOMESYSTEMS, INC. 11 Match Enhancements ● IPv6 label and neighbour discovery ● Maskable match of MPLS LSE fields ● GENEVE options
  • 12.
    © 2017 NETRONOMESYSTEMS, INC. 12 Conntrack ● Aim would be to allow enhanced rules to be written ○ By taking into account Conntrack state ● Scheme implemented by Open vSwitch datapath is: ○ Conntrack action passes packet to conntrack subsystem ○ Packet is then classified for a second time; conntrack state may form part of flow key Match Action Match Action
  • 13.
    © 2017 NETRONOMESYSTEMS, INC. Thank You