What is new in neutron QoS?

What is new in Neutron QoS?
Miguel Lavalle
miguel@mlavalle.com
IRC: mlavalle@freenode
Slawek Kaplonski
skaplons@redhat.com
IRC: slaweq@freenode

2
AGENDA
● Introduction to Neutron QoS
● Existing QoS features in Neutron
● New features added in Queens release
● Planned New features

Open vSwitch SR-IOV Linux Bridge L3 agent
Bandwidth limit Egress / Ingress Egress (1) Egress / Ingress Egress / Ingress (3)
Minimum
Bandwidth
- Egress (2) - -
DSCP Marking Egress - Egress -
8
Neutron QoS - available rule types
1. Max burst parameter is skipped because it's not supported by ip tool
2. Best effort - scheduler is not aware about this minimum bandwidth requirement for ports
3. Limit for Floating IP

sudo ovs-vsctl list qos
_uuid : a4857dc7-e64f-4370-9cf2-408566871bf1
external_ids : {id="tap9a566f82-0c"}
other_config : {max-rate="512000"}
queues : {0=671cc958-2328-4d77-9e36-91984582e5a5}
type : linux-htb
sudo ovs-vsctl list queue
_uuid : 671cc958-2328-4d77-9e36-91984582e5a5
dscp : []
external_ids : {id="tap9a566f82-0c", queue_type="0"}
other_config : {burst="128000", max-rate="512000"}
sudo ovs-vsctl list port tap9a566f82-0c | grep qos
qos : a4857dc7-e64f-4370-9cf2-408566871bf1
10
Backend implementation - Open vSwitch L2 agent - ingress traffic

sudo ovs-vsctl list interface tap9a566f82-0c | grep ingress_
ingress_policing_burst: 256
ingress_policing_rate: 1024
11
Backend implementation - Open vSwitch L2 agent - egress traffic

tc qdisc show dev tap2e939f9e-9e
qdisc tbf 8002: root refcnt 2 rate 512Kbit burst 16Kb lat 50.0ms
12
Backend implementation - Linux bridge L2 agent - ingress traffic

tc filter show dev tap2e939f9e-9e parent ffff:
filter protocol all pref 49 basic chain 0
filter protocol all pref 49 basic chain 0 handle 0x1
police 0x1 rate 1024Kbit burst 32Kb mtu 64Kb action drop overhead 0b
ref 1 bind 1
13
Backend implementation - Linux bridge L2 agent - egress traffic

ip link show enp8s0f0
3: enp8s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc mq state UP
mode DEFAULT group default qlen 1000
link/ether 90:e2:ba:5e:a6:40 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:00:00:00:00:00, tx rate 1000 (Mbps), max_tx_rate 1000Mbps,
spoof checking on, link-state auto
vf 1 MAC 56:92:c6:c6:e5:db, tx rate 2000 (Mbps), max_tx_rate 2000Mbps,
spoof checking on, link-state auto
14
Backend implementation - SR-IOV L2 agent - egress traffic

sudo ovs-ofctl show br-int | grep tap9a566f82-0c
141(tap9a566f82-0c): addr:fe:16:3e:47:85:27
sudo ovs-ofctl dump-flows br-int | grep in_port=141 | grep tos
cookie=0x66a4b613766ef2e1, duration=70.889s, table=0, n_packets=0,
n_bytes=0, idle_age=405, priority=65535,reg2=0,in_port=141
actions=mod_nw_tos:64,load:0x37->NXM_NX_REG2[0..5],resubmit(,0)
16
Backend implementation - Open vSwitch L2 agent

iptables -t mangle -nL neutron-linuxbri-qos-o2e939f
Chain neutron-linuxbri-qos-o2e939f (1 references)
target prot opt source destination
DSCP all -- 0.0.0.0/0 0.0.0.0/0 DSCP set 0x10
17
Backend implementation - Linux bridge L2 agent

New rule types validation and reporting

20
Rules validation - old way
Report of available rule types
Example for deployment with openvswitch and sriovnicswitch mechanism
drivers
openstack network qos rule type list
+-----------------+
| Type |
+-----------------+
| bandwidth_limit | (- supported by both drivers)
+-----------------+

21
Rules validation - old way - problems
● There was no real validation - it was only information for
user,
● Very limited range of rule types available in mixed
deployments,
● Some attributes of rules have big impact of backend
implementation (e.g. direction)

23
Rules validation - new way
Report details of rule type
drivers
+----------------+------------------------------------------------------------------+
| Field | Value |
+----------------+------------------------------------------------------------------+
| drivers | {'name': 'openvswitch', |
| | 'supported_parameters': [{'parameter_values': [0, 8, .., 56], |
| | 'parameter_type': 'choices', |
| | 'parameter_name': 'dscp_mark'}]}] |
| rule_type_name | dscp_marking |
+----------------+------------------------------------------------------------------+

24
Rules validation - new way - validation
Validation if QoS policy can be applied to port (ports) during:
● Create/Update QoS rule,
● Create/Update port,
● Update network
Returns HTTP Conflict (409) error in case if rule(s) can’t be
applied for at least one port

25
openstack network qos policy show DSCP_MARK_Policy
+-------------+------------------------------------------+
| Field | Value |
+-------------+------------------------------------------+
| name | DSCP_MARK_Policy |
| rules | [{u'type': u'dscp_marking'] |
+-------------+------------------------------------------+
openstack network qos policy show Min_BW_Policy
+-------------+------------------------------------------+
| Field | Value |
+-------------+------------------------------------------+
| name | Min_BW_Policy |
| rules | [{u'type': u'minimum_bandwidth'] |
+-------------+------------------------------------------+
openstack port show 5cac029f-dbac-4c04-ab81-a6ab8faefc43 -c binding_vif_type -c id
+------------------+--------------------------------------+
| Field | Value |
+------------------+--------------------------------------+
| binding_vif_type | ovs |
| id | 5cac029f-dbac-4c04-ab81-a6ab8faefc43 |
+------------------+--------------------------------------+

26
openstack port set 5cac029f-dbac-4c04-ab81-a6ab8faefc43 --qos-policy Min_BW_Policy
ConflictException: 409
openstack port show 5cac029f-dbac-4c04-ab81-a6ab8faefc43 -c qos_policy_id -c id
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| qos_policy_id | None |
+---------------+--------------------------------------+
Apply QoS policy with minimum bandwidth rule (NOT
supported by openvswitch driver) - Fails

27
openstack port set 5cac029f-dbac-4c04-ab81-a6ab8faefc43 --qos-policy DSCP_MARK_Policy
openstack port show 5cac029f-dbac-4c04-ab81-a6ab8faefc43 -c qos_policy_id -c id
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| qos_policy_id | 4d8cf13e-2a34-43a2-a642-53b5d4dd898d |
+---------------+--------------------------------------+
Apply QoS policy with DSCP Marking rule (supported by
openvswitch driver) - OK

New features in Queens release:
bandwidth limit rules for floating IPs

29
Overview
● Only bandwidth limit rules for floating IPs
● Both directions available: egress and ingress
● Rules are implemented as tc filters applied to:
○ qg device in qr name space for legacy and HA routers
○ rfp device in qr name space for DVR local routers
○ qg device in snat name space for DVR edge routers
● Filters are applied when floating ip is associated to a port

33
After associating floating IP to VM port
$ sudo ip netns exec qrouter-4aa23079-dfe1-4250-a788-94777b4bb13d tc qdisc show dev
> qg-e0da4660-c9
qdisc htb 1: root refcnt 2 r2q 10 default 0 direct_packets_stat 63 direct_qlen 1000
qdisc ingress ffff: parent ffff:fff1 ----------------
$ sudo ip netns exec qrouter-4aa23079-dfe1-4250-a788-94777b4bb13d tc -p -s -d filter
show dev qg-e0da4660-c9
filter parent 1: protocol ip pref 1 u32
filter parent 1: protocol ip pref 1 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 1 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid
:1
match IP src 172.24.4.8/32
police 0x5 rate 3Mbit burst 300Kb mtu 2Kb action drop overhead 0b linklayer ethernet
ref 1 bind 1
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

34
After associating floating IP to VM port (cont.)
$ sudo ip netns exec qrouter-4aa23079-dfe1-4250-a788-94777b4bb13d tc -p -s -d filter show
> dev qg-e0da4660-c9 parent ffff:fff1 prio 1
filter parent ffff: protocol ip u32
filter parent ffff: protocol ip u32 fh 800: ht divisor 1
filter parent ffff: protocol ip u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid :1
match IP dst 172.24.4.8/32
police 0x4 rate 2100Kbit burst 225Kb mtu 2Kb action drop overhead 0b linklayer ethernet
ref 1 bind 1
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)

Planned new features:
Strict minimum bandwidth support

36
What do we mean by strict minimum
bandwidth support?
● Enforcement of minimum QoS policy minimum bandwidth
rules in the data plane
○ This can be done by Neutron alone
○ Egress minimum bandwidth rules are currently available
for SR-IOV back-end
○ Plans to implement for ingress and for other back-ends:
OVS, LB, etc.

37
What do we mean by strict minimum
bandwidth support? (cont.)
● Avoid oversubscription of interfaces when scheduling VMs
and their ports in compute hosts
○ This will require cooperation between Neutron and
Nova
○ Neutron will influence Nova’s scheduling decision
○ Nova will influence Neutron’s port binding process

38
Limitation of first experimental release
● Ports will have to be created and associated to QoS policy
by user and then provided in the VM boot request
○ This is because currently, when booting an instance,
Nova creates its ports in the compute host, after the
scheduling decision has been made
○ Plans exist to move port creation by Nova to the
Conductor, i.e., before the scheduling decision

39
Components of strict minimum bandwidth
support
● Neutron communicates to Nova the interfaces and
bandwidth inventory using the Placement API
● User sets in Neutron the minimum bandwidth requirements
of a port
● Neutron communicates to Nova the minimum bandwidth
requirements of a port
● Nova communicates to Neutron the selected host and the
backend within that host

40
Placement API concepts
Resource
Provider
Source of resources
like bandwidth
Inventories
Traits
Non-consumable
qualitative aspects
of resources
Egress
bandwidth
Ingress
bandwidth
VNIC typePhysnet

41
Placement data for each compute host
Compute RP
name = hostname
Agent RP
(OVS agent)
uuid = agent_uuid
Physical NIC RP
uuid = uuid5(hostname:br0)
traits: physnet_1, VNIC_TYPE_NORMAL
inventory:
{ingress_kilobits_per_sec: 10000,
egrees_kilobits_per_sec: 10000}
Physical NIC RP
traits: physnet_2, VNIC_TYPE_NORMAL
inventory:
Agent RP
(SR-IOV agent)
uuid = agent_uuid
Physical NIC RP
traits: physnet_3, VNIC_TYPE_DIRECT
inventory:

Communicating to Nova interfaces and
bandwidth inventories
ReST
Neutron Server Placement ServiceAgent RPC
Agent heartbeat ‘configurations’ with:
- Bridge / interface mappings
- Resource providers inventories
- Resource providers traits
Config

Neutron communicates to Nova the minimum
bandwidth requirements of a port
{
“port”: {
“status”: “ACTIVE”,
“name”: “Summit_Port”,
...
“device_id”: “5e3898d7-11be-483e-9732-b2f5eccd2b2e”,
“resource_request": {
"resources": {
"NET_BANDWIDTH_INGRESS_KILOBITS_PER_SECOND": 1000,
"NET_BANDWIDTH_EGRESS_KILOBITS_PER_SECOND": 1000,
}
"required": [
"CUSTOM_PHYSNET_NET0",
"CUSTOM_VNIC_TYPE_NORMAL”
]
}
}
}

Nova communicates to Neutron the selected
host and backend
{
“port”: {
“binding:host_id”: “allinone”,
“binding:profile”: {
“allocation”: “rp uuid”
}
}
}

What is new in neutron QoS?

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to What is new in neutron QoS?

Similar to What is new in neutron QoS? (20)

Recently uploaded

Recently uploaded (20)

What is new in neutron QoS?