OpenStack Neutron SDN trends TLV Meetup Cover L3 Agent legacy centralized virtual router implementation DVR Distributed virtual router using namespaces and Dragonflow Fully Distributed Virtual router using SDN

1. Eran Gampel
Chief Architect Open Source, Huawei European Research Center
Eshed Gal-Or
Sr Research Architect, Huawei European Research Center
Dragonflow & Neutron L3 Service

2. Page 2
What is L3 Service All About?
Static routes (Extra Routes), defined inside the virtual router
Inter subnet routing (East/West)
SNAT (port mapping and masquerading the IP address)
DNAT (floating IPs, public N/S connectivity directly to VM)

3. Page 3
Typical 3-tier Topology (Web-App-DB)

4. Page 4
OpenStack Neutron Plugin Layers

5. Page 5
L3 Agent – Using Linux namespace as Virtual Router

6. Page 6
The Problem
Network Node Bottleneck
 All inter-subnet traffic goes through the network controller
 In a typical cloud deployment scenario, most East-West
application traffic is between subnets (e.g. the popular
Web→App→DB pattern)
 Current model is mimicking physical world (router) network
elements using virtual software components
 Using the Linux network namespace
 Pre-configured (regardless of actual need)

7. Page 7
Host 4
The Problem – continued (single tenant)
Host 1
VM1
WWW
Neutron
Network
Node
Host 2
VM2
App
Host 3
VM3
DB Overlay network
Logical Connection
Physical Switch
Physical network

8. Page 8
The Problem – at scale (16 tenants)
Host 4
Neutron
Controller
Host 4
Neutron
Controller
Physical Switch(es)
Host 4
Neutron
Controller
Host 4Host 1
VM1
WWW
Network
Node
Host 2
VM2
App
Host 3
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
Host 7
VM1
WWW
Host 6
VM2
App
Host 5
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
Host 11
VM1
WWW
Host 12
VM2
App
Host 13
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
Host 17
VM1
WWW
Host 16
VM2
App
Host 15
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
VM1
WWW
VM2
App
VM3
DB
Overlay network
Logical Connection
Physical network

9. Page 9
Namespace-based Distributed Virtual Router (DVR)
 Proactive approach (pre-
configuring 100% of possible
flows)
 Distribute L3 services on
compute nodes
 Linux namespace is cloned to
all compute nodes that
participate in a tenant network

10. Page 10
DVR internals

11. Page 11
DVR east-west

12. Page 12
Pros & Cons of DVR
Pros Cons
Successfully distributed the East-
West traffic and the DNAT floating
IP
Puts unreasonable load on the
main message bus (e.g. sync all
ARPs to all namespaces)
Significant reduction of Network
node contention
Very complex management
Huge code change impact
Performance impact due to added
TCP stack

13. HOW ABOUT SDN?

14. Page 14
Two Strategies
External Internal
(built-in SDN controller
in OpenStack )
Dragonflow
Networking-ovn
…
…or

15. Page 15
State of the art - Problems
SDN Controller
North-bound Interface (REST?)
South-bound Interface (Openflow)
SDN Apps
SDN DB
Neutron
DB
Neutron-server
ML2-Core-Plugin
ML2.Drivers.Mechanism.XXX
Services-Plugin
Service
Network
Neutron API Nova API
CLI / Dashboard (Horizon) / Orchestration Tool (Heat)
HW Switch
Nova
Nova Compute
VM VM
Nova Compute
VM VM
Virtual Switch (OVS?) Virtual Switch (OVS?)
Neutron
Plugin Agent
Neutron
Plugin Agent
Vendor-specific API
Message Queue (AMQP)
Neutron-L3-Agent
Neutron-DHCP-Agent
LoadBalancer
Firewall
VPN
L3Services
TopologyMgr.
OverlayMgr.
Security
1. Management data is duplicated (in Neutron and in the SDN controller)

16. Page 16
State of the art - Problems
SDN Controller
North-bound Interface (REST?)
South-bound Interface (Openflow)
SDN Apps
SDN DB
Neutron
DB
Neutron-server
ML2-Core-Plugin
ML2.Drivers.Mechanism.XXX
Services-Plugin
Service
Network
Neutron API Nova API
CLI / Dashboard (Horizon) / Orchestration Tool (Heat)
HW Switch
Nova
Nova Compute
VM VM
Nova Compute
VM VM
Virtual Switch (OVS?) Virtual Switch (OVS?)
Neutron
Plugin Agent
Neutron
Plugin Agent
Vendor-specific API
Message Queue (AMQP)
Neutron-L3-Agent
Neutron-DHCP-Agent
LoadBalancer
Firewall
VPN
L3Services
TopologyMgr.
OverlayMgr.
Security
2. Possible collision/contention between Neutron services (FWaaS, LBaaS, etc.)
and SDN Applications running on the SDN Controller

17. Page 17
State of the art - Problems
SDN Controller
North-bound Interface (REST?)
South-bound Interface (Openflow)
SDN Apps
SDN DB
Neutron
DB
Neutron-server
ML2-Core-Plugin
ML2.Drivers.Mechanism.XXX
Services-Plugin
Service
Network
Neutron API Nova API
CLI / Dashboard (Horizon) / Orchestration Tool (Heat)
HW Switch
Nova
Nova Compute
VM VM
Nova Compute
VM VM
Virtual Switch (OVS?) Virtual Switch (OVS?)
Neutron
Plugin Agent
Neutron
Plugin Agent
Vendor-specific API
Message Queue (AMQP)
Neutron-L3-Agent
Neutron-DHCP-Agent
LoadBalancer
Firewall
VPN
L3Services
TopologyMgr.
OverlayMgr.
Security
3. Neutron is reduced to an API translator, and becomes a redundant layer

18. Page 18
State of the art - Problems
SDN Controller
North-bound Interface (REST?)
South-bound Interface (Openflow)
SDN Apps
SDN DB
Neutron
DB
Neutron-server
ML2-Core-Plugin
ML2.Drivers.Mechanism.XXX
Services-Plugin
Service
Network
Neutron API Nova API
CLI / Dashboard (Horizon) / Orchestration Tool (Heat)
HW Switch
Nova
Nova Compute
VM VM
Nova Compute
VM VM
Virtual Switch (OVS?) Virtual Switch (OVS?)
Neutron
Plugin Agent
Neutron
Plugin Agent
Vendor-specific API
Message Queue (AMQP)
Neutron-L3-Agent
Neutron-DHCP-Agent
LoadBalancer
Firewall
VPN
L3Services
TopologyMgr.
OverlayMgr.
Security
4. Add another component to install, further complicating the installation

19. Page 19
State of the art - Problems
SDN Controller
North-bound Interface (REST?)
South-bound Interface (Openflow)
SDN Apps
SDN DB
Neutron
DB
Neutron-server
ML2-Core-Plugin
ML2.Drivers.Mechanism.XXX
Services-Plugin
Service
Network
Neutron API Nova API
CLI / Dashboard (Horizon) / Orchestration Tool (Heat)
HW Switch
Nova
Nova Compute
VM VM
Nova Compute
VM VM
Virtual Switch (OVS?) Virtual Switch (OVS?)
Neutron
Plugin Agent
Neutron
Plugin Agent
Vendor-specific API
Message Queue (AMQP)
Neutron-L3-Agent
Neutron-DHCP-Agent
LoadBalancer
Firewall
VPN
L3Services
TopologyMgr.
OverlayMgr.
Security
Full blown SDN is a
good architecture for
some use cases
but not for All

20. Page 20
What is Dragonflow?
 Free software: Apache license
 Fully distributed virtual router part of
OpenStack® Neutron™
 Lean internal SDN controller
 Simplified management
 Improved performance and scale
 Eliminates network node SPOF and bottleneck
 Compiles routing logic into forwarding element
flows

21. Page 21
Architecture

22. Page 22
Advantages (vs. Juno DVR)
 Simple and nimble architecture
 Very small change impact on Neutron (vs. very big
change impact)
 Higher performance (+20% from initial benchmarks)
 Simpler management (Only actual flows are distributed
to forwarding elements instead of all possibilities)
 Higher scalability and flexibility
 Elastic architecture allows scaling in and out as the
managed instance network grows/shrinks
 Utilize the power of SDN (vs. legacy hard-wired opaque
software)

23. Page 23
Control Node
Neutron
Service Plugins
Network Node
Bootstrap
L3
Service
L3 Controller
Agent
L3 App
Message Queue (AMQP)
Compute Node
Neutron
Agent
OVS
qbrXXX
VM
br-tun
br-int
vxlan
qvoXXX
patch-tun
patch-int
IPTables
Core
Plugins
ML2
IPTables
Namespace
DHCP
Agent
DHCP
Service OpenFlow
Install L3
pipline
(L3 Agent)
Legacy
SNAT/FIP

24. Page 24
Hybrid OpenFlow Switch
OVS
OpenFlow processing pipeline
Normal L2 Switch
Input Output
Packet In Packet Out
Forward to
controller
(ofp PACKET_IN)
NORMAL
Drop
OpenFlow
Controller
 Introduced in OpenFlow/1.1. Hybrid switches support concurrent
operation of both OpenFlow pipeline and normal (legacy) Ethernet
switching functionality.
The hybrid switch allows forwarding of packets from the OpenFlow
pipeline to the normal pipeline through the NORMAL and
FLOOD reserved ports.

25. Page 25
L3 Installed pipeline
VRouter using flows- All L2 is offloaded to the normal path

26. Page 26
L3 Controller Agent
L3 SDN Application Logic
Compute Node
Controller
L3 App
OVS
qbrZZZ
VMzzz
br-tun
br-int
vxlan
qvoZZZ
patch-tun
patch-int
qbrWWW
VMwww
qvoWWW
Port VLAN TAG: 2
qbrXXX
VMxxx
qvoXXX
qbrYYY
VMyyy
qvoYYY
Port VLAN TAG: 1
Neutron
DB
OpenFlowOpenFlow
VMwww first TCP
connection with
VMyyy
Tenant A, Subnet 2
Tenant A, Subnet 1
1
1st TCP_SYN
DST: VMyyy
Packet is sent to
controller
Matched as VM to VM inter
Subnet Traffic in the L3
forwarding table
2
If route
(www to yyy)
possible, install
flow and
reverse_flow
PACKET_OUT
1st TCP_SYN
DST: VMyyy
3

27. Page 27
L3 Controller Agent
L3 SDN Application Logic same compute Node
Compute Node
Controller
L3 App
OVS
qbrZZZ
VMzzz
br-tun
br-int
vxlan
qvoZZZ
patch-tun
patch-int
qbrWWW
VMwww
qvoWWW
Port VLAN TAG: 2
qbrXXX
VMxxx
qvoXXX
qbrYYY
VMyyy
qvoYYY
Port VLAN TAG: 1
Neutron
DB
OpenFlow
FLOW_MOD
match:
vid
src_mac
src_ip
dst_mac
dst_ip
action:
pop_vlan
change src_mac
change dst_mac
output:
port qvoYYY
Tenant A, Subnet 2
Tenant A, Subnet 1
FLOW_MOD
match:
vid
src_mac
src_ip
dst_mac
dst_ip
action:
pop_vlan
change src_mac
change dst_mac
output:
port qvoWWW
4
5
Install Flow and
Reverse Flow For
Inter Subnet L3
Traffic

28. Page 28
L3 Controller Agent
L3 SDN Application Logic cross compute Node
Controller
L3 App
Compute Node
OVS
qbrZZZ
VMzzz
br-tun
br-int
vxlan
qvoZZZ
patch-tun
patch-int
Port VLAN TAG: 2
qbrYYY
VMyyy
qvoYYY
Port VLAN TAG: 1
Neutron
DB
OpenFlow
Compute Node
OVS
br-tun
br-int
vxlan
patch-tun
patch-int
qbrWWW
VMwww
qvoWWW
Port VLAN TAG: 2
qbrXXX
VMxxx
qvoXXX
Port VLAN TAG: 1
OpenFlow
VMwww first TCP
connection with
VMyyy
1st TCP_SYN
DST: VMyyy
If route
(www to yyy)
possible, install
flow and
reverse_flow
FLOW_MOD
match:
vid
src_mac
src_ip
dst_mac
dst_ip
action:
pop_vlan
change src_mac
change dst_mac
output:
port qvoYYY
FLOW_MOD
match:
vid
src_mac
src_ip
dst_mac
dst_ip
action:
pop_vlan
change src_mac
change dst_mac
output:
port qvoWWW
PACKET_OUT
1st TCP_SYN
DST: VMyyy
1
2
3
4
5

29. Page 29
DragonFlow – Current Feature List (Kilo)
Feature
APIs for routing IPv4 East-West traffic
Performance improvement for inter-subnet network by removing the amount of kernel layers
(namespaces and their TCP stack overhead)
Scalability improvement for inter-subnet network by offloading L3 East-West routing from the Network
Node to all Compute Nodes
Reliability improvement for inter-subnet network by removal of Network Node from the East-West
traffic
Simplified virtual routing management
Supports all type drivers GRE/Vxlan/VLAN
Centralized North-South traffic
Support for HA, in case the connection to the Controller is lost, fall back to the legacy L3
implementation until recovery. Reused all the legacy L3 HA. (Controller HA will be supported in the
next release).
Supports Centralized IPv6

30. Page 30
Compute Node
Dragonflow – Local Controller
Dragonflow
OVS
Dragonflow Controller
Neutron-Server
OVSDB
Pluggable DB Layer
ML2
Mechanism
Driver
Services
L3 Service Plugin
OVSDB-Server
OpenFlow
Abstraction Layer
DB Drivers
OVSDB
NB/SB Distributed DB ETCD Cassandra RAMCloud
Kernel Datapath
Module
NIC
VswitchD
User Space
Kernel Space
ETCD Cass RMC
NB DB Drivers
OVSDB ETCD Cass RMC
SB DB Drivers
Mellanox OVSDB
L3
App
L2
App
DHCP
App
Faults
App
FWaaS/
SG
LBaaS …
Future

31. Page 31
DragonFlow – Planned Feature List (Liberty+)
 Compute Node Controller (connecting to OVN project)
 Distributed DB Plugin for multiple drivers (OVSDB,
ETCD, Cassandra, …)
 Multi Controller Support ( Equal and Master Slave)
 Add support for IPv6
 North-South L3 IPv4 distribution (SNAT and DNAT)
For the complete list go to the Dragonflow blueprints:
https://blueprints.launchpad.net/dragonflow

32. Page 32
External Links
 Homepage: http://launchpad.net/dragonflow
 Documentation: http://goo.gl/rq4uJC
 Source: http://git.openstack.org/cgit/stackforge/
dragonflow
 Bugs: http://bugs.launchpad.net/dragonflow
 Blog: blog.gampel.net
 IRC : #openstack-dragonflow

33. Page 33
Vote for us in OpenStack Summit Tokyo 2015
 Say Hello to 100G OpenStack Networking by
Offloading SDN flows using Dragonflow and
intelligent NICs
 Networking High Availability by Design
 DragonFlow SDN in a Box - hands on lab
 Scaling Neutron - Distributing Advanced Services
using SDN
 OpenvSwitch Performance Secrets Revealed
 Distributed local SDN Controllers with OpenStack

34. We are
Hiring!
jobs@toganetworks.com
www.toganetworks.com