Bridges and Tunnels: A Drive Through OpenStack Networking
Mark McClain

twitter: @gtwmm
Why Create Neutron?
• New Technology
• Tenant Isolation
• Rich Topologies
• Advance Services Support
• Load Balancing, VPN, Firewall
The Basics
What does the user see?
Compute API
Network API
Storage APIGUI, CLI, API Libs
KVM
ML2 Plugin
Ceph
Abstractions
Net1

10.0.0.0/24
Nova
Neutron
L2 virtual network
virtual port
virtual server
virtual interface (VIF)
virtual subnet
VM1

10.0.0.2
VM2

10.0.0.2
Architecture
Design Goals
Open vSwitch / Linux Bridge
Ryu OpenFlow	

Controller
• Unified API
• Small Core
• Pluggable Open Architecture
• Multiple Vendor Support
• Extensible
OpenStack The Operator View
Basic Deployment
neutron-server
L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent
L3 AgentL3 Agent
L3 AgentL3 Agent
Database
L3 Agent
DHCP Agent
L2 Agent
Message
Queue
Adv Services
neutron-server
neutron-server
REST API
SERVICE
RPC SERVICE
PLUGIN
REST API
SERVICE
RPC SERVICE
• REST API

• HTTP(S) Python WSGI Application

• Customary TCP port is 9696

• Exposes logical resources

• networks, subnets, ports, etc

• Request/Response Serialization

• RPC Service

• AMQP via Oslo messaging modules

• Enables bidirectional agent communication

• Optional
neutron-server
neutron-server
REST API
SERVICE
RPC SERVICE
PLUGIN
• PLUGIN

• Written in Python

• Only one active

• Must implement V2 API calls

• Optional database access

• Optional extension support
The Plugin
ML2Plugin
core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
The Plugin
NeutronPluginBaseV2
NeutronDbPluginV2
ML2Plugin
core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
Plugin Extensions
• Add logical resources to the REST API
• Discovered by server at startup
• REST: /v2.0/extensions
• Common Extensions
• Binding, DHCP, L3, Provider, Quota, Security Group
• Other Extensions
• Allowed Addresses, Extra Routes, Metering
Monolithic Plugin
• Full implementation of core resources
• Two types:
• Proxy
• Direct control PLUGIN
ML2: Modular Layer 2 Plugin
• Full V2 Plugin Implementation
• Delegates calls to proper L2 drivers
• Two kinds of drivers
• Type Driver
• Mechanism Driver Mech Mgr
PLUGIN
Type Mgr
PLUGIN
Basic Deployment
neutron-server
L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent
L3 AgentL3 Agent
L3 AgentL3 Agent
Database
L3 Agent
DHCP Agent
L2 Agent
Message
Queue
Adv Services
L2 Agent
L2 Agent
• Runs on hypervisor
• Watch and notify when devices added/removed
• Wires new devices to bridge
• Proper network segment
• Security Group Rules
Isolation
VLAN
• 802.1Q
• limited
• underlay must support
GRE/VXLAN
• L2 encapsulated in L3
• routable
• overlay independence
Tunneling
A
D
CB
Tunneling with L2 Population
A
D
CB
• Open Source Virtual Switch
• http://openvswitch.org
• Can manage
• VLAN, GRE, VXLAN OVSDB
Open vSwitch
Neutron Server
OVS Agent
OVS
RPC
Basic Deployment
neutron-server
L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent
L3 AgentL3 Agent
L3 AgentL3 Agent
Database
L3 Agent
DHCP Agent
L2 Agent
Message
Queue
Adv Services
Linux Network Namespace
• Isolated copy of network stack
• private loopback
• scope limited to namespace
• can reuse addresses
• Explicit configuration needed to connect
• Processes can spawn within namespace
lo
eth1
eth0
lo
eth1
eth0
lo
eth1
eth0
Host A B
br-int
Network
Node
L3 Agent
• Run on Network Node
• Uses Namespaces
• Metadata Agent (if enabled)
Network
Node
Core
Hypervisor Hypervisor Hypervisor
L3 Agent How it’s implemented
• Manages Collection of Network Namespaces
• Isolated IP Stacks
• Forwarding Enabled
• net.ipv4.ip_forward=1
• Static Routing
• Metadata Proxy
lo
eth1
eth0
lo
qg-2
qr-1
lo
qg-b
qr-e
Host A B
br-ex
New in Icehouse
• 4 New Plugins/Drivers

• Stability Improvements

• Externally Tested Vendor Plugins

• Performance Improvements
Looking ahead to Juno
• Nova-Network Migration

• Distributed Virtual Routers

• IPv6

• Multi Vendor Service Support
More Information
• Cloud Administrator Guide
• http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html
• Network v2.0 API
• http://developer.openstack.org/api-ref-networking-v2.html
Thank You
Mark McClain

twitter: @gtwmm
Extra Info
OSI Model
Physical Layer
Data Link Layer
Network Layer
Transport Layer
Session Layer
Presentation Layer
Application Layer
1
2
3
4
5
6
7
TCP, UDP
IPv4,IPv6, ICMP
HTTP, DNS, etc
ARP, Ethernet, VLAN
Configuration Agents
Configuration Agents: DHCP
• RPC based notifications
• dnsmasq
• Isolation Support via Network Namespaces
• Multiple copies for HA
Configuration Agents: Metadata Proxy
• Proxies Metadata requests to Nova
• Routed Networks
• process embedded in router
• Non-routed Networks
• static route redirects traffic running in DHCP namespace
Configuration Agents: Metadata Proxy
curl http://169.254.169.254/openstack/latest/meta_data.json
Tenant
VM
Nova
Metadata
Service
Meta NS Proxy
Metadata
Agent
Unix Domain Socket
Management Network
X-Router-Id: 2bc7c882-d612-438c-a334-0047f2b5c2d7
X-Forwarded-For: 10.0.0.1
X-Instance-ID: aaaaaaaa-aaaa-aaaa-aaaaaaaaaaaa

Bridges and Tunnels: A Drive Through OpenStack Networking

  • 1.
    Bridges and Tunnels:A Drive Through OpenStack Networking Mark McClain twitter: @gtwmm
  • 2.
    Why Create Neutron? •New Technology • Tenant Isolation • Rich Topologies • Advance Services Support • Load Balancing, VPN, Firewall
  • 3.
  • 4.
    What does theuser see? Compute API Network API Storage APIGUI, CLI, API Libs KVM ML2 Plugin Ceph
  • 5.
    Abstractions Net1 10.0.0.0/24 Nova Neutron L2 virtual network virtualport virtual server virtual interface (VIF) virtual subnet VM1 10.0.0.2 VM2 10.0.0.2
  • 6.
  • 7.
    Design Goals Open vSwitch/ Linux Bridge Ryu OpenFlow Controller • Unified API • Small Core • Pluggable Open Architecture • Multiple Vendor Support • Extensible
  • 8.
  • 9.
    Basic Deployment neutron-server L2 AgentL2AgentL2 AgentL2 AgentL2 AgentL2 Agent L3 AgentL3 Agent L3 AgentL3 Agent Database L3 Agent DHCP Agent L2 Agent Message Queue Adv Services neutron-server
  • 10.
    neutron-server REST API SERVICE RPC SERVICE PLUGIN RESTAPI SERVICE RPC SERVICE • REST API • HTTP(S) Python WSGI Application • Customary TCP port is 9696 • Exposes logical resources • networks, subnets, ports, etc • Request/Response Serialization • RPC Service • AMQP via Oslo messaging modules • Enables bidirectional agent communication • Optional neutron-server
  • 11.
    neutron-server REST API SERVICE RPC SERVICE PLUGIN •PLUGIN • Written in Python • Only one active • Must implement V2 API calls • Optional database access • Optional extension support
  • 12.
    The Plugin ML2Plugin core_plugin =neutron.plugins.ml2.plugin.Ml2Plugin
  • 13.
  • 14.
    Plugin Extensions • Addlogical resources to the REST API • Discovered by server at startup • REST: /v2.0/extensions • Common Extensions • Binding, DHCP, L3, Provider, Quota, Security Group • Other Extensions • Allowed Addresses, Extra Routes, Metering
  • 15.
    Monolithic Plugin • Fullimplementation of core resources • Two types: • Proxy • Direct control PLUGIN
  • 16.
    ML2: Modular Layer2 Plugin • Full V2 Plugin Implementation • Delegates calls to proper L2 drivers • Two kinds of drivers • Type Driver • Mechanism Driver Mech Mgr PLUGIN Type Mgr PLUGIN
  • 17.
    Basic Deployment neutron-server L2 AgentL2AgentL2 AgentL2 AgentL2 AgentL2 Agent L3 AgentL3 Agent L3 AgentL3 Agent Database L3 Agent DHCP Agent L2 Agent Message Queue Adv Services
  • 18.
  • 19.
    L2 Agent • Runson hypervisor • Watch and notify when devices added/removed • Wires new devices to bridge • Proper network segment • Security Group Rules
  • 20.
    Isolation VLAN • 802.1Q • limited •underlay must support GRE/VXLAN • L2 encapsulated in L3 • routable • overlay independence
  • 21.
  • 22.
    Tunneling with L2Population A D CB
  • 23.
    • Open SourceVirtual Switch • http://openvswitch.org • Can manage • VLAN, GRE, VXLAN OVSDB Open vSwitch Neutron Server OVS Agent OVS RPC
  • 24.
    Basic Deployment neutron-server L2 AgentL2AgentL2 AgentL2 AgentL2 AgentL2 Agent L3 AgentL3 Agent L3 AgentL3 Agent Database L3 Agent DHCP Agent L2 Agent Message Queue Adv Services
  • 25.
    Linux Network Namespace •Isolated copy of network stack • private loopback • scope limited to namespace • can reuse addresses • Explicit configuration needed to connect • Processes can spawn within namespace lo eth1 eth0 lo eth1 eth0 lo eth1 eth0 Host A B br-int
  • 26.
    Network Node L3 Agent • Runon Network Node • Uses Namespaces • Metadata Agent (if enabled) Network Node Core Hypervisor Hypervisor Hypervisor
  • 27.
    L3 Agent Howit’s implemented • Manages Collection of Network Namespaces • Isolated IP Stacks • Forwarding Enabled • net.ipv4.ip_forward=1 • Static Routing • Metadata Proxy lo eth1 eth0 lo qg-2 qr-1 lo qg-b qr-e Host A B br-ex
  • 28.
    New in Icehouse •4 New Plugins/Drivers • Stability Improvements • Externally Tested Vendor Plugins • Performance Improvements
  • 29.
    Looking ahead toJuno • Nova-Network Migration • Distributed Virtual Routers • IPv6 • Multi Vendor Service Support
  • 30.
    More Information • CloudAdministrator Guide • http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html • Network v2.0 API • http://developer.openstack.org/api-ref-networking-v2.html
  • 31.
  • 32.
  • 33.
    OSI Model Physical Layer DataLink Layer Network Layer Transport Layer Session Layer Presentation Layer Application Layer 1 2 3 4 5 6 7 TCP, UDP IPv4,IPv6, ICMP HTTP, DNS, etc ARP, Ethernet, VLAN
  • 34.
  • 35.
    Configuration Agents: DHCP •RPC based notifications • dnsmasq • Isolation Support via Network Namespaces • Multiple copies for HA
  • 36.
    Configuration Agents: MetadataProxy • Proxies Metadata requests to Nova • Routed Networks • process embedded in router • Non-routed Networks • static route redirects traffic running in DHCP namespace
  • 37.
    Configuration Agents: MetadataProxy curl http://169.254.169.254/openstack/latest/meta_data.json Tenant VM Nova Metadata Service Meta NS Proxy Metadata Agent Unix Domain Socket Management Network X-Router-Id: 2bc7c882-d612-438c-a334-0047f2b5c2d7 X-Forwarded-For: 10.0.0.1 X-Instance-ID: aaaaaaaa-aaaa-aaaa-aaaaaaaaaaaa