This document discusses using routed provider networks in OpenStack to address issues with large broadcast domains when using VLAN provider networks. It describes how to create a multi-segment routed provider network in OpenStack with multiple IP address ranges assigned to different network segments. This allows subnets to be created on each segment and VMs to be launched on the provider network. The Romana project is also introduced as a way to implement topology-aware IP address management (IPAM) for routed provider networks to efficiently allocate IP addresses.

1. OpenStack
Routed Provider Networks
What, why and how…..
May 2017 OpenStack Meetup

2. Agenda
• Provider Networks v. Tenant Networks
• VLAN Provider Networks
• Use Case 1: Multisegment Provider Network
• Configuration Example
• Assumption/Limitations
• Use Case 2: Layer 3 Spine/Leaf Deployment
• Romana and Topology Aware IPAM
May 2017 OpenStack Meetup Slide 1

3. Provider v. Tenant Networks
• Provider Network
• Physical datacenter network that operator provides
• Shared among OpenStack Projects (i.e. no duplicate IP addresses)
• Visible across OpenStack Projects (i.e. bridged/shared Ethernet/L2)
• Uses other datacenter infrastructure (i.e. routers, gateways, etc.)
• Tenant Network
• Isolated virtual network created by user dedicated to Project
• VXLAN layer 2 overlay network
• Overlapping IP addresses
• Uses Neutron routers for access, NAT, etc.
May 2017 OpenStack Meetup Slide 2

4. VLAN Provider Networks
May 2017 OpenStack Meetup Slide 3
L2 Bridge
DC
Resources
DC VLAN

5. ToRToRToR
Datacenter Provider Networks
• Broadcast domains grow too large
• Broadcast storms
• Single fault domain
• VLANs not trunked to all nodes
• End user confusion
• Segment1 or Segment2?
May 2017 OpenStack Meetup Slide 4
Rack 1
ToR
Segment1 (VLAN100)
Segment2 (VLAN200)
Rack 2 Rack 3 Rack 410.124.0.0/16
192.168.2.0/24

6. Routed Provider Networks
• Logically join L2 segments into one
larger L3 Networks
• Split L2 domains
• Addresses bound to rack
• No VLAN trunks
• External router provides L3 connectivity
• Need DHCP on each segment
• Users just attach to multisegment1
May 2017 OpenStack Meetup Slide 5
multisegment1
10.124.0.0/16
192.168.214.0/24
Rack 1 Rack 2 Rack 3 Rack 4
192.168.2.0/25
192.168.2.128/25
10.124.0.0/17
10.124.128.0/17

7. Create Routed Provider Network
May 2017 OpenStack Meetup Slide 6
$ openstack network create --share --provider-physical-network provider1
--provider-network-type vlan --provider-segment 2016 multisegment1
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| l2_adjacency | True |
| mtu | 1500 |
| name | multisegment1 |
| port_security_enabled | True |
| provider:network_type | vlan |
| provider:physical_network | provider1 |
| provider:segmentation_id | 2016 |
| router:external | Internal |
| shared | True |
| status | ACTIVE |
| subnets | |
| tags | [] |
+---------------------------+--------------------------------------+
$ openstack network segment list --network multisegment1
+--------------------------------------+----------+--------------------------------------+--------------+---------+
| ID | Name | Network | Network Type | Segment |
+--------------------------------------+----------+--------------------------------------+--------------+---------+
| 43e16869-ad31-48e4-87ce-acf756709e18 | None | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 | vlan | 2016 |
+--------------------------------------+----------+--------------------------------------+--------------+---------+
$ openstack network segment set --name segment1 43e16869-ad31-48e4-87ce-acf756709e18
$ openstack network create --share --external
--provider-physical-network provider
--provider-network-type flat provider

8. Create Subnets on Segments
May 2017 OpenStack Meetup Slide 7
$ openstack subnet create
--network multisegment1 --network-segment segment1
--ip-version 4 --subnet-range 10.124.0.0/17
multisegment1-segment1-v4
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 10.124.0.2/17 - 10.124.127.254 |
| cidr | 10.124.0.0/17 |
| enable_dhcp | True |
| gateway_ip | 10.124.0.1 |
| id | c428797a-6f8e-4cb1-b394-c404318a2762 |
| ip_version | 4 |
| name | multisegment1-segment1-v4 |
| network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 |
| segment_id | 43e16869-ad31-48e4-87ce-acf756709e18 |
+-------------------+--------------------------------------+
$ openstack subnet create
--network multisegment1 --network-segment segment2
--ip-version 4 --subnet-range 192.168.2.0/25
multisegment1-segment1-v4
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 192.168.2.2 - 192.168.2.127 |
| cidr | 192.168.2.0/25 |
| enable_dhcp | True |
| gateway_ip | 192.168.2.1 |
| id | c428797a-6f8e-4cb1-b394-c404318a2762 |
| ip_version | 4 |
| name | multisegment1-segment1-v4 |
| network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b9785 |
| segment_id | 43e16869-ad31-48e4-87ce-acf756709e18 |
+-------------------+--------------------------------------+
$ openstack subnet create
--network multisegment1 --network-segment segment1
--ip-version 4 --subnet-range 10.124.128.0/17
multisegment1-segment1-v4
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 10.124.128.2/17 - 10.124.128.254 |
| cidr | 10.124.128.0/17 |
| enable_dhcp | True |
| gateway_ip | 10.124.128.1 |
| id | c428797a-6f8e-4cb1-b394-c404318a2762 |
| ip_version | 4 |
| name | multisegment1-segment1-v4 |
| network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b98d9 |
| segment_id | 43e16869-ad31-48e4-87ce-acf756709e18 |
+-------------------+--------------------------------------+
$ openstack subnet create
--network multisegment1 --network-segment segment2
--ip-version 4 --subnet-range 192.168.2.128/25
multisegment1-segment1-v4
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 192.168.2.130 - 192.168.2.254 |
| cidr | 192.168.2.128/25 |
| enable_dhcp | True |
| gateway_ip | 192.168.2.129 |
| id | c428797a-6f8e-4cb1-b394-c404318a2762 |
| ip_version | 4 |
| name | multisegment1-segment1-v4 |
| network_id | 6ab19caa-dda9-4b3d-abc4-5b8f435b9785 |
| segment_id | 43e16869-ad31-48e4-87ce-acf756709e18 |
+-------------------+--------------------------------------+

9. Launch VMs on Provider Networks
May 2017 OpenStack Meetup Slide 8
$ openstack server create --flavor m1.nano --image cirros
--nic multisegment1 --security-group default
--key-name mykey provider-instance

10. Scarce Addresses Use Case
• DC1 datacenter VLAN
• Up to 1024 IPs on /22 network
• L2 lets Nova place VMs anywhere
• 1024 VMs on VLAN fragile
• Move to L3 Spine/Leaf design
• Even might want to route to host
May 2017 OpenStack Meetup Slide 9
DC1 (VLAN100) 10.124.192.0/22 Rack 1 Rack 2 Rack 3 Rack 4
ToRToRToR ToR
DC
Resources
Can Routed Provider Networks Help?

11. Scarce Addresses Use Case
• Split /22 across 4 racks
• 4x /24 CIDRs 254 IPs per rack
• What about routing to host?
• 32 hosts per rack, /29 CIDR
• 8 IPs per host
• Trade off scheduling flexibility vs
wasted IPs
• Need dynamic route updates
• Schedule VM as necessary
• Announce route to IP upstream
May 2017 OpenStack Meetup Slide 10
DC1 10.124.192.0/22
DC
Resources
10.124.192.0/24 10.124.194.0/24
10.124.193.0/24 10.124.195.0/24
Host10.124.192.128/29

12. Romana Project
• Network and security automation
• All details available at romana.io
• Kubernetes and OpenStack integration
• Applies/enforces network policy
• Hundreds of deployments
• Open source
• Apache 2.0
• www.github.com/romana
• Romana v2.0 with Topology Aware IPAM
OpenStack MeetupMay 2017 Slide 11

13. Security
Policy
Neutron Node
OpenStack Deployment
August 2016 romana.io
Neutron
ML2IPAM
Compute Node n
VM
iptables
VM
Nova
Agent
IPAM
Routes
etcd
Policy

14. L3 Provider Networks
August 2016 romana.io
Host 1
VM 1:
10.124.192.130
VM 2:
10.124.192.131
0.0.0.0 -> 10.124.192.1
G/W: 10.124.192.1/24
10.124.192.129/29
Host 2
VM 1:
10.124.192.194
VM 2:
10.124.192.195
0.0.0.0 -> 10.124.192.1
10.124.192.193/29
Host 2
VM 1:
10.124.192.226
VM 2:
10.124.192.227
0.0.0.0 -> 10.124.192.1
10.124.192.225/29
10.124.192.128/29 -> Host 1
10.124.192.192/29 -> Host 2
10.124.192.224/29 -> Host 3
…
32 Routes Max
Agent
Peer
Announce Route
Agent Agent

15. Scarce Addresses with Romana
• Split /22 across 4 racks
• 4x /24 CIDRs 254 IPs per rack
• Schedule VM to any host announce
route to leaf router
• Allocate IPs in small blocks
• Topology Aware IPAM ensures VMs
only get IPs reachable in Rack
May 2017 OpenStack Meetup Slide 14
DC1 10.124.192.0/22
DC
Resources
Host
10.124.192.0/24 10.124.194.0/24
10.124.193.0/24 10.124.195.0/24
10.124.192.128/29