OpenStack	
  Havana	
  On	
  IPv6
Shixiong	
  Shang	
  
Randy	
  Tuttle	
  
Ciprian	
  Popoviciu	
  

!
Version	
  1.9.3
©...
Agenda
§ Introduction
§ IPv6 and Cloud
§ IPv6 Refreshment
§ Proof of Concept
§ Proposed Blueprint
§ Next Steps

© 20...
Introduction
§ Nephos6

§ Ciprian Popoviciu

– Service assurance company

– Founder, CEO

– Founded in June, 2011

– IPv...
IP Comparison
IPv4

IPv6

Address

32-bit,
128-bit,
Network Address Translation Multiple Scopes

ICMP

ICMP

ICMPv6

Autoc...
IPv6 and Cloud
IPv6 Strength

Business Value

Sufficient address space
Direct access to resources
Simplified Address Assig...
IPv6 Address Auto-Configuration
Our	
  focus	
  today!

Working	
  in	
  progress!

SLAAC*

DHCPv6

Address Assignment
(no...
SLAAC
§ RFC 4861 - “Neighbor Discovery for IP Version 6 (IPv6)” and RFC
4862 - “IPv6 Stateless Address Autoconfiguration”...
SLAAC Address Calculation
§ IPv6 SLAAC = network portion (i.e. /64 Prefix in RA) + interface id
(i.e. EUI64)
FA

MAC
Inse...
OpenStack IPv6 Readiness
OpenStack Havana

OpenStack Icehouse

Limited IPv6 support out of box

Neutron will support IPv6…...
th
s wi nd
s
u c c e zz ly a
Proof Of Concept
S
i
h Gr a na!
b ot av
H
Mission Statement: To make these two inflection poi...
POC Architecture
Controller Node
nova-api
nova-scheduler
nova-consoleauth

Network Node

nova-novncproxy

neutron-dhcp-age...
1.	
   All	
   OpenStack	
   infrastructure	
  
n o d e s	
   s h o u l d	
   b e	
   a b l e	
   to	
  
communicate	
  wi...
Enable IPv6 On Infrastructure
Nodes

Field

Value

Keystone

/etc/keystone/keystone.conf

bind_host

2001:7:10:180::101

M...
Enable IPv6 On Infrastructure
Nodes

/etc/cinder/cinder.conf

Value
2001:7:10:180::102

glance_host

2001:7:10:180::102

o...
2.	
   OpenStack	
   should	
   be	
   able	
   to	
  
spin	
  up	
  dual-­‐stack	
  VMs	
  in	
  multi-­‐
tenant	
  envir...
Neutron Tenant Network Provisioning
neutron router-create --tenant-id tenant2-id router2

!
neutron net-create --tenant-id...
Neutron Tenant Network
dnsmasq	
  binding	
  	
  
interface	
  (ipv4)

2.	
  OpenStack	
  needs	
  to	
  
know	
  this	
  ...
Enable RA Within Router Namespace
§ Method “spawn_process” in neutron.agent.linux.dhcp.py on Network Node
Derive	
  route...
3.	
   VMs	
   should	
   be	
   able	
   to	
   gain	
  
connectivity	
   to	
   external	
   IPv6	
  
network	
   beyond...
Dual-Stack options
§ Option #1: Use next-hop RA and SLAAC to allow external GW
interface defined IPv6 address
§ Option #...
Neutron External Network
Need	
  ip6tables	
  filter	
  
rules	
  to	
  enable	
  ICMPv6	
  
at	
  inbound	
  direction

N...
Dual-stack options
§ For Option #2, there exists a limitation on static IP address
assignment for dual-stack implementati...
Dual-stack solution

To	
  accomplish	
  a	
  static	
  dual-­‐stack	
  
arrangement,	
  ip_version,	
  cidr,	
  
ip_addre...
Dual-stack configuration
§ For the tenant router, learn the default route from the upstream
router through RA. When addin...
Summary
Findings

Fixes

RA is not sent to IPv6 enabled internal
tenant network by default

Enable RA on dnsmasq

DHCP pro...
Proposed Blueprint
§ From openstack-dev mailer:
– Short term, my goal is to get provider networks up and running, where
i...
Our Next Step
Tactical

Strategical

DHCPv6

IPv6 mindset

Migration Strategy

IPv6 understanding / education

SLAAC + DHC...
© 2013 nephos6 and/or its affiliates. All rights reserved.
Upcoming SlideShare
Loading in...5
×

OpenStack Havana over IPv6

2,413

Published on

This slide is presented in Dec., 2013 as part of Triangle OpenStack meet up sponsored by Cisco System in Raleigh-Durham area, North Carolina.

We did proof of concept back in June, 2013 to evaluate IPv6 readiness of OpenStack as the initial step to make IPv6 and Cloud work together seamlessly.

After 6-week of intensive efforts, we enabled OpenStack Grizzly release over IPv6. Later on, we also successfully launched dual-stack VM in Havana release. This slide summarized what problems we tried to tackle and how we resolved them. The presentation is based on the whitepaper we published at:

http://www.nephos6.com/pdf/OpenStack-Havana-on-IPv6.pdf.

The ideas captured in this slide will be leveraged by OpenStack Neutron IPv6 sub team to fulfill mid-term goals suggested by Neutron IPv6 roadmap. The target release is IceHouse in April, 2014.

We will publish more white papers and slides when we reach next milestone. Stay tuned!

Published in: Technology
3 Comments
1 Like
Statistics
Notes
No Downloads
Views
Total Views
2,413
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
84
Comments
3
Likes
1
Embeds 0
No embeds

No notes for slide

OpenStack Havana over IPv6

  1. 1. OpenStack  Havana  On  IPv6 Shixiong  Shang   Randy  Tuttle   Ciprian  Popoviciu   ! Version  1.9.3 © 2013 nephos6 and/or its affiliates. All rights reserved.
  2. 2. Agenda § Introduction § IPv6 and Cloud § IPv6 Refreshment § Proof of Concept § Proposed Blueprint § Next Steps © 2013 nephos6 and/or its affiliates. All rights reserved. 2
  3. 3. Introduction § Nephos6 § Ciprian Popoviciu – Service assurance company – Founder, CEO – Founded in June, 2011 – IPv6 expert – Twitter: @Nephos6 – Twitter: @Nephos6 – Web: http://www.nephos6.com – Email: chip@nephos6.com ! ! § Shixiong Shang § Randy Tuttle – Head of Engineering – Network Consulting Engineer – Twitter: @shshang – Twitter: @randyttl – Email: shshang@nephos6.com – Email: rantuttl@cisco.com © 2013 nephos6 and/or its affiliates. All rights reserved. 3
  4. 4. IP Comparison IPv4 IPv6 Address 32-bit, 128-bit, Network Address Translation Multiple Scopes ICMP ICMP ICMPv6 Autoconfiguration DHCP SLAAC, DHCPv6, DHCP-PD Routing RIPv2, OSPFv2, ISIS, MPBGP, EIGRP RIPng, OSPFv3, ISIS-ST/ MT, MP-BGP, EIGRPv6 IP Multicast IGMP/PIM/Multicast BGP MLD/PIM/Multicast BGP, Scope Identifier “IPv6  Is  an  Evolution,  Not  a  Revolution  of  the  Internet  Protocol” © 2013 nephos6 and/or its affiliates. All rights reserved. 4
  5. 5. IPv6 and Cloud IPv6 Strength Business Value Sufficient address space Direct access to resources Simplified Address Assignment Native support of multicast and flow label New architectural models } } Easier management and lower operational cost Great opportunity for innovation “The  promise  of  Cloud  cannot  be  fully  met  without  IPv6” © 2013 nephos6 and/or its affiliates. All rights reserved. 5
  6. 6. IPv6 Address Auto-Configuration Our  focus  today! Working  in  progress! SLAAC* DHCPv6 Address Assignment (non-link-local) By exchanging Router Solicitation and Router Advertisement messages with neighboring routers. From DHCPv6 server Additional Information None From DHCPv6 server Default Gateway The only way to announce default route is using Router Advertisement! Pros Plug and play IPv4-like approach, but better More control Cons Doesn’t provide Hostname, DNS server, WINS, etc. Operational overhead (extra DHCP server, HA, etc.) * StateLess Address AutoConfiguration © 2013 nephos6 and/or its affiliates. All rights reserved. 6
  7. 7. SLAAC § RFC 4861 - “Neighbor Discovery for IP Version 6 (IPv6)” and RFC 4862 - “IPv6 Stateless Address Autoconfiguration” § Rely on ICMPv6 (IPv6 control plane!) Host Router  Solicitation  (RS) Router  Advertisement  (RA)          subnet  prefix          lifetime          autoconfig  flag Router Solicitation (RS) Router Advertisement (RA) ICMPv6 Type 133 ICMPv6 Type 134 IPv6 Source A Link Local IPv6 Source A Link Local IPv6 Destination Link-local scope all-routers address (FF02::2) IPv6 Destination Router Link-local scope all-nodes address (FF02::1) § VM sends Router Solicitation at boot time to solicit Router Advertisement § Router sends RA to all-nodes address periodically § Default route points to router’s link-local address § Router can also unicast RA back to VM upon receiving RS © 2013 nephos6 and/or its affiliates. All rights reserved. 7
  8. 8. SLAAC Address Calculation § IPv6 SLAAC = network portion (i.e. /64 Prefix in RA) + interface id (i.e. EUI64) FA MAC Insert  0xFFFE  in   the  middle FA 1111 EUI-­‐64 IPv6 address 73 83 D9 16 3E FF FE 73 83 D9 16 3E FF FE 73 83 D9 1000 F8 = 3E 1010 1111 Change  7th  bit   in  OUI  part 16 2001:7:10:180:F816:3EFF:FE73:83D9 © 2013 nephos6 and/or its affiliates. All rights reserved. 8
  9. 9. OpenStack IPv6 Readiness OpenStack Havana OpenStack Icehouse Limited IPv6 support out of box Neutron will support IPv6… Neutron IPv6 roadmap is still in preliminary stage Blueprint: IPv6 Feature Parity (working in progress…) No clear IPv6 roadmap for other OpenStack projects Neutron-IPv6-Subteam (ongoing) Very limited documentation Biggest risk of all: IPv4 way of thinking © 2013 nephos6 and/or its affiliates. All rights reserved. 9
  10. 10. th s wi nd s u c c e zz ly a Proof Of Concept S i h Gr a na! b ot av H Mission Statement: To make these two inflection points, IPv6 and Cloud work together seamlessly! Motivation Goals We are believers All OpenStack infrastructure nodes should be able to communicate with each other by IPv6 What it is v.s. What it should be OpenStack should be able to spin up dual-stack VMs in multi-tenant environment We are doers…but we are not hackers, or developers :) VMs should be able to gain connectivity to external IPv6 network beyond OpenStack’s control © 2013 nephos6 and/or its affiliates. All rights reserved. 10
  11. 11. POC Architecture Controller Node nova-api nova-scheduler nova-consoleauth Network Node nova-novncproxy neutron-dhcp-agent Common Node nova-cert neutron-l3-agent horizon nova-conductor neutron-metadataagent keystone cinder openvswitch nova-compute mysql db glance neutronopenvswitch-agent neutronopenvswitch-agent rabbitmq neutron-server dnsmasq openvswitch eth0 eth0 7.10.180.101 2001:7:10:180::101 7.10.180.102 2001:7:10:180::102 Management and API network 7.10.180.0/24 2001:7:10:180::/64 Management  and   API  network eth0 eth1 eth2 Compute Node eth3 vlan 511 vlan 512 eth0 eth3 vlan 511 vlan 512 7.10.180.104 2001:7:10:180::104 7.10.180.103 2001:7:10:180::103 Tenant Data Networks (Tenant 1: VLAN 511) (Tenant 2: VLAN 512) External   Network Tenant 2 External Network 172.26.185.0/24 2001:172:26:185::/64 Tenant 1 External Network 172.26.184.0/24 2001:172:26:184::/64 Data   Network Router © 2013 nephos6 and/or its affiliates. All rights reserved. 11
  12. 12. 1.   All   OpenStack   infrastructure   n o d e s   s h o u l d   b e   a b l e   to   communicate  with  each  other  by   IPv6 -­‐  IT  IS  ALL  ABOUT  CONFIGURATION © 2013 nephos6 and/or its affiliates. All rights reserved. 12
  13. 13. Enable IPv6 On Infrastructure Nodes Field Value Keystone /etc/keystone/keystone.conf bind_host 2001:7:10:180::101 MySQL DB /etc/mysql/my.cnf bind-address :: Apache /etc/apache2/ports.conf Listen 80 my_ip 2001:7:10:180::102 use_ipv6 true osapi_compute_listen 2001:7:10:180::102 metadata_listen Common Components Configuration Files 7.10.180.102 novncproxy_host 2001:7:10:180::102 bind_host 2001:7:10:180::102 registry_host net-glance.sandbox.com bind_host 2001:7:10:180::102 Nova /etc/nova/nova.conf Controller /etc/glance/glance-api.conf Glance /etc/glance/glanceregistry.conf © 2013 nephos6 and/or its affiliates. All rights reserved. 13
  14. 14. Enable IPv6 On Infrastructure Nodes /etc/cinder/cinder.conf Value 2001:7:10:180::102 glance_host 2001:7:10:180::102 osapi_volume_listen Cinder Field my_ip Controller Components Configuration Files 2001:7:10:180::102 Neutron 2001:7:10:180::102 Neutron /etc/neutron/neutron.conf bind_host 2001:7:10:180::103 2001:7:10:180::102 use_ipv6 Compute bind_host my_ip Network /etc/neutron/neutron.conf true osapi_compute_listen 2001:7:10:180::102 metadata_listen 7.10.180.102 novncproxy_host 2001:7:10:180::102 bind_host 2001:7:10:180::103 Nova Neutron /etc/nova/nova.conf /etc/neutron/neutron.conf © 2013 nephos6 and/or its affiliates. All rights reserved. 14
  15. 15. 2.   OpenStack   should   be   able   to   spin  up  dual-­‐stack  VMs  in  multi-­‐ tenant  environment -­‐  IT  IS  ALL  ABOUT  IPV6  ADDRESS  ASSIGNMENT © 2013 nephos6 and/or its affiliates. All rights reserved. 15
  16. 16. Neutron Tenant Network Provisioning neutron router-create --tenant-id tenant2-id router2 ! neutron net-create --tenant-id tenant2-id net2_192_168_2 -provider:network_type vlan --provider:physical_network physnet3 --provider:segmentation_id 512 ! IPv6  tenant  subnet Specify  IP  version  6 neutron subnet-create --tenant-id tenant2-id --ip-version 4 -name sub2_192_168_2 net2_192_168_2 192.168.2.0/24 neutron subnet-create —tenant-id tenant2-id --ip-version 6 -name sub2_2001_192_168_2 net2_192_168_2 2001:192:168:2::/64 Port  is  associated   ! with  tenant  subnet neutron router-interface-add router2 sub2_192_168_2 neutron router-interface-add router2 sub2_2001_192_168_2 © 2013 nephos6 and/or its affiliates. All rights reserved. 16
  17. 17. Neutron Tenant Network dnsmasq  binding     interface  (ipv4) 2.  OpenStack  needs  to   know  this  self-­‐calculated   IPv6  SLAAC  address… qdhcp  namespace ns-­‐74f270ff-­‐01   (192.168.2.2) 3.  Need  dnsmasq  to   send  RA  from  default   gateway  interface 1.  Need  ip6tables  filter   rules  to  enable  ICMPv6   at  inbound  direction VM   192.168.2.3   (ipv6  address) tap-­‐intf tap74f270ff-­‐01 RA qrouter  namespace br-­‐eth2 eth2 qr-­‐6dbfb73d-­‐89   (2001:192:168:2::1) Default  Gateway   Interface  (ipv4) To  External  Network Default  Gateway   Interface  (ipv6) br-­‐eth3 Compute  Node qr-­‐2f573f07-­‐d9   (192.168.2.1) Network  Node br-­‐int eth3 br-­‐int br-­‐eth3 eth3 Tenant  2  Network © 2013 nephos6 and/or its affiliates. All rights reserved. 17
  18. 18. Enable RA Within Router Namespace § Method “spawn_process” in neutron.agent.linux.dhcp.py on Network Node Derive  router’s   namespace  and   gateway  interface   Enable  dnsmasq  with   RA  and  SLAAC     Specify  IPv6  DHCP   range.  Taken  from   CLI Add  IP  version  check Bind  to  IPv6  qr-­‐  interface Launch  dnsmasq  in   router’s  namespace © 2013 nephos6 and/or its affiliates. All rights reserved. 18
  19. 19. 3.   VMs   should   be   able   to   gain   connectivity   to   external   IPv6   network   beyond   OpenStack’s   control -­‐  Support  dual-­‐stack  on  a  single  external  interface       -­‐  Utilize  existing  VLAN/Segmentation  ID   ! -­‐  Eliminate  NAT  and  GARP  for  IPv6  subnets © 2013 nephos6 and/or its affiliates. All rights reserved. 19
  20. 20. Dual-Stack options § Option #1: Use next-hop RA and SLAAC to allow external GW interface defined IPv6 address § Option #2: Statically assign IPv6 address to external GW interface for the router – neutron router-gateway-set router2 ext-net-185 © 2013 nephos6 and/or its affiliates. All rights reserved. 20
  21. 21. Neutron External Network Need  ip6tables  filter   rules  to  enable  ICMPv6   at  inbound  direction Namespace:  qdhcp-­‐bfc3d877-­‐   44b6-­‐4879-­‐a83e-­‐d37455e77f71 dnsmasq  binding     interface  (ipv4) ns-­‐74f270ff-­‐01   (192.168.2.2) dnsmasq  binding   interface  (ipv6) VM   192.168.2.3   (2001:192:168:2::1) br-­‐int br-­‐int qr-­‐2f573f07-­‐d9   (192.168.2.1) qr-­‐6dbfb73d-­‐89   (2001:192:168:2::1) qg-­‐3dac3be9-­‐1b   (172.26.185.70)   (SLAAC  or  statically  assigned) br-­‐eth2 br-­‐eth3 Compute  Node tap-­‐intf Network  Node tap74f270ff-­‐01 br-­‐eth3 Namespace:  qrouter-­‐94662c71-­‐ bf80-­‐4c2f-­‐9841-­‐09a2112e3f58 eth2 eth3 RA To  External  Network Disable  NAT  and   GARP  for  IPV6 eth3 Tenant  2  Network © 2013 nephos6 and/or its affiliates. All rights reserved. 21
  22. 22. Dual-stack options § For Option #2, there exists a limitation on static IP address assignment for dual-stack implementation. § The L3 (server and agent) only allows a single IP address per network (VLAN) within the Linux namespace representing the tenant's router. § This limitation precluded the possibility of a dual-stack arrangement utilizing static assignments without code changes. © 2013 nephos6 and/or its affiliates. All rights reserved. 22
  23. 23. Dual-stack solution To  accomplish  a  static  dual-­‐stack   arrangement,  ip_version,  cidr,   ip_address  and  gateway_ip,  was   essential  for  L3  agent  to  build  dual-­‐ stack  interface  inside  router’s   namespace. © 2013 nephos6 and/or its affiliates. All rights reserved. 23
  24. 24. Dual-stack configuration § For the tenant router, learn the default route from the upstream router through RA. When adding an external gateway – net.ipv6.conf.<gateway_interface>.accept_ra=2 – net.ipv6.conf.<gateway_interface>.forwarding=1 – net.ipv6.conf.<gateway_interface>.accept_ra_defrtr=1 § Prevent learning a default route from RA from internal tenant network – net.ipv6.conf.<internal_interface>.accept_ra_defrtr=0 § When the subnet assigned is an IPv6, don’t apply NAT configuration or perform GARP. © 2013 nephos6 and/or its affiliates. All rights reserved. 24
  25. 25. Summary Findings Fixes RA is not sent to IPv6 enabled internal tenant network by default Enable RA on dnsmasq DHCP process is bound to interface other than default gateway of tenant network IPv6 address chosen by OpenStack is not based on SLAAC standard Launch dnsmasq process inside router namespace Neighbor Discovery packet is dropped by ip6tables filter rules Add ip6tables rules to allow ND related ICMPv6 packets NAT and GARP are turned on for IPv6 subnets. Not desirable! Only perform NAT and GARP for IPv4 subnets Calculate VM’s IPv6 address based on unique MAC address Whitepaper: http://www.nephos6.com/pdf/OpenStack-Havana-on-IPv6.pdf © 2013 nephos6 and/or its affiliates. All rights reserved. 25
  26. 26. Proposed Blueprint § From openstack-dev mailer: – Short term, my goal is to get provider networks up and running, where instances can get RA's from an upstream router outside of OpenStack and configure themselves. – Medium term, we want to make dnsmasq configuration more flexible. – More long term, I'd like to make it so that if there is an upstream router doing RA's - Neutron should send a PD automatically on network creation, and populate a subnet from the response given by the upstream router. § Service Provider focused; may not work entirely with L3 Agent without revisions § Integrate this PoC work with Blueprint to address broader OpenStack community and address L3 Agent © 2013 nephos6 and/or its affiliates. All rights reserved. 26
  27. 27. Our Next Step Tactical Strategical DHCPv6 IPv6 mindset Migration Strategy IPv6 understanding / education SLAAC + DHCPv6 Participation in IPv6 + Cloud efforts Support for dual-stack infrastructure Icehouse release validation © 2013 nephos6 and/or its affiliates. All rights reserved. 27
  28. 28. © 2013 nephos6 and/or its affiliates. All rights reserved.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×