The document discusses the implementation of a multi-site and multi-OpenStack cloud solution, focusing on the benefits of consolidation, improved resource management, and enhanced orchestration across different data centers. It presents a case study involving Vodafone, emphasizing the importance of maintaining open APIs while integrating multiple OpenStack instances from various vendors. The proposed OpenStack cascading solution aims to facilitate scalable cloud architectures, ensuring fault isolation, ease of management, and streamlined operations across a distributed cloud environment.

1. Building Multi-site & Multi-OpenStack Cloud
with OpenStack Cascading
Chaoyi Huang Huawei Technologies
Jiongjiong Gu Huawei Technologies
Alexis Susset Vodafone Group
Last update Jan.12, 2016

2. Page 1
Agenda
 Requirement and driving forces
 Vodafone Case Study
 Solution proposal of OpenStack cascading
 Live Demo

3. Page 2
Consolidation: The essence of Cloud
Benefits by higher level of consolidation :
- Improved business Agility & Management Simplicity
- Larger resource pooling and optimized Resource utilization
- Enlarged range of resource automation
Cloud OS
(Openstack)
App1 App2 Appn
…
Isolated Resource & Data Silos Dedicated for certain App Consolidated resource & data pooling Shared by Multiple Apps
…
OS1 OS2 OSn
Big
Data
DC1 DC2 DCk…
OSn
Appn
DB
Server
Storage
Network
OSn
Appn
DB
Server
Storage
Network
OSn
Appn
DB
Server
Storage
Network
…
DC1 DCk
Cloud
Transforming

4. Page 3
Multi-site Openstack Consolidation with unified Openstack API
 Due to maturity and difference on service cloud transformation ,
Openstack is normally built with standalone multiple DC sites in a
step by step way;
 How to achieve unified orchestration & automation across multiple
Openstack & DC sites?
 How maintain the continuity of OpenStack ecosystem?
OpenStack1
Host1 Hostk…
DC1
OpenStack2
Host1 Hostj…
DCn
DC Mngt. …
DC mngt. Cloud App…
…
Openstack EcosystemOpenstack Ecosystem
Openstack APIOpenstack API
Cloud App
OpenStack1
Host1 Hostk…
DC1
OpenStack2
Host1 Hostj…
DCn
…
Openstack API
DC Mngt. …
Openstack Ecosystem
Cloud App
OpenStack1
Host1 Hostk…
DC1
OpenStack2
Host1 Hostj…
DCn
…
Proprietory API
?

5. Page 4
Multi- Openstack PoD consolidation within the same DataCenter
Multiple modular “Openstack Cloud in Box”
consolidated into larger resource pooling;
Expanding the Openstack resource pooling by means
of adding more modular pre-integrated pooling to the
system;
OpenStack Pooling
(100 -> 1000 Hosts)
Openstack API
OpenStack1
Host1 Hostk
…
…
Consolidation &
Orchestration
Openstack API
Cloud Portal
& Orchestration
3rd party / Existing
DC mng.t
Host1 Host100…
Host1 Host100…
OpenStack Pooling
Openstack API
Hostk
OpenStack-n
Host1 Hostj
…
Pre-integrated Pooling
Hostn…

6. Page 5
Overlay network automation for VDC deployed across multiple DCs
Workloads of the same cloud tenants might span across multiple Openstack / DC sites, but
network connectivity still need to be automated rather than configured manually
Cross DC L2 connectivity Cross data center L3 networking Cross DC traffic optimization
DE
VM1 VM2 VM3 VM4
UK
Subnet1
Virtual Router
DE
VM1 VM2 VM3 VM4
UK
Subnet1
R
VR VR
Subnet2
DE
VM1 VM2 VM3 VM4
UK
Subnet1
VR VR
Subnet2
External Network External Network
Pluggable
External
Network
Pluggable
External
Network
**DVR – distributed virtual router

7. Page 6
Challenges of deploying Single Openstack across multi DC sites
No fault isolation zones, data center is in risk of out of management if controller failed, no API/CLI
available
Difficult for cross-DC integration via RPC for multi-vendor cluster
Challenge for trouble shooting/configuration change/rolling upgrade/patching… cross data center via RPC.
No clear responsibility boundary
Risk of vendor-lock in for one OpenStack instance
Challenge for performance tuning cross data center via RPC…
Nova/Cinder/Neutron
Controller
Nova/Cinder/Neutron
Agent/Driver
Message Queue
DC1
Nova/Cinder/Neutron
Agent/Driver
WAN
DC2
Nova/Cinder/Neutron
Agent/Driver

8. Page 7
Agenda
 Requirement and driving forces of multi-site cloud
 Vodafone Case Study
 Solution proposal of OpenStack cascading
 Live Demo

9. Page 8
47 OpCOs and counting ..
Mostly Acquired: Each
site can use one or multiple Vendors
which leads to better use of local
resources and capabilities
Diverse: Any site can connect
to each other, therefor establishing
globally addressable tenants which
result in global services
Local relationships:
Each site with its own pace and
requirements while maintaining
standard OpenStack APIs

10. Page 9
Vodafone use-case, World-wide OpenStack Orchestration
Multi-Vendor : Each site
can use one or multiple Vendors
which leads to better use of local
resources and capabilities
Global Networking: Any
site can connect to each other,
therefor establishing globally
addressable tenants which result in
global services
API Standard and
Multi-Version co-
existence: Each site with its
own pace and requirements while
maintaining standard OpenStack APIs
OpenStack APIs
Global Standard – Global Access

11. Page 10
Agenda
 Requirement and driving forces of multi-site cloud
 Vodafone Case Study
 Solution proposal of OpenStack cascading
 Live Demo

12. Page 11
Requirement summary for multi-site cloud
Multi-site, Multi-Vendor OpenStack distribution, Multi- OpenStack instance, Multi- OpenStack Version
Multi-vendor: anti-vendor lock in business policy.
Multi-instance: each vendor has his own OpenStack solution distribution
Multi-version: stepwise cloud construction, upgrade gradually
Restful open API /CLI for each site
OpenStack API in each site: Open, de facto standard API
make the cloud always workable and manageable standalone in each site
each site installation/upgrade/maintenance independently by different vendor or cloud admin.
Ecosystem friendly open API for the unified multi-site cloud
Ecosystem friendly open API : It takes almost 4 years for OpenStack to grow the eco-system, the
OpenStack API must be retained for distributed but unified multi-site cloud.
Cross DC orchestration: globally addressable tenants which result in global services. tenant virtual
resources will be distributed in multi-site but connected by L2/L3 networking

13. Page 12
Why not use Openstack itself as
multi-site/multi-Openstack orchestration layer?
OpenStack
( multi-site orchestrator )
…
OpenStack API
OpenStack APIOpenStack API
AZ1 AZn
DC 1 OpenStack DC n OpenStack
(Vendor1 /
Version 2.0)
(Vendor x /
Version 2.1)
 Openstack API & Ecosystem kept untouched after multi OpenStack consolidation
 multi-site orchestration become a part of OpenStack upstream which will benefits the whole
industry rather than specific vendor

14. Page 13
Good idea, but is it feasible?
OpenStack
…OpenStack OpenStack
OpenStack APIOpenStack API
Controller Node
Computer … Computer
OpenStack is generally used to
manage lots of compute nodes
OpenStack API
RPC RPC
Want to use OpenStack to orchestrator
OpenStacks?
?
Intuition - mapping the underlying OpenStack to a
compute node
OpenStack API

15. Page 14
Magic happens by just consider Nova/Cinder/Neutron
as his own backend !
Neutron Server
OVSAgent
Cinder API
Rabbit-MQ
Cinder
Volum
e
Cinder
Volum
e
Nova API
RabbitMQ
Nova
Compute
Nova
Compute
Libvirt
Driver
Nova
Driver
LVM
Driver
Cinder
Driver
KVM Nova LVM Cinder
RabbitMQ
OVS
L3Agent
Linux
Router
Scheduler
Scheduler
Neutron
Agent
Neutron
**Architecture simplified for illustration onlyFor Ceilometer and Glance, please refer to wiki link in the slides…

16. Page 15
Concept and architecture – OpenStack cascading solution
DB
Message Bus
Nova-API
Nova-Scheduler
Nova-Conductor
DB
Message Bus
Cinder-API
Cinder-Scheduler
DB
Message Bus
Neutron-API
Neutron-Plug-in
DB
Message Bus
Nova-API
Nova-Scheduler
Nova-Conductor
DB
Message Bus
Cinder-API
Cinder-Scheduler
DB
Message Bus
Neutron-API
Neutron-Plug-in
Cascaded OpenStack 1 Cascaded OpenStack x
…
Controller Node
Compute 1 Compute n
…
Compute 1 Compute n
DB
Message Bus
Nova-API
Nova-Scheduler
Nova-Conductor
DB
Message Bus
Cinder-API
Cinder-Scheduler
DB
Message Bus
Neutron-API
Neutron-Plug-in
Cascading OpenStack
Nova-API Cinder-API Neutron-API Nova-API Cinder-API Neutron-API
Controller Node Compute x
Nova-API Cinder-API Neutron-API
Introduced for
OpenStack
cascading
solution
Nova-Proxy
Cinder-Proxy
L2-Proxy
L3-Proxy
LB-Proxy
VPN-Proxy
Nova-Proxy
Cinder-Proxy
L2-Proxy
L3-Proxy
LB-Proxy
VPN-Proxy
FW-Proxy
FW-Proxy
Compute 1
VM
VM
VM
VM
Controller Node
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM

17. Page 16
OpenStack cascading works like a fractal
Fractal: Recursive self-similar, growth to scale
http://ethshar.wikia.com/wiki/File:Tricircle_fractal.gif
Cascading: Also recursive self-similar, growth to scale.
Just treat the child OpenStack as a super huge compute

18. Page 17
For example, cross OpenStack/DC L2 VxLAN
networking
Neuton-API
L2/L3-Proxy
L2/L3-Proxy
Neutron Neutron
VM1 VM2
VxLAN0
VM3 VM4
AZ1 AZ2
VM1 VM2
VxLAN0 DVR
VM3 VM4
VxLAN0
DVR
DVR
2.Periodic polling port
status( for example
VM2 port)
3. VM2 Port
status up
4. L2 population
5. fdb_add ( Port for VM2 IP / VM 2 mac / Host IP )
6. Create virtual remote Port for VM2
(with VM2 IP / VM2 mac / VM2 host IP)
VM2
7. Internal L2 population for
virtual remote port for VM2
Virtual remote port
Cascading OpenStack
Cascaded OpenStack
1. Internal L2 population for VM2
If VM2 was added to VxLAN0…
Self similar L2 population
happened in the cascading level

19. Page 18
Benefit summary of OpenStack cascading •Open-source multi-site cloud
orchestration layer with OpenStack API
•Fault isolation AZ by AZ
•Reduce Upgrade/O&M challenge in AZ
granularity
•Plug & Play Fast integration by
OpenStack API
•No vendor-lock in, Fast Integration for
multi-vendor/multi-site/multi-version
cloud infrastructure
•Scale out architecture. Horizontal
scalability, even cross multiple data
centers
•Expansion with replication of AZ by AZ
OpenStack
Vendor1
OpenStack2.0
OpenStack API
OpenStack API
…
DC1
Availability
Zone 1
Vendor n
OpenStack2.1
Vendor2
OpenStack2.
0
Vendor1
OpenStack2.
1
OpenStack APIOpenStack API
DCn
OpenStack API
Availability
Zone 2
Availability
Zone 3
Availability
Zone n ***One AZ can still integrate heterogeneous
hypervisors and heterogeneous physical
resources
…
… … … …
OpenStack cascading solution makes the distributed but unified multi-
site cloud always workable and manageable standalone in each site.

20. Page 19
Other benefits: Building ultra-large scale Cloud with 1 million VMs
OpenStack …
…
1 2 100
1 2 1000
OpenStack
…
1 2 1000
OpenStack
…
1 2 1000
OpenStack
Max. 100 data centers
Max. 100k server nodes
Max. 1 million VMs
We are working to make it happen, kindly invite you to join us…

21. Page 20
Agenda
 Requirement and driving forces of multi-site cloud
 Vodafone Case Study
 Solution proposal of OpenStack cascading
 Live Demo

22. Page 21
Use case1: Consolidating new OpenStack to the Cloud
Cascaded
OpenStack
Xi’an Hangzhou
Shenzhen
Shenzhen Hangzhou Xi’an
Dynamically
adding New
OpenStack

23. Page 22
Use case2: Single VDC deployed across multiple DCs
Cascading
OpenStack
Shenzhen DC (AZ1) Xi’an DC (AZ2)
Step1 : Creat Network (VxLAN ID1)
VM 1
VxLAN
Step2 : Creat VM(VM1 (VM2/VM3), VM Image1, AZ1, VxLAN ID1)
Step3 : Creat VM (VM4, VM Image2, AZ2, VxLAN ID1)
VM 2 VM 3
VxLAN ID1 VxLAN ID1 VxLAN ID1
Shenzhen Branch office
VM 4
Xi’an Branch Office

24. Page 23
Use case 3: Zero interruption DR across DCs
Cascading
OpenStack
Shenzhen DC Xi’an DC
vIMS
active
Step1.1 : Creat Volume (A, vIMS Image, AZ1)
Step1.2 : Creat Volume (B, vIMS Image, AZ2)
A B
Step2: Creat Shadow Volume (A, B)
Step3: Creat network (VxLAN1 & VxLAN2)
Step1 : Detach Volume (vMS active, VolA)
DR Preparation: DR execution:
Step2 : CreateVM (vIMS standby, VolB, AZ2,
VxLAN1, VxLAN2)
VxLAN ID1 vIMS
Standby
LB for vIMS
X
Data Replication
Openstack API
Step4 : Creat VM (vIMS active, VolA, AZ1, VxLAN1, VxLAN2)

25. Page 24
Use case4: VM Mobility across multiple DC
Cascaded
OpenStack
Shenzhen DC Xi’an DC
Step1.1 : Creat Volume (VolA, VDI Image, AZ1)
VDI 1
VDI 4
Step1.2 : Creat Volume (VolB, VDI Image, AZ2)
A B
Step2: Creat Shadow Volume (A, B)
Step3 : Creat VM (VDI1, VolA, AZ1, VxLAN 1)
Step1 : Detach Volume (VDI1, VolA)
Before migration: Perform migration:
Step2 : CreateVM (VDI1, VolB, AZ2, VxLAN 1)
VDI3
VDI2
VxLAN ID1
Data Replication
VxLAN ID1
VxLAN

26. Page 25
Welcome to join the design summit session:
Time: Tuesday November 4, 2014 14:50 - 15:30
Room: Manet
Topic:Approaches for scaling out

27. Page 26
Visit Huawei booth for live demo
More information:
[1] wiki: https://wiki.openstack.org/wiki/OpenStack_cascading_solution
[2] source code: https://github.com/openstack/tricircle
[3] wiki: https://wiki.openstack.org/wiki/Tricircle
[4] Live demo video at YouTube: https://www.youtube.com/watch?v=OSU6PYRz5qY

28. THANK YOU
www.huawei.com
Copyright©2014 Huawei Technologies Co., Ltd. All Rights Reserved.
The information in this document may contain predictive statements including, without limitation, statements regarding the
future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could
cause actual results and developments to differ materially from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.