The Why, When and wHow
of CloudStack Networking
Paul Angus
VP Technology & Cloud Architect
paul.angus@shapeblue.com
Twitter: @CloudyAngus
@ShapeBlue
Who am I
VP Technology & Cloud Architect with
ShapeBlue
Worked with CloudStack since 2.2.13
Specialising in deployment of CloudStack
and surrounding infrastructure
USP, Georgian Ministry of Justice, Orange,
TomTom, PaddyPower, Ascenty, BSkyB
About Me
@ShapeBlue
@ShapeBlue
“ShapeBlue are expert builders of public &
private clouds. They are the leading global
independent CloudStack / CloudPlatform
integrator & consultancy”
About ShapeBlue
@ShapeBlue
The What, When and wHow
Physical networking
Storage networks
Guest networking
Basic networking
Advanced networking
CloudStack Networking
@ShapeBlue
CloudStack Networking
Physical Networking
@ShapeBlue
Why separate networks?
Security
Bandwidth improvement
Bandwidth contention
protection
Converged networking simplifies
cabling but contention still
needs to be controlled.
Physical Networking
@ShapeBlue
Physical networks are described (per-zone)
through two constructs:
‘Physical Networks’
‘Network labels’
Physical Networking
@ShapeBlue
Physical Networks
Confusingly named – may be better to call them Network
types or groups.
Physically independent network interfaces don’t have to be
different ‘physical networks’ unless:
They use different separation techniques VLAN vs VXLAN
You have multiple physical guest networks
Physical Networking
@ShapeBlue
Physical Networking
@ShapeBlue
A word or two on Blade Chassis
16 hosts sharing a 10Gb connection for storage and mgmt ?
Often chassis present virtual interfaces, using these enable
bandwidth control
Some chassis present virtual interfaces on a per-uplink
module basis. These still need to be bonded by the
hypervisor
Physical Networking
@ShapeBlue
Network Labels
Describe how CloudStack’s network types
map to the hypervisor naming of
interfaces/bonds.
Leaving as ‘default’ not advised
How the labels are used is subtlety
different between hypervisors
Physical Networking
@ShapeBlue
vSphere
Example Mappings
CloudStack
Label
Hypervisor
Interfaces
Hypervisor
interface
Mgmt NIC 1+NIC4 vSwitch0
Public NIC2+NIC5 vSwitch1
Guest NIC2+NIC5 vSwitch1
Storage NIC3+NIC6 vSwitch2
@ShapeBlue
KVM (Ubuntu)
Example Mappings
CloudStack
Label
Hypervisor
Interfaces
Hypervisor
interface
Mgmt em1+em3 cloudbr-mgmt
Public em2+em4 cloudbr-guest-pub
Guest em2+em4 cloudbr-guest-pub
@ShapeBlue
Example Mappings# This file describes the network interfaces
auto lo
iface lo inet loopback
auto em1
iface em1 inet manual
auto em2
iface em2 inet manual
auto em3
iface em3 inet manual
auto em4
iface em4 inet manual
auto cloudbr-mgmt
iface cloudbr-mgmt inet static
bridge_ports em1 em3
address 192.168.1.78
netmask 255.255.255.0
gateway 192.168.1.1
bridge_fd 5
bridge_stp off
bridge_maxwait 1
auto cloudbr-guest-public
iface cloudbr-guest-public inet manual
bridge_ports em2 em4
bridge_fd 5
bridge_stp off
bridge_maxwait 1
@ShapeBlue
XenServer
Example Mappings
CloudStack
Label
Hypervisor
Interfaces
Hypervisor
interface
Mgmt NIC 1+NIC4 Mgmt
Public NIC2+NIC5 Public-Guest
Guest NIC2+NIC5 Public-Guest
Storage NIC3+NIC6 Storage
@ShapeBlue
XenServer with Storage VLAN
Example Mappings
@ShapeBlue
XenServer with Storage VLAN
Example Mappings
@ShapeBlue
XenServer/KVM with (secondary) Storage VLAN
• When adding into CloudStack, Storage VLAN is UNTAGGED
so that it is not tagged twice.
• Can co-exist with ESXi, but must be in different pods so that
storage network port group can be tagged with VLAN.
Example Mappings
@ShapeBlue
CloudStack Networking
Storage Networking
@ShapeBlue
Storage networks
Mgmt & Secondary
Storage traffic
NIC0
Host
192.168.1.1
/24
Hypervisor
Primary
Storage traffic
Primary
Storage
192.168.99.2
/24
Management
Server
192.168.1.2
/24
Secondary
Storage
192.168.1.3
/24
NIC1
192.168.99.0/24
192.168.1.0/24
192.168.1.0/24
192.168.99.1
/24
Switch
@ShapeBlue
Storage networks
Mgmt traffic
NIC0
Host
192.168.1.1
/24
Hypervisor
Primary
Storage traffic
Secondary
Storage
192.168.10.3
/24
Primary
Storage
192.168.99.2
/24
Management
Server
192.168.1.2
/24
NIC1
192.168.99.0/24
192.168.1.0/24
192.168.1.0/24
192.168.99.1
/24
Switch
192.168.10.1
/24
Secondary
Storage traffic
NIC2
@ShapeBlue
CloudStack Networking
Guest Networking
@ShapeBlue
Why multiple physical guest networks?
Shared vs Isolated networks
Guest Networking
@ShapeBlue
Hypervisor
SSVM
VR
Public Traffic
VLAN 99 Mgmt traffic
Storage traffic
Mgmt traffic
Mgmt / Storage traffic
Public Traffic
VLAN 99
NIC0
NIC1
Host
Public Traffic
VLAN 99
Public TrafficVLAN 99
Guest Traffic VLAN 2001
Guest Traffic
VLAN 2001
Guest Traffic
VLAN 2001
cloudbr0 /
Xenbr0 /
vSwitch0 -
mgmt &
storage
traffic
Guest Traffic
VLAN 2002
Guest Traffic VLAN 2002
cloudbr1 /
xenbr1 /
vSwitch1 -
guest &
public traffic
Guest
Guest
Switch
Trunked
(VLAN)
Port
Access
Port
@ShapeBlue
Multiple Physical Guest Networks
Guest iSCSI
Secure backend services
A number of use cases have been replaced by VPC private
gateway
Guest Networking
@ShapeBlue
Isolated networks give... er,
isolation.
Additional network services:
load-balancing
Auto-scaling
Firewalling
Port-forwarding
Multi-tiered networks
Private gateways
VPN
Isolated vs Shared
@ShapeBlue
Isolated networks are NATed and therefore (direct) inbound
routing is not possible.
This makes PaaS problematic
Isolated network VR can be a bottleneck and or perceived as
a weak link.
Isolated vs Shared
@ShapeBlue
Shared networks can run at physical wire speeds.
VMs in shared networks can easily be routed to.
Built-in CloudStack integrated network services not available
Isolated vs Shared
@ShapeBlue
OSPF and Routed VPC
Coming Soon…
10.1.1.0/24
.1
Other
Networks
VR1-VPC
Tier 1 Tier X
Virtual instances
Tier 1 Tier X
.1 .1 .1 .1
BGP
Backbone
.2
OSPF Area 0
Other
Networks
Super CIDR
Ex: 10.10.10.0/
23
Subnet 10.10.10.0/24 Subnet 10.10.11.0/24
VPC VR advertise routes
(redistribute connected and
static) via OSPF and receiving
routes from another's VPC
VRs and default route from
Border Routers
Super CIDR
Ex: 10.20.20.0/
23
Subnet 10.20.20.0/24 Subnet 10.20.21.0/24
Virtual instances Virtual instances Virtual instances
.1
0
.1
1
VR2-VPC
CORE-
ROUTER1
CORE-
ROUTER2
Other
Networks
@ShapeBlue
Questions
?
@ShapeBlue
Slides: www.slideshare.net/shapeblue
Blogs: http://shapeblue.com/blog/
Email: paul.angus@shapeblue.com
Twitter: @CloudyAngus
Web: http://shapeblue.com
http://cloudstack.apache.org/
Resources
The Why, When and wHow
of CloudStack Networking
Paul Angus
VP Technology & Cloud Architect
paul.angus@shapeblue.com
Twitter: @CloudyAngus

CloudStack networking

Editor's Notes

  • #7 What kind of uses are we talking about
  • #21 That’s the generalities Now for the specifics
  • #24 That’s the generalities Now for the specifics
  • #25 Facts from http://www.tradermedia.co.uk/media-centre/key-facts.aspx and annual report 2012 CloudStack environment is still in development Note MASSIVE scale