Xen*, SDN andApache Cloudstack    Sebastien Goasguen,Apache CloudStack Citrix EMEA      August 28th 2012        Xen Summit
Outline•   A bit about CloudStack•   A bit about SDN•   A bit about OpenVswitch•   Some bits about “SDN” in CloudStack• Sl...
Apache CloudStack• IaaS solution to build a   • Java application  private/public cloud       • Ant build but moving to• Hy...
Participating in CloudStack•   Apache incubator project•   http://www.cloudstack.org•   #cloudstack on irc.freenode.net•  ...
A Very Flexible IaaS PlatformCompute      Hypervisor              XenServer       VMware         Oracle VM       KVM      ...
NaaS ?• “Cloud Servers”  – On-demand, Elastic, Measured server provisioning• Cloud Storage  – Scalable/fault tolerant stor...
A very extensive API           • CloudStack orchestrates             your network:              – Provisioning            ...
Software Defined Networking• Enable innovation, experimentation,  optimization and customization of networks• Move control...
OpenFlow                        • Leading SDN protocol                        • Decouples control and                     ...
OpenFlow• OpenFlow rules can drop, rewrite, forward packets           Rule           Action        Stats                  ...
OF Controllers        • Several controllers out          there (NOX,          POX,Trema…)        • Floodlight from Big    ...
OpenVSwitch      • “Open vSwitch is a        production quality,        multilayer virtual switch        licensed under th...
OpenVSwitch      • Default bridge in XenServer        and XCP      • Supported in Xen but not        integrated in toolsta...
e.g OVS rate limiting• Can enforce QoS with rate limiting controls•   ovs-vsctl set Interface tap0 ingress_policing_rate=1...
e.g OVS VLAN tagging•   ovs-vsctl add-br br0•   ovs-vsctl add-port br0 eth0•   ovs-vsctl add-port br0 tap0 tag=1•   ovs-vs...
e.g OVS and GRE tunnels• No Cookbook on OVS page•   ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre    options:...
OVS and Openflow• Point OVS switches to an OF controller:  $ovs_vsctl set-controller br0 tcp 192.168.1.33:6633• Install ru...
OpenNebula     • Supports VLAN tagging       and rate limiting       through “hooks” that       call ovs_vsctl     • Scrip...
CloudStack Nicira Support             • https://cwiki.apache.or               g/confluence/display/CL               OUDSTA...
API key to customization of the            network                • You dream it,                  CloudStack orchestrates...
Terminology         Zone: Availability zone,         aka Regions. Could be         worldwide. Different data         cente...
Physical Network       Operations                                               Users       Admin and       Cloud API     ...
Layer-2 Guest Virtual Network                             1 VLAN per guest network  CS Virtual Router provides Network Ser...
Opportunity for Xen• Opportunity to create highly specialized  networking services appliances using  – OpenMirage VMs  – H...
Networking challenges in a private             Cloud• Multi-tenants on hypervisors => isolation  between guest networks• V...
Networking trend• Move to software switches• Move to L3 isolation• Use tunnels between OVS (GRE tech preview)• Program the...
Back of the enveloppe• ~10,000 hypervisors in your data center• ~100,000 VMs  – x10 or x100 if you use HalVM or Openmirage...
Slide from Chiradeep Vittal        Layer 3 cloud networking           Web                                 DB              ...
L3 isolation with distributed firewalls                                                                  Tenant   10.1.0.2...
L3 isolation with distributed firewalls                                                                  Tenant   10.1.0.2...
L3 isolation with distributed firewalls                                                                  Tenant   10.1.0.2...
A Million Firewalls?VM      VM       VM…       …                   VM                 VM                 …          …VM   ...
Problem:Manage the state of 100s of thousands of firewallsSolution:Well-known software scaling techniques• Message queues•...
Problem:Firewall (iptables) rules explosion on the host firewallSolution:Use ipsets:   ipset –N web_sg iptreemap   ipset –...
Conclusions• Programmable networking is here• Software switches are key enabler to network  virtualization• Opens the door...
Participating in CloudStack•   Apache incubator project•   http://www.cloudstack.org•   #cloudstack on irc.freenode.net•  ...
Upcoming SlideShare
Loading in...5
×

Xen*, SDN, and Apache CloudStack

2,381

Published on

Apache CloudStack has already proven its abilities to scale to tens of thousands of physical XenServer nodes, but a number of traditional networking schemes don't work. VLANs max out at 4096, Layer3 isolation (aka Security Groups) helps scale to a degree, but you lose a lot of flexibility in the process. XCP and XS6.0 have the ability to make use of openvswitch to build software defined networks for increasing flexible networks with little or none of the legacy restrictions.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,381
On Slideshare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
106
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Xen*, SDN, and Apache CloudStack

  1. 1. Xen*, SDN andApache Cloudstack Sebastien Goasguen,Apache CloudStack Citrix EMEA August 28th 2012 Xen Summit
  2. 2. Outline• A bit about CloudStack• A bit about SDN• A bit about OpenVswitch• Some bits about “SDN” in CloudStack• Slides are on slideshare for download:http://www.slideshare.net/sebastiengoasguen/c loudstack-and-sdn
  3. 3. Apache CloudStack• IaaS solution to build a • Java application private/public cloud • Ant build but moving to• Hypervisor agnostic: maven (via new – Xen, XS, XCP contributor) – KVM – VMware• Object store support • First Apache release 4.0 – Swift coming Sept 26th – Upcoming support from Caringo• EC2/S3 compatibility
  4. 4. Participating in CloudStack• Apache incubator project• http://www.cloudstack.org• #cloudstack on irc.freenode.net• @CloudStack on Twitter• http://cloudstack.org/discuss/mailing-lists.html Welcoming contributions and feedback, Join the fun !
  5. 5. A Very Flexible IaaS PlatformCompute Hypervisor XenServer VMware Oracle VM KVM Bare metalStorage Block & Object Fiber Local Disk iSCSI NFS Swift Channel Primary Storage Secondary StorageNetwork Network & Network Services Network Load Isolation Firewall VPN Type balancer http://www.slideshare.net/cloudstack/cloudstack-architecture
  6. 6. NaaS ?• “Cloud Servers” – On-demand, Elastic, Measured server provisioning• Cloud Storage – Scalable/fault tolerant storage with object stores• Cloud Networks – How to do on-demand, elastic, measured networking provisioning ? – How to program the network ?
  7. 7. A very extensive API • CloudStack orchestrates your network: – Provisioning – Configuration – Updates • For multi-tenants isolation • Using hardware and software devices • At scale: O(10^4) Hyp, O(10^5) VMs…
  8. 8. Software Defined Networking• Enable innovation, experimentation, optimization and customization of networks• Move control of the network to software. i.e Programmable network• Virtualize the network• Vendor-agnostic, standard protocol for control: OpenFlow
  9. 9. OpenFlow • Leading SDN protocol • Decouples control and data plane by giving a controller the ability to install flow rules on switches. • Hardware or software• Google achieved 95% switches can use utilization of WAN OpenFlow backbone by using SDN • Spec driven by ONF
  10. 10. OpenFlow• OpenFlow rules can drop, rewrite, forward packets Rule Action Stats Packet + byte counters 1. Forward packet to port(s) 2. Encapsulate and forward to controller 3. Drop packet 4. Send to normal processing pipeline Switch MAC MAC Eth VLAN IP IP IP TCP TCP Port src dst type ID Src Dst Prot sport dport Diagram Src: http://www.openflow.org/wp/documents/10 September 2012 10
  11. 11. OF Controllers • Several controllers out there (NOX, POX,Trema…) • Floodlight from Big Switch. Apache license
  12. 12. OpenVSwitch • “Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license. It is designed to enable the massive network automation through programmatic extension…”
  13. 13. OpenVSwitch • Default bridge in XenServer and XCP • Supported in Xen but not integrated in toolstack • Enables: – VLAN tagging – Rate limiting – GRE tunnels – OpenFlow controller – … • High Performance ( http://networkheresy.com/category/o pen-vswitch )
  14. 14. e.g OVS rate limiting• Can enforce QoS with rate limiting controls• ovs-vsctl set Interface tap0 ingress_policing_rate=1000• ovs-vsctl set Interface tap0 ingress_policing_burst=100
  15. 15. e.g OVS VLAN tagging• ovs-vsctl add-br br0• ovs-vsctl add-port br0 eth0• ovs-vsctl add-port br0 tap0 tag=1• ovs-vsctl add-port br0 tap1 tag=2• Complement on host2…
  16. 16. e.g OVS and GRE tunnels• No Cookbook on OVS page• ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.152
  17. 17. OVS and Openflow• Point OVS switches to an OF controller: $ovs_vsctl set-controller br0 tcp 192.168.1.33:6633• Install rules on switch – Proactively (before any packet flows) – Reactively (unknown packets forwarded to controller, who pushes flow mod on switch, then operates at line rate)• Can do SDN with OpenFLOW but also with straight up OVS and managing mappings/rules in CloudStack db.
  18. 18. OpenNebula • Supports VLAN tagging and rate limiting through “hooks” that call ovs_vsctl • Scripts executed on an hypervisor before a VM is launched • Potentially also executed after VM shutdown for cleanup • Also supports OpenFlow
  19. 19. CloudStack Nicira Support • https://cwiki.apache.or g/confluence/display/CL OUDSTACK/Feature+Nic ira+NVP+integration • By Hugo Trippaers, Schuberg Philis
  20. 20. API key to customization of the network • You dream it, CloudStack orchestrates it 
  21. 21. Terminology Zone: Availability zone, aka Regions. Could be worldwide. Different data centers Pods: Racks or aisles in a data center Clusters: Group of machines with a common type of Hypervisor Host: A Single server Primary Storage: Shared storage across a cluster Secondary Storage: Shared storage in a single Zone
  22. 22. Physical Network Operations Users Admin and Cloud API CloudStack Mgmt Server Cluster Router MySQL Load Balancer Availability Zone L3 Core Switch Access LayerSwitches Secondary Servers … … … … … Storage Pod 1 Pod 2 Pod 3 Pod N Slide from Chiradeep Vittal, http://www.slideshare.net/cloudstack/cloudstack-networking
  23. 23. Layer-2 Guest Virtual Network 1 VLAN per guest network CS Virtual Router provides Network Services External Devices provide Network Services Network Hardware exposing API can be controlled Guest Virtual Network 10.1.1.1/8 Guest Virtual Network 10.1.1.1/8 VLAN 100 VLAN 100Public PublicNetwork/Intern Network/Internet Guest et Guest Public IP Private IP 10.1.1.1 10.1.1.1 VM 1 10.1.1.111 VM 1 Gateway 65.37.141.11 JuniperPublic IP 1 SRX address65.37.141.11 CS Firewall 10.1.1.1 Guest Guest Virtual 10.1.1.3 VM 2 10.1.1.3 VM 2 Router Public IP Private IP DHCP, DNS 65.37.141. NetScaler 10.1.1.112 NAT Guest 112 Load Guest Load Balancing 10.1.1.4 VM 3 Blancer VM 3 10.1.1.4 VPN Guest Guest 10.1.1.5 VM 4 10.1.1.5 VM 4 CS DHCP, Virtual Router DNS Slide from Chiradeep Vittal, http://www.slideshare.net/cloudstack/cloudstack-networking
  24. 24. Opportunity for Xen• Opportunity to create highly specialized networking services appliances using – OpenMirage VMs – HalVM• See talks in Monday’s session
  25. 25. Networking challenges in a private Cloud• Multi-tenants on hypervisors => isolation between guest networks• VLANs in the datacenter is hard and limit at 4096 VLANs.• Hardware switches may not do it very well or have a lower limit
  26. 26. Networking trend• Move to software switches• Move to L3 isolation• Use tunnels between OVS (GRE tech preview)• Program the network through API• Encapsulation virtualizes the network, between overlays on overlays on overlays..• L3 on L2 on GRE on L3 on L2…• Then you bring the WAN and you have:• L3 on L2 on GRE on L3 on L2 on GRE on L3 on L2 ….Euhhhh !!!
  27. 27. Back of the enveloppe• ~10,000 hypervisors in your data center• ~100,000 VMs – x10 or x100 if you use HalVM or Openmirage.org • (10,000*9,999)/2 tunnels for a full mesh – 50x10^6 tunnels to keep track of ?
  28. 28. Slide from Chiradeep Vittal Layer 3 cloud networking Web DB Web VM VM VM Web DB Security Security Group Group Web Web DB VM VM VM … … … Web Web VM VMIngress Rule: Allow VMs in Web Security Group access to VMs in DB Security Group on Port 3306
  29. 29. L3 isolation with distributed firewalls Tenant 10.1.0.2Public Public IP address 1 VM 1Internet 65.37.141.11 65.37.141.24 65.37.141.36 10.1.0.1 Pod 1 L2 Tenant 10.1.0.3 65.37.141.80 Switch 2 VM 1 Tenant 10.1.0.4 1 VM 2 L3 Core Pod 2 L2 Switch 10.1.8.1 … 10.1.16.1 Load Pod 3 L2 Balancer Switch … Slide from Chiradeep Vittal
  30. 30. L3 isolation with distributed firewalls Tenant 10.1.0.2Public Public IP address 1 VM 1Internet 65.37.141.11 65.37.141.24 65.37.141.36 10.1.0.1 Pod 1 L2 Tenant 10.1.0.3 65.37.141.80 Switch 2 VM 1 Tenant 10.1.0.4 1 VM 2 L3 Core Pod 2 L2 Switch 10.1.8.1 … 10.1.16.1 Load Pod 3 L2 Balancer Switch … Tenant 1 VM 3 10.1.16.47 Tenant 10.1.16.85 1 VM 4 Slide from Chiradeep Vittal
  31. 31. L3 isolation with distributed firewalls Tenant 10.1.0.2Public Public IP address 1 VM 1Internet 65.37.141.11 65.37.141.24 65.37.141.36 10.1.0.1 Pod 1 L2 Tenant 10.1.0.3 65.37.141.80 Switch 2 VM 1 Tenant 10.1.0.4 1 VM 2 L3 Core Pod 2 L2 Switch 10.1.8.1 … Tenant 10.1.16.12 10.1.16.1 2 VM 2 Load Pod 3 L2 Balancer Switch Tenant 2 VM 3 10.1.16.21 … Tenant 1 VM 3 10.1.16.47 Tenant 10.1.16.85 1 VM 4 Slide from Chiradeep Vittal
  32. 32. A Million Firewalls?VM VM VM… … VM VM … …VM VM … VM VMVM VM VM VM VM VMVM VM VM… … VM VM … …VM VM … VM VMVM VM VM VM VM VMVM VM VM… … VM VM … …VM VM … VM VMVM VM VM VM VM VMVM VM VM… … VM VM … …VM VM … VM VMVM VM VM VM VM VMVM VM VM… … VM VM … …VM VM … VM VMVM VM VM VM VM VMVM…VM VM … VM VM … VM VM … … VM … VM VMVM VM VM VM VMVM VM VM… … VM VM … …VM VM … VM VMVM VM VM VM VM VMVM VM VM… … VM VM … …VM VM … VM VMVM VM VM VM VM VM Slide from Chiradeep Vittal
  33. 33. Problem:Manage the state of 100s of thousands of firewallsSolution:Well-known software scaling techniques• Message queues• Consistency tradeoffs• Idempotent configuration & retriesCloudStack uses• special purpose queues• optimized for large security groups• eventual consistency for rule updates Slide from Chiradeep Vittal
  34. 34. Problem:Firewall (iptables) rules explosion on the host firewallSolution:Use ipsets: ipset –N web_sg iptreemap ipset –A web_sg 10.1.16.31 ipset –A web_sg 10.1.16.112 ipset –A web_sg 10.1.189.5 … ipset –A web_sg 10.21.9.77 -A FORWARD –p tcp –m tcp –dport 3060 –m set –match-set web_sg src -j ACCEPT Slide from Chiradeep Vittal
  35. 35. Conclusions• Programmable networking is here• Software switches are key enabler to network virtualization• Opens the door for scalable, on-demand, ephemeral networks• OVS is the default switch in Xen, and supported in XenServer and XenCP.• CloudStack implements highly scalable network structures and leverages OVS capabilities
  36. 36. Participating in CloudStack• Apache incubator project• http://www.cloudstack.org• #cloudstack on irc.freenode.net• @CloudStack on Twitter• http://cloudstack.org/discuss/mailing-lists.html Welcoming contributions and feedback, Join the fun !
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×