Networking in the Cloud Age (LISA 2012 Tutorial)

  • 4,792 views
Uploaded on

I (along with David Nalley - ke4qqq) presented a tutorial on Cloud Networking for cloud operators (not users) at LISA 2012

I (along with David Nalley - ke4qqq) presented a tutorial on Cloud Networking for cloud operators (not users) at LISA 2012

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,792
On Slideshare
0
From Embeds
0
Number of Embeds
8

Actions

Shares
Downloads
0
Comments
0
Likes
14

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Networking in the Cloud Age! With references to Apache CloudStack! ! December 11 2012! ! Chiradeep Vittal! @chiradeep! David Nalley! @ke4qqq!
  • 2. Agenda!•  Why virtual networks?!•  Basic principles of Cloud Networking!•  Service insertion in virtual networks!•  Virtual Networking using L3 isolation!•  Networking in Apache CloudStack!•  Software Defined Networking!•  Wrap-up!
  • 3. Apache CloudStack! •  Secure, multi-tenant cloud orchestration platform! –  Turnkey platform for delivering IaaS clouds! –  Over 150 commercial Build your cloud the way deployments: private and public!the world’s most successful –  Full featured GUI, end-user API clouds are built! and admin API!
  • 4. Apache CloudStack! •  Open Source! •  Apache License! •  Incubating in the Apache Software Foundation since April 2012! Build your cloud the waythe world’s most successful •  Open Source since May clouds are built! 2010! •  In production since 2009!
  • 5. Networking in the Cloud Age!DRIVERS!
  • 6. Drivers! New-style!IAAS! Workload! Agility! Application owns availability! Virtualization! API! High bandwidth! Self-service! Elasticity! Scale! Low cost! Distributed! L3! Cookie cutter! Multi-tenancy!These classes of drivers (IAAS and new-style workloads) are highlycomplementary and therefore most new-style applications operate on IAAS!
  • 7. Traditional Style! Traditional!IAAS! Workload! Agility! Infra owns availability! Virtualization! API! Complex Packet Filters! Elasticity! Scale! High cost! Self-service! Gold-plated! Multi-tenancy! Infra! L2!It is possible to realize some of the benefits of IAAS for traditional workloads !
  • 8. Traditional infra can be IAAS! IAAS! Agility! Gold-plated! Virtualization! Infra! API! Infra owns availability! Elasticity! Scale! High cost! Self-service! Multi-tenancy! L2! Complex Packet Filters!It is possible to realize some of the benefits of IAAS for traditional infrastructure!
  • 9. Traditional! Cloud! •  10x more scaleable! •  2-5x lower cost! •  100% more open!Built for traditional Designed around big data,enterprise apps & client- massive scale & next-genserver compute! apps!•  Enterprise arch for 100s of •  Cloud architecture for 1000s hosts! of hosts!•  Scale-up (server clusters) ! •  Scale-out (multi-site server•  Apps assume reliability! farms)!•  IT Mgmt-centric [1:Dozens]! •  Apps assume failure!•  Proprietary vendor stack! •  Autonomic [1:1,000’s]! •  Open, value-added stack!
  • 10. Defining Cloud Computing (IAAS)!•  Agility! –  Re-provision complex infrastructure topologies in minutes, not days•  API! –  Automate complex infrastructure tasks•  Virtualization! –  Enables workload mobility and load sharing•  Multi-tenancy! –  Share resources and costs
  • 11. Defining Cloud Computing (IAAS)!•  Scalability! –  Ability to consume resources limited by budget, not by infrastructure•  Elasticity! –  Scale up and down on demand –  Reduce need to engineer for peak load•  Self-service! –  No IT assistance!
  • 12. Cloud Networking Requirements!•  Agile! –  Complex networking topologies created by non- network engineers•  API! –  Language to talk with the network infrastructure layer (not CLI)•  Virtualization! –  Hypervisor-level switches work together with physical infrastructure
  • 13. Cloud Networking Requirements!•  Scalability! –  Usually means L3 in the physical infrastructure•  Elasticity! –  Release resources when not in use –  Introduce new resources on demand•  Self-service! –  Novices deploying, maintaining, troubleshooting virtual networks
  • 14. Cloud-Style Workloads!•  Low cost! –  Standardized, cookie cutter infrastructure –  Highly automated and efficient•  L3! –  Applications do not need persistent ip/mac –  L2 adjacency not required•  Application owns availability! –  At scale everything breaks –  Focus on MTTR instead of MTBF
  • 15. Scale!“At scale, everything breaks”! -­‐  Urs  Hölzle,  Google! " " "! Server failure comes from:! ᵒ  70% - hard disk! 8%   ᵒ  6% - RAID controller! ᵒ  5% - memory! ᵒ  18% - other factors! Application can still fail for Annual  Failure  Rate  of  servers   other reasons:! ᵒ  Network failure!Kashi  Venkatesh  Vishwanath  and  Nachiappan  Nagappan,  Characterizing   ᵒ  Software bugs!Cloud  Compu3ng  Hardware  Reliability,   ᵒ  Human admin error!SoCC’10  
  • 16. Redundancy helps a little! • Bugs in failover 40%! mechanism! • Incorrect configuration! • Protocol issues such as TCP back-off, timeouts, and Effectiveness of network redundancy in reducing spanning tree failures! reconfiguration! Phillipa Gill, Navendu Jain & Nachiappan Nagappan, Understanding Network Failures in Data Centers: Measurement, Analysis and Implications, SIGCOMM 2011 !16!
  • 17. Reliability Strategies! Cloud workloads! Traditional-Style! New (“Amazon”) Style! Reliable hardware, backup Tell users to expect failure. entire cloud, and restore for Users to build apps that can users when failure happens! withstand infrastructure failure!Both styles of workloads must run reliably in the cloud!
  • 18. Reliability Styles! Traditional workload! Cloud workload! Link aggregation! VM backup/snapshots ! Storage multi-pathing! Ephemeral resources! VM HA, fault tolerance! Chaos monkey! VM live migration! Multi-site redundancy!Expect reliability. Back-up entire Expect failure. Design app for failure. cloud. Admin controlled failure Self-service failure handling! handling! Think Amazon Web Services! Think Server Virtualization!
  • 19. Traditional Enterprise network! Backbone/ Internet! Core Routers!N-S traffic! …! Access Routers! Packet Filters! Aggregation Switches! Load Balancers! …! Top of Rack Switches! Servers!
  • 20. Enterprise networks!•  Hierarchical tree structure! –  Assumes N-S traffic predominant•  L2 domains! –  Susceptible to flooding –  Wasted capacity due to STP•  Services provided by redundant HW appliances! –  Firewall, IDS, ACL, Loadbalancer –  Often need L2 adjacency!•  Complex engineering, limited scale!
  • 21. Scaled out network! Backbone/ Internet! Spine Routers! Leaf Routers! …! Servers!Host-based! Server Load Balancing!firewalls and ACL!
  • 22. Scaled out network!•  L3 (routed) network! –  ECMP for increased bandwidth/redundancy•  No oversubscription! –  Uniform access to bandwidth•  Predominantly east-west traffic!•  Commodity hardware!•  Services provided at the host / vm level! –  Firewall, IDS, load balancing.
  • 23. Networking in the Cloud Age!VIRTUAL NETWORKINGPRINCIPLES!
  • 24. The illusion of isolated networks on top ofshared physical infrastructure!
  • 25. Usually requires!•  Hypervisors! –  To share the same host with multiple tenants•  Virtual (software) switches! –  Port-level control to provide isolation•  Services provided in software / virtual contexts! –  Loadbalancer / firewall virtual appliances –  Host-based firewalls
  • 26. Virtual-to-Physical Mapping!•  Option 1: VLAN! –  1 virtual network = 1 VLAN in physical infra !•  Option 2: Tunnels! –  VxLAN! –  (NV) GRE! –  STT! –  Others: MAC-in-MAC, NVO3, MPLS!
  • 27. Virtual-to-Physical Mapping!•  Option 3: IP address re-write! –  1 tenant address mapped to 1 different provider address –  Hyper-V only (possible with KVM/Xen)•  Option 4: No mapping ! –  Tenant address is present on physical network –  Tenants isolated from each other and physical network using packet filters in hypervisor –  L3 isolation is CloudStack’s term for this mode –  Also called “Basic Networking”.
  • 28. Virtual Switches!•  Linux bridge! –  KVM, XenServer, XCP, Oracle VM•  Open vSwitch (OVS)! –  KVM, XenServer, XCP•  VMware options! –  vSphere –  Distributed vSwitch (DVS) –  Cisco Nexus 1000v
  • 29. Virtual Switches! Hypervisor Host! VM A1! VM A2! VM B1! VM C1! untagged (usually)!Virtual Nics! vswitch! vswitch! vswitch! Physical ! Nics! 192.168.1.0/24! VLAN TRUNK! VLAN 10! 192.168.1.0/24! VLAN 20! 10.1.1.0/24! VLAN 30!
  • 30. Egress Traffic from VM! Ethernet frame from VM A1 to vswitch (untagged) Payload (IP Packet) 06:00:01:AA:BB:CC 06:02:12:1D:1E 0x800 46-1500 octets Dest, addr Src, addr Type Ethernet frame from vswitch to physical nic( tagged) Payload (IP Packet) 06:00:01:AA:BB:CC 06:02:12:1D:1E 0x8100 0xA 0x800 46-1500 octets Dest, addr Src, addr 802.1Q Tag Type*not all fields shown for clarity!
  • 31. Ingress Traffic to VM! From physical nic to vswitch( tagged) Payload (IP Packet)06:02:12:1D:1E:1F 06:00:01:AA:BB:CC0x8100 0xA 0x800 46-1500 octets From vswitch to VM A1 (untagged) Payload (IP Packet)06:02:12:1D:1E:1F 06:00:01:AA:BB:CC 0x800 46-1500 octets
  • 32. VLAN networking!Trunks! Trunks! Trunks! User   A   User   A   User   A   User   User   B   A   User   B   …  
  • 33. 12 bits tag =4094 virtual networks
  • 34. VLANs – other problems!•  Configuration complexity! –  Need to program switches carefully•  Large L2 domains! –  Broadcast in one VLAN can cause unintended load on unrelated hypervisors•  Live migration limited to a single VLAN!•  Limited mac table sizes in L2 switches! –  100s of vms per hypervisor –  1000s of mac addresses on uplink port
  • 35. Tunnels!•  Map VM address (Tenant Address) to Physical address (PA) of Hypervisor! –  Software IPv4 tunnels between hypervisors –  Tunnel endpoints are PA of hypervisor –  Discriminator in tunnel header identifies tenant/ network •  GRE key in (NV) GRE tunnels (24-32 bits) •  VxLAN Network Identifier (VNI) in VxLAN (24 bits) •  Context ID in STT (64 bits)
  • 36. GRE tunnel example! GRE  Key  1   GRE  Key  2   OVS   User   1  OVS   User   1   User   1   OVS   User  OVS   User   OVS   2   1   User   2  …   …   …  
  • 37. GRE Example!Hypervisor 1! Hypervisor 2! VM   VM   VM   VM   A1   B1   A2   B2   192.168.10.55! 192.168.20.88! 192.168.10.5! 192.168.20.8! vswitch   vswitch   10.10.10.5! 10.20.20.9! 10.20.20.9! 10.10.10.5! GRE key=10! MAC A2! MAC A1! 192.168.10.5! 192.168.10.55!A1->A2!B1->B2! 10.20.20.9! 10.10.10.5! GRE key=20! MAC B2! MAC B1! 192.168.20.8! 192.168.20.88! Physical Address! Tenant L2 header! Tenant L3 header! Wire format!
  • 38. Layer 3 cloud networking! Web   DB   Web   VM   VM   VM   Web   DB     Security   Security   Group   Group   Web   Web   DB   VM   VM   VM   …   …   …   Web   Web   VM   VM  
  • 39. L3 isolation with distributed firewalls! ! Tenant 10.1.0.2Public Public IP 1 VM 1Internet address 65.37.141.11! 65.37.141.24! 10.1.0.1 ! Pod 1 Tenant 10.1.0.3 65.37.141.36! ! Leaf 2 VM 1 65.37.141.80! Switch ! ! Tenant 10.1.0.4 1 VM 2 L3 Core ! Pod 2 10.1.8.1 …! ! Leaf Switch 10.1.16.1 ! Load Pod 3 ! Balancer Leaf Switch … !
  • 40. Networking in the Cloud Age!SERVICES!
  • 41. Virtual Network Services!•  Provide L2-L7 network services that applications expect:! –  Load balancing, firewall, IDS, VPN, NAT, etc.•  Services are inserted in the virtual network topology! –  usually in the path to the public network•  Services are on-demand (api-driven), scalable, elastic!
  • 42. Virtual Network Appliances!Network services are often provided by virtual appliances.!These are either commercial appliances in the virtual form factor orLinux-based networking appliances! Virtual Router! Public Network Nic! Virtual Network Nic! Control Network Nic!
  • 43. Service insertion example! Tenant 1 Virtual Network 10.1.1.0/24 Public Public IP ! Tenant 10.1.1.2 Network address Gateway 1 VM 1 65.37.141.11! address 10.1.1.1 65.37.141.36 ! Tenant 1 ! Tenant 10.1.1.3 Edge 1 VM 2 Services ! Appliance(s) NAT!Internet! ! Tenant 10.1.1.4 DHCP! 1 VM 3 FW ! Tenant 10.1.1.5 1 VM 4
  • 44. Service insertion with VLAN !Trunks! Trunks! Trunks! Tena nt  1   Tena nt  1   Tena nt  1   Tena Tena nt  2   Rout nt  1   er   VM  1   …   Public VLAN! Public VLAN! Public VLAN! Rout er   VM  2  
  • 45. Network Services! Network Services!•  L2 connectivity!•  IPAM!•  DNS!•  Routing!•  ACL!•  Firewall!•  NAT!•  VPN!•  LB!•  IDS!•  IPS!!
  • 46. Network Services! Network Service Services! Providers!•  L2 ü  Virtual connectivity! appliances!•  IPAM! ü  Hardware•  DNS! firewalls!•  Routing! ü  LB•  ACL! appliances!•  Firewall! ü  SDN•  NAT! controllers!•  VPN! ü  IDS /IPS•  LB! appliances!•  IDS! ü  VRF!•  IPS! ü  Hypervisor!!
  • 47. Network Services! Network Service Network Services! Providers! Isolation!•  L2 ü  Virtual •  No connectivity! appliances!•  IPAM! isolation! ü  Hardware•  DNS! firewalls! •  VLAN•  Routing! ü  LB isolation!•  ACL! appliances!•  Firewall! ü  SDN •  Overlays!•  NAT! controllers! •  L3•  VPN! ü  IDS /IPS isolation!•  LB! appliances!•  IDS! ü  VRF!•  IPS! ü  Hypervisor!!
  • 48. Service Catalog!•  Cloud users are not exposed to the nature of the service provider!•  Cloud operator designs a service catalog and offers them to end users.! –  Gold = {LB + FW, using virtual appliances} –  Platinum = {LB + FW + VPN, using hardware appliances} –  Silver = {FW using virtual appliances, 10Mbps}
  • 49. End-user experience!•  Deploy a VM in a network! –  VM Template = Windows 2008 with Joomla on VMWare! –  Service offering {m1.large} = 2 x CPU x 2.0Ghz, 8 GB RAM! –  Disk Offering {Super fast}! –  Network Offering {Gold} = Source NAT + LB+ FW + 20 Mbps Internet access!
  • 50. End-user experience!•  Deploy a VM in a network! –  VM Template = Windows 2008 with Joomla on VMWare –  Service offering {m1.large} = 2 x CPU x 2.0Ghz, 8 GB RAM –  Disk Offering {Super fast} –  Network Offering {Gold} = Source NAT + LB+ FW + 20 Mbps Internet access•  Network Offering Gold is realized by! –  VLAN isolation –  Source NAT & FW on Juniper SRX –  LB on F5 BigIp –  DHCP, DNS on virtual appliance
  • 51. End-user experience!•  CloudStack orchestration:! –  Pick a free VLAN, pick a free public IP, free private IP –  Pick hypervisor with spare capacity –  Pick primary storage of SSD type accessible in hypervisor cluster –  Pick a Juniper SRX and F5 with spare capacity –  Spin up a new virtual appliance if necessary that runs DHCP and DNS service •  Pick hypervisor, call hypervisor APIs to provision virtual appliance on selected VLAN –  Call hypervisor APIs to provision VM on selected VLAN –  Call SRX and F5 APIs to place their internal interfaces on the VLAN, public interfaces on public VLAN –  Call SRX API to provision source NAT, default FW rules
  • 52. Network services with VLANs! Tenant 1 Virtual Network 10.1.1.0/24 ! Tenant 10.1.1.2 Gateway 1 VM 1 address 10.1.1.1 ! Tenant 10.1.1.3 1 VM 2Internet! ! Tenant 10.1.1.4 1 VM 3 ! Tenant 10.1.1.5 1 VM 4
  • 53. Network virtualization with VLANs! Tenant 1 Virtual Network 10.1.1.0/24 Public Public IP ! Tenant 10.1.1.2 Network address Gateway 1 VM 1 65.37.141.11! address 10.1.1.1 65.37.141.36 ! Tenant 1 ! Tenant 10.1.1.3 Edge 1 VM 2 ServicesInterne ! Appliance(s) NAT! ! Tenant t! DHCP! 1 VM 3 10.1.1.4 FW ! Tenant 10.1.1.5 1 VM 4
  • 54. Network virtualization with VLANs! Tenant 1 Virtual Network 10.1.1.0/24 Public Public IP ! Tenant 10.1.1.2 Network address Gateway 1 VM 1 65.37.141.11! address 10.1.1.1 65.37.141.36 ! Tenant 1 ! Tenant 10.1.1.3 Edge 1 ! Tenant 1 VM 2 Edge Services Services Appliance(s) NAT! ! ! Appliance(s)Internet! ! Tenant 10.1.1.4 DHCP! 1 VM 3 FW Load Balancing! ! VPN Tenant 10.1.1.5 1 VM 4
  • 55. Service insertion with VLANs! Tenant 1 Virtual Network 10.1.1.0/24 Public Public IP ! Tenant 10.1.1.2 Network address Gateway 1 VM 1 65.37.141.11! address 10.1.1.1 65.37.141.36 ! Tenant 1 ! Tenant 10.1.1.3 Edge 1 ! Tenant 1 VM 2 Edge Services Services Appliance(s) NAT! ! !Internet! Appliance(s) ! Tenant 10.1.1.4 DHCP! 1 VM 3 FW Load Balancing! ! Tenant 10.1.1.5 1 VM 4 Tenant 2 Virtual Network 10.1.1.0/24 Public IP address 65.37.141.24! Gateway address Tenant 2 VM 1 ! 10.1.1.2 65.37.141.80 10.1.1.1 ! Tenant 2 ! Tenant 10.1.1.3 Edge 2 VM 2 ! Services Appliance VPN! NAT! DHCP Tenant 2 VM 3 ! 10.1.1.4
  • 56. Scaling services with VLANs! Scale out edge services using virtual appliances! 10.1.1.0/24! VLAN 100 VM 1! 10.1.1. 265.37.141.1 10.1.1.111! CS!65.37.141.1 Virtual VM 2!12 Router! 10.1.1. 3 DHCP, DNS! NAT! Load 10.1.1.4 VM 3! Balancing! VPN VM 4! 10.1.1.5
  • 57. Scaling services with VLANs! Scale out edge services using virtual appliances! Scale up using hardware devices! 10.1.1.0/24! 10.1.1.0/24! VLAN 100 VLAN 100 VM 1! 65.37.141.11 10.1.1.1 10.1.1.2 VM 1! 10.1.1. 1 Juniper 2 SRX!65.37.141.1 10.1.1.111! CS! Firewall! NAT,65.37.141.1 Virtual VM 2! VPN! VM 2! 10.1.1. 10.1.1.312 Router! 3 65.37.141.11 10.1.1.112 DHCP, DNS! 2 Netscaler! NAT! Load Load 10.1.1.4 VM 3! VM 3! Balancer! 10.1.1.4 Balancing! VPN VM 4! VM 4! 10.1.1.5 10.1.1. 5 CS! DHCP, Virtual Router! DNS!
  • 58. Multi-tier virtual networking! Internet! ! Loadbalancer Virtual appliance/! Hardware Devices! (virtual or HW)!Network Services!•  IPAM!•  DNS! Web VM 1!•  LB [intra]!•  S-2-S VPN!•  Static Routes! Web VM•  ACLs! 2!•  NAT, PF!•  FW [ingress & egress]! Web VM 3! Web VM 4! Web subnet ! 10.1.1.0/24! VLAN 101
  • 59. Multi-tier virtual networking! Internet! ! Loadbalancer Virtual appliance/! Hardware Devices! (virtual or HW)!Network Services! App VM•  IPAM! 1!•  DNS! Web VM 1!•  LB [intra]!•  S-2-S VPN! App VM•  Static Routes! Web VM 2! VLAN 2724•  ACLs! 2!•  NAT, PF!•  FW [ingress & egress]! VLAN 353 Web VM DB VM•  BGP! 3! 1! Web VM 4! Web subnet ! App subnet DB Subnet! 10.1.1.0/24! VLAN 101 10.1.2.0/24! 10.1.3.0/24!
  • 60. Multi-tier virtual networking! Internet! IPSec or SSL site-to-site VPN! ! Custome Loadbalancer Virtual appliance/! r! Hardware Devices! (virtual or HW)! Premises! MPLS VLAN!Network Services! App VM•  IPAM! 1!•  DNS! Web VM 1!•  LB [intra]!•  S-2-S VPN! App VM•  Static Routes! Web VM 2! VLAN 2724•  ACLs! 2!•  NAT, PF!•  FW [ingress & egress]! VLAN 353 Web VM DB VM•  BGP! 3! 1! Web VM 4! Web subnet ! App subnet DB Subnet! 10.1.1.0/24! VLAN 101 10.1.2.0/24! 10.1.3.0/24!
  • 61. Multi-tier networking with Overlay! Internet! IPSec or SSL site-to-site VPN! Loadbalancer ! Custome (virtual Virtual Router! r! Premises! appliance)! MPLS VLAN!Network Services! App VM•  IPAM! Web VM 1!•  DNS! 1!•  LB [intra]! App VM•  S-2-S VPN! 2!•  Static Routes! Web VM GRE Key 2724 2!•  ACLs!•  NAT, PF!•  FW [ingress & egress]! Web VM GRE Key 353 DB VM•  BGP! 3! 1! Web VM 4! Web subnet ! App subnet DB Subnet! 10.1.1.0/24! GRE Key 101 10.1.2.0/24! 10.1.3.0/24!
  • 62. Multi-tier networking with Overlay! Internet!Loadbalancer vswitches! (virtual appliance)! App VM 1! Web VM 1! App VM Web VM 2! GRE Key 2724 2! Web VM GRE Key 353 DB VM 3! 1! Web VM 4! Web subnet ! App subnet DB Subnet! 10.1.1.0/24! GRE Key 101 10.1.2.0/24! 10.1.3.0/24!
  • 63. Networking in the Cloud Age!LAYER 3 ISOLATION!
  • 64. Layer 3 cloud networking! Web DB Web VM! VM! VM! Web! DB ! Security Security Group! Group! Web Web DB VM! VM! VM! …! …! …! Web Web VM! VM!Ingress Rule: Allow VMs in Web Security Group access to VMs in DB Security Group on Port 33
  • 65. L3 isolation with distributed firewalls! ! Tenant 10.1.0.2Public Public IP 1 VM 1Internet address 65.37.141.11! 65.37.141.24! 10.1.0.1 ! Pod 1 Tenant 10.1.0.3 65.37.141.36! ! Leaf 2 VM 1 65.37.141.80! Switch ! ! Tenant 10.1.0.4 1 VM 2 L3 Core ! Pod 2 10.1.8.1 …! ! Leaf Switch 10.1.16.1 ! Load Pod 3 ! Balancer Leaf Switch … !
  • 66. L3 isolation with distributed firewalls! ! Tenant 10.1.0.2Public Public IP 1 VM 1Internet address 65.37.141.11! 65.37.141.24! 10.1.0.1 ! Pod 1 Tenant 10.1.0.3 65.37.141.36! ! Leaf 2 VM 1 65.37.141.80! Switch ! ! Tenant 10.1.0.4 1 VM 2 L3 Core ! Pod 2 10.1.8.1 …! ! Leaf Switch 10.1.16.1 ! Load Pod 3 ! Balancer Leaf Switch … ! Tenant 1 VM 3 ! 10.1.16.47 ! Tenant 10.1.16.85 1 VM 4
  • 67. L3 isolation with distributed firewalls! ! Tenant 10.1.0.2Public Public IP 1 VM 1Internet address 65.37.141.11! 65.37.141.24! 10.1.0.1 ! Pod 1 Tenant 10.1.0.3 65.37.141.36! ! Leaf 2 VM 1 65.37.141.80! Switch ! ! Tenant 10.1.0.4 1 VM 2 L3 Core ! Pod 2 10.1.8.1 …! ! Leaf Switch ! Tenant 10.1.16.12 10.1.16.1 2 VM 2 ! Load Pod 3 ! Balancer Leaf ! Switch Tenant 2 VM 3 10.1.16.21 … ! Tenant 1 VM 3 ! 10.1.16.47 ! Tenant 10.1.16.85 1 VM 4
  • 68. 1 Firewall per Virtual Machine
  • 69. A Million Firewalls?!VM! VM! VM!…! …! VM! VM! …! …!VM! VM! …! VM! VM!VM! VM! VM! VM! VM! VM!VM! VM! VM!…! …! VM! VM! …! …!VM! VM! …! VM! VM!VM! VM! VM! VM! VM! VM!VM! VM! VM!…! …! VM! VM! …! …!VM! VM! …! VM! VM!VM! VM! VM! VM! VM! VM!VM! VM! VM!…! …! VM! VM! …! …!VM! VM! …! VM! VM!VM! VM! VM! VM! VM! VM!VM! VM! VM!…! …! VM! VM! …! …!VM! VM! …! VM! VM!VM! VM! VM! VM! VM! VM!VM!…!VM! VM! …! VM! VM! …! VM! VM! …! …! VM! …! VM! VM!VM! VM! VM! VM! VM!VM! VM! VM!…! …! VM! VM! …! …!VM! VM! …! VM! VM!VM! VM! VM! VM! VM! VM!VM! VM! VM!…! …! VM! VM! …! …!VM! VM! …! VM! VM!VM! VM! VM! VM! VM! VM!
  • 70. Networking in the Cloud Age!SOFTWARE DEFINEDNETWORKING!
  • 71. Definition!•  Separation of Control Plane from the hardware performing the forwarding function!•  Control plane is logically centralized!
  • 72. SDN Advantages!•  Centralized control makes it easier to configure, troubleshoot and maintain•  Eliminates ‘box’ mode of configuration•  Enables control at a high level
  • 73. Related to SDN!•  API layer over a collection of ‘boxes’! –  API layer communicates with boxes using box-level APIs / ssh / telnet•  OpenFlow! –  Standard protocol for the centralized control plane to talk to the forwarding elements.•  Tunnels / overlays! –  SDN is valuable for virtual topologies –  Initial target of SDN implementation
  • 74. Centralized control plane!Admin/User  API   Controller  Cluster   MySQL/NoSQL   Openflow/ssh/netconf/other! Boxes!
  • 75. SDN problems!•  Discovery of virtual address -> physical address mapping! –  VxLAN = multicast –  GRE = programmed by control plane –  L3 isolation = no mapping, no discovery
  • 76. SDN problems!•  State maintenance! –  Large number of endpoints + flows –  High arrival rate of new flows –  Needs fast and scalable storage and processing
  • 77. CloudStack and SDN! Hypervisor   Hypervisor   Resource   5 4 Resource   Hyperviso Hyperviso r  Plugins   r  Plugins   Plugin   Framew 6 ork   Network   API   7 SDN   Resource   Network     API   Network   controller   OrchestraSon  Engine   Plugins  1   API   Plugins     2 8 Allocator   9 3 Storage   Plugins   Plugins   Storage   Storage   Resource   Resource   Allocator   Allocator   Plugins   Plugins   Physical Resources ! Network plugin is the glue that understands the SDN controller’s API!
  • 78. Virtual Networking "!NETWORKING IN APACHECLOUDSTACK!
  • 79. Regions and Zones!•  A cluster of CloudStack management servers manage the physical resources of a region –  Single API endpoint per region•  Each region consists of zones•  Zones are physically proximate, but provide distinct failure domains (e.g., flood, earthquake, power)•  Zones are interconnected with high speed low latency links
  • 80. Region “West” Region “East” Geographic separation InternetLow Latency Region “South”
  • 81. Region “West” Zone “West-Beta” Zone “West-Alpha” High Speed Backbone (e.g., SONET ring) Zone “West-Delta” Zone “West-Gamma”
  • 82. Inside a zone! Admin/User  API   End  users   CloudStack  Cluster   DC  Edge   MySQL   L2/L3  core   Leaf  Sw  Hypervisor  (Xen  /VMWare/KVM)   Secondary  Storage  Primary  Storage  NFS/ISCSI/FC   Pod   Pod   Pod   Pod   Pod  
  • 83. Orchestration!•  Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware and services –  Wikipedia!
  • 84. CloudStack Architecture! Hypervisor   Hypervisor   Plugins   Plugins   Plugin   Framework   Network  Plugins   OrchestraSon  Engine   Network  Plugins   Allocator   Allocator   Plugins   Plugins   Storage  Plugins  
  • 85. CloudStack Architecture! •  XenServer   • VMWare   • KVM   • OracleVM   Hypervisor     Hypervisor   Plugins   Plugins   Plugin   Framework   Nicira   •  • Netscaler   • Brocade   Network  Plugins   OrchestraSon  Engine   Network  Plugins   idoNet   • M   Allocator   •  Random   Allocator   • User-­‐ Plugins   Plugins   concentrated   • Intel  TXT   • Affinity    
  • 86. CloudStack Architecture! Hypervisor   Hypervisor   Resource   5 4 Resource   Hyperviso Hyperviso r  Plugins   r  Plugins   Plugin   Framew 6 ork   Network   API   7 Network   Resource   Network     API   Network   Resource   OrchestraSon  Engine   Plugins  1   API   Plugins     2 8 Allocator   9 3 Storage   Plugins   Plugins   Storage   Storage   Resource   Resource   Allocator   Allocator   Plugins   Plugins   Physical Resources ! Orchestration steps can be executed in parallel or in sequence!
  • 87. Problem:Manage Configuration of! 1000s of virtual appliances (or VRF) Dozens of HW appliancesSolution:Database-driven state management ofappliances! Message queues + Retry Logic Idempotent updates, Recreatable virtual appliances
  • 88. Problem:Manage Configuration of! 1000s of virtual appliances (or VRF) Dozens of HW appliancesSolution:Database-driven state management ofappliances! Message queues + Retry Logic! Idempotent updates,! Recreatable virtual appliances! !
  • 89. Problem:!Single-tenant HW appliances!Solution:!CloudStack API layers multi-tenancy, providesabstraction! No direct access to devices!
  • 90. Problem:Hardware appliances with no APIs CLI only Limited concurrent login sessionsSolution: Recommend appliances with APIs Integrate with Network Orchestrators !
  • 91. Problem:Manage the configuration of 100s of thousands of firewallsSolution:Well-known software scaling techniques•  Message queues•  Consistency tradeoffs•  Idempotent configuration & retriesCloudStack uses •  special purpose queues•  optimized for large security groups•  eventual consistency for rule updates
  • 92. Problem:Firewall (iptables) rules explosion on the host firewall! Allow Security Group {Web} on TCP port 3060 ! !-A FORWARD -m tcp –p tcp –dport 3060 –src 10.1.16.31 – j ACCEPT-A FORWARD -m tcp –p tcp –dport 3060 –src 10.1.45.112 – j ACCEPT-A FORWARD -m tcp –p tcp –dport 3060 –src 10.1.189.5 – j ACCEPT …!-A FORWARD -m tcp –p tcp –dport 3060 –src 10.21.9.77 – j ACCEPT For large security groups, performance suffers
  • 93. Problem: Firewall (iptables) rules explosion on the host firewall ! Solution:! Use ipsets: !ipset –N web_sg iptreemapipset –A web_sg 10.1.16.31ipset –A web_sg 10.1.16.112ipset –A …! web_sg 10.1.189.5ipset –A web_sg 10.21.9.77-A FORWARD –p tcp –m tcp –dport 3060 –m set –match-set web_sg src -j ACCEPT
  • 94. Apache CloudStack!•  Apache CloudStack! –  http://www.cloudstack.org/! –  Download it! –  Use it! –  Contribute to it!•  Citrix CloudPlatform! –  Based on Apache CloudStack! –  Commercial support!