Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Openstack@ebay: Practical SDN deployment with Quantum


Published on

  • Be the first to comment

Openstack@ebay: Practical SDN deployment with Quantum

  1. Prod Prod QA DEV PCI Secure DEVQA QA DEV Copyright eBay Inc. 2012 2
  2.  Any Application Anywhere  Dedicated physical environments cause fragmentation Soft Cabling  Datacenter reconfiguration is costly and cannot be automated Shared Standardized Infrastructure  Simplifies automation and improves supply chain efficiency Virtualize everything  White space between applications and infrastructure helps agility Automate everything  Automation helps agility and efficiency Copyright eBay Inc. 2012 3
  3. • Translation of physical environment properties into configurations • Assigned to projects (logical environments), drives scheduling and policies • For example, network selection Production DEVObligations Restrictions Capabilities Obligations Restrictions CapabilitiesQA Approved Builds No Login Access Core DB access Certified OS versions Limited Prod Full root AccessProd OS version No Corp Access 24/7 Incident Mgt Limited QA AccessMonitoring No QA Access Site traffic Access No site Traffic Filtered Internet External Obligations Restrictions Capabilities No Prod Access Private DB Certified OS Versions No Corp Access 24/7 Incident Mgt Monitoring No QA Access Site traffic Access Copyright eBay Inc. 2012 4
  4. Core4 spines(Nx10Gb) SpineN leaves(48x1Gb) Leaves 48 -> N “½ racks”M servers2x1Gb Flat L3 (all switches are routers too) Line rate from any server to any server (oversubscription = 48/40) OSPF/ECMP to advertise routes Copyright eBay Inc. 2012 5
  5. Dedicated Network VLAN Based VLAN trunk vlan 1 Prod QA vlan n Production QA- physical network build out + Physical isolation - Limited scale (n = 4096) + L2 isolation- Fragmentation + fool proof - Large fault domain (STP) + somewhat soft Cabling- coarse grained isolation Copyright eBay Inc. 2012 6
  6. Security Groups or Virtual Firewall+ no/minimal infrastructure requirement - Difficult to combine provider policies and user policies+ good for user policies (ip tables) - Management of rules - Impact of group membership modification - Aggregation/summarization difficult/impossible Copyright eBay Inc. 2012 7
  7. Virtual Networks using Software Defined Networks Overlay 1 Prod OtherNetworks QA Overlay n Cloud Fabric + L2 isolation + Can complement L3 isolation + compatible with large scale + large number of networks (n>4096) + can be fully automated - Tunnel overhead + firewall can be interposed between - L2 size limited by # of tunnels virtual networks Copyright eBay Inc. 2012 8
  8. Traditional SDN The The Network Network Network protocols Network protocolsRouting/switching engine Routing/switching engine controls The Switch/Router controls Logic Logic API The Switch/Router Controller Copyright eBay Inc. 2012 9
  9. Wizard Physical Switches OSPF/ECMP,… Traffic Engineering Virtual + Physical switchesNinja Overlay Networks Virtual Switches ARP + L2 protocolsNerdy Overlay Networks Copyright eBay Inc. 2012 10
  10.  A logical environment defined as a class of service on top of shared infrastructure  Self Service VM for developers.  Access must be similar to their desktops (access to QA, Corp, …)  Should allow collaboration Implemented as a set of L2 networks (/24) with in a given L3 (/20)  No private networks : all developers on same shared networks  No private IP space: traffic is routed within core, no need for floating Ips Isolated from infrastructure  Overlay network using OpenVswitch / STT tunneling  Nicira NVP controllers integrated with Quantum (Essex)  Routed out through perimeter firewall Copyright eBay Inc. 2012 11
  11. From default-> -> From default-> Standby Gateway Eth1/vlan 1 Dev Cloud : Eth0/vlan 2 Corp10.9.1.0/24 N gtw-xxxx trunk gtw-xxxx Internet10.9.2.0/24 N M gtw-xxxx QA vswitch M Eth1/vlan 1 Eth0/vlan 2 vswitch Nicira default-> Nicira Nicira Active Gateway Service Nicira Service controllers Nodes controllers Nodes vif K C Hypervisor S A Q N:Nova-network+dnsmasq K:Ubuntu + KVM vswitch C:Nova-compute A:Nova-api S:Nova-scheduler Q:Quantum M:Metadata Infrastructure/Internal Virtual network Infrastructure/External Copyright eBay Inc. 2012 12
  12. Developer Admin Create network (project = admin, Create routes eBay Cloud Portal Cidr=10.9.x.0/24) Create instance 1 (COS,OS, size) Nova-manage Gateway 2 Get Free Networks eBay IaaS Create DNS Boot Instance Nova Network (A,PTR) (Image ID,Flavor, NIC) Create 4 3 gtw-xxxx DNS Nova API Quantum novaManagement db Create Create Nova Scheduler port lswitch 13 Get IP Create port Nicira Controller Nova Compute Copyright eBay Inc. 2012
  13. 250 100 Instance200 80 Requests150 60 Success Failed100 40 rate 50 20 0 0 Copyright eBay Inc. 2012 14
  14.  Perimeter firewalls configured once, not  No capacity/policy based assignment of dependent on the instance networks – had to be implemented outside. creation/deletion/movement Moving it to nova scheduler. Network are pre-created using nova-  One network flavor supported in Essex. manage, good for provider networks Cannot have, e.g., one gateway per network, with different behavior (dhcp) Can be extended with other COS using same pattern  Scale out requires bigger links out of the gateway, or more gateways Stability of both Nicira NVP and Openstack + Ubuntu + KVM  Upset the separation of concern Looking forward to new features in Folsom – requirement: Netsec + Networking + Sys Quantum v2 Admins in same box = ‘interesting’ 15
  15.  New classes of service  External : private networks + VIP and Floating IP on the Internet  Production : Bridged network Scale out  80 today, going to a lot more  More gateways/10Gb Folsom upgrade  L3 Routers  Load Balancers Cleaner Openstack integration  Network Allocation  DNS configuration  AuthN/AuthZ 16
  16. We are Hiring ! Copyright eBay Inc. 2012 17