Marriage of ESX and OpenStack - PayPal - VMWorld US 2013
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Marriage of ESX and OpenStack - PayPal - VMWorld US 2013

on

  • 675 views

VSVC4994 - Marriage of ESX and OpenStack at PayPal ...

VSVC4994 - Marriage of ESX and OpenStack at PayPal

PayPal is quickly moving forward to utilize open source and open standards based technologies in the build-out of our private cloud. With our internal release of OpenStack software based on 'Grizzly' we have integrated ESX 5 support and now can deploy workloads against ESX as well as against KVM.

Statistics

Views

Total Views
675
Views on SlideShare
669
Embed Views
6

Actions

Likes
1
Downloads
21
Comments
0

2 Embeds 6

https://twitter.com 5
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Marriage of ESX and OpenStack - PayPal - VMWorld US 2013 Presentation Transcript

  • 1. VSVC4994 Marriage of ESX and OpenStack at PayPal Scott Carlson, PayPal #VSVC4994
  • 2. THE MARRIAGE OF ESX AND OPENSTACK AT PAYPAL MULTI-VENDOR AGILITY VMWORLD 2013
  • 3. ABOUT PAYPAL PayPal offers flexible and innovative payment solutions for consumers and merchants of all sizes. • 132,000,000 Users. • $300,000 Payments processed by PayPal each minute. • 193 markets / 25 currencies. • PayPal is the World’s Most Widely Used Digital Wallet. 3
  • 4. WHY WE VIRTUALIZED ON ESX • 80% of the PayPal front-end is virtualized on VSphere 5.0u1 • Primary Criteria − Stability, performance, industry expertise, availability of experts • Standardized on VCE VBLOCK© for initial implementation • Fully consumable API • Load-test harness well understood in industry (specInt & vMark) − Predictable scaling pattern for horizontally scaled workloads 4
  • 5. CLOUD 5
  • 6. PAYPAL INTERNAL CLOUD 2012/2013 Shift toward an internal cloud model • Shift from Enterprise design model to cloud-based design • Elastically scale and self-heal infrastructure to accommodate unpredictable usage patterns of customers and internet commerce • Separate rapidly iterating customer experiences from core services • reduce overall cost per transaction within the environment 6
  • 7. CLOUD IS THE GREAT ENABLER ENABLE THE DEVELOPER ENABLE THE BUSINESS One-Click Developer Self Service Global Compute & Data Fulfillment Payment Delivery SelfOrganizing & Optimizing Infrastructure System Intelligence Driven Operation Code 7 Deploy Enjoy
  • 8. PAYPAL CLOUD PLATFORM – GUIDING PRINCIPLES • Technology − Adopt Open Source Solutions where ever possible − No Vendor Lock-in − Industry Best Practices − Leverage Industry/ebay Inc Investments • Functionality − Self-Service tool for application life cycle management. − Robust Automation & Orchestration − Seamless On-Demand Capacity Fulfillment 8
  • 9. OPENSTACK PayPal deploying Openstack in order to help transform our global infrastructure into an agile and open cloud platform. Agility - time to market for customer facing services Agility - speed to service developer requests for VM resources Agility – utilize the engineering culture of PayPal to deliver specialized cloud services where needed 9
  • 10. INCREDIBLE INDUSTRY SUPPORT 10
  • 11. TECHNOLOGY STACK User Interface Operations Portal DEVS Deployment Portal Horizon, Ceilometer Traffic Mgmt Monitoring Metering Stages Workflow Monitoring Orchestration Engine Orchestration Cloud Formation (Heat) Foundational Services Nova, Cinder, Swift, Keystone, Quantum, Horizon Software Infrastructure Cobbler ISC DHCP Hardware Infrastructure x86 Compute Salt BIND Local Storage RHEL 6.x Network LBaaS, DNSaaS FWaaS Hypervisor Zabbix Load Balancer PP Specific 11
  • 12. CLOUD BEFORE INTEGRATION WEB F Z F Z F Z “Stateless & Disposable” F Z F Z KVM Local Disk F Z F Z F Z ESX 5.0u2 KVM Local Disk F Z MID Cloud Management Zone VCenter Management F Z F Z ESX 5.0u2 ESX 5.0u2 Shared Storage Shared Storage Shared Storage Physical Non-virtualized F Z ESX 5.0u2 Shared Storage Physical Non-virtualized DATABASE & RESTRICTED ZONE FZ = Logical Fault Zones
  • 13. SIDE-BY-SIDE 13
  • 14. CLOUD AFTER INTEGRATION F Z F Z F Z F Z F Z WEB F Z KVM Local Disk ESX 5.0u2 Shared Storage Physical Non-virtualized MID Cloud Management Zone F Z KVM Local Disk ESX 5.0u2 Shared Storage Physical Non-virtualized DATABASE & RESTRICTED ZONE
  • 15. COMPARING But isn’t Openstack a direct replacement for ESX? Why would you keep them both? ESX/Vsphere != Openstack NOVA != vSphere || vCenter || ESXi NOVA =~ vCD, vCAC KVM =~ ESX To connect to any hypervisor, the Openstack cloud ‘proxies’ connections to any supported hypervisor via Nova. That abstracts the ‘Cloud’ from the hypervisor 15
  • 16. BRINGING ESX ‘INTO’ THE CLOUD • Equivalent functionality on KVM and ESX • Full birth to death lifecycle management of virtual machines − Build new, power on, power off, console, rebuild, delete • Auto-configuration of host resources following t-shirt sizes standards − CPU, RAM, NIC, IP, OS Version • IP Address Management • Build from “Snapshot”/”Template” • Deploy resources following appropriate fault zone model • Must work from within single Horizon/Asgard interface 16
  • 17. HYPERVISOR REQUIREMENTS • VSphere/ESX 5.1 − 5.0 works but many, many, many back-ports / tweaks • Single security zone per hypervisor − No sharing of confidential & non-confidential on same hardware (PCI) • Openstack management network communication − This is NOT necessarily the VKERNEL network 17
  • 18. STORAGE REQUIREMENTS • Shared storage required − Data Store Cluster − Single Data Store support [ bug fix coming ] • DRS Enabled with auto-placement • Data Stores must be created in advance − No Cinder support 18
  • 19. OPENSTACK GRIZZLY ⁃ O penS ack C t ommand Line T ools (nova-client, swif t-client, et c.) ⁃ C loud M anagement T ools (Right scale, E rat ius, et c.) nst ⁃ G t ools (C UI yberduck, iPhone client, et c.) Int er net O penS ack t O bject API O penS ack t C omput e API O penS ack t Image API O penStack Identity AP I O penStack D ashboard H (S) TTP Amazon Web Ser vices E 2 API C VNC VMRC / / Spice O penS ack t Block S orage API t Hor izon O penS ack t Net wor k API O penStack O bject AP I O penStack Image AP I O penS ack t O bject API swif t-proxy O penStack C ompute AP / I Admin AP I O penStack Identity AP I O penS ack t O penS ack t Block S orage API Block S orage API t t nova-api O penS ack t Image API glance-api (O E 2, Met adat a, Admin) S, C nova-comput e nova-cert/ objectstore glance-regist r y cont ainer object cinder-api nova-console nova-*proxy O penS ack t Image API memcached account O penS ack t Net wor k API O penS ack t Net wor k API H TTP(S) cinder-volume quant um agent (s) nova dat abase object st ore O penStack Identity AP I O penS ack O t bject St ore Queue net wor k provider quant um plugin(s) quant um dat abase Queue volume provider Queue hyper visor cont ainer D B cinder-backup libvirt, XenAPI, et c. glance dat abase account D B quant um-ser ver cinder dat abase nova-conduct or nova-consoleauth cinder-scheduler http://www.solinea.com O penStack Identity AP I O penS ack t Image Ser vice nova-scheduler O penS ack C t omput e O penS ack t Block S orage t O penS ack t Net wor k Ser vice O penStack Identity AP I O penStack Identity AP I keyst one (ser vice & admin APIs) O penStack Identity Service t oken backend cat alog backend policy backend O penStack O bject AP I O penStack Identity AP I O penStack Identity AP I ident it y backend
  • 20. ITS ALL ABOUT NOVA
  • 21. CONFIG OF NOVA Nova is the project name for OpenStack Compute, a cloud computing fabric controller, the main part of an IaaS system. Individuals and organizations can use Nova to host and manage their own cloud computing systems. #compute_driver = libvirt.LibvirtDriver compute_driver = vmwareapi.VMwareVCDriver Can be multiple vmwareapi_host_ip=192.168.20.50 clusters now! vmwareapi_host_username=root vmwareapi_host_password=vmware vmwareapi_cluster_name=openstack_test vmwareapi_wsdl_loc=https://192.168.20.50/sdk/vimService.wsdl Vcenter 5.1 Appliance 21 Confidential and Proprietary
  • 22. GLANCE AND IMAGES Rules for Glances images for VMWare • Saved in VMDK Format • Imported as VMDK Format • Thick Provisioned VMDK Required • No split VMDK allowed (must be merged) • In a multi-hypervisor cloud, all images are separate ‘per hypervisor’ (no launching KVM VM’s on ESX) glance add name=”MYMACHINE.vmdk" disk_format=vmdk container_format=bare is_public=true vmware_adaptertype="lsiLogic" vmware_disktype="preallocated" vmware_ostype="otherGuest" < /path/to/MYMACHINE.vmdk 22 Confidential and Proprietary
  • 23. BUILDING AND INSTALLING OS • Kickstart • Build a small root disk • Use kickstart to image machine • Post-install with puppet to customize machine and add additional mount points depending on application requirements • Image Deploy • Currently does not support ‘config-drive’ • Need Guest Tools to ‘duplicate’ functionality 23 Confidential and Proprietary
  • 24. WHAT ABOUT THE NETWORK • 24 Quantum requires NVP 3.2 • Cannot talk directly to VSphere API to allocate VDS Port to NIC • Implemented via vAPP – integration bridge • Configured as separate transport zone within Nicira Confidential and Proprietary
  • 25. WHAT’S LEFT • Component “at-scale” testing Currently manage “tens” at a time, need to move to “hundreds” or “thousands” • Most fixes scheduled to go into Havanna, every bug-fix needs to be reviewed and possible backported to Grizzly • Multiple Data Store enumeration on a cluster • Full Certification on VCE© VBLOCK with Vision Intelligent Operations, auto-upgrades, and full Openstack support of all components 25 Confidential and Proprietary
  • 26. READING MATERIALS • • • • • • http://www.solinea.com/2013/06/15/openstack-grizzly-architecture-revisited/ - Ken Pepple http://www.slideshare.net/kenhui65/getting-started-with-openstack?ref=http://cloudarchitectmusings.com/2013/06/16/getting-started-with-openstack/ - Kenneth Hui http://docs.openstack.org/trunk/openstack-compute/admin/content/config-drive.html - config-drive doc http://docs.openstack.org/trunk/openstack-compute/admin/content/vmware.html - Openstack VMWARE doc http://www.ebay.com - Buy It Now http://www.paypal.com - and then Pay for it Here!
  • 27. Interested? DL-PayPal-Cloud-Hiring@ebay.com
  • 28. THANK YOU
  • 29. VSVC4994 Marriage of ESX and OpenStack at PayPal Scott Carlson, PayPal #VSVC4994