Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PayPal's Private Cloud @ Scale

2,187 views

Published on

Learn about OpenStack at PayPal.

Published in: Technology
  • Be the first to comment

PayPal's Private Cloud @ Scale

  1. 1. PayPal’s Private Cloud @ Scale
  2. 2. Table of contents ©2015 PayPal Inc. Confidential and proprietary. 2 Introduction OpenStack@PayPal Our Journey Lessons Learned Future Q&A
  3. 3. Introduction to PayPal 3
  4. 4. About PayPal ©2015 PayPal Inc. Confidential and proprietary. 4 PayPal is a leading technology platform company that enables digital and mobile payments on behalf of consumers and merchants worldwide. We put our customers at the center of everything we do. We strive to increase our relevance for consumers, merchants, friends and family to access and move their money anywhere in the world, anytime, on any platform and through any device.
  5. 5. Architecture @ PayPal ©2015 PayPal Inc. Confidential and proprietary. 5 Robust Infrastructure, Reusable Platforms, Payments Operating System, Delightful Experiences CustomerFacingInfrastructure Analytics Platform HadoopTeradata FPTIBI Technology Platform Framework Data Access Tools MediaMessagingSOA CrawlerCache Infrastructure & Operations Layer Power Data Center Hardware Network Database OperationsToolsCloudStorage Monitoring Experience Payments Operating System LoginIdentity EmailCart CouponsPayment Ship SupportRisk InvoiceWallet POS Notify Merchant
  6. 6. OpenStack@PayPal 6
  7. 7. © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. x 100K Cores x 10 PB Storage OpenStack Cloud > 10K Physical Servers x 10K VMs x 1000 engineers x 10M Lines x 1000 Releases /year > 1000 Services x 1000 Builds/d ay x 1000 Deploy/day Key Statistics Developer Statistics 9 Availability Zones 3 Regions
  8. 8. Our OpenStack Journey ©2015 PayPal Inc. Confidential and proprietary. 8 2012 2013 2014 2015 • OpenStack POC with 16 servers • Built 2 AZs with OpenStack Essex release (96 hypervisors) • 5 apps in production • Migrated from Essex to Folsom • Built 3 more Azs • 350 apps in production • Support for dev and QA environments • Upgrade to Havana • Migration of Nova to Neutron • 1000+ apps in production • 100% of web and mid tier apps on OpenStack • Deployed our largest AZ with 2500 nodes using Nova cells • Kilo upgrade in progress
  9. 9. The Current Stack ©2015 PayPal Inc. Confidential and proprietary. 9 Datacenters (Geographic regions, Availability Zones) OpenStack Cloud (Infrastructure as a Service) Platform as a Service Web/Mid Tier Applications Common Infrastructure Common Platforms and Services • 100% of web and mid tier applications on OpenStack • Proprietary PaaS on top of OpenStack to automate E2E application development life cycle • Support for polyglot applications • Test and production environments available on- demand
  10. 10. © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. Steps to Production 25 Manual Steps Automated Time to Market QA Deployment Build Time Prod Deployment Infrastructure 2 years back Now 8 Weeks 5 Hours 2.5 Hours 4 Hours 1 Month 1 Day 30 Minutes 5 Minutes < 30 Minutes 30 Minutes Impact On Life Of A Developer
  11. 11. Challenges And Lessons Learned 11
  12. 12. Challenges And Lessons Learned • Operating OpenStack at scale requires building an entire ecosystem of tools and back office functionality like monitoring, configuration management, log aggregation, hardware remediation etc. • We faced several control plane reliability issues as our deployment grew from several hundreds of compute to several thousands specially around RabbitMQ and services interaction • Scaling OpenStack to 10s of thousands of nodes is quite challenging specially around Neutron and RabbitMQ. • Security is evolving but not mature enough yet • Upgrading to a newer version of OpenStack is not straightforward specially with internal patches and takes months for larger deployments with multiple regions and availability zones • Distributed nature of OpenStack results into challenges with debug-ability of system and identifying the root cause • OpenStack is a journey and commitment. It’s success depends on building a talented team of engineers that takes pride in leveraging and contributing to OpenStack community©2015 PayPal Inc. Confidential and proprietary. 12
  13. 13. The Future 13
  14. 14. The Future Stack ©2015 PayPal Inc. Confidential and proprietary. 14 Datacenters (Geographic regions, Availability Zones) OpenStack Cloud (Infrastructure as a Service) Web and Mid Tier Applications Common Infrastructure Common Platforms and Services • New PaaS is being built using Mesos and Docker on top of OpenStack • Goal is to achieve industry leading datacenter efficiency and utilization • High application resiliency against infrastructure failures • Hybrid cloud POC • PaaS support for stateful applications Public Clouds
  15. 15. Questions? Jdesai at paypal dot com

×