DockerCon SF 2015: Beyond CI to Production Scale PaaS with Docker
Report
Docker, Inc.Follow
Jun. 25, 2015•0 likes•3,913 views
DockerCon SF 2015: Beyond CI to Production Scale PaaS with Docker
Jun. 25, 2015•0 likes•3,913 views
Download to read offline
Report
Technology
Talk Description: What a difference a year makes. Last year Paypal presented on scaling a CI infrastructure with Docker, Mesos and Jenkins. Over the past year we worked on integrating Docker to the core of our PaaS ecosystem. We are Dockerizing the entire PayPal application ecosystem consisting of polyglot Java, Node.js, Scala, C++ and Python frameworks. In this journey we have introduced HA Docker Registry with ElasticSearch search plugin using both OpenStack Swift and Cinder storage, cross-data center ATS server smart image caching along with a developer-friendly boot2docker fig/compose workflow for Kraken (open source nodeJS) stack. This we believe has the potential to become one of the largest financial services production deployment of Docker.
Docker, Inc.Follow
Recommended
DockerCon SF 2015: From Months to MinutesDocker, Inc.
DockerCon SF 2015: Docker at LyftDocker, Inc.
DockerCon SF 2015: How to Build a Secure DevOps Environment for the GovernmentDocker, Inc.
DCSF 19 Microservices API: Routing Across Any InfrastructureDocker, Inc.
How to build your containerization strategyDocker, Inc.
DockerCon 15 Keynote - Day 2Docker, Inc.
More Related Content
What's hot
An Integrated Pipeline for Private and Public Clouds with Jenkins, Artifactor...VMware Tanzu
Automating CICD Pipeline with GitLab and Docker Containers for Java ApplicationsJelastic Multi-Cloud PaaS
DockerCon SF 2015: Docker in the New York Times NewsroomDocker, Inc.
DCEU 18: Desigual Transforms the In-Store Experience with Docker Enterprise C...Docker, Inc.
DCEU 18: 5 Patterns for Success in Application TransformationDocker, Inc.
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelDocker, Inc.
Viewers also liked
【HinemosWorld2015】A1-3_コンテナ技術Dockerの導入事例と完全運用自動化Hinemos
Docker国内外本番環境サービス事例のご紹介ThinkIT_impress
Dockerの事例紹介Hiroki Endo
Infrastructure as CodeでReNom環境構築入門ReNom User Group
鯨物語～Dockerコンテナとオーケストレーションの理解Masahito Zembutsu
Dockerを活用したリクルートグループ開発基盤の構築Recruit Technologies
Similar to DockerCon SF 2015: Beyond CI to Production Scale PaaS with Docker
Dockercon2015_paypalahunnargikar
2 万林涛Jiang Shang
DevOps with Azure, Kubernetes, and Helm WebinarCodefresh
Oracle virtualbox basic to rac attackBobby Curtis
Containers and Nutanix - Acropolis Container ServicesNEXTtour
321 codeincontainer brewboxLino Telera
More from Docker, Inc.
Containerize Your Game Server for the Best Multiplayer Experience Docker, Inc.
How to Improve Your Image Builds Using Advance Docker BuildDocker, Inc.
Build & Deploy Multi-Container Applications to AWSDocker, Inc.
Securing Your Containerized Applications with NGINXDocker, Inc.
How To Build and Run Node Apps with Docker and ComposeDocker, Inc.
Hands-on Helm Docker, Inc.
Recently uploaded
Product Research Presentation-Maidy Veloso.pptxMaidyVeloso
UiPath Tips and Techniques for Debugging - Session 3DianaGray10
Unleashing Innovation: IoT Project with MicroPythonVubon Roy
Accelerating Data Science through Feature Platform, Transformers, and GenAIFeatureByte
The Flutter Job Market At The MomentAhmed Abu Eldahab
Document Understanding as Cloud APIs and Generative AI Pre-labeling Extractio...DianaGray10
DockerCon SF 2015: Beyond CI to Production Scale PaaS with Docker
- 1. Beyond CI to Production Scale PaaS with Docker
- 2. Platform Engineering @ PayPal ➢ Several thousand OpenStack servers across multiple data centers ➢ More than 3000 PayPal developers supported ➢ Thousands of application deployments performed every day ➢ 165 Million active PayPal customer accounts ➢ Presence in 203 markets and 100 currencies ➢ $235 Billion payment volume ➢ 12.5 million payment transactions every day Boost developer productivity Support ever increasing scale of operations
- 3. Last year at DockerCon 2014… ➢ Thousands of VMs dedicated to run Jenkins! ➢ Utilization is less than 5% ☹ ➢ VMs idle most of time (no one runs CI builds frequently) VM sprawl and poor resource utilization ➢ Polyglot application stacks (Java, C++, Node.js, Python, Scala) ➢ Different OS flavors (Ubuntu & RHEL) ➢ Software version conflicts ➢ Special hardware requirements Build slave management and maintenance is a nightmare
- 4. How PaaS Orchestrates The Docker PDLC
- 5. Building & Storing Docker Images
- 6. Docker Registry HA Setup ➢ Supervisord as the process manager ➢ Logrotate for registry and nginx logs ➢ Elasticsearch plugin for indexing ➢ Swift plugin for storage ➢ Basic authentication ➢ Ansible playbook for setting up the registry ➢ HA running behind F5 load balancer ➢ Docker load used to deploy the registry for the first time ➢ Swift auto-sync between data centers Features
- 7. Docker Image Index Challenges ➢ Production firewalls block multicast clustering protocol ➢ ES sniffing timeout issues when ES nodes were unavailable ➢ ES split-brain problems with clustering Solutions ➢ ES indexing plugin for the Docker Registry ➢ Zookeeper transport plugin for ES Python client ➢ Persisting ES index data using OpenStack Cinder
- 9. Deploying Docker Images In Production ➢ DNS-based ATS discovery per DC ➢ Header rewrite plugin ➢ Custom Cache rules ➢ Custom SSL certs ➢ ATS Ansible deployment Features
- 10. Dockerized Development Environments ➢ Building an application stack should be simple, but it’s not! ➢ Development environments are snow-flakes ➢ Development environments should be self-contained
- 11. Container with Dev configuration
- 12. Container with Stage configuration
- 13. Demo
- 14. Thank you Ashish Hunnargikar Software Engineer @hunnarg Mohit Soni Software Engineer @mosoni #dockercon