Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Couchbase-as-a-Service – Connect Silicon Valley 2017

189 views

Published on

Speaker: Naveen Kumar Eppa, DecOps Engineer, Dreamworks Animation

Learn how DreamWorks Animation leverages DevOps, Docker, and the Cloud Native Application model to deploy Couchbase on demand. Couchbase-as-a-Service has revolutionized the way in which DreamWorks manages data, allowing them to reduce costs, cut delivery times, and dramatically increase efficiency.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Couchbase-as-a-Service – Connect Silicon Valley 2017

  1. 1. Couchbase-as-a-Service Naveen Kumar Eppa
  2. 2. Who I am Naveen Kumar Eppa DevOps Engineer, Data Services Digital Operations
  3. 3. Who we are
  4. 4. What we do
  5. 5. ● 22,000-core render farm ● 80 Million compute hours per movie ● 350 TB of data per movie ● Half Billion files per movie ● 250+ Billion pixels per movie ● Backend services receive over 3.6 Billion hits per day ● At peak, 110,000 transactions per second Data is at our core
  6. 6. ● Manual deployments ● Dedicated hardware ● Single point of failure ● Inconsistencies between environments ● Cron jobs ● Scaling ● Static infrastructure Past problems
  7. 7. ● Automated deployments ● Efficient resource usage and tracking ● Multi-tenancy ● Ease of management ● Monitoring and alerting ● Automated backup and restore ● Elasticity Solution: Couchbase-as-a-Service
  8. 8. Achieving CBaaS ● We treat our data center as a cloud ● We treat our databases as cloud native applications
  9. 9. Cloud Native? ● Cloud-native is an approach to building and running applications that fully exploit the advantages of the cloud computing delivery model.
  10. 10. Cloud Native Applications 1. Containerized 2. Dynamically orchestrated 3. Microservices-oriented
  11. 11. 1. Containerized ● Containerization is a technique of encapsulating applications in containers with their own operating environment, abstracting them from the operating system ● We use Docker
  12. 12. Challenge: Docker containers are stateless ○ Databases need to be stateful Making stateful containers (v1) ● Explicit host directory mapping (docker run -v) ● Problem: data local to the host Making stateful containers (v2) ● Docker Volume Plugins: Portworx ○ Portworx provides distributed shared storage ○ Allows portable volumes 1. Containerized
  13. 13. Challenge: Docker containers are not routable ○ Databases need to be routable Making routable containers (v1) ● Explicit port forwarding (docker run -p) ● Problem: port conflicts Making routable containers (v2) ● Docker user-defined networks ○ We use Quagga to advertise routes ○ Allows routable containers 1. Containerized
  14. 14. 2. Dynamically orchestrated ● Scheduling ○ Custom scheduler ● Deployment ○ Chef ● Discovery ○ DNS ● Monitoring ○ Datadog auto discovery ● Backup scheduling ○ Rundeck
  15. 15. 10,000 ft view Host 1 Host 2 DNS Host 3
  16. 16. 3. Microservices-oriented Traditional microservices: ● The micro-service architecture is a style of building applications as a suite of small, independently deployable services rather than building a large, monolithic application
  17. 17. 3. Microservices-oriented Microservices applied to Databases: ● Our “microclusters” philosophy is to deploy a suite of small, independently deployable clusters rather than deploying a large, monolithic database cluster
  18. 18. 3. Microservices-oriented ● 5-7 movies at a time ● Each movie is a “tenant” ● 25+ services per movie ● Dedicated cluster per service ● Smaller clusters means less impact ● Microclusters are fast and efficient ● Microservices allow each movie to evolve independently
  19. 19. ● Automated deployments ● Efficient resource usage and tracking ● Ease of management ● Multi-tenancy ● Monitoring and alerting ● Automated backup and restore ● Elasticity Couchbase-as-a-Service
  20. 20. Time’s UP http://www.lifesmith.com/jbhumor1. html
  21. 21. Questions? Naveen Kumar Eppa naveen.kumar.eppa@dreamworks.com

×