Recommended
More Related Content
What's hot
What's hot (20)
Similar to Scaling MongoDB with Docker and cgroups
Similar to Scaling MongoDB with Docker and cgroups (20)
Recently uploaded
Recently uploaded (20)
Scaling MongoDB with Docker and cgroups
- 1. Scaling MongoDB with Docker and cgroups Marco Bonezzi TechnicalServicesEngineer,MongoDB marco@mongodb.com @marcobonezzi
- 2. #MDBW16 About the speaker I am Marco Bonezzi, TSE at MongoDB TSE = helping customers with a variety of issues Based in Dublin, Ireland Experience in distributed systems, high availability:
- 3. #MDBW16 How many of you have ever... 1. … manually deployed a MongoDB replica sets or sharded clusters? 2. ... had issues with resource allocation? 3. ... used Docker? 4. … used MongoDB running on Docker?
- 4. #MDBW16 We know how it feels… Different architectures in Development and Production Co-located MongoDB processes Production != docker run mongodb
- 5. #MDBW16 What are the pain points? Deployment Complex architectures and deployment patterns Resource Contention Multiple instances competing for resources MongoDB & Docker Combining both: network configuration, container location, other
- 6. #MDBW16 How to solve this? Deployment Using predefined cluster patterns Replicating environments Resource Control Setting limits to key resources MongoDB & Docker Create once, deploy everywhere Deploy patterns, not processes Orchestration Resource Management Automate for scaling
- 7. #MDBW16 About this talk Patterns for successful deployments Difference between success and failure Orchestrating MongoDB with Docker MongoDB cluster on AWS with containers Patterns with Swarm and Compose Managing container resources with cgroups Benefits of cgroups in a MongoDB cluster P S S
- 8. #MDBW16 Redundancy and fault tolerance Deploy an odd number of voting members Members ó Majority required ó Fault tolerance High availability and resource colocation Single member of a replica set / server Shards as Replica Set Ideally: primary / secondary / secondary Deployment patterns: Replica Set and Sharded Clusters Server 3Server 2Server 1 mongos Primary Primary RS1 SecondarySecondary Secondary RS2 Secondary RS3 Secondary RS1 Primary RS2 Secondary RS3 Secondary RS1 Secondary RS2 Primary RS3 mongos mongos cfgsvr1 cfgsvr2 cfgsvr3
- 9. #MDBW16 Docker • Noun: a person employed in a port to load and unload ships (from “what is docker” on Google) Containers: Isolated process in userspace Application + dependencies Shared kernel and libraries Can run on any infrastructure (or cloud) www.docker.com
- 10. #MDBW16 44% of orgs adopting microservices Why use Docker? 41% want application portability 13x improvement in release frequency 62% MTTR on software issues 60% Using Docker to migrate to cloud Reason to run containers: SPEED Microservices architectures Efficiency Cloud (The Docker Survey, 2016)
- 11. #MDBW16 Good news: Docker can help with this! Orchestration Coordinate MongoDB containers to deploy a recommended deployment pattern Resource Control Sizing each instance (and each cluster) to avoid resource contention issues
- 12. #MDBW16 Orchestrating MongoDB with Docker How can we use Docker for MongoDB deployments? How can we deploy these patterns using Docker containers? Why should we use Docker? Our recipe:
- 13. #MDBW16 Docker ecosystem Provisioning and managing your Dockerized hosts Native clustering: turns a pool of Docker hosts into a single, virtual Docker host. Define a multi-container application with all of its dependencies in a single file S
- 14. #MDBW16 Why Docker Swarm? 5x faster than Kubernetes to spin up a new container 7x faster than Kubernetes to list all running containers Evaluating Container Platforms at Scale 1000 EC2 instances in a cluster What is their performance at scale? Can they operate at scale? What does it take to support them at scale? https://medium.com/on-docker/evaluating-container-platforms-at-scale-5e7b44d93f2c#.k2fxds8c2 hGps://www.docker.com/survey-2016
- 15. #MDBW16 swarm-node-2 swarm-node-3swarm-node-1 Swarm multi-host networking How each mongod process connects to other processes outside of the host? • Swarm overlay container-to-container network • Using the hostname defined in the Compose file
- 16. #MDBW16 Swarm filters to build our patterns Constraint filters Mark each mongod container with a label: “role=mongod” “replset=rs1”
- 17. #MDBW16 Affinity filters Prevent multiple RS members on the same host: "affinity:replset!=rs1” swarm-node-1 swarm-node-3swarm-node-2 Affinity filters for container distribution
- 18. #MDBW16 Road to container success Deploying containers to the right node is not enough… Next step: Resource control on each swarm cluster node using cgroups Maritime New Zealand
- 19. #MDBW16 Resource control with cgroups and Docker Simple parameters to add to docker run or compose: --cpu-shares --cpuset-cpus --memory --blkio-weight --net
- 20. #MDBW16 MongoDB Memory usage in 3.2 with WiredTiger MongoDB Memory: mongod process: connections, aggregations, mapReduce, etc WiredTiger cache: (0.6 x total memory) – 1 GB Total = mem(mongod) + mem(WiredTiger cache) WiredTiger cache mongod mongod memory
- 21. #MDBW16 cgroup! memory_limit! Process memory with containers and cgroups WiredTiger cache mongod mongod memory total memory (seen from mongod process)! Inside the container • Can see total memory and not memory limit WiredTiger cache: • memory_limit *0.6
- 22. #MDBW16 cgroups: Memory and CPU limits mongod! mongod! mongod! mongod! MEM = (TOTAL_MEM - OS_MEM) / NUM_MONGOD WiredTiger cache = (MEM * 0.6) CPUSET= 0, 1, …, MAX_CPU_CORES mongod! mongod! mongod! mongod!
- 23. #MDBW16 MongoDB on Docker + cgroups: Memory WiredTiger cache: 60% of the container memory limit (for each mongod) Compose: mem_limit! Docker Engine: --memory! WiredTiger cache! ! ! ! ! ! ! mongod memory! WiredTiger cache! ! ! ! ! ! mongod memory! WiredTiger cache! ! ! ! ! ! ! mongod memory! WiredTiger cache! ! ! ! ! ! ! mongod memory! OS Memory!
- 24. #MDBW16 MongoDB on Docker + cgroups: CPU mongod (and mongos) mapped to 1+ core Compose: cpuset! Docker Engine: --cpuset-cpus! mongod! rs1a! mongod! rs1a! mongod! rs2b! mongod! rs2b! core0! core1! core2! core3! OS! OS! mongod cfgsrv -c! mongos! core4! core5! core6! core7! --cpuset-mems (NUMA)!
- 25. #MDBW16 Understanding resource usage: • docker top rs1a! • docker stats rs1a! Container stats available via Docker remote API: GET /containers/(id)/stats Also available from docker-py: http://docker-py.readthedocs.org/en/latest/api/#stats Resource usage with Docker
- 26. #MDBW16 Resource usage with Docker Multiple statistics for each container: Memory limit and usage, CPU (per core level), Network, Disk Useful to combine with MongoDB metrics (like db.serverStatus())
- 27. #MDBW16 Creating a Swarm cluster on AWS to deploy MongoDB
- 28. #MDBW16 Creating a Swarm cluster on AWS to deploy MongoDB DEMO! Configure docker-machine with ec2 driver (AWS) Deploy discovery service for Swarm Master Deploy AWS instances for: • Swarm master • Swarm worker nodes Connect to the Swarm master Define compose file for deployment Define Swarm filters and constraints and cgroup limits Deploy the environment with a single command using the compose file Configure our MongoDB sharded cluster using Cloud Manager API
- 29. #MDBW16 What our dockerized sharded cluster looks like… rs1a! rs1c!rs1b! rs2b! rs2a! rs2c! rs3b! rs3a! rs3c! mongos! cfgsvr -a! cfgsvr -b! cfgsvr -c! dockerC1! dockerC2! dockerC3! PRIMARY! SECONDARY! MONGOS!
- 30. Summary
- 31. #MDBW16 Advantages of using MongoDB with Docker Speed: testing and deploying cluster patterns easily Build once, deploy everywhere Control: Resource control and utilization Key to success with containers Agility: Microservices architectures Making change less expensive Flexibility: Multi vendor cloud opportunities AWS, Azure, Google, IBM, CloudFoundry P S S
- 32. #MDBW16 How successful customers use MongoDB with Docker • Case Studies @ hGps://www.mongodb.com/blog • Whitepaper: “Enabling Microservices – Containers & Orchestration Explained” https://www.mongodb.com/collateral/microservices-containers-and-orchestration-explained
- 33. #MDBW16 Now it’s YOUR turn Share with us your use case of MongoDB & Docker: http://bit.do/DockerMongoDB @marcobonezzi You can actually try this at home: https://github.com/sisteming/mongo-swarm