MongoDB Europe 2016 - Powering Microservices with Docker, Kubernetes, and Kafka
•
2 likes•2,066 views
Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. This session introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to MongoDB Europe 2016 - Powering Microservices with Docker, Kubernetes, and Kafka
Similar to MongoDB Europe 2016 - Powering Microservices with Docker, Kubernetes, and Kafka (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
MongoDB Europe 2016 - Powering Microservices with Docker, Kubernetes, and Kafka
- 1. Powering Microservices with Docker, Kubernetes & Kafka Andrew Morgan MongoDB Product Marketing @andrewmorgan
- 2. #MDBW16 Agenda What, Why, How? Microservices 01 Kubernetes, Mesos,… Orchestration 03Docker Kafka Containers 02 Why, How? MongoDB 04 Who, Why? Use Cases 06When to Use 05
- 4. #MDBW16 Why Use Microservices? (tl;dr WebScale) Team = Components No committees Empower Build MVP quickly Speed Simple components Isolated impact Maintain Rapid iterations React to market Change Product Team Scale
- 5. #MDBW16 1990s & Earlier Monolithic Cross-Team Tight Coupling Small Change Huge Impact System Retests
- 13. #MDBW16 Alternate Microservices Example • Much larger Microservices 1. User Account 2. Product Catalog 3. Inventory 4. Orders • Best practice is each microservice has its own database
- 14. “ Conway’s Law 1967 Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure.”
- 16. Containers
- 17. #MDBW16 Containers – Powering Microservices Real world shipping containers • Road, rail & sea • Contents untouched • Ubiquitous & standardized • Simple • Contents protected • Constraints
- 18. #MDBW16 Containers – Powering Microservices Software containers • 1 image -> Many containers • Laptop, DC, cloud • Dev, QA, production, support • Simple, efficient • Isolation • Constraints
- 19. #MDBW16 VMs vs. Containers VM VMVM Bare Metal Host Operating System Hypervisor Guest OS Libraries Apps Service Guest OS Libraries Apps Service Guest OS Libraries Apps Service Container ContainerContainer Bare Metal Host Operating System Docker Engine Libraries Libraries Apps Libraries Apps Service ServiceService
- 20. #MDBW16 VMs vs. Containers VM VMVM Bare Metal Host Operating System Hypervisor Guest OS Libraries Apps Service Guest OS Libraries Apps Service Guest OS Libraries Apps Service Container ContainerContainer Bare Metal Host Operating System Docker Engine Libraries Libraries Apps Libraries Apps Service ServiceService
- 21. #MDBW16 VMs vs. Containers VM VMVM Bare Metal Host Operating System Hypervisor Guest OS Libraries Apps Service Guest OS Libraries Apps Service Guest OS Libraries Apps Service Container ContainerContainer Bare Metal Host Operating System Docker Engine Libraries Libraries Apps Libraries Apps Service ServiceService
- 22. #MDBW16 Docker • Simple to use • 100K+ images on Docker Hub • Build images from images • Platforms • Linux, OS X, Windows • Laptop, VM, Cloud,… • Cloud services
- 23. #MDBW16 Run MongoDB docker run -d mongo
- 24. #MDBW16 Security Opportunities • Isolation • Constrain resources • Narrow roles • Keep images/containers current • root: so restrict access
- 27. #MDBW16 Microservice Architectures Built on Containers Many small, focused containers -> sophisticated services • Well defined APIs • Independent languages & libraries • Modular: easy maintenance + reuse • Fault tolerant • Scalable
- 28. #MDBW16 Connecting the Microservices – Apache Kafka Producer 9 8 7 123... Topic A Consumer
- 29. #MDBW16 Connecting the Microservices – Apache Kafka Producer 9 8 7 123... Topic A Consumer Producer Consumer
- 30. #MDBW16 Connecting the Microservices – Apache Kafka Producer 9 8 7 123... Partition 0 Topic A Consumer Producer Consumer 4 3 5 123 ... Partition 1
- 31. #MDBW16 Connecting the Microservices – Apache Kafka Producer LEADER Topic A / Partition 0 Broker 1 FOLLOWER Topic A / Partition 1 FOLLOWER Topic A / Partition 0 Broker 2 LEADER Topic A / Partition 1
- 32. #MDBW16 Connecting the Microservices – Apache Kafka Producer Producer Producer 9 8 7 123 ... Partition 0 4 3 5 123 ... Partition 1 7 3 2 123 ... Partition N Topic A Topic B 7 6 5 123 ... Partition 0 New ß Old Consumer Consumer
- 33. Orchestration
- 34. #MDBW16 Orchestration Automated deployment, connecting, and maintenance of multiple containers • Provision hosts • Containers • Instantiate • Reschedule • Link • Scale Out/In • Expose services
- 35. #MDBW16 Kubernetes Created by Google, feature-rich and widely adopted • Deployment and ‘replication’ • On-line scale out/in • Rolling upgrades • High Availability • Persistence • Ports • Load balancing • Google Compute Engine
- 36. #MDBW16 Apache Mesos 10,000s of physical servers; used by Twitter, Airbnb & Apple • Code (“frameworks) vs. declarative • Less feature rich than Kubernetes • Kubernetes as a Mesos framework • Foundation for distributed systems • Apache Aurora, Chronos, Marathon
- 37. #MDBW16 Choosing an Orchestration Framework • What you have: • Skills? • DevOps frameworks? • Number of hosts? • Bare metal, VMs, or cloud? • Lifecycle • Features • Automated High Availability? • Grouping and load balancing? • As a service??
- 38. MongoDB
- 39. #MDBW16 Why MongoDB is a Good Fit For Microservices Scalability Monitoring & Automation Redundancy Flexible Data Model Simplicity
- 40. #MDBW16 Orchestrating MongoDB Using Kubernetes Distributed, stateful application • Persistent volumes • External IP addresses for internal comms • Init MongoDB replica set • Monitor • Backup
- 41. Volume name: mongo-persistent-storage1 pdName: mongodb-disk1 mongodb -disk1 Container name: mongo-node1 image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage1 mountPath: /data/db Docker Hub Registry Pod labels: [name: mongo-node1; instance: rod] ReplicationController name: mongo-rc1 labels: [name: mongo-rc] replicas: 1 selector: [name: mongo-node1] Service: LoadBalancer name: mongo-svc-a labels: [name: mongo-svc-a] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node, instance: rod] External IP Address 104.1.1.1 App 104.1.1.1:27017
- 42. ReplicationControler name: mongo-rc-europe labels: [name: mongo-europe] replicas: 1 selector: [name: mongo-node] Volume name: mongo-persistent-storage pdName: mongodb-disk-europe mongodb- disk-europe Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-europe labels: [name: mongo-svc-europe] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node] ReplicationControler name: mongo-rc-asia labels: [name: mongo-asia] replicas: 1 selector: [name: mongo-node] Volume name: mongo-persistent-storage pdName: mongodb-disk-asia mongodb- disk-asia Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-asia labels: [name: mongo-svc-asia] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node] ReplicationControler name: mongo-rc-us labels: [name: mongo-us] replicas: 1 selector: [name: mongo-node] Volume name: mongo-persistent-storage pdName: mongodb-disk-us mongodb- disk-us Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-us labels: [name: mongo-svc-us] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node]
- 43. When to use Microservices
- 44. #MDBW16 When to use Microservices
- 45. Use Cases
- 46. #MDBW16 MongoDB & Microservices in the Wild
- 47. #MDBW16 References • Enabling Microservices – Containers & Orchestration Explained https://www.mongodb.com/collateral/microservices-containers-and- orchestration-explained • Microservices: The Evolution of Building Modern Applications https://www.mongodb.com/collateral/microservices-the-evolution-of- building-modern-applications