Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Powering Microservices
with Docker, Kubernetes
& Kafka
Andrew Morgan
MongoDB Product Marketing
@andrewmorgan
#MDBW16
Agenda
What, Why, How?
Microservices
01 Kubernetes, Mesos,…
Orchestration
03Docker
Kafka
Containers
02
Why, How?
M...
Microservices
#MDBW16
Why Use Microservices? (tl;dr WebScale)
Team =
Components
No committees
Empower
Build MVP quickly
Speed
Simple com...
#MDBW16
1990s & Earlier
Monolithic
Cross-Team
Tight Coupling
Small
Change
Huge
Impact
System
Retests
#MDBW16
2000s
SOA
Coordination
Looser Coupling
System Retests
#MDBW16
2010s
Microservices
Decoupled
Independent Dev
Isolated Impact
#MDBW16
Microservices Example
Twitter
Ingest
Google
+
Ingest
Snapch
at
Ingest
Feed
merge
Facebo
ok
Ingest
#MDBW16
Microservices Example
Twitter
Ingest
Snapch
at
Ingest
Feed
merge
Facebo
ok
Ingest
#MDBW16
Microservices Example
Twitter
Ingest
Snapch
at
Ingest
Feed
merge
Facebo
ok
Ingest
#MDBW16
Microservices Example
Twitter
Ingest
Snapch
at
Ingest
Feed
merge
Facebo
ok
Ingest
Whatsa
pp
Ingest
#MDBW16
Microservices Example
Twitter
Ingest
Snapch
at
Ingest
Feed
merge
Facebo
ok
Ingest
Whatsa
pp
Ingest
Snapch
at
Inges...
#MDBW16
Alternate Microservices Example
• Much larger Microservices
1. User Account
2. Product Catalog
3. Inventory
4. Ord...
Conway’s Law
1967
Any organization that designs a system will
inevitably produce a design whose structure
is a copy of the...
#MDBW16
Development Teams
Containers
#MDBW16
Containers – Powering Microservices
Real world shipping containers
• Road, rail & sea
• Contents untouched
• Ubiqu...
#MDBW16
Containers – Powering Microservices
Software containers
• 1 image -> Many containers
• Laptop, DC, cloud
• Dev, QA...
#MDBW16
VMs vs. Containers
VM VMVM
Bare Metal
Host Operating System
Hypervisor
Guest OS
Libraries
Apps
Service
Guest OS
Li...
#MDBW16
VMs vs. Containers
VM VMVM
Bare Metal
Host Operating System
Hypervisor
Guest OS
Libraries
Apps
Service
Guest OS
Li...
#MDBW16
VMs vs. Containers
VM VMVM
Bare Metal
Host Operating System
Hypervisor
Guest OS
Libraries
Apps
Service
Guest OS
Li...
#MDBW16
Docker
• Simple to use
• 100K+ images on Docker Hub
• Build images from images
• Platforms
• Linux, OS X, Windows
...
#MDBW16
Run MongoDB
docker run -d
mongo
#MDBW16
Security
Opportunities
• Isolation
• Constrain resources
• Narrow roles
• Keep images/containers
current
• root: s...
#MDBW16
Title only
#MDBW16
Title only
#MDBW16
Microservice Architectures Built on Containers
Many small, focused containers -
> sophisticated services
• Well de...
#MDBW16
Connecting the Microservices – Apache Kafka
Producer
9
8
7
123...
Topic A
Consumer
#MDBW16
Connecting the Microservices – Apache Kafka
Producer
9
8
7
123...
Topic A
Consumer
Producer Consumer
#MDBW16
Connecting the Microservices – Apache Kafka
Producer
9
8
7
123...
Partition 0
Topic A
Consumer
Producer Consumer
4...
#MDBW16
Connecting the Microservices – Apache Kafka
Producer
LEADER
Topic A / Partition 0
Broker 1
FOLLOWER
Topic A / Part...
#MDBW16
Connecting the Microservices – Apache Kafka
Producer
Producer
Producer
9
8
7
123
...
Partition 0
4
3
5
123
...
Par...
Orchestration
#MDBW16
Orchestration
Automated deployment,
connecting, and maintenance of
multiple containers
• Provision hosts
• Contain...
#MDBW16
Kubernetes
Created by Google, feature-rich and
widely adopted
• Deployment and ‘replication’
• On-line scale out/i...
#MDBW16
Apache Mesos
10,000s of physical servers;
used by Twitter, Airbnb & Apple
• Code (“frameworks) vs.
declarative
• L...
#MDBW16
Choosing an Orchestration Framework
• What you have:
• Skills?
• DevOps frameworks?
• Number of hosts?
• Bare meta...
MongoDB
#MDBW16
Why MongoDB is a Good Fit For
Microservices
Scalability
Monitoring
&
Automation
Redundancy
Flexible
Data
Model
Sim...
#MDBW16
Orchestrating MongoDB Using Kubernetes
Distributed, stateful application
• Persistent volumes
• External IP addres...
Volume
name:
mongo-persistent-storage1
pdName: mongodb-disk1
mongodb
-disk1
Container
name: mongo-node1
image: mongo
comma...
ReplicationControler
name: mongo-rc-europe
labels: [name: mongo-europe]
replicas: 1
selector: [name: mongo -node]
Volume
n...
When to use Microservices
#MDBW16
When to use Microservices
Use Cases
#MDBW16
MongoDB & Microservices in the Wild
#MDBW16
References
• Enabling Microservices – Containers & Orchestration Explained
https://www.mongodb.com/collateral/micr...
Powering Microservices with MongoDB, Docker, Kubernetes & Kafka – MongoDB Europe 2016
Upcoming SlideShare
Loading in …5
×

Powering Microservices with MongoDB, Docker, Kubernetes & Kafka – MongoDB Europe 2016

7,552 views

Published on

Organisations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver.

Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Need an identical copy of your application stack in multiple environments? Build your own container image and then your entire development, test, operations, and support teams can launch an identical clone environment.

Containers are revolutionizing the entire software lifecycle: from the earliest technical experiments and proofs of concept through development, test, deployment, and support. Orchestration tools manage how multiple containers are created, upgraded and made highly available. Orchestration also controls how containers are connected to build sophisticated applications from multiple, microservice containers.

This presentation introduces you to technologies such as Docker, Kubernetes & Kafka which are driving the microservices revolution. Learn about containers and orchestration – and most importantly how to exploit them for stateful services such as MongoDB.

Published in: Software
  • How to Get Automated winning picks for NFL, NCAA and MLB? ➤➤ http://ishbv.com/zcodesys/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Powering Microservices with MongoDB, Docker, Kubernetes & Kafka – MongoDB Europe 2016

  1. 1. Powering Microservices with Docker, Kubernetes & Kafka Andrew Morgan MongoDB Product Marketing @andrewmorgan
  2. 2. #MDBW16 Agenda What, Why, How? Microservices 01 Kubernetes, Mesos,… Orchestration 03Docker Kafka Containers 02 Why, How? MongoDB 04 Who, Why? Use Cases 06When to Use 05
  3. 3. Microservices
  4. 4. #MDBW16 Why Use Microservices? (tl;dr WebScale) Team = Components No committees Empower Build MVP quickly Speed Simple components Isolated impact Maintain Rapid iterations React to market Change Product Team Scale
  5. 5. #MDBW16 1990s & Earlier Monolithic Cross-Team Tight Coupling Small Change Huge Impact System Retests
  6. 6. #MDBW16 2000s SOA Coordination Looser Coupling System Retests
  7. 7. #MDBW16 2010s Microservices Decoupled Independent Dev Isolated Impact
  8. 8. #MDBW16 Microservices Example Twitter Ingest Google + Ingest Snapch at Ingest Feed merge Facebo ok Ingest
  9. 9. #MDBW16 Microservices Example Twitter Ingest Snapch at Ingest Feed merge Facebo ok Ingest
  10. 10. #MDBW16 Microservices Example Twitter Ingest Snapch at Ingest Feed merge Facebo ok Ingest
  11. 11. #MDBW16 Microservices Example Twitter Ingest Snapch at Ingest Feed merge Facebo ok Ingest Whatsa pp Ingest
  12. 12. #MDBW16 Microservices Example Twitter Ingest Snapch at Ingest Feed merge Facebo ok Ingest Whatsa pp Ingest Snapch at Ingest Snapch at Ingest
  13. 13. #MDBW16 Alternate Microservices Example • Much larger Microservices 1. User Account 2. Product Catalog 3. Inventory 4. Orders • Best practice is each microservice has its own database
  14. 14. Conway’s Law 1967 Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure.”
  15. 15. #MDBW16 Development Teams
  16. 16. Containers
  17. 17. #MDBW16 Containers – Powering Microservices Real world shipping containers • Road, rail & sea • Contents untouched • Ubiquitous & standardized • Simple • Contents protected • Constraints
  18. 18. #MDBW16 Containers – Powering Microservices Software containers • 1 image -> Many containers • Laptop, DC, cloud • Dev, QA, production, support • Simple, efficient • Isolation • Constraints
  19. 19. #MDBW16 VMs vs. Containers VM VMVM Bare Metal Host Operating System Hypervisor Guest OS Libraries Apps Service Guest OS Libraries Apps Service Guest OS Libraries Apps Service Container ContainerContainer Bare Metal Host Operating System Docker Engine Libraries Libraries Apps Libraries Apps Service ServiceService
  20. 20. #MDBW16 VMs vs. Containers VM VMVM Bare Metal Host Operating System Hypervisor Guest OS Libraries Apps Service Guest OS Libraries Apps Service Guest OS Libraries Apps Service Container ContainerContainer Bare Metal Host Operating System Docker Engine Libraries Libraries Apps Libraries Apps Service ServiceService
  21. 21. #MDBW16 VMs vs. Containers VM VMVM Bare Metal Host Operating System Hypervisor Guest OS Libraries Apps Service Guest OS Libraries Apps Service Guest OS Libraries Apps Service Container ContainerContainer Bare Metal Host Operating System Docker Engine Libraries Libraries Apps Libraries Apps Service ServiceService
  22. 22. #MDBW16 Docker • Simple to use • 100K+ images on Docker Hub • Build images from images • Platforms • Linux, OS X, Windows • Laptop, VM, Cloud,… • Cloud services
  23. 23. #MDBW16 Run MongoDB docker run -d mongo
  24. 24. #MDBW16 Security Opportunities • Isolation • Constrain resources • Narrow roles • Keep images/containers current • root: so restrict access
  25. 25. #MDBW16 Title only
  26. 26. #MDBW16 Title only
  27. 27. #MDBW16 Microservice Architectures Built on Containers Many small, focused containers - > sophisticated services • Well defined APIs • Independent languages & libraries • Modular: easy maintenance + reuse • Fault tolerant • Scalable
  28. 28. #MDBW16 Connecting the Microservices – Apache Kafka Producer 9 8 7 123... Topic A Consumer
  29. 29. #MDBW16 Connecting the Microservices – Apache Kafka Producer 9 8 7 123... Topic A Consumer Producer Consumer
  30. 30. #MDBW16 Connecting the Microservices – Apache Kafka Producer 9 8 7 123... Partition 0 Topic A Consumer Producer Consumer 4 3 5 123... Partition 1
  31. 31. #MDBW16 Connecting the Microservices – Apache Kafka Producer LEADER Topic A / Partition 0 Broker 1 FOLLOWER Topic A / Partition 1 FOLLOWER Topic A / Partition 0 Broker 2 LEADER Topic A / Partition 1
  32. 32. #MDBW16 Connecting the Microservices – Apache Kafka Producer Producer Producer 9 8 7 123 ... Partition 0 4 3 5 123 ... Partition 1 7 3 2 123 ... Partition N Topic A Topic B 7 6 5 123 ... Partition 0 New  Old Consumer Consumer
  33. 33. Orchestration
  34. 34. #MDBW16 Orchestration Automated deployment, connecting, and maintenance of multiple containers • Provision hosts • Containers • Instantiate • Reschedule • Link • Scale Out/In • Expose services
  35. 35. #MDBW16 Kubernetes Created by Google, feature-rich and widely adopted • Deployment and ‘replication’ • On-line scale out/in • Rolling upgrades • High Availability • Persistence • Ports • Load balancing • Google Compute Engine
  36. 36. #MDBW16 Apache Mesos 10,000s of physical servers; used by Twitter, Airbnb & Apple • Code (“frameworks) vs. declarative • Less feature rich than Kubernetes • Kubernetes as a Mesos framework • Foundation for distributed
  37. 37. #MDBW16 Choosing an Orchestration Framework • What you have: • Skills? • DevOps frameworks? • Number of hosts? • Bare metal, VMs, or cloud? • Lifecycle • Features • Automated High Availability? • Grouping and load balancing? • As a service??
  38. 38. MongoDB
  39. 39. #MDBW16 Why MongoDB is a Good Fit For Microservices Scalability Monitoring & Automation Redundancy Flexible Data Model Simplicity
  40. 40. #MDBW16 Orchestrating MongoDB Using Kubernetes Distributed, stateful application • Persistent volumes • External IP addresses for internal comms • Init MongoDB replica set • Monitor • Backup
  41. 41. Volume name: mongo-persistent-storage1 pdName: mongodb-disk1 mongodb -disk1 Container name: mongo-node1 image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage1 mountPath: /data/db Docker Hub Registry Pod labels: [name: mongo-node1; instance: rod] ReplicationController name: mongo-rc1 labels: [name: mongo-rc] replicas: 1 selector: [name: mongo -node1] Service: LoadBalancer name: mongo-svc-a labels: [name: mongo-svc-a] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node, instance: rod] ExternalIP Address 104.1.1.1 App 104.1.1.1:27017
  42. 42. ReplicationControler name: mongo-rc-europe labels: [name: mongo-europe] replicas: 1 selector: [name: mongo -node] Volume name: mongo-persistent-storage pdName: mongodb-disk-europe mongodb- disk-europe Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-europe labels: [name: mongo-svc-europe] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node] ReplicationControler name: mongo-rc-asia labels: [name: mongo-asia] replicas: 1 selector: [name: mongo -node] Volume name: mongo-persistent-storage pdName: mongodb-disk-asia mongodb- disk-asia Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-asia labels: [name: mongo-svc-asia] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node] ReplicationControler name: mongo-rc-us labels: [name: mongo-us] replicas: 1 selector: [name: mongo -node] Volume name: mongo-persistent-storage pdName: mongodb-disk-us mongodb- disk-us Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-us labels: [name: mongo-svc-us] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node]
  43. 43. When to use Microservices
  44. 44. #MDBW16 When to use Microservices
  45. 45. Use Cases
  46. 46. #MDBW16 MongoDB & Microservices in the Wild
  47. 47. #MDBW16 References • Enabling Microservices – Containers & Orchestration Explained https://www.mongodb.com/collateral/microservices-containers-and- orchestration-explained • Microservices: The Evolution of Building Modern Applications https://www.mongodb.com/collateral/microservices-the-evolution-of-building- modern-applications • Data Streaming with Apache Kafka & MongoDB https://www.mongodb.com/collateral/data-streaming-with-apache-kafka-and- mongodb

×