Deploying & operating microservices

DEPLOYING AND
OPERATING
MICROSERVICES
Sam Newman
ThoughtWorks London 2015
1

@samnewman#thoughtworks
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS

Accounts
Returns
Invoicing
Shipping
Inventory
Customer
Service

Accounts
Returns
Invoicing
Shipping
Inventory
Customer
Service
Small Autonomous services
that work together, modelled
around a business domain

Independently
Releasable

Building

Building
Deploying

Building
Deploying
Managing

Hands up if you’re doing Continuous Integration!

Hands up if you’re doing Continuous Integration!
Now keep them up if…

You check in to mainline at least once per day

You have a suite of tests to validate your changes

When the build is broken, it is your
team’s #1 priority to ﬁx it

Single Repo
ONE GIANT BUILD
Version Control
/Service A
/Service B
/Service C

Single Repo
ONE GIANT BUILD
Version Control
/Service A
/Service B
/Service C
CI Tool
Monolithic
Build

Single Repo
ONE GIANT BUILD
Version Control
/Service A
/Service B
/Service C
CI Tool
Monolithic
Build
Service B
v123
Service A
v123
Service C
v123

Single Repo
ONE GIANT BUILD
Version Control
/Service A
/Service B
/Service C
CI Tool
Monolithic
Build
Service B
v123
Service A
v123
Service C
v123
One checkin = lots of services built

Single Repo
ONE GIANT BUILD
Version Control
/Service A
/Service B
/Service C
CI Tool
Monolithic
Build
Service B
v123
Service A
v123
Service C
v123
One checkin = lots of services built
Can’t independently release in a safe manner

Single Repo
ONE BUILD PER SERVICE, SINGLE REPO
Version Control
CI Tool
/Service A
/Service B
/Service C

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A
Service A
v123

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A
Service A
v123
Service B

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A
Service A
v123
Service B
v456
Service B

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A
Service A
v123
Service B
v456
Service B
Service C

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A
Service A
v123
Service B
v456
Service C
v789
Service B
Service C

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A
Service A
v123
Service B
v456
Service C
v789
Can build & release independently
Service B
Service C

Single Repo
Version Control
CI Tool
/Service A
/Service B
/Service C
Service A
Service A
v123
Service B
v456
Service C
v789
Can encourage making cross-repo changes
Service B
Service C

ONE BUILD & REPO PER SERVICE
Version Control CI Tool
Service A
Service B
v123
Service A
v123
Service C
v123
Service B
Service C
/Service A
/Service B
/Service C

Service A
Service B
v123
Service A
v123
Service C
v123
Service B
Service C
/Service A
/Service B
/Service C

Service A
Service B
v123
Service A
v123
Service C
v123
Firmer separation, but developer workﬂow can suﬀer
Service B
Service C
/Service A
/Service B
/Service C

Customer Service

Customer Service
S/M TestsBuild Large Tests

S/M TestsBuild Large Tests UAT ProdLarge Tests
Artifacts should be built once, and moved through
environments

More Production Like
environments

Faster Feedback
More Production Like
environments

SO, ARTIFACTS….

SO, ARTIFACTS….
▫︎Bundles-o-stuﬀ

SO, ARTIFACTS….
▫︎Tech-stack speciﬁc

SO, ARTIFACTS….
▫︎OS-speciﬁc

SO, ARTIFACTS….
▫︎OS-speciﬁc
▫︎Custom Images

SO, ARTIFACTS….
▫︎OS-speciﬁc
▫︎Custom Images
▫︎Containers

Deploying

@samnewman#thoughtworks 18
Host
Service
Which is best?

Host
Service
Host
Service Service
Service Service
VS
Which is best?

Application Containers
OSGI
Actor-based Systems

Host
Service Service
Service Service
Application Containers
OSGI
Actor-based Systems

Host
Service
Linux Containers/Docker
Custom Images

Host
Service

Host
Service
Host
Service Service
Service Service
VS

Host
Service
Host
Service Service
Service Service
Much Easier To Reason About
VS

Host
Service
Host
Service Service
Service Service
Easier To Provision
VS

Host
Service
Host
Service Service
Service Service
Easier To Provision
Fewer Side-eﬀects
VS

Host
Service
Host
Service Service
Service Service
Easier To Provision
Fewer Side-eﬀects
Lower host-management overhead
VS

Host
Service
Host
Service Service
Service Service
Easier To Provision
Fewer Side-eﬀects
Cheaper!
VS

Host
Service
Host
Service Service
Service Service
Easier To Provision
Fewer Side-eﬀects
Cheaper!
VS
Enforces Interdependence

Reducing Host
Management Overhead?

API-driven Host
Provisioning

VMWare
Virtualisation Platforms

VMWare
AWS
Digital Ocean
Virtualisation Platforms

Step 0:
Provision Host

Step 0:
Provision Host
Step 1:
Conﬁgure Host

Step 0:
Provision Host
Step 2:
Install Service
Step 1:
Conﬁgure Host

Ansible
Puppet
Chef

Ansible
Puppet
Chef
AWS

Ansible
Puppet
Chef
AWS
Digital Ocean

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack
VMWare

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack
VMWare
Vagrant

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack
VMWare
Vagrant
Immutable Servers

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack
VMWare
Vagrant
Immutable Servers
Fast Spin-up

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack
VMWare
Vagrant
Immutable Servers
Fast Spin-up
Provider Agnostic

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack
VMWare
Vagrant
Immutable Servers
Fast Spin-up
Provider Agnostic
Feedback Can Suﬀer

Ansible
Puppet
Chef
AWS
Digital Ocean
OpenStack
VMWare
Vagrant
Immutable Servers
Fast Spin-up
Provider Agnostic
Feedback Can Suﬀer
Cycle Time

Immutable Servers

Machine
Base OS
Hypervisor
VM
OS
Apps
VM
OS
Apps
VM
OS
Apps

Machine
Base OS
Hypervisor
VM
OS
Apps
VM
OS
Apps
VM
OS
Apps
Expensive!

DOCKER

DOCKER
Machine

DOCKER
Machine
Base OS

DOCKER
Machine
Base OS
Docker

DOCKER
Machine
Base OS
Docker
Apps

DOCKER
Machine
Base OS
Docker
Apps Apps

DOCKER
Machine
Base OS
Docker
Apps Apps Apps

DOCKER
Machine
Base OS
Docker
Apps Apps Apps
Docker Image Registry

Semi-permanent Servers

Image-based Deployments

Image-based Deployments
Docker

S/M TestsBuild Large Tests UAT Prod

DB
Machine CI Node
Large Tests Environment
Large Tests

DB
MachineMachine

Master
DB
Machine
Production Environment
Machine Machine Machine
Slave
DB

ENVIRONMENTS

ENVIRONMENTS
▫︎Artifact remains the same…

ENVIRONMENTS
▫︎But the environment diﬀers

ENVIRONMENTS
▫︎But the environment diﬀers
▫︎How can we handle this?

Environment Deﬁnition

http://www.ﬁg.sh/

https://www.terraform.io/

Operating

Shop
How Do We Find Things?
??? Inventory

HARDCODE IP!
Shop Inventory
inventory: 10.2.4.52

DNS
Shop Inventory
inventory: inventory.mycompany.com
DNS Service

DNS
Shop Inventory
DNS Service
Conﬁguring DNS?

DNS
Shop Inventory
DNS Service
Conﬁguring DNS?
DNS propagation time?

SERVICE DISCOVERY TOOL
Shop
Service
Discovery

Shop Inventory
Service
Discovery

Shop Inventory
Service
Discovery
I’m here!

Shop Inventory
Service
Discovery
inventory:
10.1.1.10
I’m here!

Shop
Service
Discovery
inventory:
10.1.1.10
I’m here!

Shop
Service
Discovery
inventory:
10.1.1.10
Inventory
I’m here!

Shop
Service
Discovery
Inventory
inventory:
10.1.1.25
I’m here!

DNS SRV

DNS SRV REST API

Service Discovery

Service Discovery Conﬁguration

Watches

Consul-templateWatches

Beware of writing your own service discovery system

Horizontal Duplication

Data Partitioning

Data Partitioning
Functional Decomposition

Functional decomposition = splitting microservices!

HORIZONTAL DUPLICATION - LOAD BALANCING

Client

Load Balancing
Client

Load Balancing
Client
Server or client load balancing

Load Balancing
Client
Distribution algorithms can vary

Load Balancing
Client
Distribution algorithms can vary
Software or
hardware based

HORIZONTAL DUPLICATION - COMPETING CONSUMERS

Customer
Service
create

Customer
Service
create
customer created

Customer
Service
create
customer created
Requires shared state

Customer
Service
create
customer created
Requires shared state
Lots of variations on this
‘scatter gather’ approach

DATA PARTITIONING - SHARDING BY KEY
Customer Service Nodes

A-E

A-E F-M

A-E F-M N-R

A-E F-M N-R S-Z

A-E F-M N-R S-Z
Load Balancing
Client

A-E F-M N-R S-Z
Getting shard keys right is key
- rebalancing is painful!
Load Balancing
Client

A-E F-M N-R S-Z
Getting shard keys right is key
- rebalancing is painful!
Can combine with other
scaling (e.g. load balancing)
Load Balancing
Client

AVOID THE DISTRIBUTED SINGLE POINT OF FAILURE!

Strangler App

Strangler App
Legacy App
Legacy AppLegacy App

Strangler App
Legacy App
Legacy App
Requests
Legacy App

Strangler App
Legacy App
Legacy App
Requests
Legacy App
30 - 60 Concurrent Requests

Strangler App
Legacy App
Legacy App
Requests
Legacy App
30 - 60 Concurrent Requests
> 800 Concurrent Requests

Strangler App
Legacy App Legacy App Legacy App
Thread Pool

Strangler App
Thread Pool
Failing…slowly!

Strangler App
Thread Pool
Failing…slowly!
Thread-pool
exhausted

Strangler App
Thread Pool
Failing…slowly!
Thread-pool
exhausted
No requests to other
downstream apps

Strangler App
Thread Pool
Failing…slowly!
Thread-pool
exhausted
Requests
Building Up
No requests to other
downstream apps

Strangler App
Fix Timeouts
Thread Pool

Strangler App
Fix Timeouts
Thread Pool Thread PoolThread Pool

Strangler App
Fix Timeouts
Bulkhead
Downstream
Connections

Strangler App
Fix Timeouts
Bulkhead
Downstream
Connections
Circuit Breakers

http://www.ﬂickr.com/photos/kalexanderson/5421517469/

You have to get *much*
better at monitoring

You are not a badass if you
use an SSH Multiplexer

AGGREGATE METRICS

Response Time
Response Time
Response Time
AGGREGATE METRICS

HEALTH CHECK PAGES

SYNTHETIC TRANSACTIONS

@samnewman#thoughtworkshttps://www.ﬂickr.com/photos/yercombe/9033732392/

CORRELATION IDS

ID: 123
CORRELATION IDS

ID: 123
ID: 123
ID: 123
CORRELATION IDS

Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
http://buildingmicroservices.com/
AUTHD

http://lanyrd.com/proﬁle/samnewman/

Deploying & operating microservices

More Related Content

What's hot

Viewers also liked

Similar to Deploying & operating microservices

More from Thoughtworks

Recently uploaded

Deploying & operating microservices