When Docker Engine 1.12 features unleashes software architecture

voxxeddays.com/luxembourg/ #voxxeddaysLU
When Docker Engine 1.12 features
unleashes software architecture
[Update of Dockercon EU’ 15 presentartion initially built w/ L. Grangeau]
Adrien Blind
@adrienblind

Back on Docker paradigms
‘’A universal, self-sufficient and standard artifact embedding an app module,
and its subsequent infrastructure configuration’’
 It’s mainly focused on enclosing computing
aspects of the app: what about persistence? communication? Topologies?
Immutable
Portable
Lightweight
Incremental
Versionned
Disposable

1 - Starters
Application
architecture shifts
3 - Dessert
Taste-an-app
2 - Main course
Docker networking,
service & volume
features discovered

Application architecture shifts

Security paradigms shifts

Your IT opens up
• Externalization (housing, hosting)
• Cloud (IaaS/PaaS/SaaS)
Open up your IS
• B2B, services exposition
• Multi tenancy
More & more breaches appears in your Great Wall of China!

The necessary porosity of your IS requires to stick security closer to each application:
sandbox your apps and expose protected interfaces (ciphered/auth/authz)!
• Network is now part of application topology
• Security is an app topic, not just infra. Concern
• Onboard security in feature teamSecDevOps

Fine-grained, highly decoupled and
atomic purpose centric services
Multi-versioned
Scalable
Stateless
Share-nothing
Immutable
Continuously
delivered
Distributed
Micro services

From Enterprise Services buses to full-mesh topologies
ESB
Service Service Service
Service Service
>
ServiceService
Service
Service
Service
Micro services

Service
consumer
Service
provider
Registry
2. Find 1. Publish
3. Bind
Leverage on a Service registry to discover
where services are located
Micro services

Resilience & scalability: apps problem now!
Vertical > horizontal
• Apps designed for failure & scalability
• Data to be externalized
• Dumber infrastructure
 Structured: MongoDB, Hadoop, Cassandra, Elastic Search...
 Binaries: object storage with Ceph, OpenStack Swift...
 Helpful patterns: stateless, multi-versioning, loose coupling...
 Infrastructure rationalization
 Low-cost, poor-SLA commodity

« Organizations which design systems... are constrained to
produce designs which are copies of the communication structures
of these organizations ». - M. Conway, 1968
Consider shifting your organization if you
wish to shift your architecture!
• Forget about the central architects myth of
organizing, integrating everything
• Promote feature teams
Organization

Networking, Service & Volume
features discovered
#Networking

Docker networking
The Container Network Model (CNM)
A docker container
Endpoint
A docker container
Endpoint
A docker container
EndpointEndpoint
Network sandbox Network sandbox Network sandbox
Front network Back network

Docker networking
Host Host Host Host
SDN 1 / App 1
SDN 2 / App 2
SDN 3 / App 3

Docker networking
$docker network create --driver overlay app
Bo51qvm381rrf6obchh6uw4w7
$docker network ls
NETWORK ID NAME DRIVER SCOPE
bo51qvm381rr app overlay swarm

Docker networking
$ dockerdocker network inspect app
[
{
"Name": "app",
"Id": "bo51qvm381rrf6obchh6uw4w7",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": []
},
"Internal": false,
"Containers": null,
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "258"
},
"Labels": null
}
]

docker-compose evolved to embrace new networking features:
version: '2'
services:
wordpressapp:
image: wordpress: latest
networks:
- wordpressnet
wordpressdb:
image: mysql:latest
networks:
- wordpressnet
networks:
wordpressnet:
driver: overlay
Docker networking

features discovered
#Service/Task/Stack

Docker service, tasks, stack
Docker engine 1.12 introduces a complete new swarm orchestration
framework
• Built in the engine
• Decentralized
• More secured
• More resilient
$docker node ls
ID NAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS
0cdxzmgi1a[...] m1 Accepted Ready Active Leader
4wz1zlur5c[...] * m3 Accepted Ready Active Reachable
a4v6da1yre[...] m2 Accepted Ready Active Reachable

Docker service
Depicts the desired runtime behavior of a given image : networking, resiliency, quotas...
 shift to state-machine paradigms
$docker service create --name front -–network app –replicas 3 -p 80:80/tcp nginx:latest
$docker service ls
ID NAME REPLICAS IMAGE COMMAND
9gxxdqpauq08 front 3/3 nginx:latest
$docker service scale front=10
Front scaled to 10
Attach the containers to a given network
Define the desired amount of instances for this service (named « tasks »)
Attach each instance to a transversal L4 loadbalancer instance, reachable on each node of the cluster

Docker tasks
The Swarm cluster schedules creation/deletion of tasks (aka
containers) to meet the desired state described in the service
$docker service tasks front
ID NAME SERVICE IMAGE LAST STATE DESIRED NODE
7yu4rgc23[...] front.1 front nginx:latest Running 8 hours Running m3
dj4trimu4[...] front.2 front nginx:latest Running 8 hours Running m2
7rdiv2r2e[...] front.3 front nginx:latest Running 7 hours Running m1
Should a task or a cluster node fails, swarm will fire new tasks to meet
the desired state

Docker stack
docker-compose enabled to trigger a complete containerized
app topology and its network conf., piloted from the client side
docker-compose bundle enables to create an artifact
depicting this topology, which can be consumed server-side
docker stack deploy enables to pop up the desired
topology at runtime. The state-machine behavior of swarm then
ensures this topology to be maintained

Networking, Service & volume
features discovered
#Volume

Docker volumes
Host file system Host file system
‘’Former data management locked in a host’’
• No persistent data inside app containers
• Object storage: OpenStack Swift, Ceph, Amazon
S3…
• Consider use of Docker Volumes if you need to
manage binaries on app container filesystem
• Handle configuration at run time

Docker volumes
Host file system
Container
Volume
‘’Containers mount a volume which may be backed externally’’

Docker volumes
$ docker volume create –d volplugin --name
pool/name
Cf872ca21d27843f6b6319ac1a34390dd38d94ed4649cd9
85456d523fb05d4cc
$ docker run –d –p 8080:8080 –v
pool/name:/var/jenkins_home jenkins
96aec6f4e45e050dfb4f75a1009e7f105bced5b406752e6
2d470615d07348b07

Docker volumes
$ docker volume ls
DRIVER VOLUME NAME
local cf872ca21d27843f6b6319ac1a34390…
local f19f50251f48c64a6b33a5c637c2330…
$ docker volume inspect cf872ca21d27843f6b6319ac1a34390dd38d94…
[
{
"Name": "cf872ca21d27843f6b6319ac1a34390dd38d94…",
"Driver": "local",
"Mountpoint": "/mnt/sda1/var/lib/docker/volumes/[…]/_data"
}
]

features discovered
From infrastructure containers to a complete object-oriented app

Distributed application
Compute (service/task)
Storage (volume) Transport (network)
Topology
(compose, bundle,
deploy, stack)
 Docker shifted from container infra. to object-oriented app. topologies
CaaS platform
Clustering (swarm)
Image mgmt
(registry)
Hosting (node)
Provisioning (machine)
... relying on an CaaS platform 

Taste-an-app

AppConsumers
• The NGINX reverse proxy forward app. requests on one of the python instance registered in Consul
Find
Application design in nov’ 15
Registrator agents makes Consul aware of popped up containers

Application design with Engine 1.12
AppConsumers
Routing layer
Any swarm
node IP
address
loadbalances
to all WP
instances
• Load balancing is now directly achieved at engine/swarm level

3 command lines to heaven
docker network create --driver overlay wordpressnet
docker service create --env MYSQL_ROOT_PASSWORD=plop --
env MYSQL_DATABASE=wordpress --network wordpressnet --
replicas 1 --name wordpressdb mysql:latest
docker service create --env
WORDPRESS_DB_HOST=wordpressdb --env
WORDPRESS_DB_PASSWORD=plop --network wordpressnet --
replicas 4 --name wordpressapp --publish 80:80/tcp
wordpress:latest
... Or even, even better: use docker-compose bundle & deploy through CI

Between apps, consumers may asks a service discovery where a
desired micro-service is located
Docker now exposes multi-instanciated services, leveraging on
IPVS load balancing and internal service discovery
The app. may use internally its own service service discovery to get
more higher control (Java ex.: Spring Cloud stack with Zuul/Eureka)
At infrastructure level, an internal service discovery is used by swarm
Noticed the different usages of a service discovery & name resolution mechanism?
Zoom on various service discovery usages

Conclusion

Conclusion
• Software is eating the world: application architecture is the
key, infrastructure is commodity
• Security is an app concern
• Docker shifted from universal containers to object-
oriented app. architecture

When Docker Engine 1.12 features unleashes software architecture

More Related Content

What's hot

Viewers also liked

Similar to When Docker Engine 1.12 features unleashes software architecture

More from Adrien Blind

Recently uploaded

When Docker Engine 1.12 features unleashes software architecture