Continuous Delivery at Docker Age
A DevOps story
Adrien Blind
DevOps coach @ Société Générale
Docker Paris Meetup co-manager
adrien.blind@sgcib.com
@adrienblind
The continuous delivery challenge
Promote an agile & automated approach up to production
in order to enhance time to market & quality of delivered products
 New challenges to face (non exhaustive!)
 Reconciliate app & infrastructure lifecycle : think « product »
 Raise app teams autonomy
 ... While raising the need for interactions with Ops...
 New solution elements emerge at different levels
 Organization: DevOps culture, feature-teams...
 Applicative architecture : micro-services, loose-coupling, stateless, versionned APIs…
 Infrastructure : more & more rich cloud services, infrastructure-as-code practice & tools…
Code
developed
Unit tests Integration
Acceptance
tests
Push to
production
Execution
Docker as a universal product artifact
@adrienblind
The container paradigm
 Docker provides both containers and an ecosystem enabling to handle them
Immutable Versionned
Light
Portable
Disposable
Programmatic
SocialIncremental
« A universal, self-sufficient, standard artifact, embedding an app. module
and its subsequent infrastructure configuration »
Docker is not virtualization!
IsolationVirtualisation
VM VMVM
Hypervisor
Hardware
App.
MDW
App.
MDW
App.
MDW
Kernel Kernel Kernel
OS
base
OS
base
OS
base
Kernel
Hardware or VM
Container
App.
OS
base
MDW
Container
App.
OS
base
MDW
Container
App.
MDW
OS
base
Docker as a product artifact
I hope you don’t deliver single class files
on an app server, do you?
Create immutable objects:
Rebuild vs Upgrade
Full-stack versioning
Merge app & infra conf. lifecycles
System conf v.5.79
Middleware v.69.3
App code v1.2.3
Docker image
 Product version = app version + infra version
 Whenever you change a single line of code
or a system lib., you build up a new artifact
 The dockerfile is part of the app code repo
Disposable, short term-living containers
 No persistent data inside app containers
 Object storage: OpenStack Swift, Ceph, Amazon S3…
 Consider use of Docker Volumes if you need to manage
binaries on app container filesystem
 Handle configuration at run time
App
Container
Volume
Redistributing DevOps cards
@adrienblind
Docker is not only about technical stuff
« Organizations which design systems... are constrained to produce designs
which are copies of the communication structures of these organizations ».
- M. Conway, 1968
« Organize your teams appropriatly to achieve continuous delivery »
Reorganize roles & reponsibilities
DevOps Apps team
In the container
 Ignore where the container run (dev
computer, prod…)
 Knows how to build the container and
operate application
“You build it, you run it!”
DevOps Infra team
Out of the container
 Ignore how the container images are
built
 Knows how to deal with huge amount
of containers
Infra team to deliver a CaaS product
IaaSCapacity (VM, Storage, Netwk…)
PaaSApp. (code)
CaaSContainer
Infra team to propose basic set of images
RHEL 7.2 (INFRA/system)
Tomcat 8 (INFRA/middleware)
APPx.y (APP product X)
 Infra team may propose valuable, ready to use images in the registry
 Which could serve as base image for apps team (Ex. Tomcat middleware)
 ... or which could be directly used (ex. MySQL database, REDIS instances…)
 Apps teams build their product images leveraging on underlying INFRA base images
Devs & Ops to share the same vocabulary & ecosystem :
Docker paradigms, commands and dockerfiles
Deliver continuously Docker artifacts
@adrienblind
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryDev
At the beginning, the registry contains Infrastructure base image catalog
 App team will leverage on it, to avoid reinventing the complete wheel
CaaS Platform
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryDev
Local use on developer workstation using Docker toolbox
 Reduced workstation setup time, more representative environment
CaaS Platform
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryDev
Developer commits his code to the CVS
 It may be app code or dockerfile code
 The platform auto detects the operation (ex. Webhook in github)
 It triggers a new pipeline in the software factory and starts building app
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryDev
The platform now generates the new app binary
 Evaluate code, build app, pass unit tests…
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryDev
The platform now builds the new, versioned docker image enclosing app
 The platform first pull subsequent docker image from the registry
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryDev
The platform now builds the new, versioned docker image enclosing app
 The platform first pull subsequent docker image from the registry
 It builds the docker image itself
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryDev
The platform now builds the new, versioned docker image enclosing app
 The platform first pull subsequent docker image from the registry
 It builds the docker image itself
 Push new version into registry, used as an app artifact repo (ie. Nexus)
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryApp
The platform deploys the app in DEV environment
 Achieved automatically without human interaction
 Devs now have a still-up-to-date env
 Technically image is pulled from the registry
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryApp
The platform deploys the app in UAT environment
 Triggered by human interaction
One artifact to rule them all! Do not rebuild per env, as you do for apps
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Docker continuous delivery pipeline
001101010011010110
110101111101110101
111010011
Registry
CVS+
Soft. factoryApp
The platform deploys the app in PROD environment
 Triggered by human interaction
One artifact to rule them all! Do not rebuild per env, as you do for apps
CaaS Platform
Build product
Deploy DEV
Deploy UAT
Deploy PRD
Click here to trigger next step
Notice that docker
tag contains git
hash
 Version N is in DEV
 Version N-1 is in UAT
 Version N-2 is in PROD
Tools used for the lab
 Docker toolbox, engine, registry
 Github for code repository
 Jenkins
 Delivery pipeline plugin
 Cloudbees plugin to drive Docker interactions
 Cloud to host containers (also used an
alternate local swarm cluster)
From containers to topologies
@adrienblind
Orchestration is the key
Application
Compute (Run containers)
Storage (Volumes) Transport (Network)
Topology
(Compose)
 Docker shifted from containers to object-oriented infra. topologies
CaaS platform
Clustering (Swarm, UCP)
Catalog
(Registry, DTR)
Hosting (Engine)
Provisioning (Machine)
... relying on an execution platform 
Conclusion
@adrienblind
Conclusion
Cloud DevOps
CaaS
PaaS
Continuous
Delivery
Architecture
Organization
Time to
market
Quality €
Opportunities
Thank you!
Adrien Blind
Coach DevOps @ Société Générale
adrien.blind@sgcib.com
@adrienblind

Continous delivery at docker age

  • 1.
    Continuous Delivery atDocker Age A DevOps story Adrien Blind DevOps coach @ Société Générale Docker Paris Meetup co-manager adrien.blind@sgcib.com @adrienblind
  • 2.
    The continuous deliverychallenge Promote an agile & automated approach up to production in order to enhance time to market & quality of delivered products  New challenges to face (non exhaustive!)  Reconciliate app & infrastructure lifecycle : think « product »  Raise app teams autonomy  ... While raising the need for interactions with Ops...  New solution elements emerge at different levels  Organization: DevOps culture, feature-teams...  Applicative architecture : micro-services, loose-coupling, stateless, versionned APIs…  Infrastructure : more & more rich cloud services, infrastructure-as-code practice & tools… Code developed Unit tests Integration Acceptance tests Push to production Execution
  • 3.
    Docker as auniversal product artifact @adrienblind
  • 4.
    The container paradigm Docker provides both containers and an ecosystem enabling to handle them Immutable Versionned Light Portable Disposable Programmatic SocialIncremental « A universal, self-sufficient, standard artifact, embedding an app. module and its subsequent infrastructure configuration »
  • 5.
    Docker is notvirtualization! IsolationVirtualisation VM VMVM Hypervisor Hardware App. MDW App. MDW App. MDW Kernel Kernel Kernel OS base OS base OS base Kernel Hardware or VM Container App. OS base MDW Container App. OS base MDW Container App. MDW OS base
  • 6.
    Docker as aproduct artifact I hope you don’t deliver single class files on an app server, do you? Create immutable objects: Rebuild vs Upgrade
  • 7.
    Full-stack versioning Merge app& infra conf. lifecycles System conf v.5.79 Middleware v.69.3 App code v1.2.3 Docker image  Product version = app version + infra version  Whenever you change a single line of code or a system lib., you build up a new artifact  The dockerfile is part of the app code repo
  • 8.
    Disposable, short term-livingcontainers  No persistent data inside app containers  Object storage: OpenStack Swift, Ceph, Amazon S3…  Consider use of Docker Volumes if you need to manage binaries on app container filesystem  Handle configuration at run time App Container Volume
  • 9.
  • 10.
    Docker is notonly about technical stuff « Organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations ». - M. Conway, 1968 « Organize your teams appropriatly to achieve continuous delivery »
  • 11.
    Reorganize roles &reponsibilities DevOps Apps team In the container  Ignore where the container run (dev computer, prod…)  Knows how to build the container and operate application “You build it, you run it!” DevOps Infra team Out of the container  Ignore how the container images are built  Knows how to deal with huge amount of containers
  • 12.
    Infra team todeliver a CaaS product IaaSCapacity (VM, Storage, Netwk…) PaaSApp. (code) CaaSContainer
  • 13.
    Infra team topropose basic set of images RHEL 7.2 (INFRA/system) Tomcat 8 (INFRA/middleware) APPx.y (APP product X)  Infra team may propose valuable, ready to use images in the registry  Which could serve as base image for apps team (Ex. Tomcat middleware)  ... or which could be directly used (ex. MySQL database, REDIS instances…)  Apps teams build their product images leveraging on underlying INFRA base images Devs & Ops to share the same vocabulary & ecosystem : Docker paradigms, commands and dockerfiles
  • 14.
    Deliver continuously Dockerartifacts @adrienblind
  • 15.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryDev At the beginning, the registry contains Infrastructure base image catalog  App team will leverage on it, to avoid reinventing the complete wheel CaaS Platform
  • 16.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryDev Local use on developer workstation using Docker toolbox  Reduced workstation setup time, more representative environment CaaS Platform
  • 17.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryDev Developer commits his code to the CVS  It may be app code or dockerfile code  The platform auto detects the operation (ex. Webhook in github)  It triggers a new pipeline in the software factory and starts building app CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD Build product Deploy DEV Deploy UAT Deploy PRD
  • 18.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryDev The platform now generates the new app binary  Evaluate code, build app, pass unit tests… CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD Build product Deploy DEV Deploy UAT Deploy PRD
  • 19.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryDev The platform now builds the new, versioned docker image enclosing app  The platform first pull subsequent docker image from the registry CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD Build product Deploy DEV Deploy UAT Deploy PRD
  • 20.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryDev The platform now builds the new, versioned docker image enclosing app  The platform first pull subsequent docker image from the registry  It builds the docker image itself CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD Build product Deploy DEV Deploy UAT Deploy PRD
  • 21.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryDev The platform now builds the new, versioned docker image enclosing app  The platform first pull subsequent docker image from the registry  It builds the docker image itself  Push new version into registry, used as an app artifact repo (ie. Nexus) CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD Build product Deploy DEV Deploy UAT Deploy PRD
  • 22.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryApp The platform deploys the app in DEV environment  Achieved automatically without human interaction  Devs now have a still-up-to-date env  Technically image is pulled from the registry CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD
  • 23.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryApp The platform deploys the app in UAT environment  Triggered by human interaction One artifact to rule them all! Do not rebuild per env, as you do for apps CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD
  • 24.
    Docker continuous deliverypipeline 001101010011010110 110101111101110101 111010011 Registry CVS+ Soft. factoryApp The platform deploys the app in PROD environment  Triggered by human interaction One artifact to rule them all! Do not rebuild per env, as you do for apps CaaS Platform Build product Deploy DEV Deploy UAT Deploy PRD
  • 25.
    Click here totrigger next step Notice that docker tag contains git hash  Version N is in DEV  Version N-1 is in UAT  Version N-2 is in PROD
  • 26.
    Tools used forthe lab  Docker toolbox, engine, registry  Github for code repository  Jenkins  Delivery pipeline plugin  Cloudbees plugin to drive Docker interactions  Cloud to host containers (also used an alternate local swarm cluster)
  • 27.
    From containers totopologies @adrienblind
  • 28.
    Orchestration is thekey Application Compute (Run containers) Storage (Volumes) Transport (Network) Topology (Compose)  Docker shifted from containers to object-oriented infra. topologies CaaS platform Clustering (Swarm, UCP) Catalog (Registry, DTR) Hosting (Engine) Provisioning (Machine) ... relying on an execution platform 
  • 29.
  • 30.
  • 31.
    Thank you! Adrien Blind CoachDevOps @ Société Générale adrien.blind@sgcib.com @adrienblind

Editor's Notes

  • #4 Like virtual appliance