Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

'DOCKER' & CLOUD: ENABLERS For DEVOPS

592 views

Published on

Sprekers: Stijn Van den Enden & Stijn Wijndaele (ACA IT-Solutions) DevOps is gericht op het tot stand brengen van een cultuur binnen organisaties waardoor het ontwikkelen, valideren en releasen van software sneller, meer betrouwbaar en frequenter kan verlopen. Om dit te realiseren staan het automatiseren van het 'software delivery process' en de bijhorende infrastructurele veranderingen centraal. Door de opkomst van 'Microservice Architecture' neemt het belang hiervan nog verder toe.

In deze avondconferentie werd, na een korte toelichting over DevOps, nagegaan wat Docker en de Cloud kunnen betekenen voor uw business, en hoe zij als enablers kunnen dienen voor het tot stand brengen van een DevOps-cultuur. Het container-landschap waarvan tools zoals Kubernetes, Docker Swarm, ...een belangrijk onderdeel vormen, wordt toegelicht en er wordt ingegaan op de wijze waarop deze tools aangewend kunnen worden om 'development' en 'operations' efficiënt te laten samenwerken.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

'DOCKER' & CLOUD: ENABLERS For DEVOPS

  1. 1. ACA IT-SOLUTIONS | © 2015 Stijn Wijndaele stijn.wijndaele@aca-it.be @stijnwijndaele Introduction? Your Hosts For Today Business Development Manager Cloud & Mobile Solutions ACA IT-SOLUTIONS | © 2016 Stijn Van den Enden stijn.vandenenden@aca-it.be @stieno CTO
  2. 2. Wants to be the greatest IT service provider by being fanatic about maximising customer value
  3. 3. 'DOCKER' & CLOUD: ENABLERS For DEVOPS
  4. 4. DevOps DevOps is gericht op het tot stand brengen van een cultuur binnen organisaties waardoor het ontwikkelen, valideren en releasen van software sneller, meer betrouwbaar en frequenter kan verlopen
  5. 5. CLOUD ?
  6. 6. Why the Cloud ? • Elastic Capacity • Business Agility • CapEx becomes OpEx • Enabler for DevOps
  7. 7. Why the Cloud ? • Elastic Capacity • Business Agility • CapEx becomes OpEx • Enabler for DevOps
  8. 8. Elastic Capacity Traditional: Rigid Cloud: Elastic Capacity Demand Capacity Excess Capacity Wasted $ Demand Un-met Demand
  9. 9. Why the Cloud ? • Elastic Capacity • Business Agility • CapEx becomes OpEx • Enabler for DevOps
  10. 10. Business Agility Add New Dev Environment Add New Prod Environment Add New Environment in Japan Add 1,000 Servers Remove 1,000 servers Deploy 2 PB Data warehouse Shut down 2 PB Data warehouse IAAS/PAAS: Infrastructure in Minutes Old World: Infrastructure in Weeks Everything changes with this kind of agility + weeks Service Request
  11. 11. Why the Cloud ? • Elastic Capacity • Business Agility • CapEx becomes OpEx • Enabler for DevOps
  12. 12. Capex becomes Opex • no initial investments • no termination fees • no (real) commitments • clear pricing model
  13. 13. Why the Cloud ? • Elastic Capacity • Business Agility • CapEx becomes OpEx • Enabler for DevOps
  14. 14. Enabler for devops • Automation - Less repetitive tasks • Better management tools • no need to build from scratch - reuse • Focus on what matters - your business
  15. 15. Netflix in 2015 • 69,2 million users • 42,5 billion streaming hours • 45GB bandwidth user/month Source : DMR January 2016
  16. 16. Netflix in 2015 • 10x Customers • 100x Traffic • 5x Devs • 2x Cost of IT infrastructure Source : DMR January 2016 in 2008
  17. 17. Infrastructure as Code From API accessible xAAS services to automated deployment
  18. 18. Tools can enable change in behavior and eventually change culture ‘ [patrick debois]
  19. 19. AMI, VMX, OVF, … Cloud Provider Continuous Integration Build, test and integrate applications Configuration Management Configure and setup an existing machine Orchestration Tools Provision Infrastructure and services based on a configuration template heat base image additional packages application code-artefact-765
  20. 20. 'DOCKER' & CLOUD: ENABLERS For DEVOPS
  21. 21. Docker • What is docker ? • What’s more ?
  22. 22. Docker • What is docker ? • What’s more ?
  23. 23. Docker IT works on my System
  24. 24. Supply Matrix - From Hell!
  25. 25. IT Matrix From Hell!
  26. 26. One Size - Fits All
  27. 27. Virtual Machine ?
  28. 28. Compute, Storage, Network Host OS Hypervisor VM1 MicroService Guest OS JVM VM2 MicroService Guest OS JVM Compute, Storage, Network Host OS container1 container2 container3 container4 JVM JVM JVM MicroService MicroService MicroService JVM MicroService Containers have own isolated resources Performance Comparison: http://ibm.co/V55Otq VM’s abstract underlying hardware, but limits resource utilisation
  29. 29. Ship without worrying
  30. 30. Docker - Layering • Container = Writeable layer 
 • Image = Static layers
 • Containers = Share the kernel
  31. 31. docker build -t test/a . FROM aca—base-ubuntu:1.1 RUN apt-get update RUN apt-get install -y apache2 RUN touch /opt/a.txt Let’s build Webserver A! Our Dockerfile
  32. 32. docker build -t test/a . FROM aca—base-apache:2.1 RUN touch /opt/a.txt Or even better….
  33. 33. docker history test/a IMAGE CREATED CREATED BY SIZE 4dc359259700 About a minute ago /bin/sh -c touch /opt/a.txt 8 B 9977b78fbad7 About a minute ago /bin/sh -c apt-get install -y apache2 54.17 MB e83b3bf07b42 2 minutes ago /bin/sh -c apt-get update 20.67 MB 9cd978db300e 3 months ago /bin/sh -c #(nop) ADD precise.tar.xz in / 204.4 MB 6170bb7b0ad1 3 months ago /bin/sh -c #(nop) MAINTAINER Tianon Gravi <ad 0 B 511136ea3c5a 10 months ago 0 B What is in Webserver A?
  34. 34. docker build -t test/b . Let’s build Webserver B! FROM aca—base-ubuntu:1.1 RUN apt-get update RUN apt-get install -y apache2 RUN touch /opt/b.txt Our Dockerfile
  35. 35. docker history test/b IMAGE CREATED CREATED BY SIZE c0daf4bw2ed4 5 seconds ago /bin/sh -c touch /opt/b.txt 8 B 9977b78fbad7 About a minute ago /bin/sh -c apt-get install -y apache2 54.17 MB e83b3bf07b42 2 minutes ago /bin/sh -c apt-get update 20.67 MB 9cd978db300e 3 months ago /bin/sh -c #(nop) ADD precise.tar.xz in / 204.4 MB 6170bb7b0ad1 3 months ago /bin/sh -c #(nop) MAINTAINER Tianon Gravi <ad 0 B 511136ea3c5a 10 months ago 0 B What is in Webserver B?
  36. 36. Containers are here to stay! The Open Container Initiative (OCI) is a collaborative project hosted under the Linux Foundation designed to establish common standards for containers.
  37. 37. Docker • What is docker ? • What’s more ?
  38. 38. Docker Hub - Public Library
  39. 39. Docker Registry - Private Library
  40. 40. Docker Registry - Private Library • Tightly control where your images are being stored • Fully own your images distribution pipeline • Integrate image storage and distribution 
 tightly into your in-house development workflow • Collaborate with your colleagues
  41. 41. Docker Hub • Docker HUB = Docker Registry - As A Service • Zero Maintenance • Additional Functionality to increase collaboration • Alternative Providers : Google / AWS / CoreOS
  42. 42. /preference-service Repository DockerFile Continuous Integration Infrastructure Container Image Repository Compute, Storage, Network Host OS daemon container1 JVM MicroService pull push build provision container1 JVM MicroService Source Control System
  43. 43. The real value of Docker is not technology It’s getting people to agree on something
  44. 44. Shipping Applications Beyond a single container
  45. 45. Container Image code-artefact-765
  46. 46. Minimal OS Docker Virtual Machine Hardware kubelet Pod Container proxy NodeMaster scheduler controller manager (replication controller, etc.) API endpoint distributed configuration storage kubectl (client command line tool)
  47. 47. K/V K/V K/V K/V K/V /etc distributed raft - leader election //Adding a value $ curl http://127.0.0.1:2379/v2/keys/message -XPUT -d value="Hello world” //Quering $ curl http://127.0.0.1:2379/v2/keys/message { "action": "get", "node": { "createdIndex": 2, "key": "/message", "modifiedIndex": 2, "value": "Hello world" } } //Delete $ curl http://127.0.0.1:2379/v2/keys/message -XDELETE Operations
  48. 48. apiVersion: v1 kind: ReplicationController metadata: name: es-data labels: component: elasticsearch role: data visualize: "true" spec: replicas: 2 selector: component: elasticsearch role: data template: metadata: labels: name: es-data component: elasticsearch role: data visualize: "true" spec: serviceAccount: elasticsearch containers: - name: es-data securityContext: capabilities: add: - IPC_LOCK image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.2 es-data-rc.yaml Pod Container
  49. 49. es-data-rc.yaml metadata: labels: name: es-data component: elasticsearch role: data visualize: "true" spec: serviceAccount: elasticsearch containers: - name: es-data securityContext: capabilities: add: - IPC_LOCK image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.2 env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace … ports: - containerPort: 9300 name: transport protocol: TCP volumeMounts: - mountPath: /data name: storage volumes: - name: storage persistentVolumeClaim: claimName: elasticsearch-storage-claim kind: PersistentVolumeClaim apiVersion: v1 metadata: name: elasticsearch-storage-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Gi es-claim.yaml Pod Container
  50. 50. Minimal OS Docker Virtual Machine Hardware kubelet proxy Node Pod Container kind: PersistentVolumeClaim apiVersion: v1 metadata: name: elasticsearch-storage-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Gi es-claim.yaml apiVersion: "v1" kind: "PersistentVolume" metadata: name: “ebs-volume-1“ spec: capacity: storage: "100Gi" accessModes: - "ReadWriteOnce" awsElasticBlockStore: fsType: "ext4" volumeID: "vol-f331a05cc" ebs-volume-1.yaml /data scheduler Amazon EBS
  51. 51. service apiVersion: v1 kind: Service metadata: name: elasticsearch labels: component: elasticsearch role: client visualize: "true" spec: type: LoadBalancer selector: component: elasticsearch role: client ports: - name: rest port: 9200 protocol: TCP - name: transport port: 9300 protocol: TCP es-svc.yaml Minimal OS Docker Virtual Machine Hardware kubelet Pod Container proxy Node (replication controller, etc.)
  52. 52. Minimal OS Docker Virtual Machine Hardware kubelet proxy NodeMaster scheduler controller manager (replication controller, etc.) API endpoint distributed configuration storage kubectl (client command line tool) Pod Container • cluster topology design • operational cluster management • cluster upgrade and maintenance • application • declarative resource specification DevOps • container image standardisation • deployment processes
  53. 53. Elastic LoadBalancer Node service Minimal OS Docker Virtual Machine Hardware kubelet proxy Master scheduler controller manager (replication controller, etc.) API endpoint distributed configuration storage kubectl (client command line tool) Pod Container ServerGroup
  54. 54. Open Source Commercially supported
  55. 55. kubernetes, what else?
  56. 56. Docker Swarm
  57. 57. Minimal OS Docker Virtual Machine Hardware Manager Node Container swarm manager swarm agent swarm agent distributed configuration storage docker remote API overlay network
  58. 58. Amazon ECS
  59. 59. Docker Task Container Instance Amazon ECS Container ECS Agent ELB Internet ELB User / Scheduler API Cluster Management Engine Task Container Docker Task Container Instance Container ECS Agent Task Container Docker Task Container Instance Container ECS Agent Task Container AZ 1 AZ 2 Key/Value Store Agent Communication Service * slide from Deepak Singh, General Manager, Amazon EC2 Container Service
  60. 60. Slave Master framework Virtual Pool of Resources (CPU, RAM, …)
  61. 61. • Set up in seconds, integrates in the AWS stack (ELB, CloudWatch, ECR) and yes, specific to Amazon WebServices • Basic capabilities out of the box (no Discovery, …) but extensible • Ideal for simple containerised workloads • Opinionated declarative cluster management solution • Runs on a single machine (in Docker) and on a large datacenter setup • Rich API for cluster management • Support for Secrets, Quota, Volumes, … • Provider agnostic (docker, rkt, amazon, gce, …) • Abstracts a cluster behind the Docker Remote API • Networking and scheduling support • Open plugin points for discovery, networking, … • Docker only • Lacks (being out of the box) service load balancing support • Essentially a resource abstraction • Scales to +10K nodes • Job agnostic (more than containers alone) • Support Kubernetes as one of the many frameworks
  62. 62. + Container declarative specification of operational needs Cluster continuous deployment pipeline includes quality gates: automated tests ops compliance security tests manual steps …
  63. 63. Patternsfor Continuous Deployment
  64. 64. Container Image preference-service- artefact-765 Blue Green Content Based Router Blue/Green deployments Container Image preference-service- artefact-765 Container Image preference-service- artefact-123 production traffictest traffic
  65. 65. Container Image preference-service- artefact-765 Stage 1 Stage 2 Stage 3 Content Based Router Canary staged deployment
  66. 66. 2 Container ecosystem is maturing rapidly and cAAS provides the abstraction sweet spot 1 Cloud services are a true enabler for business agility 3 Leverage Continuous Deployment as competitive advantage (as long as it lasts)
  67. 67. ACA IT-SOLUTIONS | © 2015 Stijn Wijndaele stijn.wijndaele@aca-it.be @stijnwijndaele Questions? Your Hosts For Today Business Development Manager Cloud & Mobile Solutions ACA IT-SOLUTIONS | © 2016 Stijn Van den Enden stijn.vandenenden@aca-it.be @stieno CTO

×