Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
October 24, 2017
Becoming a Plumber
Building Deployment Pipelines
Dan Barker
@barkerd427
danbarker.codes
Why?
The current data center is...challenging...
RHEL
7.4
Dev
RHEL
7.3
Test
RHEL
7.1
Prod
Dev Test Prod
RHEL
7.2
Prod
Admin Adm...
The new data center is understandable and usable.
Developer Access Production Controlled
Network
Storage
Compute
Platform
...
The value of Pipelines
● Abstract audit and compliance
● Trivialities eliminated
● Security checks occur early/often
● Tes...
The value of Pipelines
● Nimble security
● Common artifact repositories
● Standardized approval system
● Apps become secur...
Jenkins Pipelines
Two types of pipelines
Scripted:
● Very Groovy!
● More powerful
● Provides greatest level
of flexibility
Declarative:
● On...
Shared Libraries
● Global or local
● Groovy
● Third-party
● Src
● Var
● Resources
Fabric8
● Shared Library
○ Kubernetes
○ Fabric8 Jenkins
○ Specialized
● Pipelines
○ Golang
○ Node
○ .Net
Alternatives?
Screwdriver
● No orange juice
● Yahoo!
● Distributed system
● Independently
scalable
● Components
○ ReST API
○ Web UI
○ La...
Screwdriver Architecture
http://screwdriver.cd/
Concourse
● Immutable
architecture
● Ephemeral
architecture
● Tasks
● Resources
● Jobs
● Builds
Concourse Architecture
Deployment
Pipeline’s have
fallen behind
@barkerd427
Config Pipeline
App 1
Config
App 2
Config
App 3
Config
Message
Queue
Combined
Config
Repo
Pipeline
Config
Build
Config
Dep...
Pipeline Config Flow
Pipeline
Config
Message
Queue
Jenkins
Deployer
Combined Config Repo
Jenkins
GitLab MR Splunk
Nexus
Gi...
Build Config Flow
Build Config
Service
Message Queue
OpenShift
Deployer
Combined Config Repo
OpenShift
OpenShift
Listener
...
Deploy Config Flow
Deploy Config
Message Queue
OpenShift
Deployer
Combined Config Repo
OpenShift
OpenShift
Listener
GitLab...
Pipelines
● Stages
● Steps
● Environments
@barkerd427
● Application
● PipelineTemplate
● PipelineConfig
An Application includes a Pipeline, based on an opinionated PipelineTemplate. These combine as a PipelineConfig.
apiVersio...
An Application and PipelineTemplate also combine to create a DeploymentConfig.
apiVersion: v1
kind: Application
name: app1...
ImageStreams
are an image
abstraction
@barkerd427
ImageStreams
● Contains images from:
○ Integrated registry
○ Other ImageStreams
○ External registries
● Automatic event tr...
ImageStreams - Metadata
● Commands
● Entrypoint
● EnvVars
@barkerd427
● Layers
● Labels
● Ports
http://blog.openshift.com
ImageStreams
CoreOS
Operators are
magical
(not really)
@barkerd427
Operators
● Represents human
operational knowledge
in software
● Uses 3rd-party
resources
○ Controller of controllers and
...
Operators
● Deployed into k8s
cluster
● Interactions through
new controller
○ kubectl get prometheuses
○ kubectl get alert...
Thanks!
Contact me:
Dan Barker
drbarker@dstsystems.com
dan@danbarker.codes
danbarker.codes
@barkerd427
October 24, 2017
Session Title
Your Name
Your Title
Your Company
Your @TwitterHandle
October 24, 2017
Session Title
Your Name
Your Title
Your Company
Your @TwitterHandle
October 24, 2017
bit.ly/addo-slack
Find me on slack, right now!
October 24, 2017
Becoming a Plumber: Building Deployment Pipelines - All Day DevOps
Becoming a Plumber: Building Deployment Pipelines - All Day DevOps
Becoming a Plumber: Building Deployment Pipelines - All Day DevOps
Becoming a Plumber: Building Deployment Pipelines - All Day DevOps
Becoming a Plumber: Building Deployment Pipelines - All Day DevOps
Becoming a Plumber: Building Deployment Pipelines - All Day DevOps
Upcoming SlideShare
Loading in …5
×

Becoming a Plumber: Building Deployment Pipelines - All Day DevOps

243 views

Published on

A core part of our IT transformation program is the implementation of deployment pipelines for every application. Attendees will learn how to build abstract pipelines that will allow multiple types of applications to fit the same basic pipeline structure. This has been a big win for injecting change and updating legacy applications.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Becoming a Plumber: Building Deployment Pipelines - All Day DevOps

  1. 1. October 24, 2017
  2. 2. Becoming a Plumber Building Deployment Pipelines Dan Barker @barkerd427 danbarker.codes
  3. 3. Why?
  4. 4. The current data center is...challenging... RHEL 7.4 Dev RHEL 7.3 Test RHEL 7.1 Prod Dev Test Prod RHEL 7.2 Prod Admin Admin Admin Admin Admin Admin Dev RHEL 7.2 Dev RHEL 7.1 Dev RHEL 7.3 Dev Ubuntu Trusty RHEL 7.4 Dev RHEL 6.6 Dev Ubuntu Trusty RHEL 7.3 Dev RHEL 7.4 Dev RHEL 7.2 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 7.1 Dev RHEL 7.4 Dev RHEL 7.1 Dev RHEL 7.3 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 7.2 Dev @barkerd427
  5. 5. The new data center is understandable and usable. Developer Access Production Controlled Network Storage Compute Platform Deployment Pipeline RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 @barkerd427
  6. 6. The value of Pipelines ● Abstract audit and compliance ● Trivialities eliminated ● Security checks occur early/often ● Test all the things! @barkerd427
  7. 7. The value of Pipelines ● Nimble security ● Common artifact repositories ● Standardized approval system ● Apps become secure by default @barkerd427
  8. 8. Jenkins Pipelines
  9. 9. Two types of pipelines Scripted: ● Very Groovy! ● More powerful ● Provides greatest level of flexibility Declarative: ● Only a little Groovy ● Simpler to maintain ● Easier to read and understand
  10. 10. Shared Libraries ● Global or local ● Groovy ● Third-party ● Src ● Var ● Resources
  11. 11. Fabric8 ● Shared Library ○ Kubernetes ○ Fabric8 Jenkins ○ Specialized ● Pipelines ○ Golang ○ Node ○ .Net
  12. 12. Alternatives?
  13. 13. Screwdriver ● No orange juice ● Yahoo! ● Distributed system ● Independently scalable ● Components ○ ReST API ○ Web UI ○ Launcher ○ Execution Engine ○ Datastore
  14. 14. Screwdriver Architecture http://screwdriver.cd/
  15. 15. Concourse ● Immutable architecture ● Ephemeral architecture ● Tasks ● Resources ● Jobs ● Builds
  16. 16. Concourse Architecture
  17. 17. Deployment Pipeline’s have fallen behind @barkerd427
  18. 18. Config Pipeline App 1 Config App 2 Config App 3 Config Message Queue Combined Config Repo Pipeline Config Build Config Deploy Config MRTrigger
  19. 19. Pipeline Config Flow Pipeline Config Message Queue Jenkins Deployer Combined Config Repo Jenkins GitLab MR Splunk Nexus GitLab Repo
  20. 20. Build Config Flow Build Config Service Message Queue OpenShift Deployer Combined Config Repo OpenShift OpenShift Listener GitLab Repo
  21. 21. Deploy Config Flow Deploy Config Message Queue OpenShift Deployer Combined Config Repo OpenShift OpenShift Listener GitLab Repo
  22. 22. Pipelines ● Stages ● Steps ● Environments @barkerd427 ● Application ● PipelineTemplate ● PipelineConfig
  23. 23. An Application includes a Pipeline, based on an opinionated PipelineTemplate. These combine as a PipelineConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: PipelineConfig name: app1-pipeline labels: type: application pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build manager: maven builderImage: java8-builder baseImage: tomcat8 version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev @barkerd427
  24. 24. An Application and PipelineTemplate also combine to create a DeploymentConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: DeploymentConfig metadata: name: app1-pipeline type: application spec: replicas: 2 selector: name: frontend template: { ... } triggers: - type: ConfigChange - imageChangeParams: automatic: true containerNames: - helloworld from: kind: ImageStreamTag name: hello-openshift:latest type: ImageChange strategy: type: Rolling @barkerd427
  25. 25. ImageStreams are an image abstraction @barkerd427
  26. 26. ImageStreams ● Contains images from: ○ Integrated registry ○ Other ImageStreams ○ External registries ● Automatic event triggers http://blog.openshift.com @barkerd427
  27. 27. ImageStreams - Metadata ● Commands ● Entrypoint ● EnvVars @barkerd427 ● Layers ● Labels ● Ports
  28. 28. http://blog.openshift.com ImageStreams
  29. 29. CoreOS Operators are magical (not really) @barkerd427
  30. 30. Operators ● Represents human operational knowledge in software ● Uses 3rd-party resources ○ Controller of controllers and resources @barkerd427 ● Identical model to k8s controllers ○ OODA Loop ● Not supported in OpenShift
  31. 31. Operators ● Deployed into k8s cluster ● Interactions through new controller ○ kubectl get prometheuses ○ kubectl get alertmanagers @barkerd427 ● Abstraction around k8s primitives ○ Users just want to use a MySQL cluster. ● Complex tasks that can be performed ○ Rotating credentials, certs, versions, backups
  32. 32. Thanks! Contact me: Dan Barker drbarker@dstsystems.com dan@danbarker.codes danbarker.codes @barkerd427
  33. 33. October 24, 2017 Session Title Your Name Your Title Your Company Your @TwitterHandle
  34. 34. October 24, 2017 Session Title Your Name Your Title Your Company Your @TwitterHandle
  35. 35. October 24, 2017 bit.ly/addo-slack Find me on slack, right now!
  36. 36. October 24, 2017

×