Download to read offline
Uploaded byDaniel Barker
Becoming a Plumber: Building Deployment Pipelines - RevConf
The document discusses the process of building deployment pipelines in a new data center environment, emphasizing the challenges of the current setup and the benefits of using pipelines for security and compliance. It explains the types of pipelines (scripted and declarative) and introduces tools like Jenkins and Screwdriver, along with configuration examples for applications. Additionally, it covers operators in Kubernetes, image streams, and their functionalities in automating and managing application deployments.
More Related Content
Similar to Becoming a Plumber: Building Deployment Pipelines - RevConf
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Becoming a Plumber: Building Deployment Pipelines - RevConf
- 1. Becoming a Plumber BuildingDeployment Pipelines Dan Barker @barkerd427 danbarker.codes
- 3.
- 4. The current datacenter is...challenging... RHEL 6.9 Dev RHEL 6.8 Test RHEL 6.6 Prod Dev Test Prod RHEL 6.7 Prod Admin Admin Admin Admin Admin Admin Dev RHEL 6.7 Dev RHEL 6.4 Dev RHEL 6.8 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 6.6 Dev Ubuntu Trusty RHEL 6.7 Dev RHEL 6.4 Dev RHEL 6.8 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 6.6 Dev RHEL 6.7 Dev RHEL 6.4 Dev RHEL 6.8 Dev Ubuntu Trusty RHEL 6.9 Dev RHEL 6.6 Dev @barkerd427
- 6. The new datacenter is understandable and usable. Developer Access Production Controlled Network Storage Compute Platform Deployment Pipeline RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App1 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2 RHEL 6.9 App2 @barkerd427
- 7. The value ofPipelines ● Abstract audit and compliance ○ Approvals added dynamically ● Trivialities eliminated ○ Tabs vs. spaces ○ Semicolons or not ● Security checks occur early and often ○ Feedback is important @barkerd427
- 8. The value ofPipelines ● Test all the things! ● Nimble security ● Common artifact repositories ○ Restrict dependencies ○ Automated security vulnerability notification ● Standardized/Centralized approval system ● Applications will become secure by default @barkerd427
- 9.
- 10. Two types ofpipelines Scripted: ● Very Groovy! ● More powerful ● Provides greatest level of flexibility Declarative: ● Only a little Groovy ● Simpler to maintain ● Easier to read and understand
- 11. Shared Libraries ● Globalor local ● Groovy ● Third-party ● Src ● Var ● Resources
- 12. Fabric8 ● Shared Library ○Kubernetes ○ Fabric8 Jenkins ○ Specialized ● Pipelines ○ Golang ○ Node ○ .Net
- 13.
- 14. Screwdriver ● No orangejuice ● Yahoo! ● Distributed system ● Independently scalable ● Components ○ ReST API ○ Web UI ○ Launcher ○ Execution Engine ○ Datastore
- 15.
- 16.
- 17. Config Pipeline App 1 Config App2 Config App 3 Config Message Queue Combined Config Repo Pipeline Config Build Config Deploy Config MRTrigger
- 18. Build Config Flow Build Config Service Message Queue OpenShift Loader CombinedConfig Repo OpenShift OpenShift Listener Message Queue GitLab Repo
- 19. Deploy Config Flow Deploy Config Message Queue OpenShift Loader CombinedConfig Repo OpenShift OpenShift Listener Message Queue GitLab Repo
- 20. Pipeline Config Flow Pipeline Config Message Queue Jenkins Loader CombinedConfig Repo Jenkins GitLab MR Splunk Artifactory GitLab Repo
- 21. Pipelines ● Stages ● Steps ●Environments @barkerd427 ● Application ● PipelineTemplate ● PipelineConfig
- 22. An Application includesa Pipeline, based on an opinionated PipelineTemplate. These combine as a PipelineConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: PipelineConfig name: app1-pipeline labels: type: application pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build manager: maven builderImage: java8-builder baseImage: tomcat8 version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev @barkerd427
- 23. An Application andPipelineTemplate also combine to create a DeploymentConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: DeploymentConfig metadata: name: app1-pipeline type: application spec: replicas: 2 selector: name: frontend template: { ... } triggers: - type: ConfigChange - imageChangeParams: automatic: true containerNames: - helloworld from: kind: ImageStreamTag name: hello-openshift:latest type: ImageChange strategy: type: Rolling @barkerd427
- 25.
- 26. Operators ● Represents human operationalknowledge in software ● Uses 3rd-party resources ○ Controller of controllers and resources @barkerd427 ● Identical model to k8s controllers ○ OODA Loop ● Not supported in OpenShift
- 27. Operators ● Deployed intok8s cluster ● Interactions through new controller ○ kubectl get prometheuses ○ kubectl get alertmanagers @barkerd427 ● Abstraction around k8s primitives ○ Users just want to use a MySQL cluster. ● Complex tasks that can be performed ○ Rotating credentials, certs, versions, backups
- 30.
- 31. ImageStreams ● Contains imagesfrom: ○ Integrated registry ○ Other ImageStreams ○ External registries ● Automatic event triggers http://blog.openshift.com @barkerd427
- 32. ImageStreams - Metadata ●Commands ● Entrypoint ● EnvVars @barkerd427 ● Layers ● Labels ● Ports
- 33.
- 34.