Downloaded 10 times
Uploaded byDaniel Barker
Becoming A Plumber: Building Deployment Pipelines - LISA17
The document discusses the challenges and improvements in deploying applications within a new, more manageable data center compared to a traditional setup. It highlights the benefits of using deployment pipelines, including enhanced security, standardized processes, and efficient artifact management, along with examples of pipeline and application configuration using tools like Jenkins. Additionally, it introduces the concept of operators in Kubernetes that encapsulate operational knowledge for tasks such as credential management and backups.
More Related Content
Similar to Becoming A Plumber: Building Deployment Pipelines - LISA17
![[WSO2Con Asia 2018] Deploying Applications in K8S and Docker](https://cdn.slidesharecdn.com/ss_thumbnails/deployingapplicationsink8sanddocker-wso2conasia2018-sanjaya-180810103523-thumbnail.jpg?width=640&height=640&fit=bounds)
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Becoming A Plumber: Building Deployment Pipelines - LISA17
- 1. Becoming a Plumber BuildingDeployment Pipelines Dan Barker @barkerd427 danbarker.codes
- 3.
- 4. The current datacenter is...challenging... RHEL 7.4 Dev RHEL 7.3 Test RHEL 7.1 Prod Dev Test Prod RHEL 7.2 Dev Admin Admin Admin Admin Admin Admin Dev RHEL 7.2 Dev RHEL 7.1 Dev RHEL 7.3 Dev Ubuntu Trusty RHEL 7.4 Dev RHEL 6.6 Dev Ubuntu Trusty RHEL 7.3 Test RHEL 7.4 Test RHEL 7.2 Test Ubuntu Trusty RHEL 6.9 Test RHEL 7.1 Test RHEL 7.4 Prod RHEL 7.1 Prod RHEL 7.3 Prod Ubuntu Trusty RHEL 6.9 Prod RHEL 7.2 Prod @barkerd427
- 6. The new datacenter is understandable and usable. Developer Access Production Controlled Network Storage Compute Platform Deployment Pipeline RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App1 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 RHEL 7.4 App2 @barkerd427
- 7. The value ofPipelines ● Abstract audit and compliance ● Trivialities eliminated ● Security checks occur early/often ● Test all the things! @barkerd427
- 8. The value ofPipelines ● Nimble security ● Common artifact repositories ● Standardized approval system ● Apps become secure by default @barkerd427
- 9.
- 10. Two types ofpipelines Scripted: ● Very Groovy! ● More powerful ● Provides greatest level of flexibility Declarative: ● Only a little Groovy ● Simpler to maintain ● Easier to read and understand
- 11. Shared Libraries ● Globalor local ● Groovy ● Third-party ● Src ● Var ● Resources
- 12. Fabric8 ● Shared Library ○Kubernetes ○ Fabric8 Jenkins ○ Specialized ● Pipelines ○ Golang ○ Node ○ .Net
- 13.
- 15. Screwdriver ● No orangejuice ● Yahoo! ● Distributed system ● Independently scalable ● Components ○ ReST API ○ Web UI ○ Launcher ○ Execution Engine ○ Datastore
- 16.
- 17. Concourse ● Immutable architecture ● Ephemeral architecture ●Tasks ● Resources ● Jobs ● Builds
- 18.
- 19.
- 20. Config Pipeline App 1 Config App2 Config App 3 Config Message Queue Combined Config Repo Pipeline Config Build Config Deploy Config MRTrigger
- 21. Pipeline Config Flow Pipeline Config Message Queue Jenkins Deployer CombinedConfig Repo Jenkins GitLab MR Splunk Nexus GitLab Repo
- 22. Build Config Flow BuildConfig Service Message Queue OpenShift Deployer Combined Config Repo OpenShift OpenShift Listener GitLab Repo
- 23. Deploy Config Flow DeployConfig Message Queue OpenShift Deployer Combined Config Repo OpenShift OpenShift Listener GitLab Repo
- 24. Pipelines ● Stages ● Steps ●Environments @barkerd427 ● Application ● PipelineTemplate ● PipelineConfig
- 25. An Application includesa Pipeline, based on an opinionated PipelineTemplate. These combine as a PipelineConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: PipelineConfig name: app1-pipeline labels: type: application pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build manager: maven builderImage: java8-builder baseImage: tomcat8 version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev @barkerd427
- 26. An Application andPipelineTemplate also combine to create a DeploymentConfig. apiVersion: v1 kind: Application name: app1 cap: template: name: approvedTemplates/Tomcat8.yaml pipeline: notifications: mattermost: team: cloud channel: general on_success: never on_failure: always dependencies: - name: authn dnsName: authn - name: key-management username: reference_to_username password: reference_to_password stages: - name: build steps: - action: build baseImage: version: 8.0.41 - name: dev approvers: - role: app1-dev steps: - action: deploy params: environment: dev apiVersion: v1 kind: PipelineTemplate name: Tomcat8 labels: type: application build: manager: maven version: latest builderImage: java8-builder version: latest baseImage: tomcat8 version: latest deploy: deploymentType: canary maxUnavailable: 10% maxSurge: 20% apiVersion: v1 kind: DeploymentConfig metadata: name: app1-pipeline type: application spec: replicas: 2 selector: name: frontend template: { ... } triggers: - type: ConfigChange - imageChangeParams: automatic: true containerNames: - helloworld from: kind: ImageStreamTag name: hello-openshift:latest type: ImageChange strategy: type: Rolling @barkerd427
- 27.
- 28. ImageStreams ● Contains imagesfrom: ○ Integrated registry ○ Other ImageStreams ○ External registries ● Automatic event triggers http://blog.openshift.com @barkerd427
- 29. ImageStreams - Metadata ●Commands ● Entrypoint ● EnvVars @barkerd427 ● Layers ● Labels ● Ports
- 30.
- 33.
- 34. Operators ● Represents human operationalknowledge in software ● CustomResource Definitions ○ Extends Kubernetes API @barkerd427 ● Identical model to k8s controllers ○ OODA Loop ● Now in Beta!!!
- 35. Operators ● Deployed intok8s cluster ● Interactions through new API ○ kubectl get prometheuses ○ kubectl get alertmanagers @barkerd427 ● Abstraction around k8s primitives ○ Users just want to use a MySQL cluster. ● Complex tasks that can be performed ○ Rotating credentials, certs, versions, backups
- 38. What to takeaway? This is the slide you should take a picture of... Do not take a picture of this slide!!! No, stop! STOP!!! Really, STOP!!! Security! SECURITY!!!! ● Just Start ● Start Small ● Abstract Consciously ● Let Others Do The Work ● Contribute Back
- 39. Resources ● https://github.com/jenkinsci/pipeline-examples/blob/master/global-library-examples/global-function/sta ndardBuild.groovy ● https://github.com/jenkinsci/pipeline-examples/blob/master/global-library-examples/global-function/Jen kinsfile ●https://github.com/jenkinsci/pipeline-examples/blob/master/jenkinsfile-examples/nodejs-build-test-deplo y-docker-notify/Jenkinsfile ● https://github.com/jenkinsci/pipeline-examples/blob/master/declarative-examples/jenkinsfile-examples/ mavenDocker.groovy ● https://github.com/fabric8io/fabric8-pipeline-library/blob/master/src/io/fabric8/Utils.groovy ● https://github.com/fabric8io/fabric8-jenkinsfile-library/blob/master/golang/ReleaseAndStage/Jenkinsfile ● https://istio.io/ ● https://kubernetes.io/docs/getting-started-guides/minikube/ ● http://danbarker.codes
- 40.