DCSF 19 Building Your Development Pipeline
•
1 like•467 views
Oliver Pomeroy, Docker & Laura Tacho, Cloudbees Enterprises often want to provide automation and standardisation on top of their container platform, using a pipeline to build and deploy their containerized applications. However this opens up new challenges; Do I have to build a new CI/CD Stack? Can I build my CI/CD pipeline with Kubernetes orchestration? What should my build agents look like? How do I integrate my pipeline into my enterprise container registry? In this session full of examples and how-to's, Olly and Laura will guide you through common situations and decisions related to your pipelines. We'll cover building minimal images, scanning and signing images, and give examples on how to enforce compliance standards and best practices across your teams.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to DCSF 19 Building Your Development Pipeline
Similar to DCSF 19 Building Your Development Pipeline (20)
More from Docker, Inc.
More from Docker, Inc. (20)
Recently uploaded
Recently uploaded (20)
DCSF 19 Building Your Development Pipeline
- 3. What is a pipeline? A set of processes to make software development more efficient, secure, and high-quality. We want to deploy a containerized image. Our pipeline includes: ● Building the image ● Automated testing ● Security scanning ● Promoting the image and deploying to production
- 5. Why use containers for your pipeline?
- 6. Containers make it easy to create consistent, reproducible environments because your environment is declared in a Dockerfile. You know exactly what’s running, where, and can modify and reproduce environments easily It also allows for efficiency by sharing some artifacts between dev, test, and prod. It’s about what’s INSIDE the container...
- 7. Since containers are lightweight, isolated, and fast to boot, they enable different workflows that are a great fit for your pipelines ● Fanning out to run large tasks across multiple containers ● Parallelizing workflows ...And what goes on OUTSIDE in systems and workflows
- 8. Certain things will be made easier, but Docker can’t do the work for you. It’s still up to you to: ● Follow 12-factor app guidelines like pinning dependencies ● Pay attention to size of images and understand what’s in them ● Perform security and vulnerability scans But there are still no shortcuts!
- 9. Building images in a pipeline
- 10. We size our build agents for that 1 job that requires a lot of CPU. The rest of the time they are pretty idle. We don’t standardize our tools, so I need everything on every build agent. Optimizing Build Agents
- 11. We size our build agents for that 1 job that requires a lot of CPU. The rest of the time they are pretty idle. We don’t standardize our tools, so I need everything on every build agent. On Demand Build Agents! A Build Agent Image for Everyone! But is this secure? Optimizing Build Agents
- 12. Containerized Build Agents: Docker in Docker $ docker run --privileged --name dind -d docker:18.09.0-dind $ docker run -it --rm --link dind:docker docker:18.09.0 sh / # docker images REPOSITORY TAG IMAGE ID CREATED SIZE / # docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
- 13. $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:18.09.0 sh / # docker images REPOSITORY TAG IMAGE ID CREATED SIZE olly/jenkins-slave 1 fae1591c6584 3 hours ago 628MB nginx latest 62f816a209e6 6 days ago 109MB openjdk 8-stretch 954739b8bdfb 2 weeks ago 624MB golang latest 45e48f60e268 4 weeks ago 777MB / # docker ps CONTAINER ID IMAGE COMMAND STATUS NAMES b2e07edbd47e jenkins/slave:3.27-1 "sh" Up 2 hours optimistic_chandrasekhar 86f77c2b67f0 openjdk:8-stretch "sh" Up 3 hours distracted_mcnulty Containerized Build Agents: Mounted Socket
- 14. $ docker run -it --rm -v .pipedocker_engine:.pipedocker_engine docker:18.09.0 cmd C:>docker images REPOSITORY TAG IMAGE ID CREATED SIZE openjdk 1803 2f91c65915d9 2 weeks ago 5.2GB dtr.az.olly.dtcntr.net/openjdk 1803 2f91c65915d9 2 weeks ago 5.2GB docker 18.09.0 629e0258a222 2 weeks ago 5.11GB microsoft/aspnet 4.7.2-wind..cbdbd42e5a14 7 weeks ago 5.46GB windows/servercore 1803 1a4a9d0fd8af 7 weeks ago 4.93GB C:>docker ps CONTAINER ID IMAGE COMMAND STATUS NAMES a7911d0ff315 docker:18.09.0 "cmd" Up 9 seconds priceless_franklin Containerized Build Agents: Mounted Pipe (Windows)
- 15. Docker in Docker docker run dind, --privileged The container can do almost everything the host can do :( Mounted socket or pipe -v /var/run/docker.sock:/var/run/docker.sock -v .pipedocker_engine:.pipedocker_engine The container controls your Docker host. Any security applied to the socket has just been bypassed. :( Containerized Build Agents: Is this Secure?
- 16. Docker in Docker docker run dind, --privileged Rootless Docker? (Coming soon! Docker CE / EE 19.03!) Rootless building daemon? (Buildkit, Img, Kaniko) Mounted socket or pipe -v /var/run/docker.sock:/var/run/docker.sock -v .pipedocker_engine:.pipedocker_engine Dedicated build host? Dedicated build cluster? Containerized Build Agents: Is this Secure?
- 19. Build once, use everywhere Treat your Dockerfile as a shared artifact that can before different types of testing during all phases in your development process ● local testing during development ● standalone unit tests ● browser and integration tests ● promotion to staging or QA environments
- 20. Maintain flexibility in each environment ● Dev and test environments can be very different ○ Reuse the Dockerfile (or a shared base image, if that makes sense) ○ Create a specific docker-compose.yml file with your testing environment ○ Long(er)-running dev environment can coexist alongside ephemeral test environments
- 21. In practice: integration testing patterns docker-compose.yml docker-compose.test.yml version: '3' services: vote: build: ../vote/ ports: ["80"] depends_on: - redis - db networks: - front-tier - back-tier result: ... worker: ... redis: ... db: ... version: '3' services: test: build: ./tests/ depends_on: - vote - result - worker vote: ... result: ... worker: ... redis: ... db: ... Create new one-off application environment Create service to run integration tests
- 22. Configurations can be reused with many tools OSS Jenkins Jenkins X Hosted SaaS Circle CI CodeShip Travis CI Azure DevOps Supported on-prem CloudBees CloudBees Jenkins Distribution Circle CI Bamboo TeamCity GitLab and plenty more!
- 23. Parallel Testing with Docker ● Theory: employ task parallelism to split work across parallel computers (containers) ○ Think of your container as just one process, and split testing loads across processes ○ Improve performance on-demand by adding more containers ○ Manage environments simply with Docker ecosystem tools
- 24. Parallel Testing with Docker ● In practice: most CI tools do this for you i.e. declarative pipelines in Jenkinsfile, CodeShip steps, GitLab ● Use cases ○ Test against a matrix of versions ○ Cross-compile on Linux and Windows ○ Run integration tests against different browsers ● Caution: parallelism is great for testing, but not deploying.
- 25. Example: Windows & Linux Builds in Jenkins pipeline { agent none stages { stage("build and deploy on Windows and Linux") { parallel { stage("windows") { agent { label "windows" } stages { stage("build") {} stage("deploy") {} } } stage("linux") { agent { label "linux" } stages { stage("build") {} stage("deploy") {} } } } } } }
- 26. Example: Selenium Grid Selenium Hub Firefox Ubuntu Chrome MacOS Safari iOS Safari MacOS Chrome Windows
- 28. Security question Q. I’ve downloaded all of these container images, how do I know what's inside? What happens if there is an out of date package in there? How do I know which vulnerabilities are exposed? A. It's fine. They came from the DockerHub and the Dockerfile looks ok…
- 29. The Old World Host Operating System, Kernel… Devs Ops App1 App 2 App3 Java Python .Net
- 30. The New World Host Operating System, Kernel…. Devs Ops Who’s giving this TLC? App1 App 2 App3 Java Python .Net
- 31. Maybe we can check the Dockerfile? # Pull base image FROM oracle/serverjre:8 # Maintainer LABEL MAINTAINER=”bruno.borges@oracle.com” ENV ORACLE_HOME=/u01/oracle USER_MEM_ARGS="-Djava.security.egd=file:/ dev/./urandom" PATH=$PATH:/usr/java/default/bin:/u01/ora cle/oracle_common/common/bin RUN mkdir -p /u01 && ... Oracle Weblogic 12.1.3 Image FROM oraclelinux:7-slim LABEL MAINTAINER=”bruno.borges@oracle.com” ENV JAVA_PKG=server-jre-8u*-linux-x64.tar. gz JAVA_HOME=/usr/java/default ... oracle/serverjre:8 FROM scratch LABEL MAINTAINER=”ol-ovm-info_ww@oracle.com” ADD oraclelinux-7-slim-rootfs.tar.xz / oraclelinux:7-slim
- 32. Maybe we can check the Dockerfile? # Pull base image FROM oracle/serverjre:8 # Maintainer LABEL MAINTAINER=”bruno.borges@oracle.com” ENV ORACLE_HOME=/u01/oracle USER_MEM_ARGS="-Djava.security.egd=file:/ dev/./urandom" PATH=$PATH:/usr/java/default/bin:/u01/ora cle/oracle_common/common/bin RUN mkdir -p /u01 && ... FROM scratch LABEL MAINTAINER=”ol-ovm-info_ww@oracle.com” ADD oraclelinux-7-slim-rootfs.tar.xz / FROM oraclelinux:7-slim LABEL MAINTAINER=”bruno.borges@oracle.com” ENV JAVA_PKG=server-jre-8u*-linux-x64.tar. gz JAVA_HOME=/usr/java/default ... Oracle Weblogic 12.1.3 Image oracle/serverjre:8 oraclelinux:7-slim
- 33. Image Vulnerability Scanning Docker Trusted Registry
- 34. How do I control what runs in my cluster?? Enforcing “Corporate Standards” or Best Practises to all of the container images within your environment: ○ Everyone builds from “myco/base:1” or “myco/base:2” ○ Container Images shouldn’t run as root by default You can add stages in to your Pipeline to make this happen! ○ Linting Dockerfiles (fromlatest, hadolint + many more) ○ Policy Engines (Anchore Engine, used within SysDig)
- 35. How do I control what runs in my cluster?? Enforcing that the “production” images are the only images that can run on my cluster. $ kubectl apply -f exampleapp.yaml Error from server (Forbidden): error when creating "exampleapp.yaml": pods "nginx" is forbidden: one or more container images do not meet the required signing policy: [nginx: image did not meet required signing policy] $ docker run nginx:latest docker: Error response from daemon: image did not meet required signing policy.
- 36. Automating your Pipeline with a Private Registry Automated Image Promotion dev/example qa/example Developer Pushes an Image to DTR Promotion Policy verifies that the image has no vulnerabilities. Webhooks at every step
- 38. The Software Supply Chain New Code Lands in SCM Jenkins - Builds new Image from SCM Image Uploaded to Registry If Image has no vulnerabilities. Move to testing. Jenkins Pipeline runs QA on the Image. Security Team, sign off on image. Moves to Production. New Image lands in Production Jenkins Pipeline Creates the Deployment from Templates App Successfully Deployed to Docker EE Cluster. Continuous Integration Continuous Deployment
- 39. Out-of-the-Box Features Modern orchestration systems come pre-baked with many deployment features. The work is already done for you! This means that your pipeline needs to monitor for status updates, but not implement the functionality.
- 40. Deployment Strategies Rolling Update Update containers one-by-one (or in groups), so that the application has no downtime. It’s possible for two versions of the software to be deployed at the same time.
- 45. In practice: Rolling Updates and Auto-Rollbacks Swarm and Kubernetes handle this for you -- no need to custom build service: build: myapp/myservice image: ${REGISTRY-127.0.0.1:5000}/myservice:${TAG-latest} deploy: replicas: 7 update_config: delay: 5s failure_action: rollback max_failure_ratio: .5 monitor: 5s parallelism: 1 docker stack
- 46. In practice: Rolling Updates and Auto-Rollbacks ... strategy: type: RollingUpdate #this is the default rollingUpdate: maxSurge: 1 maxUnavailable: 1 Swarm and Kubernetes handle this for you -- no need to custom build kubernetes deployment
- 47. The Software Supply Chain
- 48. The Software Supply Chain New Code Lands in SCM Jenkins - Builds new Image from SCM Image Uploaded to Registry If Image has no vulnerabilities. Move to testing. Jenkins Pipeline runs QA on the Image. Security Team, sign off on image. Moves to Production. New Image lands in Production Jenkins Pipeline Creates the Deployment from Templates App Successfully Deployed to Docker EE Cluster. Continuous Integration Continuous Deployment
- 49. Thank You!
- 50. Rate & Share Rate this session in the DockerCon App Follow Laura @rhein_wein Follow Olly @ollypom Tweet #DockerCon