SlideShare a Scribd company logo
Docker-based Pipelines
with Codefresh
KOSTIS KAPELONIS - DAN GARFIELD
Dan
Garfield
Chief Technology Evangelist
About Kostis Kapelonis
Agenda
1. Docker usage in Continuous Integration
2. Dockerizing build tools as pipeline steps
3. Upgrading build tools to new versions
4. Mixing multiple versions of the same tool in
the same pipeline
5. Creating new pipeline steps on the fly
● Demos for everything using Codefresh
https://github.com/containers101/
docker-based-pipelines-webinar/
Theory: Docker-based Pipelines
“Docker-based” means 2 different things:
Using Docker as a
deployment package
(this is what most people think)
Using Docker for build Tooling
(this is not what most people think)
90% of cases: “We have migrated
to Docker in production”
The Basic Software Lifecycle
Before Docker – The Dark Ages
Docker-based Deployments - Better
Adding Docker-based Build Pipelines
Today’s Webinar
CI/CD Runtime
Old versus new
Before:
● Tools are static
● Used for all pipelines
After:
● Tools are dynamic
● Isolated to each pipeline
Pipeline
Java6
Docker Go
Docker
Pipeline
Node9
Docker
Python2
Docker
Pipeline
Python3
Docker
Node11
Docker
Resources for Docker as Deployment artifact
Docker CI/CD - benefits
Build farm before Docker
● Multiple tools on each node
● Very hard to manage
●Often nodes had different versions of the
same tool
●Developers had to choose the correct
machine for their build
Build farm after Docker
● Only Docker is installed
● Very easy to manage
●All nodes are exactly the same
●These nodes are often a Kubernetes cluster
Using Docker in Continuous Integration
● EVERY build tool is placed in a Docker
container
● The build node has only Docker installed
and nothing else
● A pipeline is a series of commands that run
inside a Docker context
● After each build the node reverts back to its
original state
● Developers don’t care about nodes
Docker images are everywhere
● They are reusable and shareable
● No need to re-invent the week (e.g. Terraform in Docker)
● Private docker images can be created with your team in mind
Container per build step
● Codefresh requires ALL tools to be dockerized
● You can use any public or private Docker
image as tooling
● Each build step has a Docker image as context
● Pipelines are described in declarative YAML
About Codefresh
● Docker based CI/CD solution
● Each build step is a Docker image
● Native support for Docker, Helm,
Kubernetes deployments
● Includes built-in Docker registry and
Helm repository
● 20,000+ users
Demo 1:
Python/Node application
https://github.com/containers101/docker-based-pipelines-webinar/tree/master/01_simple_pipeline
Traditional VM based problems
Traditional CI Platform Questions:
● Do you support my favorite version of
Node/Java/Go/Ruby/Python?
● Do you support maven, yarn, gulp, sbt, gradle, rake?
● Can I run Ansible? Terraform? GCloud? AWS CLI?
● Can I run Kubectl? Helm? Draft?
Traditional CI/CD Platforms
Demo 2:
Adding Go and AWS CLI
https://github.com/containers101/docker-based-pipelines-webinar/tree/master/02_aws_cli
Does Codefresh Support…
● Node 10?
● Perl 6?
● Python2?
● Gradle?
● Vault?
● AWS cli?
● Sonar?
● Findbugs?
● Selenium?
● Snyk?
● Clair?
Does Codefresh Support…
● Node 10?
● Perl 6?
● Python2?
● Gradle?
● Vault?
● AWS cli?
● Sonar?
● Findbugs?
● Selenium?
● Snyk?
● Clair?
YES!
Because there is a Docker
image for it
Codefresh Pipelines are Future Proof
● You can use ANY existing Docker image from Dockerhub or
any other Registry
● Every time a new tool comes out, it can be used right away if
packaged in a Docker image
Tool Upgrades and Version
Clashes
Updating a Tool in a Traditional VM Pipeline
Traditional CI Solutions
Demo:
Updating Python to 3.7
https://github.com/containers101/docker-based-pipelines-webinar/tree/master/02_aws_cli
Using Tools from Different Versions
● Version clashes are a huge pain for both developers and operators
● Legacy projects need to still use old version
● Using different versions in the same pipeline is almost impossible
● Developers want to use latest version of tool, traditional CI/CD
platforms may not be able to keep up
Wasting Effort on “Version Managers”
Wasting Effort on “Version Managers”
● They allow developers to switch between different versions
● Tied to a specific technology/programming language
● Require they own installation/ maintenance
● Must be upgraded for new versions
The Problem with Python
● Different python versions are a
notorious problem
● Until recently you needed
dedicated support from your CI
platform
● What happens if I want to test
Python 2.5?
Replacing “version managers”
with Docker
● Works for any language/framework
● Already installed on the build node
● Its own version is independent from
the tools
● Can use any public and private image
Codefresh “Python Support”
● We support EVERY container
ever made
● We support EVERY container
that you can make in the future
Demo 3:
Multiple Python/Node versions
https://github.com/containers101/docker-based-pipelines-webinar/tree/master/03_multiple_versions
Data Sharing Between
Pipeline Steps
Data Sharing
● Steps need to communicate
● Output of one step is input for the
next
● Artifacts (node modules, ruby gems,
maven caches) need to persist
● Test reports/Coverage statistics
Caches and Artifacts (Traditional CI solutions)
● “Cache” directive
● Need to be setup explicitly
● Different for each build tool
● “Artifact” directive
● Developers defines exact path of
what needs to be archived
● Used for the result of the whole build
or as shared data between steps
All Steps Share a Volume in Codefresh
Project is on the Volume
● Project is checked out in the volume
● Volume is also persisted between builds
● Any build tools that use the project folder for
artifacts will gain automatic caching
● For other tools you just need to point their
cache to /codefresh/volume
● There is no need for special “artifact settings”.
Just place files in /codefresh/volume
Demo 4 – Node Modules
https://github.com/containers101/docker-based-pipelines-webinar/tree/master/04_volume
Dynamic Docker Images
Docker Tooling on Demand – A Unique Feature
Creating Docker Images On-demand
● Create a Docker image as a step
● Use image in a later step
● Maximum flexibility for build context
● Image contents are not known in
advance
● Codefresh is the only platform at the
moment that offers this capability
Creating Docker Images On-demand
● No need for multiple Docker
images
● “Create and forget” build steps
● Useful for integration tests
● Keep your Docker registry small
and tidy
Demo 5:
Dynamic Docker Images
https://github.com/containers101/docker-based-pipelines-webinar/tree/master/05_dynamic
Codefresh Plugins
Plugins in Traditional CI/CD Platforms
● Specific to the platform (vendor lock-in)
● Tied to a specific language (e.g. Groovy)
● Developer needs to learn proprietary API
● Testing and installing them is difficult
Codefresh Plugins = Docker Images
Codefresh Plugins
● Not tied to any programming language
● Require only Docker knowledge
● Easy to test, easy to search, easy to store
● Several plugins for Codefresh already available
Case study: bintray
● JFrog bintray integration
● There is no official docker image
● A Codefresh plugin with wrap the CLI
● Plugin will be used to query Bintray
Demo 6:
Codefresh Plugins
https://github.com/containers101/docker-based-pipelines-webinar/tree/master/06_plugin
Plugin Directory
http://steps.codefresh.io/
Summary
● Docker-based pipelines use Docker
images as build steps
● Upgrading tools is easy
● Using multiple versions of the same tool is
trivial
● Can dynamically create build steps
● Codefresh plugins are Docker images
Get 120 FREE builds/month!
Signup & schedule a 1:1 at:
Codefresh.io
Build Fast,
Deploy Faster
Q ?
Codefresh.io/events
U E

More Related Content

What's hot

Docker introduction for Carbon IT
Docker introduction for Carbon ITDocker introduction for Carbon IT
Docker introduction for Carbon IT
yannick grenzinger
 
Introduction to Containers and Docker
Introduction to Containers and DockerIntroduction to Containers and Docker
Introduction to Containers and Docker
Rob Loach
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Alan Forbes
 
Microservices using relocatable Docker containers
Microservices using relocatable Docker containersMicroservices using relocatable Docker containers
Microservices using relocatable Docker containers
Mauricio Garavaglia
 
Docker and containers : Disrupting the virtual machine(VM)
Docker and containers : Disrupting the virtual machine(VM)Docker and containers : Disrupting the virtual machine(VM)
Docker and containers : Disrupting the virtual machine(VM)
Rama Krishna B
 
Understand how docker works
Understand how docker worksUnderstand how docker works
Understand how docker works
Justin Li
 
Introduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQIntroduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQ
dotCloud
 
Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015
Jonas Rosland
 
Docker 101: An Introduction
Docker 101: An IntroductionDocker 101: An Introduction
Docker 101: An Introduction
POSSCON
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Hao Fan
 
Docker HK Meetup - 201707
Docker HK Meetup - 201707Docker HK Meetup - 201707
Docker HK Meetup - 201707
Clarence Ho
 
Introduction to docker
Introduction to dockerIntroduction to docker
Introduction to docker
Christophe Muller
 
Docker
DockerDocker
Docker 101 - High level introduction to docker
Docker 101 - High level introduction to dockerDocker 101 - High level introduction to docker
Docker 101 - High level introduction to docker
Dr Ganesh Iyer
 
Docker in pratice -chenyifei
Docker in pratice -chenyifeiDocker in pratice -chenyifei
Docker in pratice -chenyifei
dotCloud
 
Docker Basics
Docker BasicsDocker Basics
Docker Basics
DuckDuckGo
 
Introduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerIntroduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and Docker
Chris Taylor
 
Introduction to Docker - VIT Campus
Introduction to Docker - VIT CampusIntroduction to Docker - VIT Campus
Introduction to Docker - VIT Campus
Ajeet Singh Raina
 
A new model for Docker image distribution
A new model for Docker image distributionA new model for Docker image distribution
A new model for Docker image distribution
Docker, Inc.
 
Docker Overview - Rise of the Containers
Docker Overview - Rise of the ContainersDocker Overview - Rise of the Containers
Docker Overview - Rise of the Containers
Ryan Hodgin
 

What's hot (20)

Docker introduction for Carbon IT
Docker introduction for Carbon ITDocker introduction for Carbon IT
Docker introduction for Carbon IT
 
Introduction to Containers and Docker
Introduction to Containers and DockerIntroduction to Containers and Docker
Introduction to Containers and Docker
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
Microservices using relocatable Docker containers
Microservices using relocatable Docker containersMicroservices using relocatable Docker containers
Microservices using relocatable Docker containers
 
Docker and containers : Disrupting the virtual machine(VM)
Docker and containers : Disrupting the virtual machine(VM)Docker and containers : Disrupting the virtual machine(VM)
Docker and containers : Disrupting the virtual machine(VM)
 
Understand how docker works
Understand how docker worksUnderstand how docker works
Understand how docker works
 
Introduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQIntroduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQ
 
Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015
 
Docker 101: An Introduction
Docker 101: An IntroductionDocker 101: An Introduction
Docker 101: An Introduction
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Docker HK Meetup - 201707
Docker HK Meetup - 201707Docker HK Meetup - 201707
Docker HK Meetup - 201707
 
Introduction to docker
Introduction to dockerIntroduction to docker
Introduction to docker
 
Docker
DockerDocker
Docker
 
Docker 101 - High level introduction to docker
Docker 101 - High level introduction to dockerDocker 101 - High level introduction to docker
Docker 101 - High level introduction to docker
 
Docker in pratice -chenyifei
Docker in pratice -chenyifeiDocker in pratice -chenyifei
Docker in pratice -chenyifei
 
Docker Basics
Docker BasicsDocker Basics
Docker Basics
 
Introduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerIntroduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and Docker
 
Introduction to Docker - VIT Campus
Introduction to Docker - VIT CampusIntroduction to Docker - VIT Campus
Introduction to Docker - VIT Campus
 
A new model for Docker image distribution
A new model for Docker image distributionA new model for Docker image distribution
A new model for Docker image distribution
 
Docker Overview - Rise of the Containers
Docker Overview - Rise of the ContainersDocker Overview - Rise of the Containers
Docker Overview - Rise of the Containers
 

Similar to Docker based-pipelines

Docker based-Pipelines with Codefresh
Docker based-Pipelines with CodefreshDocker based-Pipelines with Codefresh
Docker based-Pipelines with Codefresh
Codefresh
 
VM vs Docker-Based Pipelines
VM vs Docker-Based PipelinesVM vs Docker-Based Pipelines
VM vs Docker-Based Pipelines
Codefresh
 
Docker for developers
Docker for developersDocker for developers
Docker for developers
Anvay Patil
 
Using Docker to boost your development experience with Drupal
Using Docker to boost your development experience with DrupalUsing Docker to boost your development experience with Drupal
Using Docker to boost your development experience with Drupal
dockerizedrupal
 
Developer workflow with docker
Developer workflow with dockerDeveloper workflow with docker
Developer workflow with docker
Lalatendu Mohanty
 
QA Club Kiev #20. Making life easier with Docker
QA Club Kiev #20. Making life easier with DockerQA Club Kiev #20. Making life easier with Docker
QA Club Kiev #20. Making life easier with Docker
QA Club Kiev
 
Docker for dev
Docker for devDocker for dev
Docker for dev
Erik Talboom
 
Docker at MoneyBird
Docker at MoneyBirdDocker at MoneyBird
Docker at MoneyBird
Edwin Vlieg
 
Run automated tests in Docker
Run automated tests in DockerRun automated tests in Docker
Run automated tests in Docker
Oleksandr Metelytsia
 
DevAssistant, Docker and You
DevAssistant, Docker and YouDevAssistant, Docker and You
DevAssistant, Docker and You
BalaBit
 
Docker on Power Systems
Docker on Power SystemsDocker on Power Systems
Docker on Power Systems
Cesar Maciel
 
New Perl module Container::Buildah - SVPerl presentation
New Perl module Container::Buildah - SVPerl presentationNew Perl module Container::Buildah - SVPerl presentation
New Perl module Container::Buildah - SVPerl presentation
Ian Kluft
 
Introduction to Containers & Diving a little deeper into the benefits of Con...
 Introduction to Containers & Diving a little deeper into the benefits of Con... Introduction to Containers & Diving a little deeper into the benefits of Con...
Introduction to Containers & Diving a little deeper into the benefits of Con...
Synergetics Learning and Cloud Consulting
 
Magento Docker Setup.pdf
Magento Docker Setup.pdfMagento Docker Setup.pdf
Magento Docker Setup.pdf
Abid Malik
 
Docker 101
Docker 101Docker 101
Docker 101
Hung-Che Lo
 
[@NaukriEngineering] Docker 101
[@NaukriEngineering] Docker 101[@NaukriEngineering] Docker 101
[@NaukriEngineering] Docker 101
Naukri.com
 
Introduction to Docker - Vellore Institute of Technology
Introduction to Docker - Vellore Institute of TechnologyIntroduction to Docker - Vellore Institute of Technology
Introduction to Docker - Vellore Institute of Technology
Ajeet Singh Raina
 
Настройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'aНастройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'a
corehard_by
 
January OpenNTF Webinar: 4D - Domino Docker Deep Dive
January OpenNTF Webinar: 4D - Domino Docker Deep DiveJanuary OpenNTF Webinar: 4D - Domino Docker Deep Dive
January OpenNTF Webinar: 4D - Domino Docker Deep Dive
Howard Greenberg
 
Docker in everyday development
Docker in everyday developmentDocker in everyday development
Docker in everyday development
Justyna Ilczuk
 

Similar to Docker based-pipelines (20)

Docker based-Pipelines with Codefresh
Docker based-Pipelines with CodefreshDocker based-Pipelines with Codefresh
Docker based-Pipelines with Codefresh
 
VM vs Docker-Based Pipelines
VM vs Docker-Based PipelinesVM vs Docker-Based Pipelines
VM vs Docker-Based Pipelines
 
Docker for developers
Docker for developersDocker for developers
Docker for developers
 
Using Docker to boost your development experience with Drupal
Using Docker to boost your development experience with DrupalUsing Docker to boost your development experience with Drupal
Using Docker to boost your development experience with Drupal
 
Developer workflow with docker
Developer workflow with dockerDeveloper workflow with docker
Developer workflow with docker
 
QA Club Kiev #20. Making life easier with Docker
QA Club Kiev #20. Making life easier with DockerQA Club Kiev #20. Making life easier with Docker
QA Club Kiev #20. Making life easier with Docker
 
Docker for dev
Docker for devDocker for dev
Docker for dev
 
Docker at MoneyBird
Docker at MoneyBirdDocker at MoneyBird
Docker at MoneyBird
 
Run automated tests in Docker
Run automated tests in DockerRun automated tests in Docker
Run automated tests in Docker
 
DevAssistant, Docker and You
DevAssistant, Docker and YouDevAssistant, Docker and You
DevAssistant, Docker and You
 
Docker on Power Systems
Docker on Power SystemsDocker on Power Systems
Docker on Power Systems
 
New Perl module Container::Buildah - SVPerl presentation
New Perl module Container::Buildah - SVPerl presentationNew Perl module Container::Buildah - SVPerl presentation
New Perl module Container::Buildah - SVPerl presentation
 
Introduction to Containers & Diving a little deeper into the benefits of Con...
 Introduction to Containers & Diving a little deeper into the benefits of Con... Introduction to Containers & Diving a little deeper into the benefits of Con...
Introduction to Containers & Diving a little deeper into the benefits of Con...
 
Magento Docker Setup.pdf
Magento Docker Setup.pdfMagento Docker Setup.pdf
Magento Docker Setup.pdf
 
Docker 101
Docker 101Docker 101
Docker 101
 
[@NaukriEngineering] Docker 101
[@NaukriEngineering] Docker 101[@NaukriEngineering] Docker 101
[@NaukriEngineering] Docker 101
 
Introduction to Docker - Vellore Institute of Technology
Introduction to Docker - Vellore Institute of TechnologyIntroduction to Docker - Vellore Institute of Technology
Introduction to Docker - Vellore Institute of Technology
 
Настройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'aНастройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'a
 
January OpenNTF Webinar: 4D - Domino Docker Deep Dive
January OpenNTF Webinar: 4D - Domino Docker Deep DiveJanuary OpenNTF Webinar: 4D - Domino Docker Deep Dive
January OpenNTF Webinar: 4D - Domino Docker Deep Dive
 
Docker in everyday development
Docker in everyday developmentDocker in everyday development
Docker in everyday development
 

More from DevOps.com

Modernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source SoftwareModernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source Software
DevOps.com
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
DevOps.com
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
DevOps.com
 
Next Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykNext Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and Snyk
DevOps.com
 
Vulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudVulnerability Discovery in the Cloud
Vulnerability Discovery in the Cloud
DevOps.com
 
2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions
DevOps.com
 
A New Year’s Ransomware Resolution
A New Year’s Ransomware ResolutionA New Year’s Ransomware Resolution
A New Year’s Ransomware Resolution
DevOps.com
 
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
DevOps.com
 
Don't Panic! Effective Incident Response
Don't Panic! Effective Incident ResponseDon't Panic! Effective Incident Response
Don't Panic! Effective Incident Response
DevOps.com
 
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's CultureCreating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
DevOps.com
 
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with TeleportRole Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
DevOps.com
 
Monitoring Serverless Applications with Datadog
Monitoring Serverless Applications with DatadogMonitoring Serverless Applications with Datadog
Monitoring Serverless Applications with Datadog
DevOps.com
 
Deliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or PrivatelyDeliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or Privately
DevOps.com
 
Securing medical apps in the age of covid final
Securing medical apps in the age of covid finalSecuring medical apps in the age of covid final
Securing medical apps in the age of covid final
DevOps.com
 
How to Build a Healthy On-Call Culture
How to Build a Healthy On-Call CultureHow to Build a Healthy On-Call Culture
How to Build a Healthy On-Call Culture
DevOps.com
 
The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021
DevOps.com
 
Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?
DevOps.com
 
Secure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift EnvironmentsSecure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift Environments
DevOps.com
 
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
DevOps.com
 
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
DevOps.com
 

More from DevOps.com (20)

Modernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source SoftwareModernizing on IBM Z Made Easier With Open Source Software
Modernizing on IBM Z Made Easier With Open Source Software
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
 
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
Comparing Microsoft SQL Server 2019 Performance Across Various Kubernetes Pla...
 
Next Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and SnykNext Generation Vulnerability Assessment Using Datadog and Snyk
Next Generation Vulnerability Assessment Using Datadog and Snyk
 
Vulnerability Discovery in the Cloud
Vulnerability Discovery in the CloudVulnerability Discovery in the Cloud
Vulnerability Discovery in the Cloud
 
2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions2021 Open Source Governance: Top Ten Trends and Predictions
2021 Open Source Governance: Top Ten Trends and Predictions
 
A New Year’s Ransomware Resolution
A New Year’s Ransomware ResolutionA New Year’s Ransomware Resolution
A New Year’s Ransomware Resolution
 
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
Getting Started with Runtime Security on Azure Kubernetes Service (AKS)
 
Don't Panic! Effective Incident Response
Don't Panic! Effective Incident ResponseDon't Panic! Effective Incident Response
Don't Panic! Effective Incident Response
 
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's CultureCreating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
Creating a Culture of Chaos: Chaos Engineering Is Not Just Tools, It's Culture
 
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with TeleportRole Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
Role Based Access Controls (RBAC) for SSH and Kubernetes Access with Teleport
 
Monitoring Serverless Applications with Datadog
Monitoring Serverless Applications with DatadogMonitoring Serverless Applications with Datadog
Monitoring Serverless Applications with Datadog
 
Deliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or PrivatelyDeliver your App Anywhere … Publicly or Privately
Deliver your App Anywhere … Publicly or Privately
 
Securing medical apps in the age of covid final
Securing medical apps in the age of covid finalSecuring medical apps in the age of covid final
Securing medical apps in the age of covid final
 
How to Build a Healthy On-Call Culture
How to Build a Healthy On-Call CultureHow to Build a Healthy On-Call Culture
How to Build a Healthy On-Call Culture
 
The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021
 
Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?
 
Secure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift EnvironmentsSecure Data Sharing in OpenShift Environments
Secure Data Sharing in OpenShift Environments
 
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
How to Govern Identities and Access in Cloud Infrastructure: AppsFlyer Case S...
 
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
Elevate Your Enterprise Python and R AI, ML Software Strategy with Anaconda T...
 

Recently uploaded

HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
FODUU
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 

Recently uploaded (20)

HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
Things to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUUThings to Consider When Choosing a Website Developer for your Website | FODUU
Things to Consider When Choosing a Website Developer for your Website | FODUU
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 

Docker based-pipelines

  • 4. Agenda 1. Docker usage in Continuous Integration 2. Dockerizing build tools as pipeline steps 3. Upgrading build tools to new versions 4. Mixing multiple versions of the same tool in the same pipeline 5. Creating new pipeline steps on the fly ● Demos for everything using Codefresh https://github.com/containers101/ docker-based-pipelines-webinar/
  • 6. “Docker-based” means 2 different things: Using Docker as a deployment package (this is what most people think) Using Docker for build Tooling (this is not what most people think) 90% of cases: “We have migrated to Docker in production”
  • 7. The Basic Software Lifecycle
  • 8. Before Docker – The Dark Ages
  • 12. Old versus new Before: ● Tools are static ● Used for all pipelines After: ● Tools are dynamic ● Isolated to each pipeline Pipeline Java6 Docker Go Docker Pipeline Node9 Docker Python2 Docker Pipeline Python3 Docker Node11 Docker
  • 13. Resources for Docker as Deployment artifact
  • 14. Docker CI/CD - benefits
  • 15. Build farm before Docker ● Multiple tools on each node ● Very hard to manage ●Often nodes had different versions of the same tool ●Developers had to choose the correct machine for their build
  • 16. Build farm after Docker ● Only Docker is installed ● Very easy to manage ●All nodes are exactly the same ●These nodes are often a Kubernetes cluster
  • 17. Using Docker in Continuous Integration ● EVERY build tool is placed in a Docker container ● The build node has only Docker installed and nothing else ● A pipeline is a series of commands that run inside a Docker context ● After each build the node reverts back to its original state ● Developers don’t care about nodes
  • 18. Docker images are everywhere ● They are reusable and shareable ● No need to re-invent the week (e.g. Terraform in Docker) ● Private docker images can be created with your team in mind
  • 19. Container per build step ● Codefresh requires ALL tools to be dockerized ● You can use any public or private Docker image as tooling ● Each build step has a Docker image as context ● Pipelines are described in declarative YAML
  • 20. About Codefresh ● Docker based CI/CD solution ● Each build step is a Docker image ● Native support for Docker, Helm, Kubernetes deployments ● Includes built-in Docker registry and Helm repository ● 20,000+ users
  • 23. Traditional CI Platform Questions: ● Do you support my favorite version of Node/Java/Go/Ruby/Python? ● Do you support maven, yarn, gulp, sbt, gradle, rake? ● Can I run Ansible? Terraform? GCloud? AWS CLI? ● Can I run Kubectl? Helm? Draft?
  • 25. Demo 2: Adding Go and AWS CLI https://github.com/containers101/docker-based-pipelines-webinar/tree/master/02_aws_cli
  • 26. Does Codefresh Support… ● Node 10? ● Perl 6? ● Python2? ● Gradle? ● Vault? ● AWS cli? ● Sonar? ● Findbugs? ● Selenium? ● Snyk? ● Clair?
  • 27. Does Codefresh Support… ● Node 10? ● Perl 6? ● Python2? ● Gradle? ● Vault? ● AWS cli? ● Sonar? ● Findbugs? ● Selenium? ● Snyk? ● Clair? YES! Because there is a Docker image for it
  • 28. Codefresh Pipelines are Future Proof ● You can use ANY existing Docker image from Dockerhub or any other Registry ● Every time a new tool comes out, it can be used right away if packaged in a Docker image
  • 29. Tool Upgrades and Version Clashes
  • 30. Updating a Tool in a Traditional VM Pipeline
  • 32. Demo: Updating Python to 3.7 https://github.com/containers101/docker-based-pipelines-webinar/tree/master/02_aws_cli
  • 33. Using Tools from Different Versions ● Version clashes are a huge pain for both developers and operators ● Legacy projects need to still use old version ● Using different versions in the same pipeline is almost impossible ● Developers want to use latest version of tool, traditional CI/CD platforms may not be able to keep up
  • 34. Wasting Effort on “Version Managers”
  • 35. Wasting Effort on “Version Managers” ● They allow developers to switch between different versions ● Tied to a specific technology/programming language ● Require they own installation/ maintenance ● Must be upgraded for new versions
  • 36. The Problem with Python ● Different python versions are a notorious problem ● Until recently you needed dedicated support from your CI platform ● What happens if I want to test Python 2.5?
  • 37. Replacing “version managers” with Docker ● Works for any language/framework ● Already installed on the build node ● Its own version is independent from the tools ● Can use any public and private image
  • 38. Codefresh “Python Support” ● We support EVERY container ever made ● We support EVERY container that you can make in the future
  • 39. Demo 3: Multiple Python/Node versions https://github.com/containers101/docker-based-pipelines-webinar/tree/master/03_multiple_versions
  • 41. Data Sharing ● Steps need to communicate ● Output of one step is input for the next ● Artifacts (node modules, ruby gems, maven caches) need to persist ● Test reports/Coverage statistics
  • 42. Caches and Artifacts (Traditional CI solutions) ● “Cache” directive ● Need to be setup explicitly ● Different for each build tool ● “Artifact” directive ● Developers defines exact path of what needs to be archived ● Used for the result of the whole build or as shared data between steps
  • 43. All Steps Share a Volume in Codefresh
  • 44. Project is on the Volume ● Project is checked out in the volume ● Volume is also persisted between builds ● Any build tools that use the project folder for artifacts will gain automatic caching ● For other tools you just need to point their cache to /codefresh/volume ● There is no need for special “artifact settings”. Just place files in /codefresh/volume
  • 45. Demo 4 – Node Modules https://github.com/containers101/docker-based-pipelines-webinar/tree/master/04_volume
  • 46. Dynamic Docker Images Docker Tooling on Demand – A Unique Feature
  • 47. Creating Docker Images On-demand ● Create a Docker image as a step ● Use image in a later step ● Maximum flexibility for build context ● Image contents are not known in advance ● Codefresh is the only platform at the moment that offers this capability
  • 48. Creating Docker Images On-demand ● No need for multiple Docker images ● “Create and forget” build steps ● Useful for integration tests ● Keep your Docker registry small and tidy
  • 49. Demo 5: Dynamic Docker Images https://github.com/containers101/docker-based-pipelines-webinar/tree/master/05_dynamic
  • 51. Plugins in Traditional CI/CD Platforms ● Specific to the platform (vendor lock-in) ● Tied to a specific language (e.g. Groovy) ● Developer needs to learn proprietary API ● Testing and installing them is difficult
  • 52. Codefresh Plugins = Docker Images
  • 53. Codefresh Plugins ● Not tied to any programming language ● Require only Docker knowledge ● Easy to test, easy to search, easy to store ● Several plugins for Codefresh already available
  • 54. Case study: bintray ● JFrog bintray integration ● There is no official docker image ● A Codefresh plugin with wrap the CLI ● Plugin will be used to query Bintray
  • 57. Summary ● Docker-based pipelines use Docker images as build steps ● Upgrading tools is easy ● Using multiple versions of the same tool is trivial ● Can dynamically create build steps ● Codefresh plugins are Docker images
  • 58. Get 120 FREE builds/month! Signup & schedule a 1:1 at: Codefresh.io Build Fast, Deploy Faster Q ?