This document provides an overview of continuous integration (CI) and continuous delivery (CD) using Concourse. It introduces Concourse and discusses why containers and pipelines are useful for CI/CD. It then demonstrates how to use Concourse in practice with examples and explains how to get started using Concourse for CI/CD workflows. The key aspects covered are Concourse's simple and scalable pipeline model using resources, tasks, and jobs, how containerization provides dependable and isolated executions, and how pipelines allow flexible yet efficient software testing and deployment.

1. © Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0
October 25 2017
CI and CD Using
Concourse For PCF

2. Cover w/ Image
Agenda
■ Introduction to CI & CD
■ Concourse Overview
■ Why Containers and Pipelines are
Useful
■ Concourse in practice
■ How to get started using Concourse for
CI/CD
■ Q & A

3. Introduction to CI & CD
CONCOURSE FOR PCF

4. What is CI &
CD ?
Continuous Integration is the practice of merging
your changes back into the main branch as often as
possible. Changes are continuously checked against
a suite of automated tests to ensure that the
application isn’t broken as a result of the changes
Continuous Delivery is the practice of building,
testing and releasing software into production
environments faster, and more frequently

5. What Problems
Do CI & CD
solve?
Continuous Integration:
● The application should always be in a working state
● All software is broken until you prove it works
● Write tests to capture the business requirements,
acceptance requirements, and expected functionality
Continuous Delivery:
● Code is only valuable and useful when it is in the
hands of the customer
● Bugs happen. Business requirements change. Push
to production as often as possible, as quickly as
possible

6. Concourse Overview
CONCOURSE FOR PCF

7. Concourse's user interface helps
you find out what you need to
know about your builds as quickly
as possible.
Streamlined
UI

10. Rather than a myriad of
checkboxes, pipelines are defined
as a single declarative config file,
composing together just three core
concepts.
As your project grows, your
pipeline will grow with it, and
remain understandable.
Simple and
Scalable

11. Pipelines are defined as a single
declarative config file composing
together just three core concepts.

12. Resources
● Track versions of
external artifacts used for
CI / CD
● Any entity that can be
checked for new versions,
pulled down at a specific
version, and/or pushed to
● Can by one of many types
of built in resources; git
repositories, Amazon S3
Buckets, Docker Images,
or a custon
implementation
Three Core Concepts
Tasks
● Allow the execution of
arbitrary scripts against a
set of resources.
● Can output directories
representing a new
resource version.
● Run in a container using a
configurable container
image.
Jobs
● Represent the plan for a
build step within a
pipeline.
● Can contain operations
against resources, or
tasks as steps.
● Builds of a job’s plan can
be triggered manually or
trigger on new versions of
resource.

13. jobs:
- name: run-unit-tests
plan:
- get: source-code
- task: run-tests
file: source-code/unit.yml
resources:
- name: source-code
type: git
source:
uri: ((repo_uri))
- name: release
type: cf
source:
...
- put: release

14. As your project grows, your
pipeline will grow with it, and
remain understandable.

17. resources:
...
jobs:
- name: run-unit-tests
plan:
- get: source-code
- task: run-tests
file: source-code/unit.yml
- name: deploy-to-staging
plan:
- get: source-code
passed: [run-unit-tests]
- put: release

19. Why Containers Are Useful
CONCOURSE FOR PCF

20. Containers are a form of virtualization using Linux
features which allow the creation of effectively a
separate Linux machine running on a host.
Concourse creates containers on Workers which run
the same containerization APIs used by Cloud
Foundry.
Containerization is accomplished using Linux
features (chroot jails, namespaces, mounts) to isolate
a process, and create a new directory system, which
can be based off of a container image from Docker
Hub or other registry. This allows Concourse users to
use public, OSS container images for whichever
programming language their software is using, or
build their own.
What are
Containers ?

21. Concourse controls the inputs to
your pipeline so that the results are
repeatable every time.
Rather than sharing state, every
task runs in its own container,
controlling its own dependencies.
Dependable
Results

22. Rather than sharing state, every
task runs in its own container,
controlling its own dependencies.

23. Dependability
Through
Containerization
> test.sh

25. git@my/repo.git@d4e5f6

26. Why Pipelines Are Useful
CONCOURSE FOR PCF

27. Developing CI and CD pipelines in Concourse allows
users to efficiently and resiliently test and ship
software.
Since resources are the connective tissue between
jobs, test and release pipelines can be separated into
smaller units, or connected as one large pipeline.
Using the passed semantics for resources, pipeline
developers can perfom fan-out and fan-in style logic
to parallelize their testing. Fan-in logic can be used to
ensure only a tested set of microservices reach a
deployment job.
Why Pipelines
are Useful

28. a1b2c3
d4e5f6
d4e5f6

30. Concourse In Practice
CONCOURSE FOR PCF

31. Concourse’s simple core concepts ammount to a CI/
CD system which allows immense flexibility for users.
The use of containerization to isolate and virtualize
the environment for testing allows for flexibility while
retaining reliability.
Concourse does not need to support - or even have
an opinion on - specific programming languages, build
systems, or deployment automation, or cloud lifecycle
management tools. If it can run in a container, you
can put it in your pipeline.
Concourse in
Practice

32. Concourse is purpose-built for
Microservices CI and CD

34. ---
platform: linux
image_resource:
type: docker-image
source:
repository: ruby
tag: '2.1'
inputs:
- name: 😼-source-code
run:
path: my-app/scripts/test

35. ---
platform: linux
image_resource:
type: docker-image
source:
repository: ruby
tag: '2.1'
inputs:
- name: 😼-source-code
run:
path: my-app/scripts/test

36. The git resource can pull and push to
git repositories.
The hg resource can pull and push to
Mercurial repositories.
The s3 resource can fetch from and
upload to S3 buckets.
The github-release resource can
fetch and publish versioned GitHub
resources.
The docker-image resource can fetch,
build, and push Docker images
The cf resource can deploy an
application to Cloud Foundry.
The bosh-deployment resource can
deploy BOSH stemcells and releases.

37. How to Get Started Using
Concourse
CONCOURSE FOR PCF

38. https://ci.concourse.ci

39. https://concourse.ci/tutorials.html

40. Q & A
CONCOURSE FOR PCF

41. Transforming How The World Builds Software
© Copyright 2017 Pivotal Software, Inc. All rights Reserved.