An introduction to Concourse CI

1. CONCOURSE CI
AN INTRODUCTION
MATTEO GAZZETTA
08/02/2018

2. KEY FACTOR
• Pipeline first (Declarative with yaml)
• Container based (Language Agnostic)
• Configuration as code
• Zero Snowflake-able configuration
• Good visibility of the «flows»
• Written in Go

3. ARCHITECTURE
• Web
• ATC:
• API, web UI, pipeline orchestration
• TSA:
• SSH server for authorizing worker registration
• Worker
• Garden:
Container runtime
• Baggageclaim:
Manages volumes (caches, folders)
• Fly
• CLI for interacting and configuring pipelines

4. CONCEPTS
• Resource:
• Any entity that can be checked for new versions, pulled down at a specific version, and/or
pushed up to idempotently create new versions.
• Task:
• Execution of a script in an isolated environment with dependent resources available to it
• Job:
• Describes some actions to perform when dependent resources change
• Pipeline:
• Composition of Task - Resource - Job

5. PIPELINE TIPS
• Use groups for large pipeline
• Use aggregate for running in parallel (useful for resources)
• Use serial when you need atomic changes (like updating version)
• Use “[ci skip]” keyword to commit message when Concourse commits/push to git repo
• on_success/on_failure hook is useful for notification (slack, email, gitlab)
• input_mapping/output_mapping is useful for shared task definition
• Use attempts for task due to random failures (network, …)
• Define caches in task definition to increase performance

6. DEMO
• Hello World
• Base Pipelines
• Integration testing with DCINDING
• Credential Management

7. HELLO WORLD
fly -t main set-pipeline -p hello-world -c hello.yml

8. DCINDING

9. CONCLUSION
• Concourse is pipeline oriented
• Workers are tools agnostic (container centric solution)
• Process is defined as code
• Concourse only has 3 simple abstractions (task, job, resource)
• Reduces operational load
• Helps agile development of deployment/testing process

10. FUTURE
• Spaces (#1707)