Submit Search
Upload
Lessons Learned: Using Concourse In Production
•
11 likes
•
10,662 views
Shingo Omura
Follow
Lessons Learned: Using Concourse In Production
Read less
Read more
Software
Report
Share
Report
Share
1 of 30
Download now
Download to read offline
Recommended
Apache Presentation
Apache Presentation
Manish Bothra
Demystifying MS17-010: Reverse Engineering the ETERNAL Exploits
Demystifying MS17-010: Reverse Engineering the ETERNAL Exploits
Priyanka Aash
From Zero to Hero with Kafka Connect
From Zero to Hero with Kafka Connect
confluent
Nfs
Nfs
tmavroidis
Using cgroups in docker container
Using cgroups in docker container
Vinay Jindal
Corosync and Pacemaker
Corosync and Pacemaker
Marian Marinov
Learning postgresql
Learning postgresql
DAVID RAUDALES
CloudStack - Top 5 Technical Issues and Troubleshooting
CloudStack - Top 5 Technical Issues and Troubleshooting
ShapeBlue
Recommended
Apache Presentation
Apache Presentation
Manish Bothra
Demystifying MS17-010: Reverse Engineering the ETERNAL Exploits
Demystifying MS17-010: Reverse Engineering the ETERNAL Exploits
Priyanka Aash
From Zero to Hero with Kafka Connect
From Zero to Hero with Kafka Connect
confluent
Nfs
Nfs
tmavroidis
Using cgroups in docker container
Using cgroups in docker container
Vinay Jindal
Corosync and Pacemaker
Corosync and Pacemaker
Marian Marinov
Learning postgresql
Learning postgresql
DAVID RAUDALES
CloudStack - Top 5 Technical Issues and Troubleshooting
CloudStack - Top 5 Technical Issues and Troubleshooting
ShapeBlue
Treinamento Elasticsearch - Parte 1
Treinamento Elasticsearch - Parte 1
Luiz Henrique Zambom Santana
Properly Use Parallel DML for ETL
Properly Use Parallel DML for ETL
Andrej Pashchenko
Cilium - BPF & XDP for containers
Cilium - BPF & XDP for containers
Docker, Inc.
Automating AWS Infrastructure Provisioning Using Concourse and Terraform
Automating AWS Infrastructure Provisioning Using Concourse and Terraform
Cesar Rodriguez
Containerd Internals: Building a Core Container Runtime
Containerd Internals: Building a Core Container Runtime
Phil Estes
PostgreSQL on EXT4, XFS, BTRFS and ZFS
PostgreSQL on EXT4, XFS, BTRFS and ZFS
Tomas Vondra
Unix operating system
Unix operating system
midhunjose4u
Container security
Container security
Anthony Chow
Docker introduction
Docker introduction
Julien Maitrehenry
Workshop on CIFS / SMB Protocol Performance Analysis
Workshop on CIFS / SMB Protocol Performance Analysis
PerformanceVision (previously SecurActive)
CloudInit Introduction
CloudInit Introduction
Publicis Sapient Engineering
Step by Step to Install oracle grid 11.2.0.3 on solaris 11.1
Step by Step to Install oracle grid 11.2.0.3 on solaris 11.1
Osama Mustafa
Docker Swarm for Beginner
Docker Swarm for Beginner
Shahzad Masud
Making MySQL highly available using Oracle Grid Infrastructure
Making MySQL highly available using Oracle Grid Infrastructure
Ilmar Kerm
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast Storage
Kernel TLV
DevOps - Interview Question.pdf
DevOps - Interview Question.pdf
MinhTrnNht7
Key aggregate cryptosystem for scalable data sharing in cloud storage
Key aggregate cryptosystem for scalable data sharing in cloud storage
Shruthi Iyer
How to size up an Apache Cassandra cluster (Training)
How to size up an Apache Cassandra cluster (Training)
DataStax Academy
Lisa 2015-gluster fs-introduction
Lisa 2015-gluster fs-introduction
Gluster.org
Building Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCC
Kernel TLV
LAS16-209: Finished and Upcoming Projects in LMG
LAS16-209: Finished and Upcoming Projects in LMG
Linaro
Kubernetes 101
Kubernetes 101
Stanislav Pogrebnyak
More Related Content
What's hot
Treinamento Elasticsearch - Parte 1
Treinamento Elasticsearch - Parte 1
Luiz Henrique Zambom Santana
Properly Use Parallel DML for ETL
Properly Use Parallel DML for ETL
Andrej Pashchenko
Cilium - BPF & XDP for containers
Cilium - BPF & XDP for containers
Docker, Inc.
Automating AWS Infrastructure Provisioning Using Concourse and Terraform
Automating AWS Infrastructure Provisioning Using Concourse and Terraform
Cesar Rodriguez
Containerd Internals: Building a Core Container Runtime
Containerd Internals: Building a Core Container Runtime
Phil Estes
PostgreSQL on EXT4, XFS, BTRFS and ZFS
PostgreSQL on EXT4, XFS, BTRFS and ZFS
Tomas Vondra
Unix operating system
Unix operating system
midhunjose4u
Container security
Container security
Anthony Chow
Docker introduction
Docker introduction
Julien Maitrehenry
Workshop on CIFS / SMB Protocol Performance Analysis
Workshop on CIFS / SMB Protocol Performance Analysis
PerformanceVision (previously SecurActive)
CloudInit Introduction
CloudInit Introduction
Publicis Sapient Engineering
Step by Step to Install oracle grid 11.2.0.3 on solaris 11.1
Step by Step to Install oracle grid 11.2.0.3 on solaris 11.1
Osama Mustafa
Docker Swarm for Beginner
Docker Swarm for Beginner
Shahzad Masud
Making MySQL highly available using Oracle Grid Infrastructure
Making MySQL highly available using Oracle Grid Infrastructure
Ilmar Kerm
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast Storage
Kernel TLV
DevOps - Interview Question.pdf
DevOps - Interview Question.pdf
MinhTrnNht7
Key aggregate cryptosystem for scalable data sharing in cloud storage
Key aggregate cryptosystem for scalable data sharing in cloud storage
Shruthi Iyer
How to size up an Apache Cassandra cluster (Training)
How to size up an Apache Cassandra cluster (Training)
DataStax Academy
Lisa 2015-gluster fs-introduction
Lisa 2015-gluster fs-introduction
Gluster.org
Building Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCC
Kernel TLV
What's hot
(20)
Treinamento Elasticsearch - Parte 1
Treinamento Elasticsearch - Parte 1
Properly Use Parallel DML for ETL
Properly Use Parallel DML for ETL
Cilium - BPF & XDP for containers
Cilium - BPF & XDP for containers
Automating AWS Infrastructure Provisioning Using Concourse and Terraform
Automating AWS Infrastructure Provisioning Using Concourse and Terraform
Containerd Internals: Building a Core Container Runtime
Containerd Internals: Building a Core Container Runtime
PostgreSQL on EXT4, XFS, BTRFS and ZFS
PostgreSQL on EXT4, XFS, BTRFS and ZFS
Unix operating system
Unix operating system
Container security
Container security
Docker introduction
Docker introduction
Workshop on CIFS / SMB Protocol Performance Analysis
Workshop on CIFS / SMB Protocol Performance Analysis
CloudInit Introduction
CloudInit Introduction
Step by Step to Install oracle grid 11.2.0.3 on solaris 11.1
Step by Step to Install oracle grid 11.2.0.3 on solaris 11.1
Docker Swarm for Beginner
Docker Swarm for Beginner
Making MySQL highly available using Oracle Grid Infrastructure
Making MySQL highly available using Oracle Grid Infrastructure
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast Storage
DevOps - Interview Question.pdf
DevOps - Interview Question.pdf
Key aggregate cryptosystem for scalable data sharing in cloud storage
Key aggregate cryptosystem for scalable data sharing in cloud storage
How to size up an Apache Cassandra cluster (Training)
How to size up an Apache Cassandra cluster (Training)
Lisa 2015-gluster fs-introduction
Lisa 2015-gluster fs-introduction
Building Network Functions with eBPF & BCC
Building Network Functions with eBPF & BCC
Similar to Lessons Learned: Using Concourse In Production
LAS16-209: Finished and Upcoming Projects in LMG
LAS16-209: Finished and Upcoming Projects in LMG
Linaro
Kubernetes 101
Kubernetes 101
Stanislav Pogrebnyak
GeoServer Developers Workshop
GeoServer Developers Workshop
Jody Garnett
Why kernelspace sucks?
Why kernelspace sucks?
OpenFest team
Road to sbt 1.0 paved with server
Road to sbt 1.0 paved with server
Eugene Yokota
Road to sbt 1.0: Paved with server (2015 Amsterdam)
Road to sbt 1.0: Paved with server (2015 Amsterdam)
Eugene Yokota
OpenDataPlane Testing in Travis
OpenDataPlane Testing in Travis
Dmitry Baryshkov
My "Perfect" Toolchain Setup for Grails Projects
My "Perfect" Toolchain Setup for Grails Projects
GR8Conf
Leveraging Android's Linux Heritage at AnDevCon IV
Leveraging Android's Linux Heritage at AnDevCon IV
Opersys inc.
Parallelizing CI using Docker Swarm-Mode
Parallelizing CI using Docker Swarm-Mode
Akihiro Suda
HPC on OpenStack
HPC on OpenStack
Erich Birngruber
Rejekts 24 EU No GitOps Pain, No Platform Gain
Rejekts 24 EU No GitOps Pain, No Platform Gain
Łukasz Piątkowski
Deep dive - Concourse CI/CD and Pipelines
Deep dive - Concourse CI/CD and Pipelines
Syed Imam
Ci for android OS
Ci for android OS
Jarek Potiuk
Real-World Docker: 10 Things We've Learned
Real-World Docker: 10 Things We've Learned
RightScale
Leveraging Android's Linux Heritage at AnDevCon VI
Leveraging Android's Linux Heritage at AnDevCon VI
Opersys inc.
Leveraging Android's Linux Heritage at AnDevCon V
Leveraging Android's Linux Heritage at AnDevCon V
Opersys inc.
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Ambassador Labs
LMG Lightning Talks - SFO17-205
LMG Lightning Talks - SFO17-205
Linaro
Devops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShift
Yaniv cohen
Similar to Lessons Learned: Using Concourse In Production
(20)
LAS16-209: Finished and Upcoming Projects in LMG
LAS16-209: Finished and Upcoming Projects in LMG
Kubernetes 101
Kubernetes 101
GeoServer Developers Workshop
GeoServer Developers Workshop
Why kernelspace sucks?
Why kernelspace sucks?
Road to sbt 1.0 paved with server
Road to sbt 1.0 paved with server
Road to sbt 1.0: Paved with server (2015 Amsterdam)
Road to sbt 1.0: Paved with server (2015 Amsterdam)
OpenDataPlane Testing in Travis
OpenDataPlane Testing in Travis
My "Perfect" Toolchain Setup for Grails Projects
My "Perfect" Toolchain Setup for Grails Projects
Leveraging Android's Linux Heritage at AnDevCon IV
Leveraging Android's Linux Heritage at AnDevCon IV
Parallelizing CI using Docker Swarm-Mode
Parallelizing CI using Docker Swarm-Mode
HPC on OpenStack
HPC on OpenStack
Rejekts 24 EU No GitOps Pain, No Platform Gain
Rejekts 24 EU No GitOps Pain, No Platform Gain
Deep dive - Concourse CI/CD and Pipelines
Deep dive - Concourse CI/CD and Pipelines
Ci for android OS
Ci for android OS
Real-World Docker: 10 Things We've Learned
Real-World Docker: 10 Things We've Learned
Leveraging Android's Linux Heritage at AnDevCon VI
Leveraging Android's Linux Heritage at AnDevCon VI
Leveraging Android's Linux Heritage at AnDevCon V
Leveraging Android's Linux Heritage at AnDevCon V
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
LMG Lightning Talks - SFO17-205
LMG Lightning Talks - SFO17-205
Devops with Python by Yaniv Cohen DevopShift
Devops with Python by Yaniv Cohen DevopShift
Recently uploaded
HR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.com
Fatema Valibhai
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software Solutions
Mehedi Hasan Shohan
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
gurkirankumar98700
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
AxelRicardoTrocheRiq
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
kalichargn70th171
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
joe51371421
What is Binary Language? Computer Number Systems
What is Binary Language? Computer Number Systems
JheuzeDellosa
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
ICS
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Alberto González Trastoy
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanation
kaushalgiri8080
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The Ugly
Frank van der Linden
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
Tier1 app
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
soniya singh
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
OnePlan Solutions
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Christina Lin
EY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
Neo4j
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
Ortus Solutions, Corp
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
soniya singh
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
MyIntelliSource, Inc.
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
kalichargn70th171
Recently uploaded
(20)
HR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.com
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software Solutions
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽❤️🧑🏻 89...
Salesforce Certified Field Service Consultant
Salesforce Certified Field Service Consultant
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
What is Binary Language? Computer Number Systems
What is Binary Language? Computer Number Systems
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanation
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The Ugly
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
EY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
Lessons Learned: Using Concourse In Production
1.
Using Concourse in
Production - Lessons Learned - Shingo Omura(@everpeace) omura@chatwork.com ChatWork, Inc. Concourse Meetup #5 2017/03/13
2.
© ChatWork All
rights reserved.© ChatWork All rights reserved. Outline ● About ChatWork ● Our Context From the Point of View of Infrastructure ● Our Use Case ● Good parts ● Pipeline Tips ● Small Bad parts (expect to improve) 2
3.
© ChatWork All
rights reserved. Group Chat File Sharing Task Management Video Conference About ChatWork ~Group Chat for Global Teams~ 3
4.
© ChatWork All
rights reserved. ChatWork is growing rapidly ● 127,000 organizations ○ number of users is not opened ● 205 countries or regions ● 6 languages supported as of 2017/02 4
5.
Our Context From the
Point of View of Infrastructure
6.
© ChatWork All
rights reserved. New Infrastracture Project (1/2) ● Current Infra ○ EC2 based apps, deploy servers(for capistorano) ○ Jenkins servers for CI/CD ● Pain points ○ Ops team doesn’t scale ■ release always have to be done with Infra team members ○ AWS env and Jenkins are hard to sandboxing ■ part of aws resouces are managed by terraform, but not all ■ deployment flow is hard to develop and testing 6
7.
© ChatWork All
rights reserved. New Infrastracture Project (2/2) ● Next Infra ○ Kubernetes and Helm with Dockerized apps ○ Concourse CI for CI/CD ● Benefits ○ Kubernetes accelarate DevOps ■ App team can fully manage their deployment cycle by themselves. ■ minikube is really helpful for local dev environemnt. ■ kubernetes team can focus on reliability of Kuberentes. ○ Concourse CI does too! ← Today’s Focus ■ reduces operational load ■ helps agile development of deployment/testing process ● Status ○ Using from new messaging backend (released the last december) ○ Current system is planned to migrate to this next infra 7
8.
Our Use Case
9.
© ChatWork All
rights reserved. Overview of deployment system ● Concourse is deployed by concourse-aws ○ maintained by @mumoshu (my-colleague) and @everpeace (me) ● Branching model is Gitlab flow with Environment Branches ● chatwork-notify-resource for notification staging branch staging environment production environment master branch push im age build and deploy helm package build and deploy helm packagepush image pull image pull image notify 9
10.
© ChatWork All
rights reserved. Our build pipeline environment can be split by ‘groups’ notification resource 10
11.
© ChatWork All
rights reserved. Our build pipeline test&build jobs deploy jobs rollback jobs 11
12.
Good Parts Learned
13.
© ChatWork All
rights reserved. Good Parts ● concourse.ci is extemely well-documented ○ You can start trying concourse in 5 min. ■ virtualbox and vagrant: just ‘vagrant up’!! ■ docker-compose support!! ○ easty to write pipelines thanks to comrehensive reference ● easy to deploy & version up (thanks to concourse-aws :-P ) ○ initial deploy: 3 steps ■ ‘build-amis.sh’ → edit ‘cluster.yml’ → ‘concourse-aws up’ ○ version up: similar 3 steps ■ ‘build-amis.sh’(new version) → edit ‘cluster.yml’(new ami) → ‘concourse-aws up’ 13
14.
© ChatWork All
rights reserved. Good Parts (cont.) ● Concourse frees us from ”plugin hell” ○ all resource is provided by docker image ○ task environment can be injected by docker image too ○ no need to manage backups of CI servers!! ● Multi tenancy ‘team’ support ■ multiple team can share CI server resources ■ but isolated appropriately ■ each app team can have controll in their team ● Various authentication scheme support ■ concourse need not to have user database ■ we use github authentication 14
15.
© ChatWork All
rights reserved. ● easy to develop pipelines ○ Pipeline developed & tested in local env can be deployed directly to production concourse ■ Concourse CI’s pipeline is stateless and reproductive ■ Concourse & Kubernetes both supports local env (minikube & concourse vagrant box) Good Parts (cont.) 15
16.
© ChatWork All
rights reserved. Good Parts (cont.) ● easy to extend/custom ○ easy to develop custom resource. ■ you only need to develop 3 commands(check, in, out) whose returns json objects. ■ language agnostic! you can choose your own language!! ○ easy to prepare task environment ■ when you need some task environment in which some toolkit is installed, you just push docker image to any repository and specify the image to your task definition task.yml --- platform: linux image_resource: type: docker-image source: repository: /yourown/image tag: '1.1' 16
17.
Pipeline tips Learned
18.
© ChatWork All
rights reserved. Pipeline tips: summary ● Use groups for large pipeline ● Use aggregate for running in parallel (useful for resources) ● Use “[ci skip]” keyword to commit message when Concourse commits/push to git repo ● on_success/on_failure hook is useful for notification ● input_mapping/output_mapping is useful for shared task definition ● use attempts for deployment task due to intermittent network failure ● @making’s trick is helpful for build caches(sbt, ivy, maven) 18
19.
© ChatWork All
rights reserved. Pipeline Tips ● Use groups for large pipeline to group many jobs ● Use aggregate for multiple resources (useful for resources) pipeline.yml groups: - name: master jobs: - job-for-master - name: production jobs: - job-for-production pipeline.yml plan: - aggregate: - get: app-repo trigger: true - get: tool-repo - get: sbt-ivy-cache those 3 get runs in parallel 19
20.
© ChatWork All
rights reserved. Pipeline Tips ● Use “[ci skip]” keyword to commit message when Concourse commits/push to git repo ○ git resource skip commits with [ci skip] keywords ○ It’s really useful when ■ back merge: “merging release branch to develop branch” ● the commit is wanted to skip CI process ■ the commit bumping versions ● when using sbt, version number is embedded to repo 20
21.
© ChatWork All
rights reserved. ● on_success/on_failure hook is useful for notification Pipeline Tips pipeline.yml - task: deploy-write-api-to-dev-kube file: foo/task.yml on_success: task: chatwork-notification file: tasks/notify_chatwork.yml on_failure: task: chatwork-notification file: tasks/notify_chatwork.yml on_failure on_success 21
22.
© ChatWork All
rights reserved. ● input_mapping/output_mapping is useful for shared task definition Pipeline Tips pipeline.yml - task: test-pull-request file: pull-request/ci/tasks/unit.yml input_mapping: { repo: pull-request } - task: unit file: master/ci/tasks/unit.yml input_mapping: { repo: master } ci/tasks/unit.yml --- platform: linux image_resource: type: docker-image source: repository: yourown/toolbox inputs: - name: repo run: path: /bin/bash args: - repo/ci/tasks/unit.sh 22
23.
© ChatWork All
rights reserved. ● use attempts for deployment task due to intermittent network failure Pipeline Tips pipeline.yml ... - task: deploy-write-api-to-dev-kube file: ..snip../deploy-to-kube-helm.yml attempts: {{attempts}} attempts=3 23
24.
© ChatWork All
rights reserved. ● @making’s trick is helpful for build caches(sbt, ivy, maven) ○ prepare own cache docker image repo (anywhere) ○ archives cache files as rootfs.tar and push it directly to the image repo ○ related issue is now open: Caching directories between runs of a task #230 Pipeline Tips 24
25.
Small Bad Parts (expect
to improve)
26.
© ChatWork All
rights reserved. Small Bad Parts (expect to improve) ● No fine-grained authorization (No role based aaccess control) ○ every team member can take full controll in the team ○ ‘fly get-pipeline’ exposes all creadentials embedded in pipelines ○ We sometime want to split ■ people who can write/read pipeline ■ people who can just view logs and trigger jobs (no rights to change pipelines but can just operate the pipeline) ○ related issues are open ■ Credential management #19 ■ Individual/fine-grained access control #23 26
27.
© ChatWork All
rights reserved. Small Bad Parts (expect to improve)(cont.) ● No parameterized job ○ we would like to deploy specific feature branch to shared dev environment ○ How could do this with Concourse?? Any Idea?? ○ git-multibranch-resource could achive similar thing ■ branch name convention which will be deployed to shared dev env should be agreed ○ Perhaps `fly exec` prompts user input? 27
28.
© ChatWork All
rights reserved. Small Bad Parts (expect to improve)(cont.) ● No Docker Compose in task ○ the issue is now open: Docker Compose support in Task definitions #324 ■ integration test task with app & local db containers ● FYI: various improvements are disscued in https://github.com/concourse/design-notes/issues 28
29.
Thank you for
Listening!!
30.
We’re Hiring!!! Search “ChatWork”
in Wantedly https://www.wantedly.com/companies/chatwork/projects
Download now