1.
Install
Concourse CI
with BOSH
Toshiaki Maki (@making)
Sr. Solutions Architect @Pivotal
2016-‐‑‒05-‐‑‒19
Concourse CI Tokyo Meetup

2.
Who am I ?
•Toshiaki Maki (@making)
•https://blog.ik.am
•Sr. Solutions Architect
•Spring Framework enthusiast
Spring
Framework
徹底⼊入⾨門
(Coming
Soon?)
Perfect
Java EE
(Coming
Soon?)

3.
How to Install
•Vagrant Image
•Executable Binaries
•BOSH Release
https://concourse.ci/installing.html
Easy
Hard Production
Ready
Local
Dev

4.
Vagrant
$ vagrant init concourse/lite
$ vagrant up

5.
Vagrant

6.
BOSH
•Powerful Provisioning Tool to deploy distributed
system
• Monitor / Health Check
• Rolling Updates
• Networking
• Storage Management
• VM Provisioning
• IaaS Agnostic
• ...

7.
Do we really need BOSH?
"If you're not yet familiar
with BOSH, learning it will be
a bit of an investment, but it
should pay off in spades"
http://concourse.ci/clusters-‐‑‒with-‐‑‒bosh.html

8.
BOSH components
https://bosh.io/docs/bosh-‐‑‒components.html

9.
How to install with BOSH
1. Set up IaaS
2. Install BOSH Director using bosh-‐‑‒init
3. Upload Stemcell
4. Upload Releases
5. Create manifest file
6. Deploy the system using manifest file

10.
•Deploy Concourse to AWS
•Deploy Concourse to Azure

11.
Manifest samples
https://github.com/making/concouse-‐‑‒ci-‐‑‒manifest-‐‑‒samples

12.
Deploy Concourse to AWS

13.
Prepare VPC

14.
VPC (10.0.0.0/16)
public(10.0.0.0/24) private(10.0.1.0/24)
NATEIP

15.
Install BOSH Director

16.
Create Keypair

17.
Security Group

18.
Manifest for bosh-‐‑‒init
• https://bosh.io/docs/init-‐‑‒aws.html#create-‐‑‒manifest
Copy & Paste
to bosh.yml

19.
Replace placeholders
Name Value
ELASTIC-‐‑‒IP <Your New EIP for Director> there are 3 places to
replace
SUBNET-‐‑‒ID subnet-‐‑‒xxxxxxxx public subnet
AVAILABILITY-‐‑‒ZONE ap-‐‑‒northeast-‐‑‒1a
ACCESS-‐‑‒KEY-‐‑‒ID xxxxxxxx
SECRET-‐‑‒ACCESS-‐‑‒KEY xxxxxxxx
REGION ap-‐‑‒northeast-‐‑‒1

20.
Deploy
Download from https://bosh.io/docs/install-‐‑‒bosh-‐‑‒init.html
$ bosh-init deploy bosh.yml

21.
After 20min

22.
VPC (10.0.0.0/16)
public(10.0.0.0/24) private(10.0.1.0/24)
NAT
BOSH
Director
EIP
EIP

23.
$ sudo gem install bosh_cli --no-ri --no-rdoc

24.
$ bosh target 52.196.159.101
Target set to `my-bosh'
Your username: admin
Enter password:
Logged in as `admin'

25.
IaaS Specific Configuration
az: ...
vm_types: ...
disk_types: ...
networks: ...
compilation: ...
aws.yml

26.
IaaS Specific Configuration
azs:
- name: z1
cloud_properties:
{availability_zone: ap-northeast-1a}
aws.yml

27.
IaaS Specific Configuration
vm_types:
- name: web
cloud_properties:
instance_type: t2.micro
ephemeral_disk: {size: 3000, type: gp2}
security_groups: [web, bosh]
- name: database
cloud_properties: ...
aws.yml

28.
IaaS Specific Configuration
disk_type:
- name: database
disk_size: 50_000
cloud_properties: {type: gp2}
aws.yml

29.
Set Iaas Config
$ bosh update cloud-config aws.yml
Acting as user 'admin' on 'my-bosh'
Successfully updated cloud config

30.
Upload Stemcell
$ bosh upload stemcell
https://bosh.io/d/stemcells/bosh-aws-xen-
hvm-ubuntu-trusty-go_agent?v=3232.3

31.
Upload Releases (garden,
concourse)
$ bosh upload release
https://bosh.io/d/github.com/cloudfoundry-
incubator/garden-linux-release?v=0.337.0
$ bosh upload release
https://bosh.io/d/github.com/concourse/con
course?v=1.2.0

32.
name: concourse
director_uuid: ...
releases: ...
stemcells: ...
instance_groups: ...
update: ...
Create manifest for Concourse
concourse.yml

33.
Create manifest for Concourse
instance_groups: ...
- name: web
...
- name: db
...
- name: worker
...
concourse.yml

34.
Deploy Concourse CI
$ bosh deployment concourse.yml
$ bosh -n deploy

35.
VPC (10.0.0.0/16)
public(10.0.0.0/24) private(10.0.1.0/24)
NAT
BOSH
Director
EIP
EIP
WebEIP
Worker
DB

36.
Scale out
- name: worker
instances: 3
vm_type: worker
concourse.yml
$ bosh -n deploy

37.
VPC (10.0.0.0/16)
public(10.0.0.0/24) private(10.0.1.0/24)
NAT
BOSH
Director
EIP
EIP
WebEIP
WorkerWorkerWorker
DB

38.
Health Check

39.
Health Check

40.
Version up
$ bosh upload release
https://bosh.io/d/github.com/concourse/con
course?v=1.3.0
$ bosh deploy

41.
Deploy Concourse to Azure

42.
Use template
https://azure.microsoft.com/en-‐‑‒us/documentation/templates/bosh-‐‑‒setup/

43.
After 1 hour ...

44.
Already prepared👍

45.
$ ssh bosh@40.115.251.81
bosh@bosh:~$ bosh target
Current target is https://10.0.0.4:25555 (bosh)

46.
IaaS Specific Configuration
az: ...
vm_types: ...
disk_types: ...
networks: ...
compilation: ...
azure.yml

47.
IaaS Specific Configuration
vm_types:
- name: web
cloud_properties:
instance_type: Standard_D1
- name: database
cloud_properties: ...
azure.yml

48.
Set Iaas Config
bosh@bosh:~$ bosh update cloud-config azure.yml
Acting as user 'admin' on 'bosh'
Successfully updated cloud config

49.
Upload Stemcell
$ bosh upload stemcell
https://bosh.io/d/stemcells/bosh-azure-
hyperv-ubuntu-trusty-go_agent?v=3232.4

50.
Upload Releases (garden,
concourse)
$ bosh upload release
https://bosh.io/d/github.com/cloudfoundry-
incubator/garden-linux-release?v=0.337.0
$ bosh upload release
https://bosh.io/d/github.com/concourse/con
course?v=1.2.0

51.
Deploy Concourse CI
$ bosh deployment concourse.yml
$ bosh -n deploy
Manifest file is as
same as AWS's

52.
AWS Azure
IaaS Setup Create
VPC, NAT, Keypair, SG, EIP
Use "BOSH Setup"
Template
(other than SG)Director • Create Manifestfile (bosh.yml)
• bosh-‐‑‒init
Cloud Config • Create IaaS config (aws.yml)
• bosh update cloud-‐‑‒cofig
• Create IaaS config
(azure.yml)
• bosh update cloud-‐‑‒cofig
Manifest for
Concourse
Create manifest file (IaaS Independent)
Stemcell Upload for AWS Xen Upload for Azure Hyper-‐‑‒V
Release Upload
• garden-‐‑‒linux
• concourse
Deploy bosh deploy

53.
Installation Article (Japanese)
• AWS https://blog.ik.am/entries/383
• Azure https://blog.ik.am/entries/384

54.
Cloud Foundry Tokyo Meetup #2
http://www.meetup.com/ja-‐‑‒JP/Cloud-‐‑‒Foundry-‐‑‒Tokyo-‐‑‒Meetup/events/230285366/
Gwenn
(@The_̲Shinji62)
will talk about
BOSH 2.0