Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Paul	Bonaud
@paulrb_r
Maxime	Visonneau
@mvisonneau
A	
A	
A	SMOOTH
SMOOTH
SMOOTH	MIGRATION	TO
	MIGRATION	TO
	MIGRATION	TO
D...
↓
↓
Paul	Bonaud	
⌨	+	⚙	+	 	
▶	Infra	&	Ops	Engineer	
⏸	Software	Developer	
Maxime	Visonneau	
⌨	+	 	+	 	
Infrastructure	Engineer
Meet	Maurice
Meet	Maurice
Meet	Maurice
Meet	Maurice
Meet	Maurice
Meet	Maurice
aka	
aka	
aka	build2
build2
build2
Manually	administrated
Building	all	projects
Android,	JS	(Ember),	C++,	Ruby
Deploying	everything
Nginx,	apps,	infrastructure
Single	architecture	(Debian	7)
New	era
New	era
New	era
Docker
Docker
Docker
Open	Source	❤
Isolated
Immutable
Stateless
Version	controlled
	Dockerfile
Dockerfile
FROM	ruby:2.4-jessie
Dockerfile
RUN	apt-get	update
RUN	apt-get	-y	install	libpq-dev	ghostscript
Dockerfile
RUN	apt-get	update	&&	
				apt-get	-y	install	libpq-dev	ghostscript	&&	
				rm	-rf	/var/lib/apt/lists/*
Dockerfile
WORKDIR	/opt/app
COPY	Gemfile	Gemfile.lock	/opt/app
RUN		bundle	install
Dockerfile
VOLUME	/opt/app
CMD	[	"make",	"run"	]
Dockerfile
FROM	ruby:2.4-jessie
RUN	apt-get	update	&&	
				apt-get	-y	install	libpq-dev	ghostscript	&&	
				rm	-rf	/var/li...
GitLab
GitLab
GitLab
Open	Source	❤
Integrated	Docker	registry!
registry.gitlab.com
Gitlab-CI
Gitlab-CI
Gitlab-CI
Open	Source	❤
Declarative
Integrated
Version	controlled
	.gitlab-ci.yml
.gitlab-ci.yml
stages:
		-	build
		-	test
		-	package
		-	deploy
.gitlab-ci.yml
variables:
		IMAGE:	${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
		LATEST_IMAGE:	${CI_REGISTRY}/${CI_PROJECT_PATH}...
.gitlab-ci.yml
build:docker:
		stage:	build
.gitlab-ci.yml
		tags:	[	privileged	]
		image:	"docker:latest"
.gitlab-ci.yml
		before_script:
				-	docker	login	-u	gitlab-ci-token
																	 	-p	${CI_JOB_TOKEN}	${CI_REGISTRY}...
.gitlab-ci.yml
image:	${IMAGE}
.gitlab-ci.yml
test:
		stage:	test
		script:
				-	make	test
.gitlab-ci.yml
package:
		stage:	package
		script:
				-	make	package	>	release.tar.gz
		artifacts:
				paths:	[	release.t...
.gitlab-ci.yml
deploy:integration:
		stage:	deploy
		image:	${CI_REGISTRY}/infra/ansible:latest
		variables:
				ENV:	inte...
.gitlab-ci.yml
deploy:integration:	&deploy
		stage:	deploy
		image:	${CI_REGISTRY}/infra/ansible:latest
		variables:
				E...
WHAT	ABOUT	EASY	
WHAT	ABOUT	EASY	
WHAT	ABOUT	EASY	OS
OS
OS	TESTING	AND	UPGRADES?!
	TESTING	AND	UPGRADES?!
	TESTING	AND	UPG...
Dockerfile
FROM	ruby:2.4-jessie
Dockerfile
FROM	ruby:2.4-stretch
.gitlab-ci.yml
variables:
		IMAGE:	${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
#	(...)
.gitlab-ci.yml
variables:
		IMAGE:	${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA}
#	(...)
.gitlab-ci.yml
variables:
		IMAGE:	${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA}
#	(...)
test:jessie:
		stage:	test
		variab...
.gitlab-ci.yml
variables:
		IMAGE:	${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA}
#	(...)
test:jessie:	&test
		stage:	test
		...
.gitlab-ci.yml
package:jessie:	&package
		stage:	package
		variables:
				OS:	jessie
		script:
				-	make	package	>	releas...
.gitlab-ci.yml
deploy:integration:	&deploy
		stage:	deploy
		image:	${CI_REGISTRY}/infra/ansible:latest
		variables:
				O...
Runners
Runners
Runners
Different	Executors	/	Multiple	OSes
Shell
Docker
VirtualBox
Parallels
SSH
Kubernetes
Linux
Windows
Mac	OS	X
FreeBSD
Different	Executors	/	Multiple	OSes
Shell
Docker
VirtualBox
Parallels
SSH
Kubernetes
Linux
Windows
Mac	OS	X
FreeBSD
Portable
Light	system	dependencies
Easily	configurable	and	maintainable
---
classes:
		-	docker
		-	gitlab::ci
gitlab::ci::runners:
		"%{::fqdn}_docker":
				executor:	docker
				docker-image:	d...
---
classes:
		-	docker
		-	gitlab::ci
gitlab::ci::runners:
		"%{::fqdn}_docker":
				executor:	docker
				docker-image:	d...
AWS
AWS
AWS
Terraform
Terraform
Terraform
Open	Source	❤
terraform.tf
resource	"aws_instance"	"gitlab_runner"	{
		count									=	10
		ami											=	"ami-6b2cd712"
		instance_ty...
MISSINGDEP
MISSINGDEP
MISSINGDEP
Who	had	any	clue	this	system	library	was	required?
MISSINGDEP
MISSINGDEP
MISSINGDEP
Update	Dockerfiles	with	missing	dependencies
ENOSPC
ENOSPC
ENOSPC
No	space	left	on	device
ENOSPC
ENOSPC
ENOSPC
Increase	space	/	Automate	cleanup	of	old	containers	and
images
UNAUTH_DEPLOY
UNAUTH_DEPLOY
UNAUTH_DEPLOY
Where	are	my	SSH	keys	for	Ansible?
UNAUTH_DEPLOY
UNAUTH_DEPLOY
UNAUTH_DEPLOY
Enhance	security	by	creating	individual	keys	per
project/env
SHM_OOM
SHM_OOM
SHM_OOM
Less	than	64MB	of	free	space	in	temporary	directory	for
shared	memory	files
SHM_OOM
SHM_OOM
SHM_OOM
Configure	max	SHM	size
IOWAIT
IOWAIT
IOWAIT
On	heavy	load	tests
IOWAIT
IOWAIT
IOWAIT
Bigger	disks
Provisioned	IOPS
Leverage	tmpfs
AWS
AWS
AWS
Docker
Docker
Docker
Terraform
Terraform
Terraform
terraform.tf
resource	"aws_autoscaling_group"	"gitlab_runner"	{
		desired_capacity					=	10
		min_size													=	4
		m...
We	can	do	better	and	easier!
	We	can	do	better	and	easier!
	We	can	do	better	and	easier!
Ourselves
Ourselves
Ourselves
with	an	interesting	challenge
Software	engineers
Software	engineers
Software	engineers
with	a	reliable	CI	implementation
Finance
Finance
Finance
by	making	some	savings
docker-machine	integrated
out-of-the-box	with	gitlab-runner
scheduling	jobs	on	spot	instances
gitlab::ci::runners:
		'aws_spot_docker_machine':
gitlab::ci::runners:
		'aws_spot_docker_machine':
				executor:	docker+machine
gitlab::ci::runners:
		'aws_spot_docker_machine':
				executor:	docker+machine
				limit:	24
gitlab::ci::runners:
		'aws_spot_docker_machine':
				executor:	docker+machine
				limit:	24
				machine-IdleCount:	4
				...
gitlab::ci::runners:
		'aws_spot_docker_machine':
				executor:	docker+machine
				limit:	24
				machine-IdleCount:	4
				...
gitlab::ci::runners:
		'aws_spot_docker_machine':
				executor:	docker+machine
				limit:	24
				machine-IdleCount:	4
				...
gitlab::ci::runners:
		'aws_spot_docker_machine':
				executor:	docker+machine
				limit:	24
				machine-IdleCount:	4
				...
A	small	drawback..
A	small	drawback..
A	small	drawback..
Scheduling
Scheduling
Scheduling
Kubernetes
	Kubernetes
	Kubernetes
---
apiVersion:	v1
kind:	ConfigMap
metadata:
		name:	gitlab-runner
		namespace:	gitlab-ci
data:
		config.toml:	|
				[[run...
~$	kubectl	-n	gitlab-ci	get	po
NAME																				READY					STATUS				RESTARTS			AGE
runner-315e4d80-0qj79g		2/2					...
Definitely	promising
Definitely	promising
Definitely	promising
Still	a	work	in	progress	for	us
Still	a	work	in	progress	for	us
Still	a	work	in	progress	for	us
QUESTIONS?
QUESTIONS?
QUESTIONS?
REFERENCES
REFERENCES
REFERENCES
·		Dockerfile	documentation
·		.gitlab-ci.yml	documentation
·		Slides
https://docs.docker...
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker
Upcoming SlideShare
Loading in …5
×

A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker

536 views

Published on

by Paul Bonnaud and Maxime Visonneau, Trainline
TIAD Camp Docker 6 Octobre 2017

Published in: Technology
  • Be the first to comment

A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker

  1. 1. Paul Bonaud @paulrb_r Maxime Visonneau @mvisonneau A A A SMOOTH SMOOTH SMOOTH MIGRATION TO MIGRATION TO MIGRATION TO DOCKER DOCKER DOCKER FOCUSING ON FOCUSING ON FOCUSING ON BUILD PIPELINES BUILD PIPELINES BUILD PIPELINES
  2. 2. ↓ ↓
  3. 3. Paul Bonaud ⌨ + ⚙ + ▶ Infra & Ops Engineer ⏸ Software Developer Maxime Visonneau ⌨ + + Infrastructure Engineer
  4. 4. Meet Maurice Meet Maurice Meet Maurice
  5. 5. Meet Maurice Meet Maurice Meet Maurice aka aka aka build2 build2 build2
  6. 6. Manually administrated
  7. 7. Building all projects Android, JS (Ember), C++, Ruby
  8. 8. Deploying everything Nginx, apps, infrastructure
  9. 9. Single architecture (Debian 7)
  10. 10. New era New era New era
  11. 11. Docker Docker Docker
  12. 12. Open Source ❤
  13. 13. Isolated
  14. 14. Immutable
  15. 15. Stateless
  16. 16. Version controlled Dockerfile
  17. 17. Dockerfile FROM ruby:2.4-jessie
  18. 18. Dockerfile RUN apt-get update RUN apt-get -y install libpq-dev ghostscript
  19. 19. Dockerfile RUN apt-get update && apt-get -y install libpq-dev ghostscript && rm -rf /var/lib/apt/lists/*
  20. 20. Dockerfile WORKDIR /opt/app COPY Gemfile Gemfile.lock /opt/app RUN bundle install
  21. 21. Dockerfile VOLUME /opt/app CMD [ "make", "run" ]
  22. 22. Dockerfile FROM ruby:2.4-jessie RUN apt-get update && apt-get -y install libpq-dev ghostscript && rm -rf /var/lib/apt/lists/* WORKDIR /opt/app COPY Gemfile Gemfile.lock /opt/app RUN bundle install VOLUME /opt/app CMD [ "make", "run" ]
  23. 23. GitLab GitLab GitLab
  24. 24. Open Source ❤
  25. 25. Integrated Docker registry! registry.gitlab.com
  26. 26. Gitlab-CI Gitlab-CI Gitlab-CI
  27. 27. Open Source ❤
  28. 28. Declarative
  29. 29. Integrated
  30. 30. Version controlled .gitlab-ci.yml
  31. 31. .gitlab-ci.yml stages: - build - test - package - deploy
  32. 32. .gitlab-ci.yml variables: IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA} LATEST_IMAGE: ${CI_REGISTRY}/${CI_PROJECT_PATH}:latest
  33. 33. .gitlab-ci.yml build:docker: stage: build
  34. 34. .gitlab-ci.yml tags: [ privileged ] image: "docker:latest"
  35. 35. .gitlab-ci.yml before_script: - docker login -u gitlab-ci-token   -p ${CI_JOB_TOKEN} ${CI_REGISTRY} script: - docker pull ${LATEST_IMAGE} || true - docker build --cache-from ${LATEST_IMAGE}   -t ${IMAGE} -t ${LATEST_IMAGE} . - docker push ${IMAGE} - docker push ${LATEST_IMAGE}
  36. 36. .gitlab-ci.yml image: ${IMAGE}
  37. 37. .gitlab-ci.yml test: stage: test script: - make test
  38. 38. .gitlab-ci.yml package: stage: package script: - make package > release.tar.gz artifacts: paths: [ release.tar.gz ]
  39. 39. .gitlab-ci.yml deploy:integration: stage: deploy image: ${CI_REGISTRY}/infra/ansible:latest variables: ENV: integration script: - make deploy file=release.tar.gz env=${ENV}
  40. 40. .gitlab-ci.yml deploy:integration: &deploy stage: deploy image: ${CI_REGISTRY}/infra/ansible:latest variables: ENV: integration script: - make deploy file=release.tar.gz env=${ENV} deploy:production: <<: *deploy when: manual variables: ENV: production
  41. 41. WHAT ABOUT EASY WHAT ABOUT EASY WHAT ABOUT EASY OS OS OS TESTING AND UPGRADES?! TESTING AND UPGRADES?! TESTING AND UPGRADES?!
  42. 42. Dockerfile FROM ruby:2.4-jessie
  43. 43. Dockerfile FROM ruby:2.4-stretch
  44. 44. .gitlab-ci.yml variables: IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA} # (...)
  45. 45. .gitlab-ci.yml variables: IMAGE: ${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA} # (...)
  46. 46. .gitlab-ci.yml variables: IMAGE: ${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA} # (...) test:jessie: stage: test variables: OS: jessie script: - make test
  47. 47. .gitlab-ci.yml variables: IMAGE: ${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA} # (...) test:jessie: &test stage: test variables: OS: jessie script: - make test test:stretch: <<: *test variables: OS: stretch
  48. 48. .gitlab-ci.yml package:jessie: &package stage: package variables: OS: jessie script: - make package > release-${OS}.tar.gz artifacts: paths: [ release-${OS}.tar.gz ] package:stretch: <<: *package variables: OS: stretch
  49. 49. .gitlab-ci.yml deploy:integration: &deploy stage: deploy image: ${CI_REGISTRY}/infra/ansible:latest variables: OS: stretch ENV: integration script: - make deploy file=release-${OS}.tar.gz env=${ENV} deploy:production: <<: *deploy when: manual variables: OS: jessie ENV: production
  50. 50. Runners Runners Runners
  51. 51. Different Executors / Multiple OSes Shell Docker VirtualBox Parallels SSH Kubernetes Linux Windows Mac OS X FreeBSD
  52. 52. Different Executors / Multiple OSes Shell Docker VirtualBox Parallels SSH Kubernetes Linux Windows Mac OS X FreeBSD
  53. 53. Portable
  54. 54. Light system dependencies
  55. 55. Easily configurable and maintainable
  56. 56. --- classes: - docker - gitlab::ci gitlab::ci::runners: "%{::fqdn}_docker": executor: docker docker-image: docker:latest
  57. 57. --- classes: - docker - gitlab::ci gitlab::ci::runners: "%{::fqdn}_docker": executor: docker docker-image: docker:latest docker-volumes: "/var/run/docker.sock:/var/run/docker.sock" docker-privileged: true
  58. 58. AWS AWS AWS
  59. 59. Terraform Terraform Terraform
  60. 60. Open Source ❤
  61. 61. terraform.tf resource "aws_instance" "gitlab_runner" { count = 10 ami = "ami-6b2cd712" instance_type = "m4.large" [..] }
  62. 62. MISSINGDEP MISSINGDEP MISSINGDEP Who had any clue this system library was required?
  63. 63. MISSINGDEP MISSINGDEP MISSINGDEP Update Dockerfiles with missing dependencies
  64. 64. ENOSPC ENOSPC ENOSPC No space left on device
  65. 65. ENOSPC ENOSPC ENOSPC Increase space / Automate cleanup of old containers and images
  66. 66. UNAUTH_DEPLOY UNAUTH_DEPLOY UNAUTH_DEPLOY Where are my SSH keys for Ansible?
  67. 67. UNAUTH_DEPLOY UNAUTH_DEPLOY UNAUTH_DEPLOY Enhance security by creating individual keys per project/env
  68. 68. SHM_OOM SHM_OOM SHM_OOM Less than 64MB of free space in temporary directory for shared memory files
  69. 69. SHM_OOM SHM_OOM SHM_OOM Configure max SHM size
  70. 70. IOWAIT IOWAIT IOWAIT On heavy load tests
  71. 71. IOWAIT IOWAIT IOWAIT Bigger disks Provisioned IOPS Leverage tmpfs
  72. 72. AWS AWS AWS
  73. 73. Docker Docker Docker
  74. 74. Terraform Terraform Terraform
  75. 75. terraform.tf resource "aws_autoscaling_group" "gitlab_runner" { desired_capacity = 10 min_size = 4 max_size = 24 launch_configuration = "${aws_launch_configuration.runner.name}" [..] }
  76. 76. We can do better and easier! We can do better and easier! We can do better and easier!
  77. 77. Ourselves Ourselves Ourselves with an interesting challenge
  78. 78. Software engineers Software engineers Software engineers with a reliable CI implementation
  79. 79. Finance Finance Finance by making some savings
  80. 80. docker-machine integrated out-of-the-box with gitlab-runner scheduling jobs on spot instances
  81. 81. gitlab::ci::runners: 'aws_spot_docker_machine':
  82. 82. gitlab::ci::runners: 'aws_spot_docker_machine': executor: docker+machine
  83. 83. gitlab::ci::runners: 'aws_spot_docker_machine': executor: docker+machine limit: 24
  84. 84. gitlab::ci::runners: 'aws_spot_docker_machine': executor: docker+machine limit: 24 machine-IdleCount: 4 machine-IdleTime: 600
  85. 85. gitlab::ci::runners: 'aws_spot_docker_machine': executor: docker+machine limit: 24 machine-IdleCount: 4 machine-IdleTime: 600 machine-OffPeakPeriods: - "* * 0-8,18-23 * * mon-fri *" - "* * * * * sat,sun *" machine-OffPeakIdleCount: 0 machine-OffPeakIdleTime: 600
  86. 86. gitlab::ci::runners: 'aws_spot_docker_machine': executor: docker+machine limit: 24 machine-IdleCount: 4 machine-IdleTime: 600 machine-OffPeakPeriods: - "* * 0-8,18-23 * * mon-fri *" - "* * * * * sat,sun *" machine-OffPeakIdleCount: 0 machine-OffPeakIdleTime: 600 machine-MaxBuilds: 30
  87. 87. gitlab::ci::runners: 'aws_spot_docker_machine': executor: docker+machine limit: 24 machine-IdleCount: 4 machine-IdleTime: 600 machine-OffPeakPeriods: - "* * 0-8,18-23 * * mon-fri *" - "* * * * * sat,sun *" machine-OffPeakIdleCount: 0 machine-OffPeakIdleTime: 600 machine-MaxBuilds: 30 machine-MachineDriver: amazonec2 machine-MachineOptions: - amazonec2-instance-type=m4.large - amazonec2-request-spot-instance=true - amazonec2-spot-price=0.10 [..]
  88. 88. A small drawback.. A small drawback.. A small drawback..
  89. 89. Scheduling Scheduling Scheduling
  90. 90. Kubernetes Kubernetes Kubernetes
  91. 91. --- apiVersion: v1 kind: ConfigMap metadata: name: gitlab-runner namespace: gitlab-ci data: config.toml: | [[runners]] name = "k8s_runner" url = "https://gitlab.example.com/" executor = "kubernetes" [runners.kubernetes] cpu_limit = "1" memory_limit = "256Mi" [..]
  92. 92. ~$ kubectl -n gitlab-ci get po NAME READY STATUS RESTARTS AGE runner-315e4d80-0qj79g 2/2 Running 0 3m runner-315e4d80-9kdkee 2/2 Running 0 2m runner-315e4d80-kdfdfe 2/2 Running 0 1m runner-315e4d80-ldorpk 2/2 Running 0 1m runner-315e4d80-pleofi 2/2 Running 0 20s
  93. 93. Definitely promising Definitely promising Definitely promising
  94. 94. Still a work in progress for us Still a work in progress for us Still a work in progress for us
  95. 95. QUESTIONS? QUESTIONS? QUESTIONS?
  96. 96. REFERENCES REFERENCES REFERENCES · Dockerfile documentation · .gitlab-ci.yml documentation · Slides https://docs.docker.com/engine/reference/builder/ https://gitlab.com/help/ci/yaml/README.md https://paulrbr.gitlab.io/ci-migration-slides/

×