This document discusses migrating build pipelines to Docker and GitLab CI/CD. It begins with an introduction to Docker and its benefits for building isolated, immutable, and versioned applications. It then covers using GitLab for integrated Docker registry and GitLab CI/CD for running pipelines in a declarative configuration. The rest of the document discusses strategies for testing across OSes, handling errors and upgrades, and scheduling jobs on Kubernetes or AWS spot instances for improved efficiency and cost savings.

1. Paul Bonaud
@paulrb_r
Maxime Visonneau
@mvisonneau
A
A
A SMOOTH
SMOOTH
SMOOTH MIGRATION TO
MIGRATION TO
MIGRATION TO
DOCKER
DOCKER
DOCKER FOCUSING ON
FOCUSING ON
FOCUSING ON
BUILD PIPELINES
BUILD PIPELINES
BUILD PIPELINES

2. ↓
↓

3. Paul Bonaud
⌨ + ⚙ +
▶ Infra & Ops Engineer
⏸ Software Developer
Maxime Visonneau
⌨ + +
Infrastructure Engineer

4. Meet Maurice
Meet Maurice
Meet Maurice

6. Meet Maurice
Meet Maurice
Meet Maurice
aka
aka
aka build2
build2
build2

8. Manually administrated

10. Building all projects
Android, JS (Ember), C++, Ruby

12. Deploying everything
Nginx, apps, infrastructure

14. Single architecture (Debian 7)

16. New era
New era
New era

18. Docker
Docker
Docker

19. Open Source ❤

20. Isolated

21. Immutable

22. Stateless

23. Version controlled
Dockerfile

24. Dockerfile
FROM ruby:2.4-jessie

25. Dockerfile
RUN apt-get update
RUN apt-get -y install libpq-dev ghostscript

26. Dockerfile
RUN apt-get update &&
apt-get -y install libpq-dev ghostscript &&
rm -rf /var/lib/apt/lists/*

27. Dockerfile
WORKDIR /opt/app
COPY Gemfile Gemfile.lock /opt/app
RUN bundle install

28. Dockerfile
VOLUME /opt/app
CMD [ "make", "run" ]

29. Dockerfile
FROM ruby:2.4-jessie
RUN apt-get update &&
apt-get -y install libpq-dev ghostscript &&
rm -rf /var/lib/apt/lists/*
WORKDIR /opt/app
COPY Gemfile Gemfile.lock /opt/app
RUN bundle install
VOLUME /opt/app
CMD [ "make", "run" ]

30. GitLab
GitLab
GitLab

31. Open Source ❤

34. Integrated Docker registry!
registry.gitlab.com

35. Gitlab-CI
Gitlab-CI
Gitlab-CI

36. Open Source ❤

37. Declarative

38. Integrated

39. Version controlled
.gitlab-ci.yml

40. .gitlab-ci.yml
stages:
- build
- test
- package
- deploy

41. .gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
LATEST_IMAGE: ${CI_REGISTRY}/${CI_PROJECT_PATH}:latest

42. .gitlab-ci.yml
build:docker:
stage: build

43. .gitlab-ci.yml
tags: [ privileged ]
image: "docker:latest"

44. .gitlab-ci.yml
before_script:
- docker login -u gitlab-ci-token
-p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${LATEST_IMAGE} || true
- docker build --cache-from ${LATEST_IMAGE}
-t ${IMAGE} -t ${LATEST_IMAGE} .
- docker push ${IMAGE}
- docker push ${LATEST_IMAGE}

45. .gitlab-ci.yml
image: ${IMAGE}

46. .gitlab-ci.yml
test:
stage: test
script:
- make test

47. .gitlab-ci.yml
package:
stage: package
script:
- make package > release.tar.gz
artifacts:
paths: [ release.tar.gz ]

48. .gitlab-ci.yml
deploy:integration:
stage: deploy
image: ${CI_REGISTRY}/infra/ansible:latest
variables:
ENV: integration
script:
- make deploy file=release.tar.gz env=${ENV}

49. .gitlab-ci.yml
deploy:integration: &deploy
stage: deploy
image: ${CI_REGISTRY}/infra/ansible:latest
variables:
ENV: integration
script:
- make deploy file=release.tar.gz env=${ENV}
deploy:production:
<<: *deploy
when: manual
variables:
ENV: production

50. WHAT ABOUT EASY
WHAT ABOUT EASY
WHAT ABOUT EASY OS
OS
OS TESTING AND UPGRADES?!
TESTING AND UPGRADES?!
TESTING AND UPGRADES?!

51. Dockerfile
FROM ruby:2.4-jessie

52. Dockerfile
FROM ruby:2.4-stretch

53. .gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
# (...)

54. .gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA}
# (...)

55. .gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA}
# (...)
test:jessie:
stage: test
variables:
OS: jessie
script:
- make test

56. .gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA}
# (...)
test:jessie: &test
stage: test
variables:
OS: jessie
script:
- make test
test:stretch:
<<: *test
variables:
OS: stretch

57. .gitlab-ci.yml
package:jessie: &package
stage: package
variables:
OS: jessie
script:
- make package > release-${OS}.tar.gz
artifacts:
paths: [ release-${OS}.tar.gz ]
package:stretch:
<<: *package
variables:
OS: stretch

58. .gitlab-ci.yml
deploy:integration: &deploy
stage: deploy
image: ${CI_REGISTRY}/infra/ansible:latest
variables:
OS: stretch
ENV: integration
script:
- make deploy file=release-${OS}.tar.gz env=${ENV}
deploy:production:
<<: *deploy
when: manual
variables:
OS: jessie
ENV: production

60. Runners
Runners
Runners

61. Different Executors / Multiple OSes
Shell
Docker
VirtualBox
Parallels
SSH
Kubernetes
Linux
Windows
Mac OS X
FreeBSD

62. Different Executors / Multiple OSes
Shell
Docker
VirtualBox
Parallels
SSH
Kubernetes
Linux
Windows
Mac OS X
FreeBSD

63. Portable

64. Light system dependencies

65. Easily configurable and maintainable

66. ---
classes:
- docker
- gitlab::ci
gitlab::ci::runners:
"%{::fqdn}_docker":
executor: docker
docker-image: docker:latest

67. ---
classes:
- docker
- gitlab::ci
gitlab::ci::runners:
"%{::fqdn}_docker":
executor: docker
docker-image: docker:latest
docker-volumes: "/var/run/docker.sock:/var/run/docker.sock"
docker-privileged: true

68. AWS
AWS
AWS

69. Terraform
Terraform
Terraform

70. Open Source ❤

71. terraform.tf
resource "aws_instance" "gitlab_runner" {
count = 10
ami = "ami-6b2cd712"
instance_type = "m4.large"
[..]
}

74. MISSINGDEP
MISSINGDEP
MISSINGDEP
Who had any clue this system library was required?

75. MISSINGDEP
MISSINGDEP
MISSINGDEP
Update Dockerfiles with missing dependencies

76. ENOSPC
ENOSPC
ENOSPC
No space left on device

77. ENOSPC
ENOSPC
ENOSPC
Increase space / Automate cleanup of old containers and
images

78. UNAUTH_DEPLOY
UNAUTH_DEPLOY
UNAUTH_DEPLOY
Where are my SSH keys for Ansible?

79. UNAUTH_DEPLOY
UNAUTH_DEPLOY
UNAUTH_DEPLOY
Enhance security by creating individual keys per
project/env

80. SHM_OOM
SHM_OOM
SHM_OOM
Less than 64MB of free space in temporary directory for
shared memory files

81. SHM_OOM
SHM_OOM
SHM_OOM
Configure max SHM size

82. IOWAIT
IOWAIT
IOWAIT
On heavy load tests

83. IOWAIT
IOWAIT
IOWAIT
Bigger disks
Provisioned IOPS
Leverage tmpfs

87. AWS
AWS
AWS

88. Docker
Docker
Docker

89. Terraform
Terraform
Terraform

90. terraform.tf
resource "aws_autoscaling_group" "gitlab_runner" {
desired_capacity = 10
min_size = 4
max_size = 24
launch_configuration = "${aws_launch_configuration.runner.name}"
[..]
}

91. We can do better and easier!
We can do better and easier!
We can do better and easier!

92. Ourselves
Ourselves
Ourselves
with an interesting challenge

93. Software engineers
Software engineers
Software engineers
with a reliable CI implementation

94. Finance
Finance
Finance
by making some savings

95. docker-machine integrated
out-of-the-box with gitlab-runner
scheduling jobs on spot instances

96. gitlab::ci::runners:
'aws_spot_docker_machine':

97. gitlab::ci::runners:
'aws_spot_docker_machine':
executor: docker+machine

98. gitlab::ci::runners:
'aws_spot_docker_machine':
executor: docker+machine
limit: 24

99. gitlab::ci::runners:
'aws_spot_docker_machine':
executor: docker+machine
limit: 24
machine-IdleCount: 4
machine-IdleTime: 600

100. gitlab::ci::runners:
'aws_spot_docker_machine':
executor: docker+machine
limit: 24
machine-IdleCount: 4
machine-IdleTime: 600
machine-OffPeakPeriods:
- "* * 0-8,18-23 * * mon-fri *"
- "* * * * * sat,sun *"
machine-OffPeakIdleCount: 0
machine-OffPeakIdleTime: 600

101. gitlab::ci::runners:
'aws_spot_docker_machine':
executor: docker+machine
limit: 24
machine-IdleCount: 4
machine-IdleTime: 600
machine-OffPeakPeriods:
- "* * 0-8,18-23 * * mon-fri *"
- "* * * * * sat,sun *"
machine-OffPeakIdleCount: 0
machine-OffPeakIdleTime: 600
machine-MaxBuilds: 30

102. gitlab::ci::runners:
'aws_spot_docker_machine':
executor: docker+machine
limit: 24
machine-IdleCount: 4
machine-IdleTime: 600
machine-OffPeakPeriods:
- "* * 0-8,18-23 * * mon-fri *"
- "* * * * * sat,sun *"
machine-OffPeakIdleCount: 0
machine-OffPeakIdleTime: 600
machine-MaxBuilds: 30
machine-MachineDriver: amazonec2
machine-MachineOptions:
- amazonec2-instance-type=m4.large
- amazonec2-request-spot-instance=true
- amazonec2-spot-price=0.10
[..]

105. A small drawback..
A small drawback..
A small drawback..

106. Scheduling
Scheduling
Scheduling

107. Kubernetes
Kubernetes
Kubernetes

108. ---
apiVersion: v1
kind: ConfigMap
metadata:
name: gitlab-runner
namespace: gitlab-ci
data:
config.toml: |
[[runners]]
name = "k8s_runner"
url = "https://gitlab.example.com/"
executor = "kubernetes"
[runners.kubernetes]
cpu_limit = "1"
memory_limit = "256Mi"
[..]

109. ~$ kubectl -n gitlab-ci get po
NAME READY STATUS RESTARTS AGE
runner-315e4d80-0qj79g 2/2 Running 0 3m
runner-315e4d80-9kdkee 2/2 Running 0 2m
runner-315e4d80-kdfdfe 2/2 Running 0 1m
runner-315e4d80-ldorpk 2/2 Running 0 1m
runner-315e4d80-pleofi 2/2 Running 0 20s

110. Definitely promising
Definitely promising
Definitely promising

111. Still a work in progress for us
Still a work in progress for us
Still a work in progress for us

113. QUESTIONS?
QUESTIONS?
QUESTIONS?

114. REFERENCES
REFERENCES
REFERENCES
· Dockerfile documentation
· .gitlab-ci.yml documentation
· Slides
https://docs.docker.com/engine/reference/builder/
https://gitlab.com/help/ci/yaml/README.md
https://paulrbr.gitlab.io/ci-migration-slides/