The document discusses the importance of reproducible builds using Docker, emphasizing that all servers should be treated equally regardless of their function. It explores various build agents, contrasting complex and micro build agents, and illustrates the benefits of containerized build pipelines. The talk concludes with a focus on build configuration management and the use of scripts to streamline the build process.

1. R A P I D
REPRODUCIBLE BUILDS
treating everything like production
Dewald Viljoen
@dewald_v

2. who am i?
2

3. HOW'S THIS TALK GOING TO WORK
and what are we actually talking
about?
3

4. "a server is a server is a server, there
really are no test servers, build
servers, dev servers and production
servers... and spoons of course"
- @dewald_v
4

5. DOCKER IN 5
MINUTES
a quick and dirty look into images, containers and all that lies below
5

6. DOCKER IN 5 MINUTES - IMAGES
6
NGINX IMAGE
UBUNTU 14.04
ADD NGINX
ADD NGINX CONFIGLayer 3
Layer 2
Base (Layer 1)

7. DOCKER IN 5 MINUTES - CONTAINERS
7
NGINX CONTAINER
UBUNTU 14.04
ADD NGINX
ADD NGINX CONFIGLayer 3
Layer 2
Base (Layer 1)
CONTAINER R/W FILESYSTEM
Layer 4
(Only in the
Container)

8. NGINX-1
CONTAINER R/W
NGINX-2
CONTAINER R/W
NGINX-3
CONTAINER R/W
NGINX IMAGE
DOCKER IN 5 MINUTES - CONTAINERS AND LAYERS
8

9. DOCKER IN 5 MINUTES - CONTAINER VS VM
9
Image credit: Docker Inc.

10. what can we use containers for?
10

11. WORKSTATIONS
AND BUILD AGENTS
the darkest corners of the mind
11

12. what's on a workstation?
12

13. A TYPICAL WORKSTATION
13
WORKSTATION

14. A TYPICAL WORKSTATION
14
WORKSTATION
Basically everything
and anything you would
need to work on your
stack

15. what's on a build agent?
15

16. A TYPICAL BUILD AGENT
16
(SUPER) BUILD AGENT

17. A TYPICAL BUILD AGENT
17
(SUPER) BUILD AGENT
Basically everything
and anything you would
need to build your app

18. what's the alternative?
18

19. DOCKER-POWERED
BUILDS
applying images and containers to build pipelines
19

20. NODEJS BUILD IMAGE
THE BUILD IMAGE - NODE + GRUNT
20
UBUNTU 14.04
ADD NODE 4.2.4
ADD GRUNT
ADD PACKAGE.JSON &
NPM INSTALL
Base Image
Build Runtime
Build Config &
Dependencies
Build Tool

21. MAVEN BUILD IMAGE
THE BUILD IMAGE - JAVA + MAVEN
21
UBUNTU 14.04
ADD JDK 1.8
ADD MAVEN 3.3.3
ADD MAVEN SETTINGS.XML
Base Image
Build Runtime
Build Config
Build Tool

22. THE BUILD CONTAINER
22
MAVEN BUILD IMAGE
+
=
SOURCE CODE
BUILD ARTIFACT

23. JAVA-APP-BUILD-1
CONTAINER R/W
JAVA-APP-BUILD-3
CONTAINER R/W
JAVA-APP-BUILD-2
CONTAINER R/W
THE BUILD CONTAINER - MULTIPLIED
23
MAVEN BUILD IMAGE

24. ENVIRONMENTS, ENVIRONMENTS, ENVIRONMENTS...
24
LOCAL MACHINE BUILD AGENT
DOCKER REGISTRY

25. THE BUILD CONTAINER - EVEN MORE MULTIPLIED
25
MAVEN 3 MAVEN 2 NODEJS RUBY
JAVA-APP-
BUILD
JAVA-APP-
BUILD
NODE-APP-
BUILD
RUBY-APP-
BUILD
BUILD AGENT
ARTIFACT ARTIFACT ARTIFACT ARTIFACT

26. OF PIPELINES AND
CONTAINERS
how do we use this then?
26

27. enterprises build all the things...
27

28. THE SUPER BUILD AGENT
28
BUILD AGENT
• Complex
• Can be containerised but needs a lot of config
• Holds all build tools and configuration for any type
of build it can service
• Results in high build-contention
• Difficult to scale
• Pushes artifacts to repositories
MAVEN 3
MAVEN 2
JDK 1.5
JDK 1.6
JDK 1.7
RUBY
PYTHON
GRADLE

29. THE MICRO BUILD AGENT
29
BUILD AGENT
• Very simple
• Can be easily containerised
• Holds build tools and configuration for only the
type of build it can service
• Easier to scale
• Can be managed with configuration as code
• Pushes artifacts to repositories
MAVEN 3
JDK 1.7

30. THE CONTAINER BUILD AGENT
30
BUILD AGENT
DOCKER
• Very simple
• Can be containerised it self (Docker in Docker)
• Doesn't have any build dependencies inside, only
docker
• Pulls build containers from a registry
• Simple to scale
• Pushes artifacts to repositories
• Basically a micro agent running only Docker

31. so what does a containerised build
pipeline look like?
31

32. PIPELINING CONTAINERS
32
Application build directly on agent
Failed due to environmental issues

33. PIPELINING CONTAINERS
33
Build Container
(Maven & JDK on Ubuntu)
Application build in container

34. PIPELINING CONTAINERS
34
Build Container
(Maven & JDK on Ubuntu)
Application Builds in containers

35. THE GO SCRIPT
not configuring the build on the server
35

36. what happens when the build
server gets killed?
36

37. BUILD CONFIGURATION
37
Backup your configuration
Configuration as code
Keep as little configuration on the server as possible
Keep build configuration next to the app it builds

38. THE GO SCRIPT
38
A Script for your build actions
Knows the commands to build your app
Keep it simple. Bash is a good start.
Everything that works on the build server should
work UNMODIFIED on the local environment.
Build server should just call this script
Use environment variables for environment
changes

39. PUTTING IT ALL
TOGETHER
demo time
39

40. live demo time...
(cross your fingers)
40

41. DEWALD VILJOEN
@dewald_v
dewaldv@thoughtworks.com
THANKS!
QUESTIONS?