Source-to-image - How to transform any application code into immutable deployments

1. Red Hat Forum Brasil 2016
Source to Image - builds reproduzíveis
Filippe Spolti
Software Engineer
@filippespolti

2. Red Hat Forum Brasil 2016
Developer Workflow
IDE
Git Repo/
Local source
Builder
Image
Build
Docker
Registry
Invoke
Build
Commit App Image
Push
Run
Running
Docker
Container

3. Red Hat Forum Brasil 2016
Project Goals
• Ecosystem
• Any existing Docker image should be easily adaptable to S2I
• Create an ecosystem of reusable builder images
• Ease of Use
• End users don’t need to learn/write Dockerfiles
• Enable existing source build tools, e.g. maven, rake, etc
• Reproducibility
• Minimize inputs into the build process (just source)
• Speed
• Perform build operations as a single layer/single commit
• Security
• Run the assemble process as a non-privileged user

4. Red Hat Forum Brasil 2016
Demo

5. Red Hat Forum Brasil 2016
S2I vs Docker build
• S2I can enforce non-root permissions
• S2I encourages fast builds by separating dependencies into parent
layers
• Use case is targeted at anything that needs to build from source but
the layering approach can help create the desired ecosystem in a lot
of scenarios.
• Reduces number of layers committed during build and in final image
• Typical use case is to build parent layers with dockerfiles and final layer
with S2I
RHEL7 + JBoss + My
App Dependencies
RHEL7 + JBoss Dockerbuild
RHEL7 + JBoss + My
App Dependencies +
My WAR
S2I

6. Red Hat Forum Brasil 2016
Anatomy of an S2I Builder Image
● Base language runtime+build tools
● Assemble script
○ Converts source code into a runnable application
○ Downloads dependencies
○ Injects configuration
● Run script
○ Responsible for starting the application in the final image
○ S2I will set this as the RUN command in the final image
● Save-artifacts script
○ Optional: extracts reusable dependencies from previous application image

7. Red Hat Forum Brasil 2016
S2I Build Steps
1. Create a container based on the build image and pass it a tar file that contains:
a. The application source in src
b. The build artifacts in artifacts (incremental builds)
2. Set the environment variables from .s2i/environment (optional)
3. Start the container and run its assemble script
4. Wait for the container to finish
5. Commit the container, setting the CMD for the output image to be the run script and tagging
the image with the name provided.

8. Red Hat Forum Brasil 2016
Sample Assemble Script
#!/bin/sh
echo "---> Installing application source"
cp -Rf /tmp/src/. ./
# Change the npm registry mirror if provided
if [ -n "$NPM_MIRROR" ]; then
npm config set registry $NPM_MIRROR
fi
echo "---> Building your Node application from source"
npm install -d

9. Red Hat Forum Brasil 2016
Sample Run Script
#!/bin/sh
echo "Launching via npm..."
exec npm run -d $NPM_RUN

10. Red Hat Forum Brasil 2016
Source can be more than “source”
● Framework configuration
○ Php.ini, EAP/Wildfly standalone.xml, etc
○ Baked in config can be better than env variable config
● Database images
○ Configuration for the DB
○ Schema initialization scripts
○ NOT: actual DB content (do not bake into the image)
● Jenkins
○ Plugins (actual binaries or list of plugins to install)
○ General Jenkins configuration
○ Jenkins job definitions

11. Red Hat Forum Brasil 2016
S2I Incremental Builds
When s2i detects:
● A builder image is compatible with incremental building (save-artifacts exists)
● A previous image exists, with the same name as the output name for this build
● Incremental build is enabled
An incremental build is performed:
1. Creates a new Docker container from the prior build image
2. Run save-artifacts in this container (streams out a tar of the artifacts to stdout)
3. Build the new output image:
i. The artifacts from the previous build will be in the artifacts directory of the tar passed to the build
ii. The build image's assemble script is responsible for detecting and using the build artifacts

12. Red Hat Forum Brasil 2016
Demo

13. Red Hat Forum Brasil 2016
S2I Layered Builds
When s2i detects:
● Builder image does not contain ‘sh’ or ‘tar’ binaries
An intermediate layered build is performed:
1. Generate a Dockerfile FROM the builder image
2. ADD the local source working directory
3. Docker build an intermediate image
4. S2I build using the resulting image (contains input files already)
Caution: will invoke ONBUILD commands from the builder image

14. Red Hat Forum Brasil 2016
S2I Extended Builds (Tech Preview)
Source
Builder
Image
Build
Docker
Registry
Start Build Commit
Artifact
Image
Push
Run
Running
Docker
Container
Build
Extract
Runtime
Image
Commit
Application
Image

15. Red Hat Forum Brasil 2016
Options
● Assemble/Run/Save-artifacts script overriding
○ Via source repo or http url
● Pull policies
○ Never/Always/IfNotPresent
● Environment variables
○ Via environment file in source repo or command line args
○ Used during assemble and included in final image
● Incremental build enabled/disabled
● Inject files
○ Make local files available during assemble
○ File contents will not be included in the final application image

16. Red Hat Forum Brasil 2016
Future
● Fit and finish on Extended Builds
● Ability to override part of assemble/run script
○ Hook points (pre-assemble/post-assemble, pre-run, etc)
● Remove git support?
○ Make s2i purely a local filesystem tool, use your own repo tools
● Abstract the Docker layer
○ Support other container platforms

17. Red Hat Forum Brasil 2016
Resources
● Source-To-Image Repository - https://github.com/openshift/source-to-image
● Community images - https://github.com/openshift-s2i/
● SCL/OpenShift images - https://github.com/sclorg/s2i-xxxx-container
○ nodejs, perl, php, python, ruby
● Jenkins image - https://github.com/openshift/jenkins