Serverless Container with Source2Image

QAware GmbH
QAware GmbHQAware GmbH
Dr. Josef Adersberger, CTO & Co-Founder QAware
Serverless containers
… with source-to-image
https://github.com/adersberger/source2image
Inventor's workshop and assembly line
ready?
ready!
The evolution of software delivery
The dark ages:
Export JAR, upload to deployment server, write ticket, wait until
application is deployed to multi-project application server by far
shore ops team.
The container era:
Build application, package with runtime into container image,
push to image registry, deploy to container manager.
PaaS & Serverless heaven:
git push + magic happens here
industrialization process:
1. lower change lead time
2. higher quality confidence
3. lower vertical integration
Rule #1: Avoid too much magic at early stages
Trust me...
Serverless flavors
git push functions
git push something in a container
generic container
CI/CD pipeline
FULL SERVERLESS
MILD SERVERLESS
serverlessy: black box
application runtime and
infrastructure resources
Why you might need mild serverless: regulatory
compliance, shift left quality checks & automated tests,
complex staging and deployment patterns, decoupling
from cloud vendors or immature open source projects
serverlessy: scale-to-zero, elastic
The anatomy of a mild serverless toolchain
Watch for code
changes
Choose compilation
method and base
image
Compile code,
prepare image, inject
binaries
Deploy image to
target container
manager
Source-to-Image workflow
Developer's workspace
aka inventor's workshop
CI/CD pipeline
aka assembly line
Static analysis, test automation,
staging and promotion, image
scanning, ...
Image builders
8
With the volkswagen CI plugin you can completely focus
on source-to-image
https://github.com/auchenberg/volkswagen
The source-to-image challengers:
WORKFLOW TOOLS (inner loop & outer loop)
● Skaffold (https://skaffold.dev)
● Tilt (https://tilt.dev)
● Garden (https://garden.io)
BUILDER TOOLS
● OpenShift Source2Image
(https://github.com/openshift/source-to-image)
● buildpacks.io (https://buildpacks.io)
● Draft (https://draft.sh)
● Jib
(https://github.com/GoogleContainerTools/jib)
10
SHOOT OUT !
Serverless Container with Source2Image
# install pack tool (buildpack reference implementation)
brew tap buildpack/tap
brew install pack
# get suggested builders for sample application
# build image for sample application
Buildpack internals
Builder Image
(e.g. heroku/buildpacks or
cloudfoundry/bionic)
App Image
StackBuild Base Image Run Base Image
Lifecycle
Buildpack 1
Buildpack n
...
Detection
Analysis
Build
Export
bin/detect
bin/build
Runtime Layer
Dependency Layer
App Layer
Serverless Container with Source2Image
# install s2i
brew install source-to-image
# get and build source2image for springboot & java
git clone https://github.com/ganrad/openshift-s2i-springboot-java.git
docker build --build-arg MAVEN_VER=3.6.2 --build-arg GRADLE_VER=5.6.3
-t springboot-java .
# build image for sample application
s2i build --incremental=true . springboot-java skaffold-example-god
S2I internals
BUILDER IMAGE
Pre-defined scripts:
APP IMAGE
Build Base Image
building the application
artifacts from source and
placing them into the
appropriate directories
inside the app image
executing the
application
(entrypoint)
Runtime Layer
Build Layer Artifact Layer
CLI tool:
entrypoint: run
Serverless Container with Source2Image
# install draft along with helm
brew install kubernetes-helm
helm init
brew install azure/draft/draft
# create draft files for application (Helm chart, draft.toml,
Dockerfile)
draft create
--> Draft detected Shell (46.149372%)
--> Could not find a pack for Shell. Trying to find the next likely language
match...
--> Draft detected Batchfile (28.163621%)
--> Could not find a pack for Batchfile. Trying to find the next likely language
match...
--> Draft detected Java (12.213444%)
--> Ready to sail
# build image for sample application and deploy application to
k8s
draft up
# connect to the application endpoint
draft connect
Draft internals
BUILDER HELM CHART APPLICATION HELM CHART
[environments]
[environments.development]
name = "god"
namespace = "default"
wait = true
watch = false
watch-delay = 2
auto-connect = false
dockerfile = "Dockerfile"
chart = ""
draft.toml
java
primary language detection by github linguist and
mapped to chart directory by language name
generated by draft create
Draft flatline sadness
Serverless Container with Source2Image
./mvnw compile jib:dockerBuild -Dimage=skaffold-example-god
pom.xml
Serverless Container with Source2Image
Custom
(SH)
# install skaffold
brew install skaffold
# build & deploy image (once)
skaffold run
# build & deploy image (everytime the code changes)
skaffold dev
apiVersion: skaffold/v1beta16
kind: Config
build:
artifacts:
- image: skaffold-example-god
context: .
jib: {}
deploy:
kubectl:
manifests:
- src/k8s/*.yaml
apiVersion: skaffold/v1beta16
kind: Config
build:
artifacts:
- image: skaffold-example-god
custom:
buildCommand: ./build-buildpacks.sh
dependencies:
paths:
- .
deploy:
kubectl:
manifests:
- src/k8s/*.yaml
#!/bin/bash
set -e
images=$(echo $IMAGES | tr " " "n")
for image in $images
do
pack build $image --builder
cloudfoundry/cnb:bionic
if $PUSH_IMAGE
then
docker push $image
fi
done
driven by skaffold.yaml:
Builder performance comparison with Skaffold
Builder Time
s2i (--incremental=true) 1:23m
Draft 1:14m
Buildpacks 0:42m
jib 0:21m
median of 3 runs timed by "time" command after an initial warming run and a code change between each run -
build and caching behaviour not optimized
time skaffold run -f=skaffold-s2i.yml
time skaffold run -f=skaffold-buildpacks.yml
time skaffold run -f=skaffold-jib.yml
time draft up
Builder shootout (lower is better)
Criteria Buildpacks.io s2i Draft Jib
Speed ● lead time to
change
● image size
(docker image ls)
● rebasing
2 4 3 1
Supported
application
technologies
Java, Node.JS,
Python, GoLang, ... 2 3 1
4
(k.O. if non-Java)
Auto-detection
of application
technologies
yes / no
1 3 1 3
Maturity /
future proof
3 2
4
(k.O.)
1
8 12 9 (k.O.) 9
Serverless Container with Source2Image
# install Tilt
brew tap windmilleng/tap
brew install windmilleng/tap/tilt
# build & deploy image (with every change)
tilt up
# Deploy: tell Tilt what YAMLs to deploy
k8s_yaml('src/k8s/pod-god.yaml')
# Build: tell Tilt what images (name) to build from which directories
docker_build('skaffold-example-god', '.')
# Watch: tell Tilt how to connect locally (optional)
k8s_resource('web', port_forwards=8080)
driven by Tiltfile (Starlark, a Python dialect):
Tilt UI
TERMINAL UI
WEB UI
Serverless Container with Source2Image
# install Garden
brew tap garden-io/garden
brew install garden-cli
# build & deploy image (once)
garden build
# build & deploy image (with
# every change)
garden dev
kind: Project
name: god-project
environments:
- name: local
providers:
- name: local-kubernetes
context: docker-desktop
---
kind: Module
name: god
description: God service
type: container
services:
- name: god
ports:
- name: http
containerPort: 8080
healthCheck:
httpGet:
path: /
port: http
ingresses:
- path: /
port: http
driven by garden.yml containing
garden-defined resource types as
abstractions for k8s primitives:
Garden UI
TERMINAL UIWEB UI
Workflow shootout (lower is better)
Criteria ⇒ Position Skaffold Tilt Garden
Pipeline
integratability
● As pipeline tasks in Jenkins
Pipelines, Tekton, Build tools
● Support for container testing
● Deployment options: Helm,
Kustomize, kubectl
1 3 2
Supported image
builders
● Plain Docker
● Daemon-less builds
● Builders: Buildpacks, Draft, s2i,
Jib
2 3 1
Multi-environments Support for multiple environments
like local, dev, prod
1 3 1
Multi-image
projects
Support for code repositories
containing multi-image projects
1 1 1
Local dev support Local build, local run,
build-on-change
1 1 1
Maturity / future
proof
1 2 2
7 13 8
1. The way from source to image can be done in a generic way
2. If you're doing Java then go for the Google guys: Skaffold and Jib
3. If you're polyglot then go for Skaffold and buildpacks.io
4. Use the same workflow & builder tool for local builds and CI/CD builds
5. Optimize the change lead time for features and the local round trip time for developers
5 things:
@adersberger
37
A possible journey towards full serverless as commodity
Serverless Build Serverless Run
Bonus slide: Change lead time optimization
1. Use well-architectured, security-hardened and minimal base images like:
a. Google Distroless Images (https://github.com/GoogleContainerTools/distroless)
b. RedHat Universal Base Images (https://developers.redhat.com/products/rhel/ubi)
2. Use a Docker daemon-less image builder with excessive caching:
a. Google Kaniko (https://github.com/GoogleContainerTools/kaniko)
b. uber Makiso (https://github.com/uber/makisu)
c. Docker BuildKit (https://github.com/moby/buildkit)
d. Google Bazel (https://bazel.build)
3. Use an efficient pipeline orchestrator with task parallelization capabilities:
a. Tekton (https://tekton.dev)
b. Argo CD (https://argoproj.github.io/argo-cd)
1 of 38

Recommended

CI/CD Development in Kubernetes - Skaffold by
CI/CD Development in Kubernetes -  SkaffoldCI/CD Development in Kubernetes -  Skaffold
CI/CD Development in Kubernetes - SkaffoldSuman Chakraborty
348 views10 slides
Serverless containers … with source-to-image by
Serverless containers  … with source-to-imageServerless containers  … with source-to-image
Serverless containers … with source-to-imageJosef Adersberger
129 views38 slides
CI/CD Development in Kubernetes - Skaffold by
CI/CD Development in Kubernetes -  SkaffoldCI/CD Development in Kubernetes -  Skaffold
CI/CD Development in Kubernetes - SkaffoldSuman Chakraborty
41 views13 slides
Continuous Delivery to Kubernetes Using Helm by
Continuous Delivery to Kubernetes Using HelmContinuous Delivery to Kubernetes Using Helm
Continuous Delivery to Kubernetes Using HelmAdnan Abdulhussein
3.2K views24 slides
Continuous Delivery to Kubernetes with Jenkins and Helm by
Continuous Delivery to Kubernetes with Jenkins and HelmContinuous Delivery to Kubernetes with Jenkins and Helm
Continuous Delivery to Kubernetes with Jenkins and HelmDavid Currie
2.7K views30 slides
Telepresence - Seamless Development Environments on Kubernetes by
Telepresence - Seamless Development Environments on KubernetesTelepresence - Seamless Development Environments on Kubernetes
Telepresence - Seamless Development Environments on KubernetesAdnan Abdulhussein
597 views17 slides

More Related Content

What's hot

Spring on Kubernetes by
Spring on KubernetesSpring on Kubernetes
Spring on KubernetesJay Lee
96 views36 slides
Continuous Delivery with Jenkins & Kubernetes @ Sky by
Continuous Delivery with Jenkins & Kubernetes @ SkyContinuous Delivery with Jenkins & Kubernetes @ Sky
Continuous Delivery with Jenkins & Kubernetes @ SkyAdriana Vasiu
516 views24 slides
Jenkins X intro (from google app dev conference) by
Jenkins X intro (from google app dev conference)Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)Michael Neale
429 views25 slides
Cloud native buildpacks-cncf by
Cloud native buildpacks-cncfCloud native buildpacks-cncf
Cloud native buildpacks-cncfSuman Chakraborty
116 views20 slides
Dockercon 2019 Developing Apps with Containers, Functions and Cloud Services by
Dockercon 2019 Developing Apps with Containers, Functions and Cloud ServicesDockercon 2019 Developing Apps with Containers, Functions and Cloud Services
Dockercon 2019 Developing Apps with Containers, Functions and Cloud ServicesPatrick Chanezon
1.2K views77 slides
Javaone 2014 - Git & Docker with Jenkins by
Javaone 2014 - Git & Docker with JenkinsJavaone 2014 - Git & Docker with Jenkins
Javaone 2014 - Git & Docker with JenkinsAndy Pemberton
2.4K views16 slides

What's hot(20)

Spring on Kubernetes by Jay Lee
Spring on KubernetesSpring on Kubernetes
Spring on Kubernetes
Jay Lee96 views
Continuous Delivery with Jenkins & Kubernetes @ Sky by Adriana Vasiu
Continuous Delivery with Jenkins & Kubernetes @ SkyContinuous Delivery with Jenkins & Kubernetes @ Sky
Continuous Delivery with Jenkins & Kubernetes @ Sky
Adriana Vasiu516 views
Jenkins X intro (from google app dev conference) by Michael Neale
Jenkins X intro (from google app dev conference)Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)
Michael Neale429 views
Dockercon 2019 Developing Apps with Containers, Functions and Cloud Services by Patrick Chanezon
Dockercon 2019 Developing Apps with Containers, Functions and Cloud ServicesDockercon 2019 Developing Apps with Containers, Functions and Cloud Services
Dockercon 2019 Developing Apps with Containers, Functions and Cloud Services
Patrick Chanezon1.2K views
Javaone 2014 - Git & Docker with Jenkins by Andy Pemberton
Javaone 2014 - Git & Docker with JenkinsJavaone 2014 - Git & Docker with Jenkins
Javaone 2014 - Git & Docker with Jenkins
Andy Pemberton2.4K views
Jenkins X - automated CI/CD solution for cloud native applications on Kubernetes by Ted Won
Jenkins X - automated CI/CD solution for cloud native applications on KubernetesJenkins X - automated CI/CD solution for cloud native applications on Kubernetes
Jenkins X - automated CI/CD solution for cloud native applications on Kubernetes
Ted Won1.5K views
Lessons Learned Building a Container App Library by Adnan Abdulhussein
Lessons Learned Building a Container App LibraryLessons Learned Building a Container App Library
Lessons Learned Building a Container App Library
Adnan Abdulhussein945 views
Docker Indy Meetup - An Opinionated View of Building Docker Images and Pipelines by Matt Bentley
Docker Indy Meetup - An Opinionated View of Building Docker Images and PipelinesDocker Indy Meetup - An Opinionated View of Building Docker Images and Pipelines
Docker Indy Meetup - An Opinionated View of Building Docker Images and Pipelines
Matt Bentley254 views
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model by Docker, Inc.
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Docker, Inc.206 views
Cicd pixelfederation by Juraj Hantak
Cicd pixelfederationCicd pixelfederation
Cicd pixelfederation
Juraj Hantak206 views
Gitlab ci e kubernetes, build test and deploy your projects like a pro by sparkfabrik
Gitlab ci e kubernetes, build test and deploy your projects like a proGitlab ci e kubernetes, build test and deploy your projects like a pro
Gitlab ci e kubernetes, build test and deploy your projects like a pro
sparkfabrik3.2K views
Continuous Integration with Cloud Foundry Concourse and Docker on OpenPOWER by Indrajit Poddar
Continuous Integration with Cloud Foundry Concourse and Docker on OpenPOWERContinuous Integration with Cloud Foundry Concourse and Docker on OpenPOWER
Continuous Integration with Cloud Foundry Concourse and Docker on OpenPOWER
Indrajit Poddar1K views
Built in ci-cd with kubernetes, jenkins and helm by Pau López
Built in ci-cd with kubernetes, jenkins and helmBuilt in ci-cd with kubernetes, jenkins and helm
Built in ci-cd with kubernetes, jenkins and helm
Pau López1.1K views
Red Hat OpenShift App Dev meetup - Operator SDK by Dmitry Kartsev
Red Hat OpenShift App Dev meetup - Operator SDK Red Hat OpenShift App Dev meetup - Operator SDK
Red Hat OpenShift App Dev meetup - Operator SDK
Dmitry Kartsev73 views
Development environments in seconds using Bitnami containers by Adnan Abdulhussein
Development environments in seconds using Bitnami containersDevelopment environments in seconds using Bitnami containers
Development environments in seconds using Bitnami containers
Adnan Abdulhussein307 views
Deploying Windows Apps to Kubernetes with Draft and Helm by Jessica Deen
Deploying Windows Apps to Kubernetes with Draft and HelmDeploying Windows Apps to Kubernetes with Draft and Helm
Deploying Windows Apps to Kubernetes with Draft and Helm
Jessica Deen465 views
Testing strategies for Docker containers by Alexei Ledenev
Testing strategies for Docker containersTesting strategies for Docker containers
Testing strategies for Docker containers
Alexei Ledenev2.7K views
KubeCon China 2019 - Building Apps with Containers, Functions and Managed Ser... by Patrick Chanezon
KubeCon China 2019 - Building Apps with Containers, Functions and Managed Ser...KubeCon China 2019 - Building Apps with Containers, Functions and Managed Ser...
KubeCon China 2019 - Building Apps with Containers, Functions and Managed Ser...
Patrick Chanezon628 views

Similar to Serverless Container with Source2Image

Настройка окружения для кросскомпиляции проектов на основе docker'a by
Настройка окружения для кросскомпиляции проектов на основе docker'aНастройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'acorehard_by
174 views25 slides
Exploring Next Generation Buildpacks - Anand Rao & Scott Deeg by
Exploring Next Generation Buildpacks - Anand Rao & Scott DeegExploring Next Generation Buildpacks - Anand Rao & Scott Deeg
Exploring Next Generation Buildpacks - Anand Rao & Scott DeegVMware Tanzu
489 views42 slides
Docker at MoneyBird by
Docker at MoneyBirdDocker at MoneyBird
Docker at MoneyBirdEdwin Vlieg
808 views17 slides
DCEU 18: Building Your Development Pipeline by
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDocker, Inc.
1.6K views57 slides
Knative build for open whisk runtimes phase 1 - 2018-02-20 by
Knative build for open whisk runtimes   phase 1 - 2018-02-20Knative build for open whisk runtimes   phase 1 - 2018-02-20
Knative build for open whisk runtimes phase 1 - 2018-02-20Matt Rutkowski
142 views16 slides
2015 DockerCon Using Docker in production at bity.com by
2015 DockerCon Using Docker in production at bity.com2015 DockerCon Using Docker in production at bity.com
2015 DockerCon Using Docker in production at bity.comMathieu Buffenoir
1.1K views36 slides

Similar to Serverless Container with Source2Image(20)

Настройка окружения для кросскомпиляции проектов на основе docker'a by corehard_by
Настройка окружения для кросскомпиляции проектов на основе docker'aНастройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'a
corehard_by174 views
Exploring Next Generation Buildpacks - Anand Rao & Scott Deeg by VMware Tanzu
Exploring Next Generation Buildpacks - Anand Rao & Scott DeegExploring Next Generation Buildpacks - Anand Rao & Scott Deeg
Exploring Next Generation Buildpacks - Anand Rao & Scott Deeg
VMware Tanzu489 views
Docker at MoneyBird by Edwin Vlieg
Docker at MoneyBirdDocker at MoneyBird
Docker at MoneyBird
Edwin Vlieg808 views
DCEU 18: Building Your Development Pipeline by Docker, Inc.
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development Pipeline
Docker, Inc.1.6K views
Knative build for open whisk runtimes phase 1 - 2018-02-20 by Matt Rutkowski
Knative build for open whisk runtimes   phase 1 - 2018-02-20Knative build for open whisk runtimes   phase 1 - 2018-02-20
Knative build for open whisk runtimes phase 1 - 2018-02-20
Matt Rutkowski142 views
2015 DockerCon Using Docker in production at bity.com by Mathieu Buffenoir
2015 DockerCon Using Docker in production at bity.com2015 DockerCon Using Docker in production at bity.com
2015 DockerCon Using Docker in production at bity.com
Mathieu Buffenoir1.1K views
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor... by Docker, Inc.
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...
Docker, Inc.33K views
Making your app soar without a container manifest by LibbySchulze
Making your app soar without a container manifestMaking your app soar without a container manifest
Making your app soar without a container manifest
LibbySchulze337 views
How to Improve Your Image Builds Using Advance Docker Build by Docker, Inc.
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
Docker, Inc.1.2K views
DockerCon EU 2015: Trading Bitcoin with Docker by Docker, Inc.
DockerCon EU 2015: Trading Bitcoin with DockerDockerCon EU 2015: Trading Bitcoin with Docker
DockerCon EU 2015: Trading Bitcoin with Docker
Docker, Inc.5.9K views
[20200720]cloud native develoment - Nelson Lin by HanLing Shen
[20200720]cloud native develoment - Nelson Lin[20200720]cloud native develoment - Nelson Lin
[20200720]cloud native develoment - Nelson Lin
HanLing Shen132 views
Knative And Pivotal Function As a Service by Jay Lee
Knative And Pivotal Function As a ServiceKnative And Pivotal Function As a Service
Knative And Pivotal Function As a Service
Jay Lee291 views
DCSF 19 Building Your Development Pipeline by Docker, Inc.
DCSF 19 Building Your Development Pipeline  DCSF 19 Building Your Development Pipeline
DCSF 19 Building Your Development Pipeline
Docker, Inc.467 views
Gitlab and Lingvokot by Lingvokot
Gitlab and LingvokotGitlab and Lingvokot
Gitlab and Lingvokot
Lingvokot597 views
Mihai Criveti - PyCon Ireland - Automate Everything by Mihai Criveti
Mihai Criveti - PyCon Ireland - Automate EverythingMihai Criveti - PyCon Ireland - Automate Everything
Mihai Criveti - PyCon Ireland - Automate Everything
Mihai Criveti120 views
2016 - Continuously Delivering Microservices in Kubernetes using Jenkins by devopsdaysaustin
2016 - Continuously Delivering Microservices in Kubernetes using Jenkins2016 - Continuously Delivering Microservices in Kubernetes using Jenkins
2016 - Continuously Delivering Microservices in Kubernetes using Jenkins
devopsdaysaustin2.3K views
TDC2016POA | Trilha Cloud Computing - Source-to-image - How to transform any ... by tdc-globalcode
TDC2016POA | Trilha Cloud Computing - Source-to-image - How to transform any ...TDC2016POA | Trilha Cloud Computing - Source-to-image - How to transform any ...
TDC2016POA | Trilha Cloud Computing - Source-to-image - How to transform any ...
tdc-globalcode1.5K views
Microservices DevOps on Google Cloud Platform by Sunnyvale
Microservices DevOps on Google Cloud PlatformMicroservices DevOps on Google Cloud Platform
Microservices DevOps on Google Cloud Platform
Sunnyvale209 views
DevOps Workflow: A Tutorial on Linux Containers by inside-BigData.com
DevOps Workflow: A Tutorial on Linux ContainersDevOps Workflow: A Tutorial on Linux Containers
DevOps Workflow: A Tutorial on Linux Containers
inside-BigData.com564 views
Simply your Jenkins Projects with Docker Multi-Stage Builds by Eric Smalling
Simply your Jenkins Projects with Docker Multi-Stage BuildsSimply your Jenkins Projects with Docker Multi-Stage Builds
Simply your Jenkins Projects with Docker Multi-Stage Builds
Eric Smalling4.9K views

More from QAware GmbH

Make Developers Fly: Principles for Platform Engineering by
Make Developers Fly: Principles for Platform EngineeringMake Developers Fly: Principles for Platform Engineering
Make Developers Fly: Principles for Platform EngineeringQAware GmbH
5 views44 slides
Der Tod der Testpyramide? – Frontend-Testing mit Playwright by
Der Tod der Testpyramide? – Frontend-Testing mit PlaywrightDer Tod der Testpyramide? – Frontend-Testing mit Playwright
Der Tod der Testpyramide? – Frontend-Testing mit PlaywrightQAware GmbH
7 views34 slides
Was kommt nach den SPAs by
Was kommt nach den SPAsWas kommt nach den SPAs
Was kommt nach den SPAsQAware GmbH
6 views47 slides
Cloud Migration mit KI: der Turbo by
Cloud Migration mit KI: der Turbo Cloud Migration mit KI: der Turbo
Cloud Migration mit KI: der Turbo QAware GmbH
18 views23 slides
Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See... by
 Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See... Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See...
Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See...QAware GmbH
17 views13 slides
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster by
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster QAware GmbH
16 views31 slides

More from QAware GmbH(20)

Make Developers Fly: Principles for Platform Engineering by QAware GmbH
Make Developers Fly: Principles for Platform EngineeringMake Developers Fly: Principles for Platform Engineering
Make Developers Fly: Principles for Platform Engineering
QAware GmbH5 views
Der Tod der Testpyramide? – Frontend-Testing mit Playwright by QAware GmbH
Der Tod der Testpyramide? – Frontend-Testing mit PlaywrightDer Tod der Testpyramide? – Frontend-Testing mit Playwright
Der Tod der Testpyramide? – Frontend-Testing mit Playwright
QAware GmbH7 views
Was kommt nach den SPAs by QAware GmbH
Was kommt nach den SPAsWas kommt nach den SPAs
Was kommt nach den SPAs
QAware GmbH6 views
Cloud Migration mit KI: der Turbo by QAware GmbH
Cloud Migration mit KI: der Turbo Cloud Migration mit KI: der Turbo
Cloud Migration mit KI: der Turbo
QAware GmbH18 views
Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See... by QAware GmbH
 Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See... Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See...
Migration von stark regulierten Anwendungen in die Cloud: Dem Teufel die See...
QAware GmbH17 views
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster by QAware GmbH
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
QAware GmbH16 views
Endlich gute API Tests. Boldly Testing APIs Where No One Has Tested Before. by QAware GmbH
Endlich gute API Tests. Boldly Testing APIs Where No One Has Tested Before.Endlich gute API Tests. Boldly Testing APIs Where No One Has Tested Before.
Endlich gute API Tests. Boldly Testing APIs Where No One Has Tested Before.
QAware GmbH20 views
Kubernetes with Cilium in AWS - Experience Report! by QAware GmbH
Kubernetes with Cilium in AWS - Experience Report!Kubernetes with Cilium in AWS - Experience Report!
Kubernetes with Cilium in AWS - Experience Report!
QAware GmbH45 views
Kontinuierliche Sicherheitstests für APIs mit Testkube und OWASP ZAP by QAware GmbH
Kontinuierliche Sicherheitstests für APIs mit Testkube und OWASP ZAPKontinuierliche Sicherheitstests für APIs mit Testkube und OWASP ZAP
Kontinuierliche Sicherheitstests für APIs mit Testkube und OWASP ZAP
QAware GmbH20 views
Blue turns green! Approaches and technologies for sustainable K8s clusters. by QAware GmbH
Blue turns green! Approaches and technologies for sustainable K8s clusters.Blue turns green! Approaches and technologies for sustainable K8s clusters.
Blue turns green! Approaches and technologies for sustainable K8s clusters.
QAware GmbH32 views
Per Anhalter zu Cloud Nativen API Gateways by QAware GmbH
Per Anhalter zu Cloud Nativen API GatewaysPer Anhalter zu Cloud Nativen API Gateways
Per Anhalter zu Cloud Nativen API Gateways
QAware GmbH30 views
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster by QAware GmbH
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
QAware GmbH22 views
How to speed up Spring Integration Tests by QAware GmbH
How to speed up Spring Integration TestsHow to speed up Spring Integration Tests
How to speed up Spring Integration Tests
QAware GmbH21 views
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster by QAware GmbH
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-ClusterAus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
Aus blau wird grün! Ansätze und Technologien für nachhaltige Kubernetes-Cluster
QAware GmbH42 views
Cloud Migration – Eine Strategie die funktioniert by QAware GmbH
Cloud Migration – Eine Strategie die funktioniertCloud Migration – Eine Strategie die funktioniert
Cloud Migration – Eine Strategie die funktioniert
QAware GmbH28 views
Policy Driven Microservices mit Open Policy Agent by QAware GmbH
Policy Driven Microservices mit Open Policy AgentPolicy Driven Microservices mit Open Policy Agent
Policy Driven Microservices mit Open Policy Agent
QAware GmbH15 views
Make Developers Fly: Principles for Platform Engineering by QAware GmbH
Make Developers Fly: Principles for Platform EngineeringMake Developers Fly: Principles for Platform Engineering
Make Developers Fly: Principles for Platform Engineering
QAware GmbH61 views
Security Lab: OIDC in der Praxis by QAware GmbH
Security Lab: OIDC in der PraxisSecurity Lab: OIDC in der Praxis
Security Lab: OIDC in der Praxis
QAware GmbH19 views
Die nächsten 100 Microservices by QAware GmbH
Die nächsten 100 MicroservicesDie nächsten 100 Microservices
Die nächsten 100 Microservices
QAware GmbH14 views
Enterprise-level Kubernetes Security mit Open Source Tools - geht das? by QAware GmbH
Enterprise-level Kubernetes Security mit Open Source Tools - geht das?Enterprise-level Kubernetes Security mit Open Source Tools - geht das?
Enterprise-level Kubernetes Security mit Open Source Tools - geht das?
QAware GmbH33 views

Recently uploaded

LIVE OAK MEMORIAL PARK.pptx by
LIVE OAK MEMORIAL PARK.pptxLIVE OAK MEMORIAL PARK.pptx
LIVE OAK MEMORIAL PARK.pptxms2332always
7 views6 slides
PRIVACY AWRE PERSONAL DATA STORAGE by
PRIVACY AWRE PERSONAL DATA STORAGEPRIVACY AWRE PERSONAL DATA STORAGE
PRIVACY AWRE PERSONAL DATA STORAGEantony420421
7 views56 slides
Ukraine Infographic_22NOV2023_v2.pdf by
Ukraine Infographic_22NOV2023_v2.pdfUkraine Infographic_22NOV2023_v2.pdf
Ukraine Infographic_22NOV2023_v2.pdfAnastosiyaGurin
1.4K views3 slides
Construction Accidents & Injuries by
Construction Accidents & InjuriesConstruction Accidents & Injuries
Construction Accidents & InjuriesBisnar Chase Personal Injury Attorneys
6 views5 slides
Shreyas hospital statistics.pdf by
Shreyas hospital statistics.pdfShreyas hospital statistics.pdf
Shreyas hospital statistics.pdfsamithavinal
5 views9 slides
[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f... by
[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f...[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f...
[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f...DataScienceConferenc1
5 views18 slides

Recently uploaded(20)

LIVE OAK MEMORIAL PARK.pptx by ms2332always
LIVE OAK MEMORIAL PARK.pptxLIVE OAK MEMORIAL PARK.pptx
LIVE OAK MEMORIAL PARK.pptx
ms2332always7 views
PRIVACY AWRE PERSONAL DATA STORAGE by antony420421
PRIVACY AWRE PERSONAL DATA STORAGEPRIVACY AWRE PERSONAL DATA STORAGE
PRIVACY AWRE PERSONAL DATA STORAGE
antony4204217 views
Ukraine Infographic_22NOV2023_v2.pdf by AnastosiyaGurin
Ukraine Infographic_22NOV2023_v2.pdfUkraine Infographic_22NOV2023_v2.pdf
Ukraine Infographic_22NOV2023_v2.pdf
AnastosiyaGurin1.4K views
Shreyas hospital statistics.pdf by samithavinal
Shreyas hospital statistics.pdfShreyas hospital statistics.pdf
Shreyas hospital statistics.pdf
samithavinal5 views
[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f... by DataScienceConferenc1
[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f...[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f...
[DSC Europe 23] Matteo Molteni - Implementing a Robust CI Workflow with dbt f...
[DSC Europe 23][AI:CSI] Aleksa Stojanovic - Applying AI for Threat Detection ... by DataScienceConferenc1
[DSC Europe 23][AI:CSI] Aleksa Stojanovic - Applying AI for Threat Detection ...[DSC Europe 23][AI:CSI] Aleksa Stojanovic - Applying AI for Threat Detection ...
[DSC Europe 23][AI:CSI] Aleksa Stojanovic - Applying AI for Threat Detection ...
[DSC Europe 23] Stefan Mrsic_Goran Savic - Evolving Technology Excellence.pptx by DataScienceConferenc1
[DSC Europe 23] Stefan Mrsic_Goran Savic - Evolving Technology Excellence.pptx[DSC Europe 23] Stefan Mrsic_Goran Savic - Evolving Technology Excellence.pptx
[DSC Europe 23] Stefan Mrsic_Goran Savic - Evolving Technology Excellence.pptx
OPPOTUS - Malaysians on Malaysia 3Q2023.pdf by Oppotus
OPPOTUS - Malaysians on Malaysia 3Q2023.pdfOPPOTUS - Malaysians on Malaysia 3Q2023.pdf
OPPOTUS - Malaysians on Malaysia 3Q2023.pdf
Oppotus30 views
Games, Queries, and Argumentation Frameworks: Time for a Family Reunion by Bertram Ludäscher
Games, Queries, and Argumentation Frameworks: Time for a Family ReunionGames, Queries, and Argumentation Frameworks: Time for a Family Reunion
Games, Queries, and Argumentation Frameworks: Time for a Family Reunion
Data about the sector workshop by info828217
Data about the sector workshopData about the sector workshop
Data about the sector workshop
info82821729 views
[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo... by DataScienceConferenc1
[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...
[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...
Dr. Ousmane Badiane-2023 ReSAKSS Conference by AKADEMIYA2063
Dr. Ousmane Badiane-2023 ReSAKSS ConferenceDr. Ousmane Badiane-2023 ReSAKSS Conference
Dr. Ousmane Badiane-2023 ReSAKSS Conference
AKADEMIYA20635 views
[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init... by DataScienceConferenc1
[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init...[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init...
[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init...
CRM stick or twist workshop by info828217
CRM stick or twist workshopCRM stick or twist workshop
CRM stick or twist workshop
info82821714 views
OECD-Persol Holdings Workshop on Advancing Employee Well-being in Business an... by StatsCommunications
OECD-Persol Holdings Workshop on Advancing Employee Well-being in Business an...OECD-Persol Holdings Workshop on Advancing Employee Well-being in Business an...
OECD-Persol Holdings Workshop on Advancing Employee Well-being in Business an...
DGST Methodology Presentation.pdf by maddierlegum
DGST Methodology Presentation.pdfDGST Methodology Presentation.pdf
DGST Methodology Presentation.pdf
maddierlegum7 views

Serverless Container with Source2Image

  • 1. Dr. Josef Adersberger, CTO & Co-Founder QAware Serverless containers … with source-to-image https://github.com/adersberger/source2image
  • 2. Inventor's workshop and assembly line ready? ready!
  • 3. The evolution of software delivery The dark ages: Export JAR, upload to deployment server, write ticket, wait until application is deployed to multi-project application server by far shore ops team. The container era: Build application, package with runtime into container image, push to image registry, deploy to container manager. PaaS & Serverless heaven: git push + magic happens here industrialization process: 1. lower change lead time 2. higher quality confidence 3. lower vertical integration
  • 4. Rule #1: Avoid too much magic at early stages
  • 6. Serverless flavors git push functions git push something in a container generic container CI/CD pipeline FULL SERVERLESS MILD SERVERLESS serverlessy: black box application runtime and infrastructure resources Why you might need mild serverless: regulatory compliance, shift left quality checks & automated tests, complex staging and deployment patterns, decoupling from cloud vendors or immature open source projects serverlessy: scale-to-zero, elastic
  • 7. The anatomy of a mild serverless toolchain Watch for code changes Choose compilation method and base image Compile code, prepare image, inject binaries Deploy image to target container manager Source-to-Image workflow Developer's workspace aka inventor's workshop CI/CD pipeline aka assembly line Static analysis, test automation, staging and promotion, image scanning, ... Image builders
  • 8. 8 With the volkswagen CI plugin you can completely focus on source-to-image https://github.com/auchenberg/volkswagen
  • 9. The source-to-image challengers: WORKFLOW TOOLS (inner loop & outer loop) ● Skaffold (https://skaffold.dev) ● Tilt (https://tilt.dev) ● Garden (https://garden.io) BUILDER TOOLS ● OpenShift Source2Image (https://github.com/openshift/source-to-image) ● buildpacks.io (https://buildpacks.io) ● Draft (https://draft.sh) ● Jib (https://github.com/GoogleContainerTools/jib)
  • 12. # install pack tool (buildpack reference implementation) brew tap buildpack/tap brew install pack # get suggested builders for sample application # build image for sample application
  • 13. Buildpack internals Builder Image (e.g. heroku/buildpacks or cloudfoundry/bionic) App Image StackBuild Base Image Run Base Image Lifecycle Buildpack 1 Buildpack n ... Detection Analysis Build Export bin/detect bin/build Runtime Layer Dependency Layer App Layer
  • 15. # install s2i brew install source-to-image # get and build source2image for springboot & java git clone https://github.com/ganrad/openshift-s2i-springboot-java.git docker build --build-arg MAVEN_VER=3.6.2 --build-arg GRADLE_VER=5.6.3 -t springboot-java . # build image for sample application s2i build --incremental=true . springboot-java skaffold-example-god
  • 16. S2I internals BUILDER IMAGE Pre-defined scripts: APP IMAGE Build Base Image building the application artifacts from source and placing them into the appropriate directories inside the app image executing the application (entrypoint) Runtime Layer Build Layer Artifact Layer CLI tool: entrypoint: run
  • 18. # install draft along with helm brew install kubernetes-helm helm init brew install azure/draft/draft # create draft files for application (Helm chart, draft.toml, Dockerfile) draft create --> Draft detected Shell (46.149372%) --> Could not find a pack for Shell. Trying to find the next likely language match... --> Draft detected Batchfile (28.163621%) --> Could not find a pack for Batchfile. Trying to find the next likely language match... --> Draft detected Java (12.213444%) --> Ready to sail # build image for sample application and deploy application to k8s draft up # connect to the application endpoint draft connect
  • 19. Draft internals BUILDER HELM CHART APPLICATION HELM CHART [environments] [environments.development] name = "god" namespace = "default" wait = true watch = false watch-delay = 2 auto-connect = false dockerfile = "Dockerfile" chart = "" draft.toml java primary language detection by github linguist and mapped to chart directory by language name generated by draft create
  • 22. ./mvnw compile jib:dockerBuild -Dimage=skaffold-example-god pom.xml
  • 25. # install skaffold brew install skaffold # build & deploy image (once) skaffold run # build & deploy image (everytime the code changes) skaffold dev apiVersion: skaffold/v1beta16 kind: Config build: artifacts: - image: skaffold-example-god context: . jib: {} deploy: kubectl: manifests: - src/k8s/*.yaml apiVersion: skaffold/v1beta16 kind: Config build: artifacts: - image: skaffold-example-god custom: buildCommand: ./build-buildpacks.sh dependencies: paths: - . deploy: kubectl: manifests: - src/k8s/*.yaml #!/bin/bash set -e images=$(echo $IMAGES | tr " " "n") for image in $images do pack build $image --builder cloudfoundry/cnb:bionic if $PUSH_IMAGE then docker push $image fi done driven by skaffold.yaml:
  • 26. Builder performance comparison with Skaffold Builder Time s2i (--incremental=true) 1:23m Draft 1:14m Buildpacks 0:42m jib 0:21m median of 3 runs timed by "time" command after an initial warming run and a code change between each run - build and caching behaviour not optimized time skaffold run -f=skaffold-s2i.yml time skaffold run -f=skaffold-buildpacks.yml time skaffold run -f=skaffold-jib.yml time draft up
  • 27. Builder shootout (lower is better) Criteria Buildpacks.io s2i Draft Jib Speed ● lead time to change ● image size (docker image ls) ● rebasing 2 4 3 1 Supported application technologies Java, Node.JS, Python, GoLang, ... 2 3 1 4 (k.O. if non-Java) Auto-detection of application technologies yes / no 1 3 1 3 Maturity / future proof 3 2 4 (k.O.) 1 8 12 9 (k.O.) 9
  • 29. # install Tilt brew tap windmilleng/tap brew install windmilleng/tap/tilt # build & deploy image (with every change) tilt up # Deploy: tell Tilt what YAMLs to deploy k8s_yaml('src/k8s/pod-god.yaml') # Build: tell Tilt what images (name) to build from which directories docker_build('skaffold-example-god', '.') # Watch: tell Tilt how to connect locally (optional) k8s_resource('web', port_forwards=8080) driven by Tiltfile (Starlark, a Python dialect):
  • 32. # install Garden brew tap garden-io/garden brew install garden-cli # build & deploy image (once) garden build # build & deploy image (with # every change) garden dev kind: Project name: god-project environments: - name: local providers: - name: local-kubernetes context: docker-desktop --- kind: Module name: god description: God service type: container services: - name: god ports: - name: http containerPort: 8080 healthCheck: httpGet: path: / port: http ingresses: - path: / port: http driven by garden.yml containing garden-defined resource types as abstractions for k8s primitives:
  • 34. Workflow shootout (lower is better) Criteria ⇒ Position Skaffold Tilt Garden Pipeline integratability ● As pipeline tasks in Jenkins Pipelines, Tekton, Build tools ● Support for container testing ● Deployment options: Helm, Kustomize, kubectl 1 3 2 Supported image builders ● Plain Docker ● Daemon-less builds ● Builders: Buildpacks, Draft, s2i, Jib 2 3 1 Multi-environments Support for multiple environments like local, dev, prod 1 3 1 Multi-image projects Support for code repositories containing multi-image projects 1 1 1 Local dev support Local build, local run, build-on-change 1 1 1 Maturity / future proof 1 2 2 7 13 8
  • 35. 1. The way from source to image can be done in a generic way 2. If you're doing Java then go for the Google guys: Skaffold and Jib 3. If you're polyglot then go for Skaffold and buildpacks.io 4. Use the same workflow & builder tool for local builds and CI/CD builds 5. Optimize the change lead time for features and the local round trip time for developers 5 things:
  • 37. 37 A possible journey towards full serverless as commodity Serverless Build Serverless Run
  • 38. Bonus slide: Change lead time optimization 1. Use well-architectured, security-hardened and minimal base images like: a. Google Distroless Images (https://github.com/GoogleContainerTools/distroless) b. RedHat Universal Base Images (https://developers.redhat.com/products/rhel/ubi) 2. Use a Docker daemon-less image builder with excessive caching: a. Google Kaniko (https://github.com/GoogleContainerTools/kaniko) b. uber Makiso (https://github.com/uber/makisu) c. Docker BuildKit (https://github.com/moby/buildkit) d. Google Bazel (https://bazel.build) 3. Use an efficient pipeline orchestrator with task parallelization capabilities: a. Tekton (https://tekton.dev) b. Argo CD (https://argoproj.github.io/argo-cd)