Dev opsec dockerimage_patch_n_lifecyclemanagement_2019

Kubernetes Application Life Cycle
(patch management)
2019/04/17
#DevoxxFR
DevSecOps
Container Image :
“Application LifeCycle Management”
sed /sLifeCycle/Patch/g
devoxx.fr
2019/04/17

(patch management)
2019/04/17
#DevoxxFR
Abstract
Lors de cette présentation, nous allons dans un premier temps rappeler la
spécificité de docker par rapport à une VM (PID, cgroups, etc) parler du
système de layer et de la différence entre images et instances puis nous
présenterons succinctement kubernetes.
Ensuite, nous présenterons un processus « standard » de propagation
d’une version CI/CD (développement, préproduction, production) à travers
les tags docker.
Enfin, nous parlerons des différents composants constituant une
application docker (base-image, tooling, librairie, code).
Une fois cette introduction réalisée, nous parlerons du cycle de vie d’une
application à travers ses phases de développement, BAU pour mettre en
avant que les failles de sécurité en période de développement sont
rapidement corrigées par de nouvelles releases, mais pas nécessairement
en BAU où les releases sont plus rares. Nous parlerons des diverses
solutions (jfrog Xray, clair, …) pour le suivie des automatique des CVE et
l’automatisation des mises à jour. Enfin, nous ferons un bref retour
d’expérience pour parler des difficultés rencontrées et des propositions
d’organisation mises en oeuvre.
Cette présentation bien qu’illustrée par des implémentations techniques et
très organisationnel
To Do
Français English

(patch management)
2019/04/17
#DevoxxFR
@kanedafromparis
Charles Sabourdin
Javaiste
Linuxien
Devoxx France
ParisJUG
OpenSource
Architect
Dev/Ops
https://github.com/kanedafromparis/
https://github.com/kanedafromparisfriends
- 180:00
Javaiste
Linuxien
Devoxx France
ParisJUG OpenSource
Architect
Dev/Ops
@jcsirot
https://github.com/jcsirot
Jenkins
Jean-Christophe Sirot

(patch management)
2019/04/17
#DevoxxFR
Sommaire
I. Reminder
A. Docker
1. intro
2. isolation
3. layers
B. Kubernetes
1. generic
2. deployment
C. Development pipeline (security focus)
II. Application life cycle
A. Scanning tools
B. Too much technos
III. Proposed solution
A. 1,2,3 Hosting
B. Pitfalls

(patch management)
2019/04/17
#DevoxxFR
Docker Engine uses namespaces such as the following on Linux:
● The pid namespace: Process isolation (PID: Process ID).
● The net namespace: Managing network interfaces (NET: Networking).
● The ipc namespace: Managing access to IPC resources (IPC:
InterProcess Communication).
● The mnt namespace: Managing filesystem mount points (MNT: Mount).
● The uts namespace: Isolating kernel and version identifiers. (UTS: Unix
Timesharing System).
Namespaces
Docker uses a technology called namespaces to provide the isolated workspace called the container.
When you run a container, Docker creates a set of namespaces for that container.
These namespaces provide a layer of isolation. Each aspect of a container runs in a separate namespace
and its access is limited to that namespace.
I. Reminder : Docker
sources : https://docs.docker.com/engine/docker-overview/#the-underlying-technology
https://en.wikipedia.org/wiki/Linux_kernel

(patch management)
2019/04/17
#DevoxxFR
Control groups
Docker Engine on Linux also relies on another technology called control groups (cgroups). A cgroup
limits an application to a specific set of resources. Control groups allow Docker Engine to share
available hardware resources to containers and optionally enforce limits and constraints. For
example, you can limit the memory available to a specific container.
Union file systems
Union file systems, or UnionFS, are file systems that operate by creating layers, making them very
lightweight and fast. Docker Engine uses UnionFS to provide the building blocks for containers.
Docker Engine can use multiple UnionFS variants, including AUFS, btrfs, vfs, and DeviceMapper.
Container format
Docker Engine combines the namespaces, control groups, and UnionFS into a wrapper called a
container format. The default container format is libcontainer. In the future, Docker may support other
container formats by integrating with technologies such as BSD Jails or Solaris Zones.
Isolation
sources : https://docs.docker.com/engine/docker-overview/#the-underlying-technology
https://en.wikipedia.org/wiki/Cgroups

(patch management)
2019/04/17
#DevoxxFR
Run simple docker file
docker-machine create dx19 --virtualbox-cpu-count=4 --virtualbox-memory=4096
eval $(docker-machine env dx19)
docker pull nginx:1.15.11
docker pull jenkins/jenkins:lts
docker pull owasp/dependency-track:3.4.0
docker pull quay.io/travelaudience/docker-nexus:3.15.2
docker pull kanedafromparis/nginx:0.1
docker run --name demo-root-nginx -d -p 8181:80 nginx:1.15.11
open http://$(docker-machine ip dx19):$(docker inspect demo-root-nginx |
jq -r '.[0].NetworkSettings.Ports."80/tcp"[0].HostPort')
docker-machine ssh dx19
ps auxf | grep -C 2 "nginx"
docker exec -it demo-root-nginx /bin/bash
cat /run/nginx.pid
ls -l /proc | grep ng
docker rm demo-root-nginx -f

(patch management)
2019/04/17
#DevoxxFR
Union file systems (again)
Union file systems, or UnionFS, are file systems that operate by creating layers,
making them very lightweight and fast. Docker Engine uses UnionFS to provide the
building blocks for containers. Docker Engine can use multiple UnionFS variants,
including AUFS, btrfs, vfs, and DeviceMapper.
Container format
A Docker image is built up from a series of layers. Each layer represents an
instruction in the image’s Dockerfile. Each layer except the very last one is read-only.
Each layer is only a set of differences from the layer before it. The layers are stacked
on top of each other. When you create a new container, you add a new writable layer
on top of the underlying layers. This layer is often called the “container layer”.
All changes made to the running container, such as writing new files, modifying
existing files, and deleting files, are written to this thin writable container layer.
Layers
sources : https://docs.docker.com/storage/storagedriver/#images-and-layers

(patch management)
2019/04/17
#DevoxxFR
Breadcrumb
Operating System
Toolings / Utils
Language Toolings
Application Toolings
Frameworks / external libs
Application code
Operating System (Docker Image)

(patch management)
2019/04/17
#DevoxxFR
Let’s dive into docker layer
brew install dive
dive nginx:1.15.11
dive jenkins/jenkins:lts
dive owasp/dependency-track:3.4.0
dive quay.io/travelaudience/docker-nexus:3.15.2
...

(patch management)
2019/04/17
#DevoxxFR
Run simple docker file
FROM alpine:3.8
LABEL Maintainer="Kanedafriends <kf@kaneda-k.net>"
Description="This is a simple image"
Version="0.1"
Name="ocp-nginx"
RUN apk --update add nginx
EXPOSE 8080
COPY nginx.conf /etc/nginx/nginx.conf
COPY nginx.vh.default.conf /etc/nginx/conf.d/default.conf
COPY run-nginx.sh /run-nginx.sh
COPY 0.1/ /var/www/localhost/htdocs/
RUN mkdir -p /var/tmp/nginx &&
chown -R 1001:0 /var/log/nginx /var/run /etc/nginx/* /var/lib/nginx/ /var/tmp/* &&
chmod -R g+rwX /var/log/nginx /var/run /etc/nginx/* /var/lib/nginx/ /var/tmp/* &&
ln -sf /dev/stdout /var/log/nginx/access.log &&
ln -sf /dev/stderr /var/log/nginx/error.log &&
ln -sf /dev/stdout /var/log/nginx/host.access.log &&
chmod -v ug+x /run-nginx.sh
USER 1001
CMD ["/run-nginx.sh"]
sources : https://raw.githubusercontent.com/kanedafromparisfriends/exo_kube_101/exercices/exo0010/Dockerfile.nginx
dive kanedafromparis/nginx:0.1

(patch management)
2019/04/17
#DevoxxFR
Docker Image Docker
Image
Docker
Image
Docker Image Docker Image
Breadcrumb
Toolings / Utils
Language Toolings
Application code
Alpine Base Image
OpenJDK 8
Tomcat Wildfly
App lib
App 3
Apache httpd 2.4
PHP 7.1 PHP 5.6
Drupal Wordpress
App 1 App 4
Springb
oot
Operating System
App
lib 2
App 2
- 150:00

(patch management)
2019/04/17
#DevoxxFR
Docker Images
A recipe or template for creating Docker containers. It includes the steps for
installing and running the necessary software
Docker Container
Like a tiny virtual machine that is created from the instructions found within the
Docker image
Docker Client
Command-line utility or other tool that takes advantage of the Docker API
(docs.docker.com/ reference/api/docker_remote_api) to communicate with a
Docker daemon
Glossary
sources : https://docs.docker.com/glossary/
https://dzone.com/refcardz/getting-started-with-docker-1
Docker Host
A physical or virtual machine that is running a Docker daemon and contains cached images as well as runnable containers created from
images
Docker Registry
A repository of Docker images that can be used to create Docker containers. Docker Hub (hub.docker.com) is the most popular social
example of a Docker repository.

(patch management)
2019/04/17
#DevoxxFR
● Agile application creation and deployment: Increased ease and efficiency
of container image creation compared to VM image use.
● Continuous development, integration, and deployment
● Dev and Ops separation of concerns
● Environmental consistency across development, testing, and production
● Application-centric management: Raises the level of abstraction from
running an OS on virtual hardware to running an application on an OS
using logical resources.
● Loosely coupled, distributed, elastic, liberated micro-services
● Resource isolation: Predictable application performance.
● Resource utilization: High efficiency and density.
Kubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both
declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely
available.
Kubernetes
I. Reminder : Kubernetes
sources : https://kubernetes.io/docs/concepts/architecture/cloud-controller/
https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/

(patch management)
2019/04/17
#DevoxxFR
Master
The machine that controls Kubernetes nodes. This is where all task assignments originate.
Node
These machines perform the requested, assigned tasks. The Kubernetes master controls
them.
Pod
A group of one or more containers deployed to a single node. All containers in a pod share an
IP address, IPC, hostname, and other resources. Pods abstract network and storage away
from the underlying container. This lets you move containers around the cluster more easily.
Kubelet
This service runs on nodes and reads the container manifests and ensures the defined
containers are started and running.
Glossary
sources : https://www.redhat.com/en/topics/containers/what-is-kubernetes
Replication controller
This controls how many identical copies of a pod should be running somewhere on the cluster.
Service
This decouples work definitions from the pods. Kubernetes service proxies automatically get service requests to the right pod—no matter
where it moves to in the cluster or even if it’s been replaced.

(patch management)
2019/04/17
#DevoxxFR
A container is an instanciate executable isolated
process
A container repository is a library of image.
A Kubernetes Pod (PO) is a group of one or
more containers.
A service (SVC) is named mapping to pod
Object
sources : https://github.com/kanedafromparisfriends/icones_ocp_kube
A ReplicatSet (RS) create and maintain the pod
declaration that will instantiate pods replica
(docker images) that will provide services.
A Deployment (Deploy) create the ReplicatSet that
will provide services.
Persistent Volume (PV) is a piece of networked
storage in the cluster.
Persistent Volume Claim (PVC) is a reservation of a
Persistent Volume into a namespace / project
...

(patch management)
2019/04/17
#DevoxxFR
The following are typical use cases for Deployments:
● Create a Deployment to rollout a ReplicaSet. The ReplicaSet creates Pods in the background. Check the status of the rollout to see if
it succeeds or not.
● Declare the new state of the Pods by updating the PodTemplateSpec of the Deployment. A new ReplicaSet is created and the
Deployment manages moving the Pods from the old ReplicaSet to the new one at a controlled rate. Each new ReplicaSet updates
the revision of the Deployment.
Deployment
sources : https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
● Rollback to an earlier Deployment revision if the current
state of the Deployment is not stable. Each rollback updates
the revision of the Deployment.
● Scale up the Deployment to facilitate more load.
● Pause the Deployment to apply multiple fixes to its
PodTemplateSpec and then resume it to start a new rollout.
● Use the status of the Deployment as an indicator that a
rollout has stuck.
● Clean up older ReplicaSets that you don’t need anymore.

(patch management)
2019/04/17
#DevoxxFR
kubernetes : set cluster
gcloud container clusters create dx19 --zone europe-west1-b --node-locations europe-west1-a,europe-west1-b,europe-west1-c
---
eksctl create cluster -p dx19 -n dx19 -r eu-central-1 --full-ecr-access
---
kops create cluster --name "dx19" --cloud=aws --zones=eu-central-1 --master-zones=""eu-central-1c" --master-size=t2.large --node-count=6
--node-size=t2.2xlarge --state=s3://dx19 --yes
---
...
sources : https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster
https://github.com/weaveworks/eksctl
https://github.com/kubernetes/kops/blob/master/docs/aws.md

(patch management)
2019/04/17
#DevoxxFR
kubernetes : imperative kubectl
gcloud auth login
gcloud container clusters get-credentials dx19 --zone europe-west1-b --project dx19
source ~/.kubectl_aliases
kubectl create ns sample2
kubectl run nginx --image=nginx -n sample2
kubectl run nx2 --generator=run-pod/v1 --image=nginx -n sample2
kgpo -n sample2
kubectl exec -it nx2 -n sample2 -- /bin/bash
kubectl get all -n sample2
kubectl get pod/nx2 -o yaml -n sample2
kubectl get pod/nginx-dbddb74b8-fwk27 -o yaml -n sample2
sources : https://ahmet.im/blog/kubectl-aliases/

(patch management)
2019/04/17
#DevoxxFR
En pratique (et Yaml)
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "3"
kubernetes.io/change-cause: "blabla..."
creationTimestamp: 2018-01-27T18:17:26Z
generation: 5
labels:
run: logversion
name: logversion
spec:
minReadySeconds: 15
progressDeadlineSeconds: 300
replicas: 3
selector:
matchLabels:
run: logversion
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
...
template:
metadata:
creationTimestamp: null
labels:
run: resourcesmonger
spec:
containers:
- image: https://myregistry.org/kanedafromparis/logversion:3.1
imagePullPolicy: IfNotPresent
name: logversion
ports:
- containerPort: 8080
protocol: TCP
resources:
...
livenessProb
...
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
...

(patch management)
2019/04/17
#DevoxxFR
kubernetes : declarative kubectl
brew install kubetail
gcloud auth login
gcloud container clusters get-credentials dx19 --zone europe-west1-b --project dx19
source ~/.kubectl_aliases
kubens sample2
kubectl apply -f
https://raw.githubusercontent.com/kanedafromparisfriends/exo_kube_101/exercices/exo0050/logsversion.deploy.011.yaml
kubetail -l run=logsversion
kubectl apply -f
kubetail -l run=logsversion
kubectl delete -f
sources : https://raw.githubusercontent.com/kanedafromparisfriends/exo_kube_101/exercices/exo0010/Dockerfile.nginx

(patch management)
2019/04/17
#DevoxxFR
Application dependency on Kubernetes
sources : https://www.infoq.com/articles/kubernetes-effect

(patch management)
2019/04/17
#DevoxxFR
Deployments Strategy
sources : https://www.infoq.com/articles/kubernetes-effect

(patch management)
2019/04/17
#DevoxxFR
Liveness : If the command returns a non-zero value, the kubelet kills the Container and restarts it.
Readiness : A pod with containers reporting that they are not ready does not receive traffic through Kubernetes Services
httpGet:
scheme: <http(s)>
host: <header hostname>
path: <uri>
port: <1 - 65 535 or name>
httpHeaders :
- name : <header name>
value :<header value>
...
exec:
command:
- cat
- /tmp/healthy
...
tcpSocket:
host: <host to test>
port: <1 - 65 535 or name>
Probs
sources : https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.14/#probe-v1-core

(patch management)
2019/04/17
#DevoxxFR
initContainers : List of initialization containers belonging to the pod.
Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and
PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in
which case the handler is aborted.
postStart : PostStart is called immediately after a container is created.
preStop : PreStop is called immediately before a container is terminated.
httpGet:
scheme: <http(s)>
host: <header hostname>
path: <uri>
port: <1 - 65 535 or name>
httpHeaders :
- name : <header name>
value :<header value>
...
exec:
command:
- cat
- /tmp/healthy
...
tcpSocket:
host: <host to test>
port: <1 - 65 535 or name>
initContainers & Container hooks
sources : kubectl explain pods.spec.containers.lifecycle
kubectl explain pods.spec.initContainers

(patch management)
2019/04/17
#DevoxxFR
Docker
Image
Docker Image Docker
Image
Docker
Image
Breadcrumb
Infrastructure
Operating System (Node OS)
Toolings / Utils
Language Toolings
Application code
Alpine Base Image
OpenJDK 8
Tomcat Wildfly
App lib
App 2
App 3
Apache httpd 2.4
PHP 7.1 PHP 5.6
Drupal Wordpress
App 1 App 4
Spring|
boot
https://myregistry.or
g/.../App1:x.y
../App2:x.y ../App3:x.y ../App4:x.y ../App5:x.y
Docker
Image
kubelet

(patch management)
2019/04/17
#DevoxxFR
I. Reminder
A. Docker
1. intro
2. isolation
3. layers
B. Kubernetes
1. generic
2. deployment
C. Development pipeline (security focus)
A. Scanning tools
B. Too much technos
A. 1,2,3 Hosting
B. Pitfalls
Sommaire

(patch management)
2019/04/17
#DevoxxFR
sources : https://github.com/jcsirot/spring-petclinic-microservices
https://github.com/spring-petclinic/spring-petclinic-microservices
https://javaetmoi.com/2018/10/architecture-microservices-avec-spring-cloud/
Notre projet de references :
I. Reminder : Development pipeline

(patch management)
2019/04/17
#DevoxxFR
Dev Local
Developper need to develop project on it,s local machine (unit test, debug,
etc…)
Dev on cluster
In order to check some integration issue, Dev can develop into a dedicated
namespace. This namespace can host component and resources identical to
production
CI on cluster
Automated build/test system (Jenkins) catch commit in order to build the future
docker artifact(s) that will be propagated to the QA namespaces
QA on cluster
Automated test are executed on a dedicated namespaces
Staging on cluster (optional)
A referential namespace can be used for load testing, human validation and
pre-visualisation
Production on cluster
A production namespace host the project with the necessary security
constraintes
s/Development/Deployment/g pipeline
sources : Containerizing Continuous Delivery in Java by Daniel Bryant

(patch management)
2019/04/17
#DevoxxFR
Docker local build
git clone https://github.com/jcsirot/spring-petclinic-microservices && cd spring-petclinic-microservices
BUILD_TAG=$(uuidgen | head -c 8)
REVISION="2.1.3-SNAPSHOT"
DOCKERREPO="kanedafromparis"
docker build -t builder:${BUILD_TAG} --target builder --build-arg REVISION=${REVISION} . &&
docker build -t base:${BUILD_TAG} --target base --build-arg REVISION=${REVISION} . &&
docker build -t ${DOCKERREPO}/spring-petclinic-admin-server:${REVISION} -f spring-petclinic-admin-server/Dockerfile
--build-arg BASE_ID=${BUILD_TAG} --build-arg REVISION=${REVISION} --build-arg EXPOSED_PORT=9090 . &&
docker build -t ${DOCKERREPO}/spring-petclinic-customers-service:${REVISION} -f spring-petclinic-customers-service/Dockerfile
docker build -t ${DOCKERREPO}/spring-petclinic-vets-service:${REVISION} -f spring-petclinic-vets-service/Dockerfile
docker build -t ${DOCKERREPO}/spring-petclinic-visits-service:${REVISION} -f spring-petclinic-visits-service/Dockerfile
docker build -t ${DOCKERREPO}/spring-petclinic-config-server:${REVISION} -f spring-petclinic-config-server/Dockerfile
docker build -t ${DOCKERREPO}/spring-petclinic-discovery-server:${REVISION} -f spring-petclinic-discovery-server/Dockerfile
docker build -t ${DOCKERREPO}/spring-petclinic-api-gateway:${REVISION} -f spring-petclinic-api-gateway/Dockerfile
docker build -t ${DOCKERREPO}/spring-petclinic-hystrix-dashboard:${REVISION} -f spring-petclinic-hystrix-dashboard/Dockerfile
--build-arg BASE_ID=${BUILD_TAG} --build-arg REVISION=${REVISION} --build-arg EXPOSED_PORT=7979 .

(patch management)
2019/04/17
#DevoxxFR
Helm
Helm is a tool for managing Kubernetes packages called charts.
Helm can do the following:
● Create new charts from scratch
● Package charts into chart archive (tgz) files
● Interact with chart repositories where charts are stored
● Install and uninstall charts into an existing Kubernetes cluster
● Manage the release cycle of charts that have been installed
with Helm
foo/
Chart.yaml # A YAML file containing
information about the chart
LICENSE # OPTIONAL: A plain text file
containing the license for
the chart
README.md # OPTIONAL: A human-readable
README file
requirements.yaml # OPTIONAL: A YAML file listing
dependencies for the chart
values.yaml # The default configuration
values for this chart
charts/ # A directory containing any
charts upon which this
chart depends.
templates/ # A directory of templates that,
when combined with values,
will generate valid Kubernetes
manifest files.
templates/NOTES.txt # OPTIONAL: A plain text file
containing short usage notes
sources : https://helm.sh/docs/developing_charts/
Chart file Structure

(patch management)
2019/04/17
#DevoxxFR
Helm : hook
Hooks allow you, the chart developer, an opportunity to perform
operations at strategic points in a release lifecycle. For example,
consider the lifecycle for a helm install. By default, the lifecycle
looks like this:
1. User runs helm install foo
2. Chart is loaded into Tiller
3. After some verification, Tiller renders the foo templates
4. Tiller loads the resulting resources into Kubernetes
5. Tiller returns the release name (and other data) to the client
6. The client exits
Hook weights can be positive or negative numbers but must be
represented as strings. When Tiller starts the execution cycle of hooks
of a particular kind (ex. the pre-install hooks or post-install hooks, etc.)
it will sort those hooks in ascending order.
sources : https://speakerdeck.com/unguiculus/helm-the-better-way-to-deploy-on-kubernetes
https://www.youtube.com/watch?v=3q0R5x6mBZg
pre-install
post-install
pre-delete
post-delete
pre-upgrade
post-upgrade
pre-rollback
post-rollback
crd-install
hoock annotations
apiVersion: batch/v1
kind: Job
metadata:
name: demo-job
annotations:
"helm.sh/hook": post-delete
"helm.sh/hook-weight": "-5"
"helm.sh/hook-delete-policy": hook-succeeded
spec:
...

(patch management)
2019/04/17
#DevoxxFR
Digression: Tillerless
That's it, tiller uses the same kubeconfig as helm to connect to
your cluster and as you see above you can pass the namespace which
tiller will use to store helm releases. You can have many
namespaces that way and just pass it when tiller starts. And a big
plus not a single tiller instance running in your Kubernetes cluster.
Your user's RBAC rules have to allow to store secrets/configmaps
(tiller's storage) in that namespace and that's it, no more service
accounts and other RBAc rules for your tiller. :)
sources : https://rimusz.net/tillerless-helm/

(patch management)
2019/04/17
#DevoxxFR
Helm
git clone https://github.com/jcsirot/spring-petclinic-microservices && cd spring-petclinic-microservices
...

(patch management)
2019/04/17
#DevoxxFR
The perfect system of record for all your software parts.
● Manage components, build artifacts, and release candidates in one
central location.
● Understand component security, license, and quality issues.
● Modernize software development with intelligent staging and release
functionality.
● Scale DevOps delivery with high availability and active/active clustering.
● Sleep comfortably with world-class support and training.
Universal support for all your favorite formats and tools.
● Store and distribute Maven/Java, npm, NuGet, RubyGems, Docker, P2,
OBR, APT and YUM and more.
● Manage components from dev through delivery: binaries, containers,
assemblies, and finished goods.
● Awesome support for the Java Virtual Machine (JVM) ecosystem,
including Gradle, Ant, Maven, and Ivy.
● Integrated with popular tools like Eclipse, IntelliJ, Hudson, Jenkins,
Puppet, Chef, Docker, and more.
Registry : Exemple nexus
sources : https://www.sonatype.com/nexus-repository-sonatype

(patch management)
2019/04/17
#DevoxxFR
Jenkins stage
def credentials = [usernamePassword(credentialsId: 'docker', usernameVariable: 'DOCKER_USER', passwordVariable:
'DOCKER_PASSWORD')]
stage("Push images") {
withCredentials(credentials) {
sh "docker login -u ${DOCKER_USER} -p ${DOCKER_PASSWORD} docker.devoxxfr.chelonix.org"
sh "docker push docker.devoxxfr.chelonix.org/jcsirot/spring-petclinic-admin-server:${revision}"
...
}
}
Jenkinsfile

(patch management)
2019/04/17
#DevoxxFR
SonarQube® is an automatic code review tool to detect bugs, vulnerabilities and
code smells in your code. It can integrate with your existing workflow to enable
continuous code inspection across your project branches and pull requests.
Continuous Inspection
SonarQube provides the capability to not only show health of an application but
also to highlight issues newly introduced. With a Quality Gate in place, you can
fix the leak and therefore improve code quality systematically.
Detect Tricky Issues
Our code analyzers are equipped with powerful path sensitive dataflow engines
to detect tricky issues such as null-pointers dereferences, logic errors, resource
leaks..
Centralize Quality
One place to provide a shared vision of code quality for developers, tech leads,
managers and executives in charge of a few to a few thousands projects and
also to act as a toll gate for application promotion or release
QA : Exemple SonarQube
sources : https://www.sonarqube.org/

(patch management)
2019/04/17
#DevoxxFR
Jenkins stage
stage("Sonar Analysis") {
withSonarQubeEnv('sonarqube') {
sh "docker build -f sonar.Dockerfile
--build-arg BASE_ID=${BUILD_TAG}
--build-arg REVISION=${revision}
--build-arg SONAR_MAVEN_GOAL=${SONAR_MAVEN_GOAL}
--build-arg SONAR_HOST_URL=${SONAR_HOST_URL}
--build-arg SONAR_AUTH_TOKEN=${SONAR_AUTH_TOKEN} ."
}
}
Jenkinsfile
ARG BASE_ID
FROM builder:${BASE_ID}
ARG REVISION
ARG SONAR_MAVEN_GOAL
ARG SONAR_HOST_URL
ARG SONAR_AUTH_TOKEN
ARG MVN_EXTRA_ARGS=""
RUNG mvn --batch-mode ${SONAR_MAVEN_GOAL}
-Dsonar.host.url=${SONAR_HOST_URL}
-Dsonar.login=$SONAR_AUTH_TOKEN -Drevision=${REVISION}
${MVN_EXTRA_ARGS}
sonar.Dockerfile

(patch management)
2019/04/17
#DevoxxFR
Some of ZAP's functionality:
● Man-in-the-middle Proxy
● Traditional and AJAX spiders
● Automated scanner
● Passive scanner
● Forced browsing
● Fuzzer
● Dynamic SSL certificates
● Smartcard and Client Digital Certificates support
● Web sockets support
● Support for a wide range of scripting languages
● Plug-n-Hack support
● Authentication and session support
● Powerful REST based API
● Automatic updating option
● Integrated and growing marketplace of add-ons
QA : Example ZappProxy
sources :
https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project#tab=Functionality

(patch management)
2019/04/17
#DevoxxFR
Continuous Integration
the practice of frequently integrating one's new or changed code with the existing code repository – should occur frequently enough that
no intervening window remains between commit and build, and such that no errors can arise without developers noticing them and
correcting them immediately.
Notice : Maturity level
sources : DevOps with OpenShift by Stefano Picozzi, Mike Hepburn, and Noel O’Connor
https://en.wikipedia.org/wiki/Continuous_delivery, https://en.wikipedia.org/wiki/Continuous_deployment, https://en.wikipedia.org/wiki/Continuous_integration
Continuous delivery (CDE)
A software engineering approach in which teams
produce software in short cycles, ensuring that the
software can be reliably released at any time and,
when releasing the software, doing so manually.
Continuous deployment (CD)
A software engineering approach in which software
functionalities are delivered frequently through
automated deployments.
- 70:00

(patch management)
2019/04/17
#DevoxxFR
DEV
(NS)
Digression: version tag versus purpose tag
img:1.2.1
QA
(NS)
STG
(NS)
PROD
(NS)
img:1.2.0 img:1.1.1 img:1.1.1
DEV
(NS)
img:dev
QA
(NS)
STG
(NS)
PROD
(NS)
img:qa img:stg img:prod
imagePullPolicy: Always

(patch management)
2019/04/17
#DevoxxFR
sources : https://devops.com.vn/2018/117/
Application lifecycle (example)

(patch management)
2019/04/17
#DevoxxFR
sources : https://www.lemondeinformatique.fr/actualites/lire-atlassian-fait-pousser-ses-offres-devops-68588.html

(patch management)
2019/04/17
#DevoxxFR
CVE happens
sources : https://www.cvedetails.com/

(patch management)
2019/04/17
#DevoxxFR
3 years Lifecycle of a website application (example)
Patch

(patch management)
2019/04/17
#DevoxxFR
Application lifecycle
Feature developpement through application configuration or coding is only a phase of the Application life cycle. This phase
can be a big or a small part of the application life.
Several phase are also necessary within the application lifecycle :
- Feature development with testing and validation
This phase can happen several time during the application life.
- Deployment
- Security update
- Decommission
This phase should only happen once.
The manpower necessary for any of theses phases vary a lot.
Depending of the complexity of the application, the disponibility of
the knowledgeable parties, the level of automatisation
This is why an application need to have a application owner
accountable over the entire application lifecycle.

(patch management)
2019/04/17
#DevoxxFR
Scan tools: downstream / upstream
Security tools can give you inside knowledge of your image and
potential security vulnerability
Most are hook on you registry or can have local agent that will
highlight most of you CVE issue.
Usually they scan package (.rpm, .deb, .apk,...) but not tar.gz
and mostly not application library.
They can have various trigger and can set various trigger
- email alert
- automatic rebuild
- block pull
- ...

(patch management)
2019/04/17
#DevoxxFR
Docker
Image
Docker Image Docker
Image
Docker
Image
Breadcump
Infrastructure
Toolings / Utils
Language Toolings
Application code
Alpine Base Image
OpenJDK 8
Tomcat Wildfly
App lib
App 2
App 3
Apache httpd 2.4
PHP 7.1 PHP 5.6
Drupal Wordpress
App 1 App 4
Spring|
boot
g/.../App1:x.y
Docker
Image
kubelet
Detected by most vulnerability
scanner
Often out of scope
- 30:00

(patch management)
2019/04/17
#DevoxxFR
Scan tools
mvn --batch-mode org.owasp:dependency-check-maven:aggregate
https://plugins.jenkins.io/dependency-track

(patch management)
2019/04/17
#DevoxxFR
Application lifecycle : PHP
PHP Component
● httpd
● nginx
● composer
● symphony
● drupal
● wordpress
● PhpUnit

(patch management)
2019/04/17
#DevoxxFR
Application lifecycle : Java
Java
● openjdk
● oraclejdk
● azul
●
● ant
● maven
● graddle
● spring boot
● vert-x
● war+tomat
● war+tomEE
● ear+tomEE
● ...

(patch management)
2019/04/17
#DevoxxFR
Application lifecycle : nodejs
Nodejs
● v8
● deno
● npm
● webpack
● bower
● espress.js
● mongoose
● lodash
● Jest
● phatomejs
● chomium
● ...

(patch management)
2019/04/17
#DevoxxFR
Application lifecycle : Scala
Sacla
● sbt
● …
● Play
● Sparks
● ...

(patch management)
2019/04/17
#DevoxxFR
It can rapidly became unmanageable

(patch management)
2019/04/17
#DevoxxFR
Docker
Image
Docker Image Docker
Image
Docker
Image
Breadcrumb
Infrastructure
Toolings / Utils
Language Toolings
Application code
Alpine Base Image
OpenJDK 8
Tomcat Wildfly
App lib
App 2
App 3
Apache httpd 2.4
PHP 7.1 PHP 5.6
Drupal Wordpress
App 1 App 4
Spring|
boot
g/.../App1:x.y
Docker
Image
kubelet
Usually ops scope
Usually Dev scop

(patch management)
2019/04/17
#DevoxxFR
Base image : A transition image destined to process program
code in order to create an application image.
Application image : An image use into kubernetes that
provide the custom service developed by the company. It can
came from a base image or a custom dockerfile.
Proposed vocabulary
Still image : An image used into kubernetes cluster with only
configuration adaptation. A still image is based on non-internal
source code or binaries.

(patch management)
2019/04/17
#DevoxxFR
Docker Image Docker
Image
Docker
Image
Breadcrumb
Infrastructure
Toolings / Utils
Language Toolings
Application code
Alpine Base Image
OpenJDK 8
Tomcat Wildfly
App lib
App 2
App 3
Apache httpd 2.4
PHP 7.1 PHP 5.6
Drupal Wordpress
App 1 App 4
Spring|
boot
g/.../App1:x.y
BaseImage
ApplicationImage
StillImage

(patch management)
2019/04/17
#DevoxxFR
1,2,3 hosting
1 : Project will use Dockerfile using our base image & still
image proposed and maintain by the host team (ops)

(patch management)
2019/04/17
#DevoxxFR
1,2,3 hosting
2 : Project will use its own Dockerfile but will be in charge of
maintenance (specially security)

(patch management)
2019/04/17
#DevoxxFR
1,2,3 hosting
2 : Project will use its own Dockerfile but will be in charge of
maintenance (specially security)
3 : Project will use third parties image (signed image)

(patch management)
2019/04/17
#DevoxxFR
Pitfalls
- 10:00

(patch management)
2019/04/17
#DevoxxFR
Pitfalls
● Absence of a project Owner

(patch management)
2019/04/17
#DevoxxFR
Pitfalls
● Tooling focus / automation freak

(patch management)
2019/04/17
#DevoxxFR
Pitfalls
● Lack of communication (specially between dev team and hosting team)

(patch management)
2019/04/17
#DevoxxFR
Pitfalls
● Build to Prod (no test)

(patch management)
2019/04/17
#DevoxxFR
Pitfalls
● Too many version, too many variante

(patch management)
2019/04/17
#DevoxxFR
Pitfalls
● Too many version, too many variante
● Lack of management support / acceptance in rules enforcement

(patch management)
2019/04/17
#DevoxxFR
Thank you
● Questions ?
- 00:00

(patch management)
2019/04/17
#DevoxxFR
kubernetes : Misc
Security check
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-hunter/master/job.yaml
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/master/job-node.yaml
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/master/job-master.yaml
sources : https://kubehunter.aquasec.com/
https://github.com/aquasecurity/kube-bench

(patch management)
2019/04/17
#DevoxxFR
kubernetes : Misc
Security check
sources : https://github.com/nicolaka/netshoot
https://github.com/aquasecurity/kube-bench

Dev opsec dockerimage_patch_n_lifecyclemanagement_2019

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Dev opsec dockerimage_patch_n_lifecyclemanagement_2019

Similar to Dev opsec dockerimage_patch_n_lifecyclemanagement_2019 (20)

More from kanedafromparis

More from kanedafromparis (7)

Recently uploaded

Recently uploaded (20)

Dev opsec dockerimage_patch_n_lifecyclemanagement_2019