Vic Iglesias
Solutions Architect
@vicnastea
Environment
Management in a
Continuously
Delivered World
Who am I?
● Solutions Architect
● Focus Areas
○ Enterprise Automation
○ Continuous Delivery
● Technology expertise
○ Kubernetes
○ Jenkins
○ Spinnaker
*
* No longer a smoker
Environment
Sprawl
Development
Developer
Laptop w/ Skaffold
Shared Kubernetes
cluster
Development Resource Configs
K8s Manifests
Staging
Developer 1
Internal-only
Kubernetes cluster
Developer 3
Developer 2
Changes for Staging
Development Resource Configs
K8s Manifests
CI/CD System
Canary + Production
Canary Cluster
Production Cluster
Delta for Canary
Development Resource Configs
K8s Manifests
Delta for Production
Code reuse in
Kubernetes
Manifests
Overlays for filesystems
ubuntu:16.04
c22013c84729
d74508fb6632
91e54dfb1179
d3a1f33e8a5a
194.5 KB
1.895 KB
0 B
188.1 MB
Container
layer
Thin R/W layer
Base Image
layers (R/O)
Overlays for Kubernetes Manifests
Security overlay
Production Overlay
Base Manifests
2O LOC
40 LOC
500 LOC
Merge in
overrides at each
layer
Kustomize
github.com/kubernetes-sigs/kustomize
Additional Features
● Uses 3-way merge to apply changes
to K8s resources
● Apply common:
○ Labels
○ Annotations
● Add a prefix to all resource names
● Inject common config maps &
secrets
Customize raw YAML files
for multiple purposes,
leaving the original YAML
untouched and usable as
is.
Kustomize
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: busybox
command: ['app']
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
spec:
containers:
- name: myapp
command: ['app', '--dev']
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: busybox
command: ['app', '--dev']
base.yaml patch.yaml output.yaml
Kustomize
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp
image: busybox
command: ['app']
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
spec:
containers:
- name: myapp
command: ['app', '--dev']
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp
image: busybox
command: ['app', '--dev']
base.yaml patch.yaml output.yaml
dev/kustomization.yaml
namespace: vic-dev
commonLabels:
owner: vic
bases:
- ../base/
patches:
- patch.yaml
configMapGenerator:
- name: application-properties
files:
- properties
~/myapp
├── base/
└── deployment.yaml
└── service.yaml
└── kustomization.yaml
├── dev/
└── patch.yaml
└── properties
└── kustomization.yaml
kustomize build dev/ | kubectl apply -f -
/me wipes brow
Now we have an easy way
to define our dev
manifests.
How do we iterate on our
Kubernetes app during
development?
Skaffold
Skaffold
● Open source project by Google
● Cluster agnostic - develop on local clusters or remote
clusters on any cloud
● Easy to use, but also features for power users
● git clone && skaffold run
● Pluggable architecture
● Run anywhere - local development or CI pipeline
Pluggable
Architecture
Skaffold Builders
● Docker
● Bazel
● In-Cluster (Kaniko)
● Google Cloud Builder
Skaffold Taggers
● Checksum of image
● Git metadata (commit SHA/tag)
● Timedate
● Static
Skaffold Deployers
● Kubectl
● Helm
● Kustomize
skaffold dev
Watch source
code
dependencies of
images
Build image
[push if remote]
Tag
Update
deployments
and apply
metadata labels
Deploy
1 2 3 4 5 6
Stream logs
from deployed
artifacts
Verify
skaffold build
Build image
[push if remote]
Tag
1
skaffold deploy
Update
deployments
and apply
metadata labels
Deploy
2 3
skaffold run
Build image
[push if remote]
Tag
Update
deployments
and apply
metadata labels
Deploy
1 2 3
Now we can iterate easily
during development.
How do we deploy to
other environments?
Spinnaker
Spinnaker
an open source,
multi-cloud
continuous delivery
platform for releasing
software changes...
Spinnaker Kubernetes Integration
Kubernetes
Manifests
Spinnaker Kubernetes Integration
Cloud
Storage
Images
Spinnaker Pipeline
Spinnaker Pipeline
Demo
Staging and Production EnvsDeveloper Env
End-to-end Flow
Container Builder
Cloud
Storage
Container
Registry
Skaffold Spinnaker
Staging Cluster
Production Cluster
Minikube
Detect
Commit
Push
Image
Kustomize
Manifests
Detect New
Image
Pull in
Latest manifest
Developer
Thank you!
Twitter: @vicnastea
Email: viglesias@google.com

Environment management in a continuous delivery world (3)