The document discusses the implementation of GitOps practices to automate Kubernetes (K8s) releases and deployments using Git as a source of truth. It highlights the challenges of tracking changes in configuration, emphasizes best engineering practices, and outlines a CI/CD pipeline employing tools like Flux and Helm for version control and deployment management. Additionally, it touches upon the evolution of software architecture, including the shift from monolithic to microservices approaches.

1. if It hurts, do it more frequently

2. Nicola Baldi
naighes the journey to git-ops
how to automate much of the process of K8S
releases and deployments by using git as a
source of truth
nic.baldi@gmail.com
https://github.com/naighes
@nicolabaldi

3. development "on time, on budget, ..."
"best engineering practices" in place
a sad story

4. development "on time, on budget, ..."
"best engineering practices" in place
it never went into production actually :-(
a sad story

5. • Can I exactly reproduce any of my environments,
including the version of the operating system, its patch
level, the network configuration, the software stack, the
applications deployed into it, and their configuration?
• Can I easily make an incremental change to any of these
individual items and deploy the change to any, and all, of
my environments?

6. • Can I exactly reproduce any of my environments,
including the version of the operating system, its patch
level, the network configuration, the software stack, the
applications deployed into it, and their configuration?
• Can I easily make an incremental change to any of these
individual items and deploy the change to any, and all, of
my environments?
• Can I easily see each change that occurred to a particular
environment and trace it back to see exactly what the
change was, who made it, and when they made it?

7. canonical CI/CD pipeline
push trigger
SCM
build/test/lint/...
package
push
deploy
apply
cloneCI

8. deployments break and it's not clear that
a code change was responsible
config changes are also hard to track
enough of tons of bash!
fallacies

9. keep absolutely everything in version control
one reason that we use the
term version control in
preference to source control is
that version control isn't just for
source code; every single
artifact related to the creation
of your software should be
under version control

10. evolution
no-ops
devops
sysadmin
containers
virtualization
physical
microservices
SOA
monoliths
minutes
days
months
hours
IT culture Infrastructure
evolution
architecture release time
long time
ago…
2019

11. brings a set of opinionated and prescriptive practices
about applying git-based workflows, devops and
infrastructure as code in the context of kubernetes
git-ops

12. flux
push trigger
SCM
build/test/lint/...
package
push
cloneCI fluxd
sync
watch
HelmResource
apply
helm operator
deploymentdeploymentdeployment

13. git clone –mirror
# HEAD revision
git rev-list --max-count 1 heads/{branch}
# current revision (flux-sync is a git tag acting
# as a pointer)
git rev-list --max-count 1 flux-sync --
# retrieve changeset
changeset = repo.CommitsBetween(currentRev, headRev)
# load all manifests by walking the file system
# searching for resources (file content is inspected)
manifests = kresource.Load(baseDir)
# takes a definition of what should be running in
# the cluster and attempts to make the cluster conform
clus.Sync(manifests)
# retrieve all cluster resources
clustRes = c.getAllowedResourcesBySelector()
# for each resource, verify it exists into the
# cluster and put the change into a stage buffer
# (enrich with a couple of annotations (e.g. GC stuff)
resBytes = applyMetadata(clustRes)
kubectl apply
# Move the revision the sync state points to
ratchet.Update(currentRev, headRev)
git sync

14. git fetch origin refs/notes/flux:refs/notes/flux
From https://github.com/naighes/flux-get-started
* [new ref] refs/notes/flux -> refs/notes/flux
git notes --ref refs/notes/flux
1c060a70c46b7dd2d87451ce493afe2b0be76cd7
293baf09d3acec58492b69c9284879a77dc9d17d
ac01f9f47d8d724e33998388ff7153532d971aa2
40f78619092b43c4cf192795f54d87d6c7fb1869
git notes --ref refs/notes/flux show
293baf09d3acec58492b69c9284879a77dc9d17d
{
"jobID": "f1f49e8f-fb27-0a06-2a72-7ba7685dc641",
"spec": {
"type": "auto",
"spec": {
"Changes": [{
"WorkloadID": "demo:deployment/podinfo",
"Container": {
"Name": "podinfod",
"Image": "stefanprodan/podinfo:2.1.0"
},
"ImageID": "stefanprodan/podinfo:2.1.3"
}
]
}
},
"result": {
"demo:deployment/podinfo": {
"Status": "success",
"PerContainer": [{
"Container": "podinfod",
"Current": "stefanprodan/podinfo:2.1.0",
"Target": "stefanprodan/podinfo:2.1.3"
}
]
}
}
}
git notes as events

15. automatic releases
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
fluxcd.io/automated: "true"
fluxcd.io/tag.init: regex:^2.1.*
with the fluxcd.io annotations we instruct flux to automate this release
when a new tag with the MAJOR=2 and MINOR=1 is pushed to the docker registry,
flux will update the image field in the yaml file, will commit and push the change to
git and finally will apply the change on the cluster

16. a new tag is pushed

17. ├── charts
│ └── podinfo
│ ├── Chart.yaml
│ ├── README.md
│ ├── templates
│ └── values.yaml
├── namespaces
│ ├── dev.yaml
│ └── stg.yaml
└── releases
├── dev
│ └── podinfo.yaml
└── stg
└── podinfo.yaml
apiVersion: flux.weave.works/v1beta1
kind: HelmRelease
metadata:
name: podinfo-dev
namespace: dev
labels:
chart: podinfo
annotations:
flux.weave.works/automated: "true"
flux.weave.works/tag.chart-image: glob:dev-*
spec:
releaseName: podinfo-dev
chart:
git: git@github.com:stefanprodan/gitops-helm
path: charts/podinfo
ref: master
values:
image: stefanprodan/podinfo:dev-kb9lm91e
replicaCount: 1
helm operator

18. --sync-state
// GitTagStateMode is a mode of state management where Flux
// uses a git tag for managing Flux state
GitTagStateMode = "git"
// NativeStateMode is a mode of state management where Flux use
// native Kubernetes resources for managing Flux state
NativeStateMode = "secret"
Default is GitTagStateMode (and it does not work in --git-readonly mode)
In this case a --git-sync-tag acts like a pointer for the revision flux is positioned at
state management

19. apply all the manifests in the git repo and give each
resource a label marking it as having been synced
from this source
ask the cluster for all the resources marked as being
from this source and delete those that were not
applied in previous step
be careful when git URL or branch are changed, path
is added and path is removed (NOTE: it's
"conservative")
garbage collection
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
fluxcd.io/automated: "true"
fluxcd.io/sync-checksum:
f388cbf998956871ac7b070a285dae684c250ca8
fluxcd.io/tag.init: regex:^2.1.*
--sync-garbage-collection (default is false)

20. where is orchestration?

21. where is orchestration?
+

22. Q&A
thank you!