1. Building the backbone of global trade,
to make shipping products as easy as sending emails.
Schlomo Schapiro, 18.11.2021, Continuous Lifecycle 2021
The Role of
GitOps in IT
Strategy

2. The Role of GitOps
In IT Strategy
The GitOps Journey to
Hands-Off Operations
18.11.2021 | Schlomo Schapiro | Principal Engineer, Forto GmbH
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0
International License (with the exception of the stock images with copyright notice) All Mountain Photos: Schlomo Schapiro / CC-BY-SA
@schlomoschapiro

3. Agenda
3
Problem
DevOps
Evolution
Automated
Governance
❤ GitOps
Hands-Off
Operations
Cost of
Compliance
Acceptable Means
of Compliance
Declarative
Descriptions

4. 4
DevOps

5. 5
Happy DevOps Campers
Engineering
Teams
git
?
CI CD

6. … if every person uses the same tool for the same job
… codified knowledge - everybody contributes his part to common automation
… if all people have the same privileges in their tooling
… if human error is equally possible for Dev and Ops
… replacing people interfaces by automated decisions and processes
... a result
DevOps is
6
bit.ly/5devops

7. 7
The Problem

8. We want digitalisation,
our IT Strategy calls for …
● IT quota grows exponentially,
no problem can be solved without IT
● All IT processes are much more integrated and
networked, API first …
● Growth factor of IT much bigger than increase in IT
staff → IT “production efficiency” must increase
● More IT in business units → decentralisation of IT
skills (BizDevOps)
● Increasing IT compliance requirements
● Utilise public cloud offerings to drive innovation –
have viable cloud exit strategy
8

9. As an IT team we want …
● Deliver great product/service
● Focus on our product/service
● Use good tools & platforms
● Know which internal processes to deal with
● Reduce overhead with internal processes
● Comply with company policies without pains
● Know about relevant company policies
● Use standard solutions for common problems
● No dependencies to other teams
9

10. Solution Approach
10
Organisational Frameworks
Technology
Frameworks
Fix the
Basics
Acceptable Means
of Compliance
Policy
as Code
Budgets for
Compliance
Standardized
Tooling
Hands-Off
Operations

11. Acceptable Means
of Compliance
Standardized
Tooling
Fix the
Basics
Policy
as Code
Solution Approach
Goal: Hands-Off Operations
11
Organisational Frameworks
Technology
Frameworks
Budgets for
Compliance
Hands-Off
Operations

12. 12
Automated
Governance

13. 13
● Problem?
● What is governance?
➢ Align IT strategy with business strategy
➢ Make sure we have and keep rules
Governance

14. 14
● What is automated?
➢ „operated automatically“
➢ Synonyms: automatic, laborsaving, robotic,
self-acting, self-operating, self-regulating
➢ Not people doing it manually
Automated
Source: https://www.merriam-webster.com/dictionary/automated

15. Automated Governance
= Compliance Automation = Very Hard!
15
Automation
friendly?
How to
check?

16. 16
GitOps to the Rescue
Engineering
Teams
git
?
CI CD

17. 17
Declarative
Descriptions

18. 18
git
?
CI CD
GitOps to the Rescue
WHAT HOW
Declarative
Descriptions
Deployment
Automation
Test for Compliance Test for Correctness
Product
Teams
Platform
Teams

19. Test for Compliance
Declarative Descriptions Example
19
stage_deploy:
script:
- ssh user@host "mkdir htdocs/_tmp"
- scp -r build/* user@host:htdocs/_tmp
- ssh user@host "mv htdocs/live htdocs/_old && mv htdocs/_tmp htdocs/live"
- ssh user@host "rm -rf htdocs/_old"
gitlab-ci.yaml
stage_deploy:
image: deploy_with_ssh
script: config.properties
gitlab-ci.yaml
#!/bin/bash
source "$1"
ssh $TARGET "mkdir $DIR/_tmp"
scp -r $SRC/* "$TARGET:$DIR/_tmp"
ssh $TARGET "mv $DIR/$NAME $DIR/_old && mv $DIR/_tmp $DIR/$NAME"
ssh $TARGET "rm -rf $DIR/_old"
Docker Image deploy_with_ssh ENTRYPOINT
TARGET=user@host
SRC=build
DIR=htdocs
NAME=live
config.properties
Test for Correctness
Source: https://docs.gitlab.com/ee/ci/examples/deployment/composer-npm-deploy.html

20. Test for Compliance
Declarative Descriptions Example
20
stage_deploy:
script:
- ssh user@host "mkdir htdocs/_tmp"
- scp -r build/* user@host:htdocs/_tmp
- ssh user@host "mv htdocs/live htdocs/_old && mv htdocs/_tmp htdocs/live"
- ssh user@host "rm -rf htdocs/_old"
gitlab-ci.yaml
stage_deploy:
image: deploy_with_ssh
script: config.properties
gitlab-ci.yaml
#!/bin/bash
source "$1"
ssh $TARGET "mkdir $DIR/_tmp"
scp -r $SRC/* "$TARGET:$DIR/_tmp"
ssh $TARGET "mv $DIR/$NAME $DIR/_old && mv $DIR/_tmp $DIR/$NAME"
ssh $TARGET "rm -rf $DIR/_old"
Docker Image deploy_with_ssh ENTRYPOINT
TARGET=user@host
SRC=build
DIR=htdocs
NAME=live
config.properties
Test for Correctness
Source: https://docs.gitlab.com/ee/ci/examples/deployment/composer-npm-deploy.html
C
o
d
e
(
H
o
w
)
Config
(W
hat)

21. Declarative Descriptions → Automated Governance
21
Config Tools
Cloud Formation aws cf
create
Kubernetes Manifest kubectl apply
Swagger YAML
Terraform YAML
…
AndroidManifest.xml
…
Test Strategy
Static Code Analysis
Linting
Integration Tests
Unit Tests

22. Declarative Descriptions → Automated Governance
22
Config
Cloud Formation
Kubernetes Manifest
Swagger YAML
Terraform YAML
…
AndroidManifest.xml
Compliance Check
cfn-nag: Linting tool for CloudFormation templates
K8S Admission Controller / OPA Gatekeeper
zally: A minimalistic, simple-to-use API linter
terraform-compliance.com, checkov.io …
. . .
Tools
aws cf create
kubectl apply
…
?
CI CD
Compliant!
Automated Compliance Checks
as Quality Gate for Deployments

23. 23
GitOps

24. 24
GitOps Tech
Engineering
Teams
git
?
CI CD

25. GitOps Tech
25
git push
State Repo
develop
Binary
Artifacts
CI
Test &
Build
State Repo
main
git push
Version 27
Merge
Request
Modify
Monitor
❶
❷ ❸
❹
❺
❺
GitOps
Controller
People Area Machine Area
Infrastructure
Environment
Systems
➏

26. 26
CI/CD
git push trigger deploy
1. CI Ops
Engineers
Git
Ops
git push trigger
deploy
delete
2. Simple GitOps
Engineers
Git
Ops
git push
monitor
git pull
deploy
delete
monitor
3. GitOps Controller
(git push)
Engineers
GitOps Controller

27. 27

28. GitOps & Compliance Automation
28
● GitOps Operations Model
provides ideal interface for
compliance automation:
A central place where
every change passes by in
a machine-readable format.
● GitOps enables true hands-off
operations, reducing IT costs
& efforts.
● Motivation to “Fix the Basics”.
Compliance
Checks

29. 29
Cost of
Compliance

30. Cost & Effort of Compliance Checking?
30
Policy 1 Policy N
…
Policy 1 Policy N
…
500+
Teams
Central
“IT Compliance”
Team
git
?
CI CD
git
?
CI CD

31. Policy 1 Policy N
…
git
?
CI CD
Policy on Paper
31
Policy on Paper (low cost)
No coordination between policies required
Every Engineering Team interprets policies anew
Every Engineering Team implements own policy checking
Distributed Cost of Compliance Checking
Linear costs scale with number of teams and
number and complexity of policies

32. Policy 1 Policy N
…
Central
“IT Compliance”
Team
git
?
CI CD
Policy as Code – Compliance
Automation
32
IT Compliance Team converts policies to code
Centralized Cost of Compliance Checking
Feedback cycle policy & code
Cost scale with number and complexity of
policies, not with number of teams
Every Engineering Team uses common policy checking
code as acceptable means of compliance

33. Platform & Compliance Engineering
33
git
?
CI CD
HOW
Deployment
Automation
Test for Correctness
Platform
Teams
Central
“IT Compliance”
Team

34. Platform & Compliance Engineering Org
34
Mission:
Compliant-by-Default IT platforms
● Create & maintain standardized
tooling for common IT tasks
● Tools are user friendly, integrate
automated compliance checks
● Educate & coach teams in
tool usage & best practices
● Cost center
● Main KPIs:
○ Productivity of product
engineering teams
○ Balancing IT compliance
risks and costs
Platform
Teams
Central
“IT Compliance”
Team
Organisational Frameworks
Technology
Frameworks
…

35. 35
Acceptable
Means of
Compliance

36. Reality Check – Food Court Example
36
All images: pixabay.com no attribution required
1. Healthy ?
2. Low Carb ?
3. Organic ?
4. Kosher ?
5. Legal ?

37. Reality Check – Product certification
37
1. Healthy ✔
2. Low Carb ✔
3. Organic ✔
4. Kosher ✔
5. Legal ✔
1. Healthy
2. Low Carb
3. Organic ✔
4. Kosher ✔
5. Legal ✔
1. Healthy ✔
2. Low Carb
3. Organic
4. Kosher
5. Legal
1. Healthy ✔
2. Low Carb ✔
3. Organic ✔
4. Kosher ✔
5. Legal ✔
1. Healthy ?
2. Low Carb ?
3. Organic ?
4. Kosher ?
5. Legal ?

38. Reality Check – Product certification
38
1. Healthy ✔
2. Low Carb ✔
3. Organic ✔
4. Kosher ✔
5. Legal ✔
1. Healthy
2. Low Carb
3. Organic ✔
4. Kosher ✔
5. Legal ✔
1. Healthy ✔
2. Low Carb
3. Organic
4. Kosher
5. Legal
1. Healthy ✔
2. Low Carb ✔
3. Organic ✔
4. Kosher ✔
5. Legal ✔
1. Healthy ?
2. Low Carb ?
3. Organic ?
4. Kosher ?
5. Legal ?

39. Toolchain Certification
39
Engineering
Teams
git
Policy 1 … N
Teams using
unmodified Toolchain
are certified to be
compliant with Policy
without further proof
Platform
Teams
Internal Toolchain Product
„Compliant-by-Default“
Provide

40. Certified Toolchains
40
1. Policy 1 ✔
2. Policy 2 ✔
3. Policy 3 ✔
4. Policy 4 ✔
5. … ✔
Product
Teams
1. Policy 1 ✔
2. Policy 2
3. Policy 3 ✔
4. Policy 4 ✔
5. …
1. Policy 1
2. Policy 2
3. Policy 3
4. Policy 4 ✔
5. …
1. Policy 1 ✔
2. Policy 2 ✔
3. Policy 3 ✔
4. Policy 4 ✔
5. … ✔
1. Policy 1 ?
2. Policy 2 ?
3. Policy 3 ?
4. Policy 4 ?
5. … ?
Platform
Team
Platform
Team
Platform
Team
Platform
Team

41. Certified Toolchains – Self-regulating
Market
41
1. Policy 1 ✔
2. Policy 2 ✔
3. Policy 3 ✔
4. Policy 4 ✔
5. … ✔
1. Policy 1 ✔
2. Policy 2
3. Policy 3 ✔
4. Policy 4 ✔
5. …
1. Policy 1
2. Policy 2
3. Policy 3
4. Policy 4 ✔
5. …
1. Policy 1 ✔
2. Policy 2 ✔
3. Policy 3 ✔
4. Policy 4 ✔
5. … ✔
1. Policy 1 ?
2. Policy 2 ?
3. Policy 3 ?
4. Policy 4 ?
5. … ?
Platform
Team
Platform
Team
Platform
Team
Platform
Team
Product
Teams

42. Acceptable Means of Compliance –
Everybody Wins!
42
● Certify tool implementations for common IT topics
around Dev & Ops to provide a compliant-by-default
usage scenario for common tasks
● Provide funding to implement compliance checks
● Ensure every policy has at least one certified
implementation (reference implementation)
● Write better policies that can be easily implemented
● Gain visibility into policy adherence
● Intrinsic motivation to prefer compliant-by-default
tools to reduce own cost of compliance
● Automated proof of compliance possible
● Focus on product development
● Can use compliance adherence to promote their tools
● Receive additional funding for implementing
non-functional requirements in tools
● Implement IT compliance together with new
functionality
● Better & central visibility for cost of compliance
● Global optimisation of compliance costs
● Global optimisation of tool costs
● Increased IT efficiency through intrinsic motivation
● Automated company-wide compliance reports
● Risk management can be based on technical KPIs
● Actual measurement of IT compliance
● Scale-out company growth with increased IT
compliance
Results:
➢ Organisational & Technology Framework
➢ More fun and happiness in IT
➢ Better IT effectiveness
Product Teams
Platform Teams
Product Engineering Teams
The Company

43. 43
IT Strategy

44. DevOps
Ops
Automation
IT Evolution Big Picture
44
Technology
Culture
Dev ⇔ Ops
CI-Ops
1
2
GitOps
Hands-Off
Operations

45. Hands-Off Operations
45
● No manual changes in production
● Dev & Ops have same permissions in
production: None by Default
● Automate the hard stuff:
○ Compliance & governance
○ Distributed rolling upgrades
○ Backup & Disaster Recovery
○ Everything in your stack
● Test Driven Everything
● Standardized Tooling
● Fix the Basics!
GitOps

46. Why GitOps?
46
Hands-Off
Operations
Impossible!
GitOps
Yes,please!
When do
we start?

47. The Role of GitOps in IT Strategy
47
Adopting GitOps practices drives
automation as the solution for
many IT strategy requirements.
● schlomo.schapiro.org/2021/04/the-gitops-journey.html read more about this
● schlomo.schapiro.org/p/5-devops-principles.html my DevOps definition
● forto.com/career join our vision:
"We are building the backbone of global trade, to make shipping goods as easy as sending emails."
Q&A
@schlomoschapiro schlomo.schapiro.org