The document discusses migrating from a complex "jungle" of tools and technologies for continuous integration and continuous delivery (CI/CD) to a cloud-native approach. It describes some of the challenges of the jungle model, including decision waste, integration pains, and lack of common standards. The document then outlines key aspects of a cloud-native CI/CD model, including using Kubernetes as the integration platform, dynamic agents and environments, and decoupling delivery from release. It provides examples of implementing such a model using shared libraries, Jenkinsfiles, containerized agents, and standardized pipelines and workflows.

1. @antweiss
Escaping the
Jungle
Or Migrating to Cloud-
Native CI/CD
Anton Weiss, Otomato Software
@antweiss, @otomato_sw
https://otomato.io
https://devopstrain.pro

2. @antweiss
Jungle

3. @antweiss
Jungle

4. @antweiss
Jungle

5. @antweiss
The Adventure Begins!

6. @antweiss
The Adventure Begins!
● Motivation!

7. @antweiss
The Adventure Begins!
● Motivation!
● Autonomy!

8. @antweiss
The Adventure Begins!
● Motivation!
● Autonomy!
● Inherent Agility!

9. @antweiss
The Adventure Begins!
● Motivation!
● Autonomy!
● Inherent Agility!
● Success!

10. @antweiss
The Adventure Begins!
● Motivation!
● Autonomy!
● Inherent Agility!
● Success!
● Explosive Growth!

11. @antweiss
The Adventure Begins!
● Motivation!
● Autonomy!
● Inherent Agility!
● Success!
● Explosive Growth!
● Everything Happens So Fast!

12. @antweiss
Welcome to the Jungle!
- All the Languages
- All the Frameworks
- All the DataBases
- All the Message Queues
- Dozens of Cloud Services
- 100s of CI Jobs
- Technical Debt All Around

13. @antweiss
Jungle Reality
● (In)Decision Waste

14. @antweiss
Jungle Reality
● (In)Decision Waste
● Integration Pains

15. @antweiss
Jungle Reality
● (In)Decision Waste
● Integration Pains
● Onboarding is Hard

16. @antweiss
Jungle Reality
● (In)Decision Waste
● Integration Pains
● Onboarding is Hard
● No common language

17. @antweiss
The Hard Choices:

18. @antweiss
The Hard Choices:
Quality
Supportability
Clarity

19. @antweiss
The Hard Choices:
Velocity
Autonomy
Creativity
Quality
Supportability
Clarity

20. @antweiss
The Golden Path
Charity Majors: Software Sprawl and the Golden Path@mipsytipsy

21. @antweiss
The Golden Path
How to:
● Build
● Test
● Package
● Version
● Configure
● Deploy
● Log
● Instrument
● Monitor
● Provision
● Upgrade
● Patches
● Security Fixes
Charity Majors: Software Sprawl and the Golden Path

22. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?

23. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?
Build: Maven vs. Gradle, Npm vs. Yarn, Cmake vs. Conan..

24. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?
Build: Maven vs. Gradle, Npm vs. Yarn, Cmake vs. Conan..
Packaging: Jars, modules, libs? Docker? Helm?

25. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?
Build: Maven vs. Gradle, Npm vs. Yarn, Cmake vs. Conan..
Packaging: Jars, modules, libs? Docker? Helm?
Binary Repo: Dockerhub? Npm.js? Artifactory? Nexus?

26. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?
Build: Maven vs. Gradle, Npm vs. Yarn, Cmake vs. Conan..
Packaging: Jars, modules, libs? Docker? Helm?
Binary Repo: Dockerhub? Npm.js? Artifactory? Nexus?
CI/CD: Jenkins Pipelines? GitLab CI? Codefresh? Github
Actions?

27. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?
Build: Maven vs. Gradle, Npm vs. Yarn, Cmake vs. Conan..
Packaging: Jars, modules, libs? Docker? Helm?
Binary Repo: Dockerhub? Npm.js? Artifactory? Nexus?
CI/CD: Jenkins Pipelines? GitLab CI? Codefresh? Github
Actions?
Platform: Kubernetes? Lambda? Fargate/Cloud Run?

28. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?
Build: Maven vs. Gradle, Npm vs. Yarn, Cmake vs. Conan..
Packaging: Jars, modules, libs? Docker? Helm?
Binary Repo: Dockerhub? Npm.js? Artifactory? Nexus?
CI/CD: Jenkins Pipelines? GitLab CI? Codefresh? Github
Actions?
Platform: Kubernetes? Lambda? Fargate/Cloud Run?
Configuration/Secrets: ?

29. @antweiss
The Delivery Toolkit
SCM: Github? Gitlab? BitBucket?
Build: Maven vs. Gradle, Npm vs. Yarn, Cmake vs. Conan..
Packaging: Jars, modules, libs? Docker? Helm?
Binary Repo: Dockerhub? Npm.js? Artifactory? Nexus?
CI/CD: Jenkins Pipelines? GitLab CI? Codefresh? Github
Actions?
Platform: Kubernetes? Lambda? Fargate/Cloud Run?
Configuration/Secrets: ?
Service Mesh: ?

30. @antweiss
The Pipeline

31. @antweiss
Cloud-Native CI/CD???
?

32. @antweiss
Cloud-Native CI/CD
● Integrated with the Platform (k8s)

33. @antweiss
Cloud-Native CI/CD
● Integrated with the Platform (k8s)
● Dynamic Agents and Environments

34. @antweiss
Cloud-Native CI/CD
● Integrated with the Platform (k8s)
● Dynamic Agents and Environments
● Decouple Delivery from Release

35. @antweiss
Standard Pipeline API
Pipeline
Agent
Definition
Flow Definition
Model
Definition
Shared Library

36. @antweiss
Jenkinsfile
In Jenkinsfile:
@Library('our-lib') _
def svcName= currentBuild.rawBuild.project.parent.displayName
def tag = "${BRANCH_NAME}${BUILD_NUMBER}"
def pod = libraryResource ‘io/otomato/data-agent.yaml'
def d = new io.otomato.data()
timestamps {
dataSvcPipeline(d, pod, svcName, tag)
}

37. @antweiss
Jenkinsfile
In Jenkinsfile:
@Library('our-lib') _
def svcName= currentBuild.rawBuild.project.parent.displayName
def tag = "${BRANCH_NAME}${BUILD_NUMBER}"
def pod = libraryResource ‘io/otomato/data-agent.yaml' <-Agent
def d = new io.otomato.data()
timestamps {
dataSvcPipeline(d, pod, svcName, tag)
}

38. @antweiss
Jenkinsfile
In Jenkinsfile:
@Library('our-lib') _
def svcName= currentBuild.rawBuild.project.parent.displayName
def tag = "${BRANCH_NAME}${BUILD_NUMBER}"
def pod = libraryResource ‘io/otomato/data-agent.yaml' <-Agent
def d = new io.otomato.data() <-Model
timestamps {
dataSvcPipeline(d, pod, svcName, tag)
}

39. @antweiss
Jenkinsfile
In Jenkinsfile:
@Library('our-lib') _
def svcName= currentBuild.rawBuild.project.parent.displayName
def tag = "${BRANCH_NAME}${BUILD_NUMBER}"
def pod = libraryResource ‘io/otomato/data-agent.yaml' <-Agent
def d = new io.otomato.data() <-Model
timestamps {
dataSvcPipeline(d, pod, svcName, tag) <-Flow
}

40. @antweiss
Agent
io/otomato/data-agent.yaml:
apiVersion: v1
kind: Pod
metadata:
labels:
product: data
spec:
serviceAccountName: jenkins
containers:
- name: maven
...
- name: helm
imagePullSecrets:
- name: regcred
volumes:
...
Defines:
● Containers
● Secrets
● Persistent Volumes

41. @antweiss
Model Definition
io.otomato.data:
def build(svcName, tag) {
container('maven'){
withEnv(["service_name=${svcName}", "environment=local"]) {
sh "mvn clean package -s settings.xml -P ci"
}
}
}

42. @antweiss
Model Definition
io.otomato.data:
def intTest(svcName, tag) {
container('maven'){
withEnv(["service_name=${svcName}", "environment=local"]) {
sh "mvn integration-test -s settings.xml -P ci"
}
}
}

43. @antweiss
Model Definition
io.otomato.serving:
def build(svcName, tag) {
container('node'){
withEnv(["service_name=${svcName}", "environment=test"]) {
sh "npm i --only=prod”
}
}
}

44. @antweiss
Flow Definition
dataSvcPipeline.groovy:
stage('build'){
steps {
script {
x.build(svcName, tag)
}
}
}

45. @antweiss
The Testing Pyramide?

46. @antweiss
The Testing Pyramide?

47. @antweiss
Helm Charts
● Support testing vs. production configuration
● Can be scaffolded
● Standard structure:
● templates/
├── deployment.yaml
├── db-configmap.yaml
├── ingress.yaml
├── monitoring-configmap.yaml
├── mq-configmap.yaml
└── service.yaml

48. @antweiss
The Vision of a Stress-free Release
Argo-Flux and the
GitOps Engine
SMI
Spec
The Meshes

49. @antweiss
Measurements & Standards:
● Measure:
○ Lead time
○ Deployment Rate
○ Mean Time to Restore
○ Change Fail Rate
● Integrate security scanning
● Define test coverage thresholds
● Define instrumentation requirements (types of metrics and
amount)

50. @antweiss
Jungle is Always Calling