The document discusses the original promise of DevOps to bridge silos between development and operations teams and enable continuous delivery. It outlines the key tools and trends that emerged over the past 5 years, including Git, Docker, Kubernetes, GitHub Actions, Terraform, and the shift to treating infrastructure as code. The document also presents examples of CI/CD pipelines used by the author's team to provision AWS accounts and deploy environments for applications.

1. Promise of DevOps
24th meeting of Cloud Native – Kubernetes - ONLINE community
20-Oct-2021
miro.adamy@gmail.com

2. Agenda
● Original promise of DevOps
● 5 years later: what worked, what changed
● Tools and recommendations that survived
● Outlook - emerging themes

3. About me
Why I think I may have something to share on the topic / my bias
● 30+ years in IT (FEI-STU, SysAdmin, Developer, Architect, SRE, DevOps)
● C, Perl, C++, Java, C#, Python / Terraform, CFN, CDK
● Cloud - 2015+ (AWS)
● Enterprise level eCommerce: Oracle Commerce (ATG), SAP (Hybris)
● I live(d) on both sides of DevOps - Dev AND Ops:
○ design and build ecommerce site, release it, maintain / deploy / monitor / enhance / support
● Back in Bratislava after 25 years abroad (A, CH, Canada, ES)
● Currently: Cloud Platform team in Spanish subs of Canadian company
○ Cloud infrastructure building (TF / CFN / CDK)
○ DevOps - CI/CD
○ 100+ AWS accounts, 4 Orgs/SSO + ecosystem

4. Sincere Apologies to Slovak Language purists
● since 1991 I used almost exclusively German, English and Spanish for work
● I am sadly not familiar with the proper technical terminology in Slovak :-(
● This talk => mixture of English and Slovak
● The word DevOps == good example

7. Roots and triggers of DevOps
● Agile ~ 2001 : Ops Gap
● ~ 2008-2009 - first mention
● complexity of Apps
● Cloud - AWS (2006+)
● option to automate
● buzzword ~ 2010+
● containers / Docker ~ 2013+

8. DevOps motivation - and original promise
● to bridge the different mindset silos (Dev / QA / Ops)
● release any time
● automated builds and tests (CI)
● automated package and release (CD&D)
● shared ownership
● shared responsibility
● speed gain / cost savings
● cultural / org structure shift
=> tools, techniques, processes

12. Plan + Code
● Source control in 2021 ~== Git (Mercurial, SVN, proprietary
) in cloud
● Github, Bitbucket, Gitlab / CodeCommit
○ all go beyond the source code: CI/CD, issue management, PM
○ Github (Microsoft) - leader, OS, open collaboration, best UX for code review and navigation
○ Bitbucket (Atlassian) - JIRA integration, free for small teams < 5,
○ GitLab - on Prem, focus on PM, minority
○ CodeCommit - limited functionality - fully enclosed inside AWS (security, cost etc)
● Dev Environment in the cloud:
○ manage dev configuration
○ CodeSpaces - GH, Cloud9 - AWS
●

13. Which to pick
See the GH - GL - BB compare:
● team structure - org structure impact
○ IDP integration
○ management cost - API
When to consider AWS CodeCommit:
● security / access management inside AWS
● cost attribution - admin overhead
○ per project account(s)
● CI/CD deeply tied to AWS services

14. Building and packaging, artifact management
● where you build:
○ cloud - part of the Source Code Management (GH / BB / GL / AWS)
○ cloud - 3rd party service
○ onPrem - usually Jenkins
● what you build: Java, Native (Go, C/C++), PHP, Javascript
○ dedicated systems: Java: Gradle, Maven, Ant => JAR/WAR/EAR
● what triggers build:
○ manual - user action
○ source code event: commit, PR, merge, tag
● resulting artifact: in most cases => Docker Image
○ where it is stored: DockerHub, ECR, own registry
○ naming and tagging => super important
● As part of the build:
○ code quality; Sonarqube - rule based, security checks on libraries
○ unit tests

16. Tools (there are too many of them …)
● Jenkins:
○ oldest, well known, Groovy as pipeline language (maintenance cost) - Java app
○ self hosted or Jenkins as a Service
○ scalability / speed / infrastructure cost
○ plugin hell - complex, fragile
● 3rd party providers: Travis CI / CircleCI
○ important in 2016-2019, fighting for relevance and extending offer, monetization challenges
● GitLab:
○ great OOTB support for typical scenarios, largest coverage - PM + VCS + CI/CD
● BitBucket Pipelines:
○ JIRA integration (update ticket), Trello, Confluence
● GitHub Actions:
○ late to the game - 2020, modern, marketplace - https://github.com/marketplace?type=actions
and many many more

18. Bitbucket Pipelines and pipes

19. BB pipeline example

20. Bitbucket pipelines ...

21. Github Actions and Workflows

22. Gitlab - .gitlab-ci.yml
●

23. GH/BB/GL pipeline pros and cons
● serverless and portable (Docker), scalable (often), stateless
● tied up with repo => access to code, can build (<=> Jenkins or CCI)
● OOTB integrations, community + marketplace
● awkward debugging (commit, push, observe logs, repeat)
● Variables / Secrets - maintenance
● BB - to set the variables, MUST be Admin of a repo => override the rules
● YML - not great for conditional tasks / branching

24. Testing
● unit testing => part of the build
● integration testing = needs deployed app
● where to deploy ?
○ depends on application - EC2, ECS, K8s
○ deployment => configuration
○ infrastructure - pre-configured or on-demand
○ OCDE example
● ROBOT framework
● ecosystem specific tools

25. Release and Deploy
● what you deploy: container ? JAR ? package ? function ?
● where you deploy: On-Prem ? EC2 ? ECS ? K8s ?
● post-deployment configuration
● who owns the environment configuration ? passwords ?
● deployment - pre-requisite for integration/full testing
● monitoring, alerting, performance / security scans ⇒ SRE
Tools:
● part of CI/CD pipeline
● Cloud providers - Azure / Google / AWS
● 3rd party: too many to list. ? Who will survive ?

26. AWS DevOps services
● CodeCommit - Git in AWS
● CodePipeline - describe the process
● CodeBuild - execute build, test
● CodeDeploy - Lambda, ECS, EC2,
On-Premises or CFN
● CodeArtifact - package mgr - JS,PY,java
● CodeStar - dashboard for projects
● Cloud9 - Dev Env in the cloud
● ElasticBeanstalk - simplified templated
environments with some CI/CD
● CloudWatch - logs
● CloudTrail - API call audit
● CFN - Infrastructure building
● Lambda + API GW
● ECS
● EKS
● ImageBuilder (Packer)
● ...

27. CI/CD example in AWS

28. AWS DevOps - To use or not to use
● low level - setup intensive
● less polished, crude UX compared to external tool
● often bumpy road compared to “works OOTB” - YMMV
Good use case:
● everything is in AWS
● all actors represented as IAM principals identities / roles
● account based cost attribution - savings of scale
● monitoring tools integration in place- CloudWatch / CloudTrail
● be prepared to invest time

29. What we learned

30. Key: automation => Everything is a Code => declarative
● scripting / procedural
● defining / declarative

31. Documentation matters
● more distributed / microservices => more complexity => larger teams
● distributed work environments => information sharing
Where:
● Wiki (dedicated, GH/BB)
● part of the source code
● => Documentation as a code https://www.writethedocs.org/guide/docs-as-code/
○ Markdown files
○ part of the repo - read/edit in GH, BB, GL
○ MD => HTML static site generators - Hugo, Gatsby, Jekyll

32. Non technical factor matter
● organization structure - silos
● org size
● process / tool overhead - ownership
● culture => next speaker

33. “Man lernt nie aus”
● CI/CD/CL - CL == continuous learning
● Docker/container - 6 years - redefined the workflows
● speed of change increases, complexity increases
● shelf life of skills decreases (except old fashion Unix skills ;-))
Best value for the cost providers (IMHO):
● OReilly Learning = books, courses, live events, Sandboxes/scenarios
● Linux Academy / ACloudGuru => Live Labs
● Udemy - frequent sales - PAYG

34. Future Themes and Challenges

35. GitOps == EiaC: Everything is a Code
● Code is a code => will be built and packaged as such (into Docker image)
● Config is code => must be versioned controlled
● Release and Deployment event should be a code
● => must be version controlled
● => must leave a trace
● => can be rolled back
Git is the only source of truth for deployed system
Everything uses Git repositories + operations on them
Declarative tools make it all work

36. Almost GitOps pipeline

37. Gitops pipeline

38. GitOps + IaaC: multiple release cycles
Cloud environment adds more challenges to GitOps:
● Environment creation
○ => must work for multiple environments (dev, qa, uat, prod)
● Environment maintenance / updates (Env release cycle)
● Application deployments and updates / deploys (app release cycle)
○ => some deployments may require environment changes
○ These environment changes must flow through environments
● Application teardown
● Environment teardown (important for QA / On Demand)
=> 2 different release cycles - different frequency / flow
=> Represented by tools used, layers and repositories involved

39. Immutable Infrastructure
● Create and destroy on demand
● “poor man’s serverless”
● “No server requires less maintenance that no server”
● servers: Pets - Cattle
Tools for IaaC
● Terraform - multi-destination (not only clouds)
● CFN - stacks and stacksets - AWS Specific
● CDK - becoming very important:
CDK, CDK8s, CDKtf - TS, Python - patterns
● 3rd parties - Pulami,

40. 2 CI/CD Examples from my team
Cloud Platform Team
https://pivotree.com/

41. AWS account provisioning
● use case - 200+ AWS accounts => IAM management is a big deal
● AWS Organizations, SSO, external IDP (OneLogin / Okta)
● Pipeline(s) to the rescue
● Pipeline calls other pipelines
●

42. One-Click deployment environment (Dev + QA)
pipeline that builds environments in EKS with full CI/CD capabilties
- deployment target for app pipeline
● BB pipeline
● uses TF
● Hybris App - Helm

43. Links
● ALL DAY DEVOPS - alldaydevops.com - 28/29 October 2021
● OReilly - https://learning.oreilly.com/home/
● ACloudGuru - https://learn.acloud.guru/dashboard?