Clean Infrastructure as Code
Software Craft Community @ DATEV
Mario-Leander Reimer
mario-leander.reimer@qaware.de
@LeanderReimer
qaware.de
“To Boldly Go
Where No DevOps Engineer
Has Gone Before”
2
QAware
QAware | 3
QAware | 4
QAware | 5
6
Mario-Leander Reimer
Geschäftsführer | CTO
@LeanderReimer
#cloudnativenerd #qaware
#gernperDude
Many of the principles on clean code and architecture also
apply to infrastructure (as) code.
QAware | 7
The 3 essential views of software architecture define the fields
of clean code, clean architecture and quality assurance.
QAware | 8
The 3 essential views of software architecture define the fields
of clean code, clean architecture and quality assurance.
QAware | 9
Biz
Dev
Ops
The infrastructure with its technologies and frameworks, as
well as the responsibilities have changed.
QAware | 10
The architecture and code for our infrastructure also needs to
address many of the 8 -illities for good software quality.
QAware | 11
Software Product
Quality
(ISO 25010)
● Modularity
● Reusability
● Analysability
● Modifiability
● Testability
● Adaptability
● Installability
● Replaceability
● Confidentiality
● Integrity
● Non-repudiation
● Authenticity
● Accountability
● Maturity
● Availability
● Fault Tolerance
● Recoverability
Maintainability
Portability
Security
Reliability
● Co-existence
● Interoperability
Compatibility
● Time Behaviour
● Resource Utilization
● Capacity
Efficiency
● Completeness
● Correctness
● Appropriateness
Functional Suitability
● Operability
● Learnability
● UI Aesthetics
● Accessibility
Usability
Clean Infrastructure Code has an impact on continuously
shipping high quality products to your end users!
QAware | 12
Continuous
Delivery
Low Risk
Releases
Less
Rework
Fast Time
to Market
Better
Products
Lower
Costs
Happier
Teams
Happier
Users
Loosely Coupled Architectures
Maintainable Code
Empowered Teams
Continuous Security from Day 1
Test
Automation
Continuous
Integration
GitOps
Deployment
Automation
Monitoring and Alerting
Trial and Error seems to be the predominant approach to get
infrastructure code up and running.
QAware | 13
Write
some
code
Run
code
Write
some
code
Run
code
Write
some
code
Run
code
?
Write
Dockerfile
docker build
docker run
Change
Dockerfile
docker build
docker run
Change
Dockerfile
docker build
docker run
?
Write
K8s YAML
kubectl
apply
Change
K8s YAML
kubectl
apply
Change
K8s YAML
kubectl
apply
?
Write
Terraform
Definition
terraform
apply
Change
Terraform
Definition
terraform
apply
Change
Terraform
Definition
terraform
apply
?
QAware | 14
Big Ball of
Infrastructure
Code Mud
Karl
Emily
Steve
The usual suspects! As always there are many excuses, myths
or reasons for bad and smelly (infrastructure) code.
QAware | 15
■ High cognitive load or missing know-how within teams
– Young and inexperienced teams are often lacking the required know-how
– High cognitive load due to many new tools and technologies
■ High (infrastructure) complexity and costs for quality assurance of IaC
– Integrative test approach required, especially for declarative tools like Terraform et.al
– Dedicated infrastructure and environments required
■ Urgent deadlines or unrealistic plans
– The standard reason excuse for bad code quality for ages!
– When will we ever learn? Learn to say No!
■ Feature greed and missing awareness from product owners
– Ops and infrastructure related tasks and acceptance criteria are often missing in user stories.
– Incomplete Definition of Done.
“Too much cognitive load will become a bottleneck for fast
flow and high productivity for many DevOps teams.”
QAware | 16
■ Intrinsic Cognitive Load
Relates to fundamental aspects and knowledge in the
problem space (e.g. used languages, APIs, frameworks)
■ Extraneous Cognitive Load
Relates to the environment (e.g. console
command, deployment, configuration)
■ Germane Cognitive Load
Relates to specific aspects of the business domain
(aka. „value added“ thinking)
A Platform team and its engineers are a key enabler for high
productivity of stream-aligned DevOps teams.
QAware | 17
■ Responsible to build and operation a platform to
enable and support the teams in their day to day
development work.
■ The platform aims to hide the inherent complexity
to reduce the cognitive load for the other teams.
– Standardization
– Self-Service
■ Fully automated software delivery is the goal!
https://hennyportman.wordpress.com/2020/05/25/review-team-topologies/
Karl & Steve
Emily
QAware | 18
You have to
work in layers!
Cloud-native
Application Engineering
Cloud-native
Platform Engineering
The 5 Layers of Cloud-native Software Engineering
QAware | 19
IaaS
Network, Compute, Storage
(VPC, EC2, NLB, ALB, ...)
CaaS
(Kubernetes Services)
PaaS
(Software Infrastructure Blueprints with Helm and
Continuous Delivery Toolchain)
Application-specific
Software Infrastructure
Cloud-friendly & cloud-native
Applications
Architect Build Run
Atlantis
The conceptual architecture of your infrastructure will have a
significant impact on the source code and its structure.
QAware | 20
■ Which cloud regions do we need to run in?
Which parts of the infrastructure and system
need to be replicated?
■ How do we model environments (INT;
PROD)? VPCs? K8s Namespace?
■ Which layers and components are required?
What are their dependencies?
■ How do we structure the Git repositories and
source code to represent the components?
SoC
Separation of Concerns.
Modules, layers and components. Loose Coupling and High Cohesion.
KISS
Keep it Simple, Stupid.
Emergent Design and Evolutionary Architecture
https://de.slideshare.net/ThoughtWorks/neal-ford-emergent-design-and-evolutionary-architecture
DRY
Don’t Repeat Yourself.
“Every piece of knowledge or logic must have a single, unambiguous representation within a system.” Andy
Hunt & Dave Thomas, “The Pragmatic Programmer”
SOLID
Single Responsibility Principle
Open Closed Principle
Liskov Substitution Principle
Interface Segregation Principle
Dependency Inversion Principle
QAware | 26
Clean Code
Good overall summary can be found at https://gist.github.com/wojteklu/73c6914cc446146b8b533c0988cf8d29
1. Follow standard conventions.
2. Keep it simple stupid. Simpler is always better.
Reduce complexity as much as possible.
General
1. Keep configurable data at high levels.
2. Prevent over-configurability.
3. Use dependency injection.
Design
1. Be consistent. If you do something a certain
way, do all similar things in the same way.
2. Use explanatory variables.
3. Avoid negative conditionals.
Understandability
1. Don’t be funny.
2. Choose descriptive and
unambiguous names.
3. Use pronounceable
names.
4. Use searchable names.
Names
1. Always try to explain yourself in code.
2. Don't be redundant.
3. Don't add obvious noise.
4. Don't comment out code. Just remove.
5. Use as clarification of code.
6. Use as warning of consequences.
Comments
1. Fast.
2. Independent.
3. Repeatable.
4. Self-validating
5. Timely
Tests
1. Keep lines short.
2. Use white space to associate related things and
disassociate weakly related.
3. Don't break indentation.
4. Dependent definitions should be close.
5. Structure should clearly express modules,
layers, components or conceptual architecture.
Source Code
1. Prefer data structures.
2. Hide internal structure.
3. Should be small.
4. Small number of variables.
Data Structures
There are many constructive and analytical options and
measures of quality assurance available.
QAware | 27
Measures of quality assurance
Constructive Analytical
Defined Processes
Document Templates
Methods and Tools
Know-How Transfer
Education
Processes
Results
Assessments
Audits
Appraisals
Static
Analysis
Reviews
Walkthroughs
Test Measure
Static
Validation
Dynamic
Validation
Demos and Examples
https://github.com/qaware/clean-infrastructure-as-code
Declarative
VS
Imperative
Imperative Tools like CDK or Pulumi enable Developers and SRE to
apply Engineering Practices to Infrastructure code.
QAware | 30
■ Tame overall complexity and reduce cognitive load.
■ One consistent approach to Infrastructure as Code and
cloud engineering for many cloud providers, Docker
and Kubernetes.
■ No breach between application development and
DevOps engineering.
■ Rich programmable cloud interfaces with abstractions
and reusable packages.
■ Apply clean code and general engineering practices to
infrastructure code: automation, modularity, testing,
and CI/CD.
qaware.de
QAware GmbH
Aschauer Straße 32
81549 München
Tel. +49 89 232315-0
info@qaware.de
twitter.com/qaware
linkedin.com/company/qaware-gmbh
xing.com/companies/qawaregmbh
slideshare.net/qaware
github.com/qaware
Meine Kontaktdaten ...

Clean Infrastructure as Code

  • 1.
    Clean Infrastructure asCode Software Craft Community @ DATEV Mario-Leander Reimer mario-leander.reimer@qaware.de @LeanderReimer qaware.de “To Boldly Go Where No DevOps Engineer Has Gone Before”
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
    6 Mario-Leander Reimer Geschäftsführer |CTO @LeanderReimer #cloudnativenerd #qaware #gernperDude
  • 7.
    Many of theprinciples on clean code and architecture also apply to infrastructure (as) code. QAware | 7
  • 8.
    The 3 essentialviews of software architecture define the fields of clean code, clean architecture and quality assurance. QAware | 8
  • 9.
    The 3 essentialviews of software architecture define the fields of clean code, clean architecture and quality assurance. QAware | 9 Biz Dev Ops
  • 10.
    The infrastructure withits technologies and frameworks, as well as the responsibilities have changed. QAware | 10
  • 11.
    The architecture andcode for our infrastructure also needs to address many of the 8 -illities for good software quality. QAware | 11 Software Product Quality (ISO 25010) ● Modularity ● Reusability ● Analysability ● Modifiability ● Testability ● Adaptability ● Installability ● Replaceability ● Confidentiality ● Integrity ● Non-repudiation ● Authenticity ● Accountability ● Maturity ● Availability ● Fault Tolerance ● Recoverability Maintainability Portability Security Reliability ● Co-existence ● Interoperability Compatibility ● Time Behaviour ● Resource Utilization ● Capacity Efficiency ● Completeness ● Correctness ● Appropriateness Functional Suitability ● Operability ● Learnability ● UI Aesthetics ● Accessibility Usability
  • 12.
    Clean Infrastructure Codehas an impact on continuously shipping high quality products to your end users! QAware | 12 Continuous Delivery Low Risk Releases Less Rework Fast Time to Market Better Products Lower Costs Happier Teams Happier Users Loosely Coupled Architectures Maintainable Code Empowered Teams Continuous Security from Day 1 Test Automation Continuous Integration GitOps Deployment Automation Monitoring and Alerting
  • 13.
    Trial and Errorseems to be the predominant approach to get infrastructure code up and running. QAware | 13 Write some code Run code Write some code Run code Write some code Run code ? Write Dockerfile docker build docker run Change Dockerfile docker build docker run Change Dockerfile docker build docker run ? Write K8s YAML kubectl apply Change K8s YAML kubectl apply Change K8s YAML kubectl apply ? Write Terraform Definition terraform apply Change Terraform Definition terraform apply Change Terraform Definition terraform apply ?
  • 14.
    QAware | 14 BigBall of Infrastructure Code Mud Karl Emily Steve
  • 15.
    The usual suspects!As always there are many excuses, myths or reasons for bad and smelly (infrastructure) code. QAware | 15 ■ High cognitive load or missing know-how within teams – Young and inexperienced teams are often lacking the required know-how – High cognitive load due to many new tools and technologies ■ High (infrastructure) complexity and costs for quality assurance of IaC – Integrative test approach required, especially for declarative tools like Terraform et.al – Dedicated infrastructure and environments required ■ Urgent deadlines or unrealistic plans – The standard reason excuse for bad code quality for ages! – When will we ever learn? Learn to say No! ■ Feature greed and missing awareness from product owners – Ops and infrastructure related tasks and acceptance criteria are often missing in user stories. – Incomplete Definition of Done.
  • 16.
    “Too much cognitiveload will become a bottleneck for fast flow and high productivity for many DevOps teams.” QAware | 16 ■ Intrinsic Cognitive Load Relates to fundamental aspects and knowledge in the problem space (e.g. used languages, APIs, frameworks) ■ Extraneous Cognitive Load Relates to the environment (e.g. console command, deployment, configuration) ■ Germane Cognitive Load Relates to specific aspects of the business domain (aka. „value added“ thinking)
  • 17.
    A Platform teamand its engineers are a key enabler for high productivity of stream-aligned DevOps teams. QAware | 17 ■ Responsible to build and operation a platform to enable and support the teams in their day to day development work. ■ The platform aims to hide the inherent complexity to reduce the cognitive load for the other teams. – Standardization – Self-Service ■ Fully automated software delivery is the goal! https://hennyportman.wordpress.com/2020/05/25/review-team-topologies/ Karl & Steve Emily
  • 18.
    QAware | 18 Youhave to work in layers!
  • 19.
    Cloud-native Application Engineering Cloud-native Platform Engineering The5 Layers of Cloud-native Software Engineering QAware | 19 IaaS Network, Compute, Storage (VPC, EC2, NLB, ALB, ...) CaaS (Kubernetes Services) PaaS (Software Infrastructure Blueprints with Helm and Continuous Delivery Toolchain) Application-specific Software Infrastructure Cloud-friendly & cloud-native Applications Architect Build Run Atlantis
  • 20.
    The conceptual architectureof your infrastructure will have a significant impact on the source code and its structure. QAware | 20 ■ Which cloud regions do we need to run in? Which parts of the infrastructure and system need to be replicated? ■ How do we model environments (INT; PROD)? VPCs? K8s Namespace? ■ Which layers and components are required? What are their dependencies? ■ How do we structure the Git repositories and source code to represent the components?
  • 21.
    SoC Separation of Concerns. Modules,layers and components. Loose Coupling and High Cohesion.
  • 22.
    KISS Keep it Simple,Stupid. Emergent Design and Evolutionary Architecture https://de.slideshare.net/ThoughtWorks/neal-ford-emergent-design-and-evolutionary-architecture
  • 23.
    DRY Don’t Repeat Yourself. “Everypiece of knowledge or logic must have a single, unambiguous representation within a system.” Andy Hunt & Dave Thomas, “The Pragmatic Programmer”
  • 24.
  • 25.
    Single Responsibility Principle OpenClosed Principle Liskov Substitution Principle Interface Segregation Principle Dependency Inversion Principle
  • 26.
    QAware | 26 CleanCode Good overall summary can be found at https://gist.github.com/wojteklu/73c6914cc446146b8b533c0988cf8d29 1. Follow standard conventions. 2. Keep it simple stupid. Simpler is always better. Reduce complexity as much as possible. General 1. Keep configurable data at high levels. 2. Prevent over-configurability. 3. Use dependency injection. Design 1. Be consistent. If you do something a certain way, do all similar things in the same way. 2. Use explanatory variables. 3. Avoid negative conditionals. Understandability 1. Don’t be funny. 2. Choose descriptive and unambiguous names. 3. Use pronounceable names. 4. Use searchable names. Names 1. Always try to explain yourself in code. 2. Don't be redundant. 3. Don't add obvious noise. 4. Don't comment out code. Just remove. 5. Use as clarification of code. 6. Use as warning of consequences. Comments 1. Fast. 2. Independent. 3. Repeatable. 4. Self-validating 5. Timely Tests 1. Keep lines short. 2. Use white space to associate related things and disassociate weakly related. 3. Don't break indentation. 4. Dependent definitions should be close. 5. Structure should clearly express modules, layers, components or conceptual architecture. Source Code 1. Prefer data structures. 2. Hide internal structure. 3. Should be small. 4. Small number of variables. Data Structures
  • 27.
    There are manyconstructive and analytical options and measures of quality assurance available. QAware | 27 Measures of quality assurance Constructive Analytical Defined Processes Document Templates Methods and Tools Know-How Transfer Education Processes Results Assessments Audits Appraisals Static Analysis Reviews Walkthroughs Test Measure Static Validation Dynamic Validation
  • 28.
  • 29.
  • 30.
    Imperative Tools likeCDK or Pulumi enable Developers and SRE to apply Engineering Practices to Infrastructure code. QAware | 30 ■ Tame overall complexity and reduce cognitive load. ■ One consistent approach to Infrastructure as Code and cloud engineering for many cloud providers, Docker and Kubernetes. ■ No breach between application development and DevOps engineering. ■ Rich programmable cloud interfaces with abstractions and reusable packages. ■ Apply clean code and general engineering practices to infrastructure code: automation, modularity, testing, and CI/CD.
  • 31.
    qaware.de QAware GmbH Aschauer Straße32 81549 München Tel. +49 89 232315-0 info@qaware.de twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware Meine Kontaktdaten ...