We have come a long way from bare metal servers and manual configuration to Serverless and GitOps. With each step of this journey, we found a new way to deal with configuration drift. Configuration management tools have been and are excellent solutions in certain situations. But too often, they act as a metaphysical hammer, and every problem becomes a nail.
We will travel through generations of solutions to combat configuration drift to understand how we got to what we have today and imagine what awaits us tomorrow.
3. www.fivexl.io | hello@fivexl.io
Typical business
needs /
problems
Not able to ship changes fast enough
Not able to scale system to meet
demand
Hard to manage / change large scale
systems
Disaster recovery / Fragile systems
Security / compliance
10. www.fivexl.io | hello@fivexl.io
Configuration Drift is the
phenomenon where servers in
an infrastructure become more
and more different from one
another as time goes on, due to
manual ad-hoc changes and
updates, and general entropy.
Keif Morris
http://kief.com/configuration-drift.html
11. www.fivexl.io | hello@fivexl.io
Configuration changes are
regularly needed to tweak the
environment so that it runs
efficiently and communicates
properly with other systems. This
requires some mix of
command-line invocations,
jumping between GUI screens, and
editing text files.
The result is a unique snowflake -
good for a ski resort, bad for a data
center.
Martin Fowler
https://martinfowler.com/bliki/SnowflakeServer.html
12. www.fivexl.io | hello@fivexl.io
More traffic
Move nginx to a
separate server,
static routing
Add more servers
Make sure that all
servers have the
same configuration
scp code
https://cdn2.iconfinder.com/data/icons/amazon-aws-stencils/100/Compute__N
etworking_copy_Amazon_EC2_Instance-512.png
NewProd,
34.45.56.78
Prod2,
34.45.60.23
Nginx,
34.44.23.67
13. www.fivexl.io | hello@fivexl.io
More traffic
More servers
Consistency
Management
Need for automation
https://www.lacisoft.com/blog/wp-content/uploads/2016/05/logo-amazon-elas
tic-load-balancing.png
NewProd,
34.45.56.78
Prod2,
34.45.60.23
ProdTmp,
34.45.80.72
Prod3,
34.50.57.71
Nginx,
34.44.23.67
14. www.fivexl.io | hello@fivexl.io
First attempts at
automation
Tool-first thinking
Replace bash with
the specialized tool
Kind of consistency
Manual scaling
https://www.lacisoft.com/blog/wp-content/uploads/2016/05/logo-amazon-elas
tic-load-balancing.png
NewProd,
34.45.56.78
Prod2,
34.45.60.23
ProdTmp,
34.45.80.72
Prod3,
34.50.57.71
Nginx,
34.44.23.67
15. www.fivexl.io | hello@fivexl.io
Adding containers
Start containers
instead of copying
code
Some would do
docker-compose
Leap to orchestrators
https://www.lacisoft.com/blog/wp-content/uploads/2016/05/logo-amazon-elas
tic-load-balancing.png
NewProd,
34.45.56.78
Prod2,
34.45.60.23
ProdTmp,
34.45.80.72
Prod3,
34.50.57.71
https://www.docker.com/sites/default/files/d8/2019-07/Moby-logo.png
Nginx,
34.44.23.67
21. www.fivexl.io | hello@fivexl.io
So if I kill my servers often enough and
provision them with Ansible then I’m doing
immutable configuration as code?
22. www.fivexl.io | hello@fivexl.io
So if I kill my servers often enough and
provision them with Ansible then I’m doing
immutable configuration as code?
Is it good enough?
25. www.fivexl.io | hello@fivexl.io
Switch over to ASG
Requires ready to
use image
Allows for scale
in/out
No ssh needed
No pet names,
dynamic
https://tudip.com/wp-content/uploads/2018/12/autoscaling-group.png
30. www.fivexl.io | hello@fivexl.io
AWS
BottleRocket
API access for configuring your system
Updates based on partition flips, for
fast and reliable system updates
Modeled configuration that's
automatically migrated through updates
Security as a top priority
Written in Rust
https://github.com/bottlerocket-os/bottlerocket
45. www.fivexl.io | hello@fivexl.io
Typical business
needs /
problems
Not able to ship changes fast enough
Not able to scale system to meet
demand
Hard to manage / change large scale
systems
Disaster recovery / Fragile systems
Security / compliance
47. Infrastructure as Code
Challenges
Server Sprawl
Configuration Drift
Snowflake Servers
Goals
IT infrastructure supports and enables change.
Changes to the system are routine, without drama or
stress for users or IT staff.
IT staff spends their time on valuable things that engage
their abilities.
Users are able to define, provision, and manage the
resources they need.
Teams are able to easily and quickly recover from failures.
Improvements are made continuously.
Solutions to problems are proven through implementing,
testing, and measuring.
Fragile Infrastructure
Automation Fear
Erosion
48. www.fivexl.io | hello@fivexl.io
Configuration
Synchronization
Still leaves the possibility of configuration drift
A first good step comparing to doing it manually
Slow scaling, far from dynamic
Often used for bare-metal setups
Apparently for K8S
Might be a necessary evil
https://www.oreilly.com/library/view/infrastructure-as-code/9781491924334/ch01.html
49. www.fivexl.io | hello@fivexl.io
Immutable
infrastructure
Great for security
Takes more work to implement
Easy to recreate systems
Resilient/self-healing dynamic systems
Focus on business goals
https://www.oreilly.com/library/view/infrastructure-as-code/9781491924334/ch01.html