Kief Morris - Infrastructure is terrible
•
3 likes•695 views
Why is nearly every infrastructure project I've run across a big ball of mud? We're still in the early days of infrastructure as code tooling, so we're struggling with messy glue code, configuration files, and weird custom scripts and tools. What can you do on your project to cope with the current state of tooling? And what should we, as an industry, do to level up?
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Kief Morris - Infrastructure is terrible
Similar to Kief Morris - Infrastructure is terrible (20)
More from Thoughtworks
More from Thoughtworks (20)
Recently uploaded
Recently uploaded (20)
Kief Morris - Infrastructure is terrible
- 3. BUILD
- 4. RUN
- 5. EVOLVE
- 9. We want Code that is: EASY TO UNDERSTAND EASY TO MAINTAIN EASY TO IMPROVE
- 10. We get HORRIFIC CODE resource "aws_subnet" "private" { count = "${length(split(",", var.availability_zones))}" cidr_block = "${cidrsubnet(var.vpc_cidr, 8, count.index + length(split(",", var.availability_zones)) + var.private_subnets_offset)}" availability_zone = "${element(split (",", var.availability_zones), count.index)}" } resource "aws_route_table_association" "private" { count = "${length(split(",",var.availability_zones))}" subnet_id = "${element(aws_subnet.private.*.id, count.index)}" route_table_id = "${aws_route_table.private.id}" } 😱
- 11. History of Infrastructure As Code PART I: PROCEDURAL SCRIPTS!
- 12. import 'cloud-api-library' app_server = CloudApi.find_server($MY_SERVER_NAME) if(app_server == null) { network_segment = CloudApi.find_network_segment('private') app_server = CloudApi.create_server( name: $MY_SERVER_NAME, image: 'base_linux', cpu: 2, ram: '2GB', network: network_segment ) while(app_server.ready == false) { wait 5 } app_server.provision( provisioner: ansible, role: tomcat_server ) } Logic ("how") overwhelms what we're actually interested in ("what") Too much code is just scaffolding 😢 😢
- 13. History of Infrastructure As Code PART II: DECLARATIVE DSL!
- 14. server: name: ${MY_SERVER_NAME} image: 'base_linux' cpu: 2 ram: 2GB network: private_network_segment provision: provisioner: ansible role: tomcat_server The tool handles the "undifferentiated heavy lifting" The language models the infrastructure Our code focuses on what we care about 🙂 🙂 🙂
- 15. History of Infrastructure As Code PART III: PROCEDURAL CODE!
- 17. It's not really "code!" 🤓 😡 😫 😭 "Serverless" uses servers! "Immutable" infrastructure can change! 🤫 🤫
- 18. IDEs don't support infrastructure DSLs very wellSyntax highlighting Code refactoring😢 😢
- 19. - name: add several users user: name: "{{ item.name }}" state: present groups: "{{ item.groups }}" fail: msg: "The command ({{ item.cmd }}) did not have a 0 return code" when: item.rc != 0 loop: - { name: 'testuser1', groups: 'wheel' } - { name: 'testuser2', groups: 'root' } Mixing procedural and declarative code is horrible
- 20. Separate concerns, and language styles [Procedural Code] [Declarative Code] [Procedural Code]
- 21. server: name: ${MY_SERVER_NAME} base_image: ${our_linux_server_image} provision: provisioner: ansible network: vpc_id: ${base_vpc.id} subnet_id: ${private_subnet.id} route: ${route.appserver_inbound} vip: ${load_balancer.appserver.vip} firewall: ${inbound.firewall.rule.allow_https} outbound: ${gateways.nat_1} ssh_key: ${ssh_keys.ops} roles: ${iam_role.appserver} Infrastructure code is too verbose
- 22. MODULES? 🤔
- 23. LIBRARIES! 😀 👍
- 24. Let's provision infrastructure from our application code!!
- 25. <table> <tr><th>Name:</th><th>Job Title:</th><tr> <% db = database.connect('dbhost.mydomain.com') rows = db.select('SELECT name, title FROM users') for row in rows %> <tr> <td><%= row.name %></td><td><%= row.title %></td> </tr> </table> <% db.close %> NO. JUST ... NO. 😱 😡 🤬
- 27. Classic test pyramid UNIT TESTS INTEGRATION TESTS JOURNEY TESTS BROADERSCOPE FASTERFEEDBACK
- 28. Infrastructure test pyramid?? UNIT(?) TESTS STACK TESTS SYSTEM TESTS Often low value, high maintenance
- 29. Testing declarative code? Prove that the tool works Prove that the cloud API works Validate contracts & policies Validate combinations of declared code Validate design constraints subnet: name: private_A address_block: 192.168.0.0/16 assert: subnet("private_A").exists assert: subnet("private_A").address_block == "192.168.0.0/16"
- 30. OFFLINE What can we test: ONLINE Server configuration Syntax and formatting (Some) logical things Interactions of multiple cloud elements Functionality of cloud elements
- 31. Swiss cheese testing model OFFLINE TESTS STACK TESTS SYSTEM TESTS MONITOR- ING RISKS
- 35. We want: To update and change EASILY and SAFELY
- 36. Monolithic infrastructure is hard to change
- 37. Decomposing infrastructure into multiple stacks is a good step
- 38. But dependencies create risk and friction for making changes Goal: Minimize coupling in our infrastructure designs and implementation
- 39. [code that declares a thing] [code that uses the thing] Coupling at the code level 😢 You need to know how I've written my code If I change my code, I may break your code You need to run my stuff in order to test your stuff
- 40. Coupling to a tool's data structures (e.g. statefiles) 😢 Coupled to the tool and its data structure formats
- 41. Integrate using a registry Coupled on named parameters, and registry product
- 42. Goal: deploy and test a standalone instance of your stuff Mock Test it in isolation Can swap out dependencies
- 44. Provisioning a new instance of infrastructure is one thing Changing infrastructure that's in use is another
- 52. Closing Thoughts
- 53. Separate concerns Find good conventions Test based on risk