Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kief Morris - Infrastructure is terrible

179 views

Published on

Why is nearly every infrastructure project I've run across a big ball of mud? We're still in the early days of infrastructure as code tooling, so we're struggling with messy glue code, configuration files, and weird custom scripts and tools. What can you do on your project to cope with the current state of tooling? And what should we, as an industry, do to level up?

Published in: Software
  • Be the first to comment

Kief Morris - Infrastructure is terrible

  1. 1. Kief Morris kief@thoughtworks.com
  2. 2. INFRASTRUCTURE IS TERRIBLE
  3. 3. BUILD
  4. 4. RUN
  5. 5. EVOLVE
  6. 6. Languages Testing Architecture Refactoring 😱
  7. 7. INFRASTRUCTURE LANGUAGES ARE TERRIBLE
  8. 8. We want Code that is: EASY TO UNDERSTAND EASY TO MAINTAIN EASY TO IMPROVE
  9. 9. We get HORRIFIC CODE resource "aws_subnet" "private" { count = "${length(split(",", var.availability_zones))}" cidr_block = "${cidrsubnet(var.vpc_cidr, 8, count.index + length(split(",", var.availability_zones)) + var.private_subnets_offset)}" availability_zone = "${element(split (",", var.availability_zones), count.index)}" } resource "aws_route_table_association" "private" { count = "${length(split(",",var.availability_zones))}" subnet_id = "${element(aws_subnet.private.*.id, count.index)}" route_table_id = "${aws_route_table.private.id}" } 😱
  10. 10. History of Infrastructure As Code PART I: PROCEDURAL SCRIPTS!
  11. 11. import 'cloud-api-library' app_server = CloudApi.find_server($MY_SERVER_NAME) if(app_server == null) { network_segment = CloudApi.find_network_segment('private') app_server = CloudApi.create_server( name: $MY_SERVER_NAME, image: 'base_linux', cpu: 2, ram: '2GB', network: network_segment ) while(app_server.ready == false) { wait 5 } app_server.provision( provisioner: ansible, role: tomcat_server ) } Logic ("how") overwhelms what we're actually interested in ("what") Too much code is just scaffolding 😢 😢
  12. 12. History of Infrastructure As Code PART II: DECLARATIVE DSL!
  13. 13. server: name: ${MY_SERVER_NAME} image: 'base_linux' cpu: 2 ram: 2GB network: private_network_segment provision: provisioner: ansible role: tomcat_server The tool handles the "undifferentiated heavy lifting" The language models the infrastructure Our code focuses on what we care about 🙂 🙂 🙂
  14. 14. History of Infrastructure As Code PART III: PROCEDURAL CODE!
  15. 15. DECLARATIVE DSLs FOR INFRASTRUCTURE ARE TERRIBLE?
  16. 16. It's not really "code!" 🤓 😡 😫 😭 "Serverless" uses servers! "Immutable" infrastructure can change! 🤫 🤫
  17. 17. IDEs don't support infrastructure DSLs very wellSyntax highlighting Code refactoring😢 😢
  18. 18. - name: add several users user: name: "{{ item.name }}" state: present groups: "{{ item.groups }}" fail: msg: "The command ({{ item.cmd }}) did not have a 0 return code" when: item.rc != 0 loop: - { name: 'testuser1', groups: 'wheel' } - { name: 'testuser2', groups: 'root' } Mixing procedural and declarative code is horrible
  19. 19. Separate concerns, and language styles [Procedural Code] [Declarative Code] [Procedural Code]
  20. 20. server: name: ${MY_SERVER_NAME} base_image: ${our_linux_server_image} provision: provisioner: ansible network: vpc_id: ${base_vpc.id} subnet_id: ${private_subnet.id} route: ${route.appserver_inbound} vip: ${load_balancer.appserver.vip} firewall: ${inbound.firewall.rule.allow_https} outbound: ${gateways.nat_1} ssh_key: ${ssh_keys.ops} roles: ${iam_role.appserver} Infrastructure code is too verbose
  21. 21. MODULES? 🤔
  22. 22. LIBRARIES! 😀 👍
  23. 23. Let's provision infrastructure from our application code!!
  24. 24. <table> <tr><th>Name:</th><th>Job Title:</th><tr> <% db = database.connect('dbhost.mydomain.com') rows = db.select('SELECT name, title FROM users') for row in rows %> <tr> <td><%= row.name %></td><td><%= row.title %></td> </tr> </table> <% db.close %> NO. JUST ... NO. 😱 😡 🤬
  25. 25. IS TERRIBLE INFRASTRUCTURE CODE
  26. 26. Classic test pyramid UNIT TESTS INTEGRATION TESTS JOURNEY TESTS BROADERSCOPE FASTERFEEDBACK
  27. 27. Infrastructure test pyramid?? UNIT(?) TESTS STACK TESTS SYSTEM TESTS Often low value, high maintenance
  28. 28. Testing declarative code? Prove that the tool works Prove that the cloud API works Validate contracts & policies Validate combinations of declared code Validate design constraints subnet: name: private_A address_block: 192.168.0.0/16 assert: subnet("private_A").exists assert: subnet("private_A").address_block == "192.168.0.0/16"
  29. 29. OFFLINE What can we test: ONLINE Server configuration Syntax and formatting (Some) logical things Interactions of multiple cloud elements Functionality of cloud elements
  30. 30. Swiss cheese testing model OFFLINE TESTS STACK TESTS SYSTEM TESTS MONITOR- ING RISKS
  31. 31. https://dobetterascode.com
  32. 32. Good testing requires good design Small, loosely coupled pieces
  33. 33. IS TERRIBLE INFRASTRUCTURE
  34. 34. We want: To update and change EASILY and SAFELY
  35. 35. Monolithic infrastructure is hard to change
  36. 36. Decomposing infrastructure into multiple stacks is a good step
  37. 37. But dependencies create risk and friction for making changes Goal: Minimize coupling in our infrastructure designs and implementation
  38. 38. [code that declares a thing] [code that uses the thing] Coupling at the code level 😢 You need to know how I've written my code If I change my code, I may break your code You need to run my stuff in order to test your stuff
  39. 39. Coupling to a tool's data structures (e.g. statefiles) 😢 Coupled to the tool and its data structure formats
  40. 40. Integrate using a registry Coupled on named parameters, and registry product
  41. 41. Goal: deploy and test a standalone instance of your stuff Mock Test it in isolation Can swap out dependencies
  42. 42. Provisioning a new instance of infrastructure is one thing Changing infrastructure that's in use is another
  43. 43. Expand and contract
  44. 44. Expand and contract v101
  45. 45. Expand and contract v101
  46. 46. Expand and contract v101
  47. 47. Expand and contract v102
  48. 48. Expand and contract v102
  49. 49. Expand and contract v102
  50. 50. Closing Thoughts
  51. 51. Separate concerns Find good conventions Test based on risk
  52. 52. THANK YOU Kief Morris kief@thoughtworks.com

×