Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Doing Infrastructure as Code? You can do better.

79 views

Published on

In this talk, Senior Consultants Shohre Mansouri and Effy Elden talk about the techniques that can be used to make developing Infrastructure as Code a better experience for developers, by establishing a faster feedback cycle and also producing higher quality well-tested code.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Doing Infrastructure as Code? You can do better.

  1. 1. Shohre Mansouri Senior Consultant Effy Elden Senior Consultant
  2. 2. Doing Infrastructure as Code? You can do better Shohre Mansouri & Effy Elden
  3. 3. My story
  4. 4. Before there was IaC…
  5. 5. Until…
  6. 6. Infrastructure as Code
  7. 7. Not quite right
  8. 8. Slow feedback…
  9. 9. Long files…
  10. 10. NO TESTS
  11. 11. NO TESTS
  12. 12. ! Lower productivity ! Lower quality ! Less confident ! No refactoring ! Less fun!
  13. 13. ! Lower productivity ! Lower quality ! Less confident ! No refactoring ! Less fun!
  14. 14. ! Lower productivity ! Lower quality ! Less confident ! No refactoring ! Less fun!
  15. 15. ! Lower productivity ! Lower quality ! Less confident ! No refactoring ! Less fun!
  16. 16. ! Lower productivity ! Lower quality ! Less confident ! No refactoring ! Less fun!
  17. 17. Effy’s story
  18. 18. ! Application code ! Infrastructure as code ! Pipelines as code ! Configuration as code ! Provisioning as code ! Deployment as code ! Compliance as code
  19. 19. ! Application code ! Infrastructure as code ! Pipelines as code ! Configuration as code ! Provisioning as code ! Deployment as code ! Compliance as code
  20. 20. ! Application code ! Infrastructure as code ! Pipelines as code ! Configuration as code ! Provisioning as code ! Deployment as code ! Compliance as code
  21. 21. Can we do better?
  22. 22. Can we do better? !Faster feedback cycle
  23. 23. Can we do better? !Faster feedback cycle !Higher quality code
  24. 24. ! Local Validation ! Good Practices ! Unit Tests ! Integration Tests
  25. 25. Syntax check everything
  26. 26. Pipelines as Code
  27. 27. Code that doesn’t run
 until we push it
  28. 28. What could I have done better?
  29. 29. ! Syntax check my pipeline ! Validate my pipeline ! Run my pipeline
  30. 30. ! Syntax check my pipeline ! Validate my pipeline ! Run my pipeline
  31. 31. ! Syntax check my pipeline ! Validate my pipeline ! Run my pipeline
  32. 32. Buildkite CircleCI GitLab Codefresh Run your pipeline locally
  33. 33. A pipeline is just
 a set of instructions
  34. 34. Infrastructure as Code
  35. 35. terraform fmt
  36. 36. terraform fmt terraform validate
  37. 37. terraform fmt terraform validate terraform plan
  38. 38. terraform apply
  39. 39. We should demand better of our infrastructure tools.
  40. 40. What could I have done better?
  41. 41. ! Extra validation of your infrastructure as code
  42. 42. tflint cfn_nag Validate your infrastructure code
  43. 43. The catch.
  44. 44. How do we make it easier to do the right thing?
  45. 45. Pre-commit Hooks
  46. 46. Hooks need to do everything
  47. 47. Hooks need to do everything Hooks become slow and overloaded
  48. 48. Hooks need to do everything Hooks become slow and overloaded Developers become frustrated
  49. 49. Hooks need to do everything Hooks become slow and overloaded Developers become frustrated Developers bypass hooks, push bad code
  50. 50. What makes a
 good pre-commit hook?
  51. 51. ! Selective ! Fast ! Consistent ! Helpful
  52. 52. ! Selective ! Fast ! Consistent ! Helpful
  53. 53. ! Selective ! Fast ! Consistent ! Helpful
  54. 54. ! Selective ! Fast ! Consistent ! Helpful
  55. 55. Therapist pre-commit Overcommit Smart hook helpers
  56. 56. Pre-commit hooks can be smarter, faster, better
  57. 57. ! Local Validation ! Good Practices ! Unit Tests ! Integration Tests
  58. 58. Treat “as code” like a
 first-class citizen
  59. 59. Clean Code
  60. 60. DRY Don’t Repeat Yourself
  61. 61. Readable
  62. 62. Small files
  63. 63. Other Good Practices
  64. 64. Use Source Control Of course!
  65. 65. All environments as close as possible
  66. 66. Pick the right
 tool/language
  67. 67. Write tests
 for everything
  68. 68. ! Local Validation ! Good Practices ! Unit Tests ! Integration Tests
  69. 69. What defines
 a unit test?
  70. 70. “unit tests are low-level, focusing on a small part of the software system” - Martin Fowler
  71. 71. “unit tests are expected to be significantly faster than other kinds of tests” - Martin Fowler
  72. 72. Unit testing IaC is like unit testing SQL queries
  73. 73. ! Mock your cloud provider ! Write convention tests ! Unit test against
 actual cloud providers
  74. 74. localstack moto Mock your cloud provider
  75. 75. ! Mock your cloud provider ! Write convention tests ! Unit test against
 actual cloud providers
  76. 76. terraform
 _validate terraform
 -compliance Write convention tests
  77. 77. ! Mock your cloud provider ! Write convention tests ! Unit test against
 actual cloud providers
  78. 78. ! Local Validation ! Good Practices ! Unit Tests ! Integration Tests
  79. 79. What defines
 an integration test?
  80. 80. “whether many separately developed modules work together as expected” - Martin Fowler
  81. 81. An example test
  82. 82. describe s3_bucket(outputs[:bucket_name][:value]), region: 'ap-southeast-2' do it { should exist } it { should have_acl_grant(grantee:
 'http://acs.amazonaws.com/groups/global/AllUsers', permission: 'READ') } it "should have public read policy for all objects" do policy = JSON.parse(subject.policy.policy.read) public_read_statement = policy["Statement"].find { |p| isPublicReadPolicy(p) } expect(public_read_statement).not_to be_nil end end
  83. 83. describe s3_bucket(outputs[:bucket_name][:value]), region: 'ap-southeast-2' do it { should exist } it { should have_acl_grant(grantee:
 'http://acs.amazonaws.com/groups/global/AllUsers', permission: 'READ') } it "should have public read policy for all objects" do policy = JSON.parse(subject.policy.policy.read) public_read_statement = policy["Statement"].find { |p| isPublicReadPolicy(p) } expect(public_read_statement).not_to be_nil end end
  84. 84. describe s3_bucket(outputs[:bucket_name][:value]), region: 'ap-southeast-2' do it { should exist } it { should have_acl_grant(grantee:
 'http://acs.amazonaws.com/groups/global/AllUsers', permission: 'READ') } it "should have public read policy for all objects" do policy = JSON.parse(subject.policy.policy.read) public_read_statement = policy["Statement"].find { |p| isPublicReadPolicy(p) } expect(public_read_statement).not_to be_nil end end
  85. 85. describe s3_bucket(outputs[:bucket_name][:value]), region: 'ap-southeast-2' do it { should exist } it { should have_acl_grant(grantee:
 'http://acs.amazonaws.com/groups/global/AllUsers', permission: 'READ') } it "should have public read policy for all objects" do policy = JSON.parse(subject.policy.policy.read) public_read_statement = policy["Statement"].find { |p| isPublicReadPolicy(p) } expect(public_read_statement).not_to be_nil end end
  86. 86. resource "aws_s3_bucket" "bucket" { bucket = "${var.bucket_name}" acl = "public-read" policy = <<-POLICY { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowPublicRead", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::${var.bucket_name}/*" } ] } POLICY }
  87. 87. resource "aws_s3_bucket" "bucket" { bucket = "${var.bucket_name}" acl = "public-read" policy = <<-POLICY { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowPublicRead", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::${var.bucket_name}/*" } ] } POLICY }
  88. 88. resource "aws_s3_bucket" "bucket" { bucket = "${var.bucket_name}" acl = "public-read" policy = <<-POLICY { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowPublicRead", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::${var.bucket_name}/*" } ] } POLICY }
  89. 89. resource "aws_s3_bucket" "bucket" { bucket = "${var.bucket_name}" acl = "public-read" policy = <<-POLICY { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowPublicRead", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::${var.bucket_name}/*" } ] } POLICY }
  90. 90. This is an integration test
  91. 91. ! Changes to infra code ! Changes to IaC tool ! Changes to your actual infra ! Changes to cloud provider’s API ! Changes to cloud provider’s offerings
  92. 92. ! Changes to infra code ! Changes to IaC tool ! Changes to your actual infra ! Changes to cloud provider’s API ! Changes to cloud provider’s offerings
  93. 93. ! Changes to infra code ! Changes to IaC tool ! Changes to your actual infra ! Changes to cloud provider’s API ! Changes to cloud provider’s offerings
  94. 94. ! Changes to infra code ! Changes to IaC tool ! Changes to your actual infra ! Changes to cloud provider’s API ! Changes to cloud provider’s offerings
  95. 95. ! Changes to infra code ! Changes to IaC tool ! Changes to your actual infra ! Changes to cloud provider’s API ! Changes to cloud provider’s offerings
  96. 96. It works and it keeps working
  97. 97. Test multiple modules
  98. 98. Test Kitchen awspec terratest InSpec Infrastructure integration testing
  99. 99. That’s all our findings
  100. 100. So…
  101. 101. Can we do better?
  102. 102. Can we do better? !Faster feedback cycle !Higher quality code
  103. 103. ! Syntax check everything ! Run your pipelines locally ! Use extra “as code” validation ! Try smart pre-commit hooks ! Pick the right tool/language
  104. 104. ! Syntax check everything ! Run your pipelines locally ! Use extra “as code” validation ! Try smart pre-commit hooks ! Pick the right tool/language
  105. 105. ! Syntax check everything ! Run your pipelines locally ! Use extra “as code” validation ! Try smart pre-commit hooks ! Pick the right tool/language
  106. 106. ! Syntax check everything ! Run your pipelines locally ! Use extra “as code” validation ! Try smart pre-commit hooks ! Pick the right tool/language
  107. 107. ! Syntax check everything ! Run your pipelines locally ! Use extra “as code” validation ! Try smart pre-commit hooks ! Pick the right tool/language
  108. 108. Can we do better? !Faster feedback cycle !Higher quality code
  109. 109. ! DRY ! Readable, maintainable ! Write tests in different levels
  110. 110. ! DRY ! Readable, maintainable ! Write tests in different levels
  111. 111. ! DRY ! Readable, maintainable ! Write tests in different levels
  112. 112. What else?
  113. 113. ! Better tooling ! Continuous improvement
  114. 114. Thank You

×