Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Gotchas using Terraform in a secure delivery pipeline

387 views

Published on

Slides from my talk at NDC Oslo, RigaDevDays, DevOps Pro during 2018

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Gotchas using Terraform in a secure delivery pipeline

  1. 1. by Anton Babenko GOTCHAS USING TERRAFORM IN A SECURE DELIVERY PIPELINE
  2. 2. AGENDA ▸ Key concepts in Infrastructure as code (IaC) and Terraform ▸ How CI/CD pipeline can look like ▸ How Terraform can be used there ▸ FTP (Frequent Terraform Problems) 2
  3. 3. ANTON BABENKO Terraform AWS fanatic Organise {HashiСorp User Group, AWS User Group, DevOpsDays} Oslo github.com/terraform-aws-modules (1M+ downloads) github.com/antonbabenko @antonbabenko linkedin.com/in/antonbabenko anton@antonbabenko.com 3
  4. 4. 4 PS: Diagrams like this (created on cloudcraft.co) can be automatically exported to usable Terraform code. Interested? Ask me how at the end :)
  5. 5. ENABLE THE RECONSTRUCTION OF THE BUSINESS FROM NOTHING BUT A SOURCE CODE REPOSITORY, AN APPLICATION DATA BACKUP, AND BARE METAL RESOURCES Jesse Robins (Founder of Chef) IF YOU’RE DOING DEVOPS TODAY, YOU’RE PROBABLY TREATING YOUR INFRASTRUCTURE AS CODE
  6. 6. WHY INFRASTRUCTURE AS CODE IS IMPORTANT? ▸ Treat infrastructure like application code ▸ Same code for all environments ▸ Anyone can build an environment anytime ▸ Validate infrastructure before deployment ▸ Always know what changed 6
  7. 7. 7 Write, plan, and create infrastructure as code www.terraform.io
  8. 8. CI/CD PIPELINE WHY CI/CD PIPELINE IS IMPORTANT? It gives you fair idea about bottlenecks that likely to occur and increase confidence when it comes to avoiding these bottlenecks. 11
  9. 9. CI/CD PIPELINE image from www.i-visionblog.com 12
  10. 10. GETTING FROM 0 TO 1 READ => WRITE => TERRAFORM APPLY 13
  11. 11. WAY AHEAD… TERRAFORM MODULES Reuse code & «not invented here» ▸ registry.terraform.io - 300+ modules ▸ github.com/terraform-aws-modules 14
  12. 12. WAY AHEAD… TERRAFORM WORKSPACE Just use terraform_remote_state instead…
  13. 13. CI/CD TOOLS ▸ CircleCI/TravisCI ▸ Plugins for Concourse/Drone/Jenkins/Ansible 16
  14. 14. ATLANTIS 17 A unified workflow for collaborating on Terraform through GitHub and GitLab www.runatlantis.io
  15. 15. 18 ▸ GUI for plan, apply, locking, state rollbacks ▸ Sentinel - policy as code ▸ Private modules registry ▸ Enterprise pricing
  16. 16. PRE-SUMMARY Terraform code CI/CD options 19
  17. 17. CI/CD GOTCHAS #1 GENERAL ▸ Remote state only ▸ Error handling: ▸ retry X times ▸ terraform plan -parallelism=1 ▸ terraform plan -detailed-exitcode 20
  18. 18. CI/CD GOTCHAS #2 TERRAFORM DEPENDENCIES & PLUGINS ▸ terraform init ▸ .terraform/* ▸ vendor, fork (modules, providers) ▸ private repositories 21
  19. 19. CI/CD GOTCHAS #3 SECRETS ▸ Part of CI system, environment variables ▸ Stored in state file ▸ Use provider specific features: ▸ PGP (aws_iam_login_profile, aws_iam_access_key) ▸ RDS (iam_database_authentication_enabled) or change outside 22
  20. 20. CI/CD GOTCHAS #3 23
  21. 21. CI/CD GOTCHAS #3 24
  22. 22. THE PRINCIPLE OF LEAST PRIVILEGE REQUIRES THAT EVERY PROCESS MUST BE ABLE TO ACCESS ONLY THE INFORMATION AND RESOURCES THAT ARE NECESSARY FOR ITS LEGITIMATE PURPOSE. Principle of least privilege ACCESS CONTROL 25
  23. 23. GRANT ONLY THE PERMISSIONS REQUIRED TO PERFORM A TASK. AWS IAM best practices ACCESS CONTROL 26
  24. 24. ACCESS CONTROL IN AWS 27
  25. 25. SECURE DELIVERY PIPELINE ▸ Access control ▸ «Do what you really want» ▸ PR approval system - https://github.com/capitalone/checks-out ▸ Depends on how you structure your code 28
  26. 26. DEEP(ER) DIVE TERRAFORM DESIGN PATTERNS ▸ Resource modules (eg, terraform-aws-modules) ▸ Infrastructure modules ▸ Composition 29
  27. 27. DESIGN PATTERNS ▸ Create resources (obviously) ▸ No relations to other modules (usually) ▸ Very flexible ▸ Versioning 30 TERRAFORM RESOURCE MODULES (EG, TERRAFORM-AWS-MODULES)
  28. 28. DESIGN PATTERNS TERRAFORM INFRASTRUCTURE MODULES ▸ Use specific version of resource modules ▸ Company-wide standards (eg, tags and names) ▸ May use code generators (jsonnet, cookiecutter, etc) ▸ Versioning 31
  29. 29. DESIGN PATTERNS TERRAFORM COMPOSITION ▸ Use specific version of infrastructure or resource modules ▸ Provide all the values for region, environment, module, etc ▸ Terragrunt is awesome 32
  30. 30. TYPES OF DELIVERY PIPELINES BASIC PIPELINE ▸ git checkout -b new-feature ▸ git commit ▸ git push origin new-feature ▸ Open a pull-request ▸ … ▸ PR is approved, new-feature is merged to master (protected branch) ▸ terraform apply in master branch 33
  31. 31. TYPES OF DELIVERY PIPELINES BASIC PIPELINE 34
  32. 32. TYPES OF DELIVERY PIPELINES PIPELINE FOR MODULES ▸ Generate missing configurations (jsonnet, for eg) ▸ Run automated tests (kitchen-terraform, terratest, for eg) ▸ Publish summary as a comment ▸ Notify downstream users - https://github.com/justwatchcom/github-releases- notifier 
 35
  33. 33. TOOLS RELATED TOOLS ▸ github.com/antonbabenko/pre-commit-terraform ▸ github.com/wata727/tflint ▸ github.com/segmentio/terraform-docs ▸ github.com/kamatama41/tfenv ▸ github.com/gruntwork-io/terragrunt 36
  34. 34. PIPELINES ARE GOOD, BUT NOT ENOUGH ▸ Refactoring ▸ Upgrades ▸ Rollbacks ▸ Force unlock 38
  35. 35. FREQUENT TERRAFORM PROBLEMS (FTP) 39 reddit.com/r/Terraform
  36. 36. FTP REFACTORING ▸ terraform state mv ▸ Upgrade versions of modules ▸ Run terraform commands recursively 40 github.com/antonbabenko/terrible
  37. 37. FTP TERRAFORM UPGRADE ▸ Enabled versioning on state bucket, right!? ▸ Use tfenv to manage Terraform version 41
  38. 38. FTP TERRAFORM LOCKS ▸ terraform force-unlock ▸ Use Atlantis ▸ terraform-aws-atlantis — Terraform configurations for running Atlantis on AWS Fargate 42
  39. 39. DEMO https://github.com/antonbabenko/terraform-deployment-pipeline-talk 43
  40. 40. FINAL SUMMARY ▸ Use CI/CD for automated tasks (check, plan, apply, destroy, merge, promotion) ▸ Use minimal roles + MFA 44
  41. 41. KEEP IT SIMPLE, STUPID 45 No workspaces and no advanced arguments in CLI
  42. 42. QUESTIONS?

×