Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DevOps Fest 2019. Mykyta Protsenko. Infrastructure-as-code: bridging the gap between Devs and Ops

Ops are overwhelmed with support. Devs are mad because their cannot deploy the changes as fast as they want. Sounds familiar?
Infrastructure-as-code can make your life easier by empowering developers and reducing operations' routine toil. It can cut down the lead time for infrastructure provisioning from hours or even days to minutes.
This talk reviews several IaC tools and approaches, showing how to integrate them into continuous delivery pipeline. It covers the problems and challenges that engineers may face while working with infrastructure-as-code tools and provides a few hands-on recipes to address them.

  • Be the first to comment

  • Be the first to like this

DevOps Fest 2019. Mykyta Protsenko. Infrastructure-as-code: bridging the gap between Devs and Ops

  1. 1. Infrastructure-as-Code bridging the gap between Devs and Ops April 6th, 2019 - DevOps Fest 2019 - Kyiv, Ukraine
  2. 2. Who am I? Mykyta Protsenko Software developer @ Netflix (Edge Developer Productivity) Twitter: @mykyta_p
  3. 3. How long does it take you to provision infrastructure?
  4. 4. How many resources do you need for one microservice?
  5. 5. How many resources do you need for one microservice?
  6. 6. ...and how many do you need for all of them?
  7. 7. Do it now and automate later?
  8. 8. Too much stuff for OPS to handle
  9. 9. Bridging the gap Empowering devs Ensuring safety
  10. 10. What is infrastructure?
  11. 11. Computing Storage Networking
  12. 12. Everything is software
  13. 13. Writing Testing Maintaining
  14. 14. 12 factor apps Codebase Dependencies Configuration Backing services Build, release, run Processes Port binding Concurrency Disposability Dev/prod parity Logs Admin processes
  15. 15. 12 factor apps Codebase Dependencies Configuration Backing services Build, release, run Processes Port binding Concurrency Disposability Logs Dev/prod parity Admin processes
  16. 16. 12 factor apps Codebase Configuration Logs Dev/Prod Parity
  17. 17. 12 factor apps Codebase Configuration Logs Dev/Prod Parity
  18. 18. 12 factor apps Codebase Configuration Logs Dev/Prod Parity
  19. 19. 12 factor apps Codebase Configuration Logs Dev/Prod Parity
  20. 20. 12 factor apps Codebase Configuration Logs Dev/Prod Parity
  21. 21. 12 factor apps Codebase Configuration Logs Dev/Prod Parity
  22. 22. Ansible Chef Puppet
  23. 23. Ansible Chef Puppet
  24. 24. Immutable infrastructure
  25. 25. What tools do we need? aws elb create-load-balancer --load-balancer-name myELB --listeners "Protocol=HTTP, LoadBalancerPort=80, InstanceProtocol=HTTP, InstancePort=80" --subnets subnet-15aaab61 --security-groups sg-a61988c3
  26. 26. Imperative tools? aws elb create-load-balancer --load-balancer-name myELB --listeners "Protocol=HTTP, LoadBalancerPort=80, InstanceProtocol=HTTP, InstancePort=80" --subnets subnet-15aaab61 --security-groups sg-a61988c3
  27. 27. Declarative tools FTW!
  28. 28. Cloudformation? "MyDNSRecord": { "Type": "AWS::Route53::RecordSet", "Properties": { "HostedZoneName": {"Fn::Join": ["", [{"Ref":"HostedZone"}, "."]]}, "Comment" : "DNS for inst.", "Name" : {"Fn::Join": ["",[{"Ref":"EC2Instance"}, ".",{"Ref": "AWS::Region"}, ".", {"Ref:"HostedZone"},"."]]}, "Type" : "A", "TTL" : "300", "ResourceRecords": [{"Fn::GetAtt" : ["EC2Instance", PublicIp"]}]} }
  29. 29. resource "aws_route53_record" "www" { zone_id = "${...}" name = "www.example.com" type = "A" ttl = "300" records = ["${aws_eip.lb.public_ip}"] } Terraform
  30. 30. Terraform Simple Human-friendly
  31. 31. What else? @RestController public class HelloWorld { @GetMapping("/") public String hello() { return "Hello World!n"; } } https://github.com/iac-demo
  32. 32. Terraform FTW! resource "aws_ecs_task_definition" "hello-world" { ... family = "hello-world" cpu = "256" // 0.25 vCPU memory = "512" // 512 MB container_definitions = <<DEF [ { ... "image": "helloworld:latest", ... } ] DEF } https://github.com/iac-demo
  33. 33. Terraform FTW! resource "aws_ecs_task_definition" "hello-world" { ... family = "hello-world" cpu = "256" // 0.25 vCPU memory = "512" // 512 MB container_definitions = <<DEF [ { ... "image": "helloworld:latest", ... } ] DEF } https://github.com/iac-demo
  34. 34. Terraform FTW! resource "aws_ecs_service" "hello-world" { cluster = "${aws_ecs_cluster.main.id}" task_definition = "${aws_ecs_task_definition.hello-world.arn}" desired_count = "2" network_configuration { subnets = ["${aws_subnet.sb_a.id}",...] security_groups = ["${aws_security_group.web_ecs.id}"] } load_balancer { target_group_arn = "${aws_alb_target_group.hello-world.id}" container_name = "helloworld" container_port = "8080" } ... https://github.com/iac-demo
  35. 35. Terraform FTW! resource "aws_ecs_service" "hello-world" { cluster = "${aws_ecs_cluster.main.id}" task_definition = "${aws_ecs_task_definition.hello-world.arn}" desired_count = "2" network_configuration { subnets = ["${aws_subnet.sb_a.id}",...] security_groups = ["${aws_security_group.web_ecs.id}"] } load_balancer { target_group_arn = "${aws_alb_target_group.hello-world.id}" container_name = "helloworld" container_port = "8080" } ... https://github.com/iac-demo
  36. 36. Terraform FTW! resource "aws_ecs_service" "hello-world" { cluster = "${aws_ecs_cluster.main.id}" task_definition = "${aws_ecs_task_definition.hello-world.arn}" desired_count = "2" network_configuration { subnets = ["${aws_subnet.sb_a.id}",...] security_groups = ["${aws_security_group.web_ecs.id}"] } load_balancer { target_group_arn = "${aws_alb_target_group.hello-world.id}" container_name = "helloworld" container_port = "8080" } ... https://github.com/iac-demo
  37. 37. Terraform FTW! resource "aws_alb" "hello-world" { name = "hello-world" subnets = [...] security_groups = [...] vpc_id = "${aws_vpc.iacdemo_vpc.id}" } https://github.com/iac-demo
  38. 38. Core Infrastructure vs Project Infrastructure
  39. 39. Core Infrastructure resource "aws_vpc" "iacdemo_vpc" { cidr_block = "10.0.0.0/16" enable_dns_hostnames = true } resource "aws_internet_gateway" "default" { vpc_id = "${aws_vpc.iacdemo_vpc.id}" } ... https://github.com/iac-demo
  40. 40. Core Infrastructure output "vpc_id" { value = "${aws_vpc.iacdemo_vpc.id}" } terraform { backend "s3" { key = "iacdemo.tfstate" region = "us-west-2" bucket = "demobucket" } } https://github.com/iac-demo
  41. 41. Project Infrastructuredata "terraform_remote_state" "core" { backend = "s3" config { key = "iacdemo.tfstate" region = "us-west-2" bucket = "demobucket" } } resource "aws_alb" "hello-world" { name = "hello-world" ... vpc_id = "${data. terraform_remote_state.core.vpc_id}" } https://github.com/iac-demo
  42. 42. Project Infrastructuredata "terraform_remote_state" "core" { backend = "s3" config { key = "iacdemo.tfstate" region = "us-west-2" bucket = "demobucket" } } resource "aws_alb" "hello-world" { name = "hello-world" ... vpc_id = "${data. terraform_remote_state.core.vpc_id}" } https://github.com/iac-demo
  43. 43. https://github.com/iac-demo
  44. 44. CELEBRATE!
  45. 45. CELEBRATE?
  46. 46. State of the world local terraform.tfstate
  47. 47. State of the world remote S3 local terraform.tfstate
  48. 48. State of the world terraform { backend "s3" { key = "iacdemo.tfstate" region = "us-west-2" bucket = "demobucket" dynamodb_table = "table_for_locking" } }
  49. 49. State of the world terraform init
  50. 50. Automated Pipeline git clone git@github.com:my/repo.git
  51. 51. Automated Pipeline git clone git@github.com:my/repo.git terraform init
  52. 52. Automated Pipeline git clone git@github.com:my/repo.git terraform init export TF_VAR_foo="bar" terraform plan
  53. 53. Automated Pipeline git clone git@github.com:my/repo.git terraform init export TF_VAR_foo="bar" terraform plan terraform apply
  54. 54. COPY-PASTE
  55. 55. Encapsulation Hiding Complexity Reusing Code
  56. 56. Terraform modules variable "service_name" {} variable "docker_image" {}
  57. 57. Terraform modules variable "service_name" {} variable "docker_image" {} resource "aws_ecs_task_definition" "service" { family = "${var.service_name}" container_definitions = <<DEF [{ ... "image": "${var.docker_image}", "name": "${var.service_name}" ... }]DEF }
  58. 58. Terraform modulesmodule "hello_world" { source = "./microservice_module" service_name = "helloworld1" docker_image= "helloworld:latest" }
  59. 59. Terraform modulesmodule "hello_world" { source = "./microservice_module" service_name = "helloworld1" docker_image= "helloworld:latest" } module "another_hello_world" { source = "./microservice_module" service_name = "helloworld2" docker_image= "helloworld:latest" }
  60. 60. Where do I store modules?
  61. 61. Terraform modules Local Storage S3 Consul Terraform Registry Git … and others module "hello_world" { source = "git::https://example.com/srv.git" ... }
  62. 62. Terraform modules Local Storage S3 Consul Terraform Registry Git … and others module "hello_world" { source = "git::https://example.com/srv.git ?ref=1.0.0" ... }
  63. 63. Terraform modulesmodule "hello_world" { source = ... service_name = "helloworld" docker_image= "helloworld:latest" }
  64. 64. Breaking Changes?
  65. 65. ...While Running 24/7?
  66. 66. Create Before Destroy resource "aws_alb" "hello-world" { ... lifecycle { create_before_destroy = "true" } }
  67. 67. Create Before Destroy resource "aws_alb" "hello-world" { ... lifecycle { create_before_destroy = "true" } } resource "aws_ecs_service" "hello-world" { ... lifecycle { create_before_destroy = "true" } }
  68. 68. Green/Blue Deployments V1
  69. 69. Green/Blue Deployments V1 V2
  70. 70. Green/Blue Deployments V1 V2
  71. 71. Safety vs Complexity/Cost
  72. 72. Life after Terraform
  73. 73. Kubernetes! And More!
  74. 74. Declarative K8S k8s_template.yaml.sh #!/bin/bash cat <<YAML apiVersion: apps/v1beta1 kind: Deployment ... spec: replicas: 1 template: spec: containers: - name: $SERVICE_NAME image: $DOCKER_IMAGE imagePullPolicy: Always ports: - containerPort: 8090 ... YAML
  75. 75. Declarative K8S k8s_template.yaml.sh #!/bin/bash cat <<YAML apiVersion: apps/v1beta1 kind: Deployment ... spec: replicas: 1 template: spec: containers: - name: $SERVICE_NAME image: $DOCKER_IMAGE imagePullPolicy: Always ports: - containerPort: 8090 ... YAML
  76. 76. Declarative K8S $ export DOCKER_IMAGE=hello:latest $ export SERVICE_NAME=helloworld $ k8s_template.yaml.sh | kubectl apply -f -
  77. 77. Still Need Core Infra provider "google" { project = "breakme-europe" region = "europe-west1" } resource "google_container_cluster" "main" { name = "k8s-cluster" zone = "europe-west1-d" initial_node_count = 4 node_config { machine_type = "n1-standard-2" ... } ... }
  78. 78. Fear of Changes
  79. 79. Show, Don’t Tell
  80. 80. 30-40 mins vs 3-4 mins 10x better!
  81. 81. Evolution
  82. 82. Small Changes
  83. 83. Learning Curve?
  84. 84. Show, Don’t Tell Be Patient
  85. 85. Tooling Issues?
  86. 86. Centralize?
  87. 87. Centralize?
  88. 88. Centralize? Vagrant?
  89. 89. Centralize? Vagrant? Package Repository?
  90. 90. Leverage Build System build.gradle buildscript { dependencies { classpath "com.roku:henka:1.0.0-RELEASE" } } task terraformPlan(type: TerraformTask) { description "Runs a terraform script" tfDir = "${projectDir}/terraform" tfAction = "plan -input=false" terraformBaseDir = "/opt/terraform" terraformVersion = "0.11.11" } // ...
  91. 91. Leverage Build System $ ./gradlew build $ ./gradlew terraformPlan
  92. 92. Best Practices Unified Build Logic
  93. 93. Best Practices Unified Build Logic Unified Monitoring
  94. 94. Best Practices Unified Build Logic Unified Monitoring Code Reviews
  95. 95. Best Practices Unified Build Logic Unified Monitoring Code Reviews CI/CD
  96. 96. Best Practices Unified Build Logic Unified Monitoring Code Reviews CI/CD No Documentation
  97. 97. Deploy Faster!
  98. 98. Terraform https://www.terraform.io 12 factor app https://12factor.net/ Kubernetes https://kubernetes.io/ Gradle https://gradle.org/ Henka https://github.com/roku-oss/henka
  99. 99. Twitter @mykyta_p Slides http://devopsfest2019.protsenko.com Sources https://github.com/iac-demo

    Be the first to comment

    Login to see the comments

Ops are overwhelmed with support. Devs are mad because their cannot deploy the changes as fast as they want. Sounds familiar? Infrastructure-as-code can make your life easier by empowering developers and reducing operations' routine toil. It can cut down the lead time for infrastructure provisioning from hours or even days to minutes. This talk reviews several IaC tools and approaches, showing how to integrate them into continuous delivery pipeline. It covers the problems and challenges that engineers may face while working with infrastructure-as-code tools and provides a few hands-on recipes to address them.

Views

Total views

95

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×