Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TIAD : Automating the aplication lifecycle

2,397 views

Published on

Integrating Jenkins, Packer and CloudFormation

  • Be the first to comment

TIAD : Automating the aplication lifecycle

  1. 1. Automating the application lifecycle 19 mars 2015 . #TIAD . @tiadparis
  2. 2. # TIAD@ tiadparis Who am I? 2 Laurent Bernaille @d2si • Linux background • Cloud enthousiast • Opensource advocate • Love discovering, building (and breaking…) new things • Passionate about the ongoing IT transformations @lbernail
  3. 3. # TIAD@ tiadparis Story behind this talk/demo 3 A classic company with a « standard » on-premise IT New business objectives in a competitive space: IOT Creation of small, independant start-up teams Use of Amazon Web Services for agility and scalability An opportunity to use a new application delivery process
  4. 4. # TIAD@ tiadparis Automating the application lifecyle: Objectives 4 Integrate a new application in hours instead of days Create a new environment in minutes instead of month Deploy a new version of the application in minutes instead of hours
  5. 5. # TIAD@ tiadparis What it was (is?) like 5 DEV Version Control System Continuous Integration Analyse code Build Repository Push code to VCS Traditional Continuous integration • Build a binary application artifact • WAR, JAR, RPM, DEB, ZIP Artifact Traditional Deployment • Build environments • Deploy the artifact in environments OPS Configured server(s) Staging Configured server(s) Production We can do better!
  6. 6. # TIAD@ tiadparis Demo outline 6 1. Infrastructure build 1. Creation of application back-ends 2. Deployment of the application 3. New release
  7. 7. # TIAD@ tiadparis Step 1: Build infrastructure automatically 7 Build a full environment from scratch • Subnets • Routing tables • Technical services
  8. 8. # TIAD@ tiadparis Step 1: Build infrastructure automatically 7 CloudFormation Build a full environment from scratch • Subnets • Routing tables • Technical services Cloudformation • Describe infrastructure components • Build them • « puppet » for infrastructure • AWS only: see also Terraform / Heat
  9. 9. # TIAD@ tiadparis Step 1: Build infrastructure automatically 7 CloudFormation Build a full environment from scratch • Subnets • Routing tables • Technical services eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets Cloudformation • Describe infrastructure components • Build them • « puppet » for infrastructure • AWS only: see also Terraform / Heat
  10. 10. # TIAD@ tiadparis Step 1: Build infrastructure automatically 7 CloudFormation NAT NATBastion Build a full environment from scratch • Subnets • Routing tables • Technical services eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets Cloudformation • Describe infrastructure components • Build them • « puppet » for infrastructure • AWS only: see also Terraform / Heat
  11. 11. # TIAD@ tiadparis Step 1: Build infrastructure automatically 7 CloudFormation NAT NATBastion Build a full environment from scratch • Subnets • Routing tables • Technical services eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets Why automate these rare actions? • Avoid errors • Create identical environments (Production, Staging) easily • Ability to deploy somewhere else very quickly • Not so rare • Easier to update Cloudformation • Describe infrastructure components • Build them • « puppet » for infrastructure • AWS only: see also Terraform / Heat
  12. 12. # TIAD@ tiadparis Under the hood 8 NAT NATBastion eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets "VPC" : { "Type" : "AWS::EC2::VPC", "Properties" : { "CidrBlock" : "10.100.0.0/16" } ”PublicSubnet1" : { "Type" : "AWS::EC2::Subnet", "Properties" : { "VpcId" : { "Ref" : "VPC" }, "AvailabilityZone" : "eu-west-1a" "CidrBlock” : "10.100.1.0/25” } } "NatInstance1" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : "ami-6e7bd919", "InstanceType" : "t2.micro" "SourceDestCheck" : "False", "UserData” : { "Fn::Base64" : { "Fn::Join" : [”n", [ "#!/bin/bash", "echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf", "sysctl -p, "iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE", "iptables-save > /etc/sysconfig/iptables" ]]}} } }
  13. 13. # TIAD@ tiadparis Step 2: Build application components 9 Build all the backends • Databases • Buckets • Cache servers • Queues & Topics NAT NATBastion eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets
  14. 14. # TIAD@ tiadparis Step 2: Build application components 9 CloudFormation Build all the backends • Databases • Buckets • Cache servers • Queues & Topics NAT NATBastion eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets DynamoDB DynamoDB • AWS NoSQL database • « Cassandra as a service »
  15. 15. # TIAD@ tiadparis Step 2: Build application components 9 CloudFormation Build all the backends • Databases • Buckets • Cache servers • Queues & Topics NAT NATBastion eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets Why separate from servers where the application run? • Different lifecycle • Can be shared between releases DynamoDB DynamoDB • AWS NoSQL database • « Cassandra as a service »
  16. 16. # TIAD@ tiadparis Difficulty: managing variables 10 VPC Addresses Subnets Instance types Infra template
  17. 17. # TIAD@ tiadparis Difficulty: managing variables 10 VPC Addresses Subnets Instance types Infra template Parameters
  18. 18. # TIAD@ tiadparis Difficulty: managing variables 10 VPC Addresses Subnets Instance types Infra template Parameters Backend template VPC Id Subnet Ids DB Name
  19. 19. # TIAD@ tiadparis Difficulty: managing variables 10 VPC Addresses Subnets Instance types Infra template Parameters Backend template VPC Id Subnet Ids DB Name
  20. 20. # TIAD@ tiadparis Difficulty: managing variables 10 VPC Addresses Subnets Instance types Infra template Parameters Backend template VPC Id Subnet Ids DB Name ? ?
  21. 21. # TIAD@ tiadparis Difficulty: managing variables 10 VPC Addresses Subnets Instance types Infra template Parameters Backend template VPC Id Subnet Ids DB Name ? ? Wrapper to manage inputs/outputs Outputs
  22. 22. # TIAD@ tiadparis Step 3: Deploy application 11 Deploy application • Load-balancers • Servers • DNS Alias NAT NATBastion eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets DynamoDB
  23. 23. # TIAD@ tiadparis Step 3: Deploy application 11 CloudFormation Deploy application • Load-balancers • Servers • DNS Alias NAT NATBastion eu-west-1a Public subnets Private subnets eu-west-1b Public subnets Private subnets DynamoDB WEB WEB Load-balancer included to allow deployment of several versions Web server is built from an image with application pre-installed
  24. 24. # TIAD@ tiadparis Difficulty: managing variables, again 12 VPC Addresses Subnets Instance types Infra template Backend template VPC Id Subnet Ids DB Name Outputs Application template Instance Types DB Name
  25. 25. # TIAD@ tiadparis Difficulty: managing variables, again 12 VPC Addresses Subnets Instance types Infra template Backend template VPC Id Subnet Ids DB Name Outputs Application template Instance Types DB Name WEB DynamoDB ?
  26. 26. # TIAD@ tiadparis Difficulty: managing variables, again 12 VPC Addresses Subnets Instance types Infra template Backend template VPC Id Subnet Ids DB Name Outputs Application template Instance Types DB Name WEB DynamoDB ? "UserData” : { "Fn::Base64" : { "Fn::Join" : ["", [ "#!/bin/bashn", "echo ", { "Ref" : "Properties" } ," >> /var/www/html/application.propertiesn” ]]}} Properties Properties
  27. 27. # TIAD@ tiadparis Updating the application: the old-fashion way 13 DEV Version Control System Continuous Integration Analyse code Build Repository Push code to VCS Traditional Continuous integration • Build a binary application artifact • WAR, JAR, RPM, DEB, ZIP Artifact Traditional Deployment • Use previously built environments • Deploy the artifact OPS Configured server(s) Staging Configured server(s) Production Environments will drift
  28. 28. # TIAD@ tiadparis Updating the application: with config management 14 Environments can still drift Not suited for autoscaling DEV Version Control System Continuous Integration Analyse code Build Repository Push code to VCS Traditional Continuous integration • Build a binary application artifact • WAR, JAR, RPM, DEB, ZIP Artifact Use config management tools • Same recipes across environments • Same artefact across environments • Manual application deployment OPS Configured server(s) Staging Configured server(s) Production
  29. 29. # TIAD@ tiadparis Updating the application: in the cloud 15 Building can take a while Some drift still possible DEV Version Control System Continuous Integration Analyse code Build Repository Push code to VCS Traditional Continuous integration • Build a binary application artifact • WAR, JAR, RPM, DEB, ZIP Artifact Use config management tools • Same recipes across environments • Same artefact across environments • Deploy application at server start OPS Configured server(s) Staging Configured server(s) Production
  30. 30. # TIAD@ tiadparis Updating the application 16 DEV Version Control System Continuous Integration Analyse code Build Repository Push code to VCS New Continuous integration • Build an application artifact • Build a server image Artifact OPS Staging Production
  31. 31. # TIAD@ tiadparis Updating the application 16 DEV Version Control System Continuous Integration Analyse code Build Repository Push code to VCS New Continuous integration • Build an application artifact • Build a server image Artifact OPS Staging Production Provision, Config, deploy Reference templatesBuild Application templates
  32. 32. # TIAD@ tiadparis Updating the application 16 DEV Version Control System Continuous Integration Analyse code Build Repository Push code to VCS New Continuous integration • Build an application artifact • Build a server image Artifact OPS Configured server(s) Staging Configured server(s) Production Provision, Config, deploy Reference templatesBuild Application templates
  33. 33. # TIAD@ tiadparis Demo 17 Push code Integration AWS images AWS Application templates Web hook Packer • Automate the creation of templates • Developped by @mitchellh / @hashicorp
  34. 34. # TIAD@ tiadparis Demo 17 Push code Integration AWS images AWS Application templates Web hook Build WEB WEB Packer • Automate the creation of templates • Developped by @mitchellh / @hashicorp
  35. 35. # TIAD@ tiadparis Packer 18 "builders": [{ "type": "amazon-ebs", "region": ”eu-west-1", "source_ami": "ami-f0b11187", "instance_type": "t2.small", "ssh_username": "ubuntu", "ami_name": "demo-{{isotime "2006-01-02T15-04-05"}}" }], "provisioners": [ { "type": "file", "source": "/jenkins_workspace/site", "destination": "/tmp/www" }, { "type": "shell", "inline": [ "sudo apt-get update", "sudo apt-get install -y apache2 php5 libapache2-mod-php5 php5-curl php5-mysql", "sudo rm /var/www/html/*", "sudo mv /tmp/www/* /var/www/html", "sudo service apache2 restart" ] }]
  36. 36. # TIAD@ tiadparis 19 Push code Continuous Integration Application templates IntegrationPerspectives
  37. 37. # TIAD@ tiadparis 19 Production Push code Continuous Integration Application templates Integration Prod DNS Perspectives
  38. 38. # TIAD@ tiadparis 19 Production Push code Continuous Integration Application templates Integration Prod DNS Perspectives
  39. 39. # TIAD@ tiadparis 19 Production Test Push code Continuous Integration Application templates Integration Prod DNS Perspectives
  40. 40. # TIAD@ tiadparis 19 Production Test Push code Continuous Integration Application templates Integration Prod DNS Perspectives
  41. 41. # TIAD@ tiadparis 19 Production Test Push code Continuous Integration Application templates Integration Prod DNS Perspectives
  42. 42. # TIAD@ tiadparis Key take-aways 20 Everything can be automated Very important change: Immutable servers • New application artifact: Images / Containers • Very challenging for organizations
  43. 43. # TIAD@ tiadparis Thank you @lbernail Fork the code of this demo on github https://github.com/lbernail/demo

×