Today’s modern infrastructure allows product teams to take full advantage of “infrastructure-as-code” and deliver value to their customers faster through a seamless & smart delivery pipeline.This delivery pipeline is built using AWS and 3rd party tools such as CloudFormation, Lambda, Terraform, Jenkins, Beanstalk, CodeDeploy, Ansible, and Docker. In the presentation we will walk you through the best practices of combining all the above into a “smart-delivery-pipeline” for your team. By Oron Adam, Emind CTO

1. Automated Agile Infrastructure
Do It Yourself
Oron Adam
CTO & Architect
Emind Cloud Experts

2. @oronoa
Started as a Developer
CTO of Emind Cloud Experts
Love to code AWS infrastructure and other things as well ...
About Myself

5. Key Challenges for Ops of Agile Product Teams
● Dev, Staging & Production Parity
● Let Dev experiment and fail fast
● Innovate while reducing risk
● Continuous uninterrupted delivery of service (and new features)
● Feedback loops, automation, metrics and goals

6. Dev/Prod Parity
● Keep development staging and production as identical as possible
● Backing services same across environments

7. Server Immutability
Phoenix servers
Fast Server Deployment
Starting from machine images
Fails sometimes, not a big deal
It’s testable and thus more stable

8. Automation Critical
Needs to be 100% reproducible
Configure Server and SW automatically
Be able test success / failure of server startup

10. AWS CloudFormation
● Declarative Language
● Integrates with git or any VC system
● Simple JSON Format
● Supports Templates and Stacks
● Supports All AWS resource types
● Part of the CI/CD Pipeline

11. AWS Cloud revolution

13. >>> from troposphere import Ref, Template
>>> import troposphere.ec2 as ec2
>>> t = Template()
>>> instance = ec2.Instance("myinstance")
>>> instance.ImageId = "ami-951945d0"
>>> instance.InstanceType = "t1.micro"
>>> t.add_resource(instance)
<troposphere.ec2.Instance object at 0x101bf3390>
>>> print(t.to_json())
{
"Resources": {
"myinstance": {
"Properties": {
"ImageId": "ami-951945d0",
"InstanceType": "t1.micro"
},
"Type": "AWS::EC2::Instance"
}
}
}
Can be programmatically generated

14. Cloud Formation Dev Workflow
Code infrastructure parallel to App
Create and Test
Iterate

15. Bootstrapping Applications & Handling Update
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[
"#!/bin/bash -ex","n",
"yum -y install gcc-c++ make","n",
"yum -y install mysql-devel sqlite-devel","n",
"yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","n",
"gem install --no-ri --no-rdoc rails","n",
"gem install --no-ri --no-rdoc mysql","n",
"gem install --no-ri --no-rdoc sqlite3","n",
"rails new myapp","n",
"cd myapp","n",
"rails server -d","n"]]}}
}
}

16. CloudFormation app deployment best practices
Use AWS::CloudFormation::Init
Use IAM roles to securely download software and data
Use Amazon CloudWatch logs for debugging
Use cfn-hup for updates
Use custom AMIs to minimize application boot times

17. Bootstrapping
Applications & Handling
Update
"Resources": {
"MyInstance": {
"Type": "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
:
},
"groups" : {
:
},
"users" : {
:
},
"sources" : {
:
},
"files" : {
:
},
"commands" : {
:
},
"services" : {
:
}
}
}
},
"Properties": {
:
}
}
}
AWS::CloudFormation::Init

18. "files" : {
"/tmp/setup.mysql" : {
"content" : { "Fn::Join" : ["", [
"CREATE DATABASE ", { "Ref" : "DBName" }, ";n",
"CREATE USER '", { "Ref" : "DBUsername" }, "'@'localhost' IDENTIFIED BY '",
{ "Ref" : "DBPassword" }, "';n",
"GRANT ALL ON ", { "Ref" : "DBName" }, ".* TO '", { "Ref" : "DBUsername" },
"'@'localhost';n",
"FLUSH PRIVILEGES;n"
]]},
"mode" : "000644",
"owner" : "root",
"group" : "root"
}
},
Creating Files

19. "rpm" : {
"epel" : "http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm"
},
"yum" : {
"httpd" : [],
"php" : [],
"wordpress" : []
},
"rubygems" : {
"chef" : [ "0.10.2" ]
}
Installing Packages

20. "services" : {
"sysvinit" : {
"nginx" : {
"enabled" : "true",
"ensureRunning" : "true",
"files" : ["/etc/nginx/nginx.conf"],
"sources" : ["/var/www/html"]
},
"php-fastcgi" : {
"enabled" : "true",
"ensureRunning" : "true",
"packages" : { "yum" : ["php", "spawn-fcgi"] }
},
"sendmail" : {
"enabled" : "false",
"ensureRunning" : "false"
}
}
}
Setting up running services

21. "sources" : {
"/etc/myapp" : "https://s3.amazonaws.com/mybucket/myapp.tar.gz"
}
"sources" : {
"/etc/puppet" : https://github.com/user1/cfn-demo/tarball/master
}
Downloading the app

22. AWS Cloud
Formation
CF Stack EC2 Instance
cfn-init
Get Metadata
Run actions
cfn-init

23. AWS Cloud Formation CF Stack EC2 Instance
cfn-signal
send signal on
setup done
cfn-signal

24. "AutoScalingGroup": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": { "Fn::GetAZs": "" },
"LaunchConfigurationName": { "Ref": "LaunchConfig" },
"DesiredCapacity": "3",
"MinSize": "1",
"MaxSize": "4"
},
"CreationPolicy": {
"ResourceSignal": {
"Count": "3",
"Timeout": "PT15M"
}
},
"UpdatePolicy" : {
"AutoScalingScheduledAction" : {
"IgnoreUnmodifiedGroupSizeProperties" : "true"
},
"AutoScalingRollingUpdate" : {
"MinInstancesInService" : "1",
"MaxBatchSize" : "2",
"PauseTime" : "PT1M",
"WaitOnResourceSignals" : "true"
}
}
},
"LaunchConfig": {
"Type": "AWS::AutoScaling::LaunchConfiguration",
"Properties": {
"ImageId": "ami-16d18a7e",
"InstanceType": "t2.micro",
"UserData": {
"Fn::Base64": {
"Fn::Join" : [ "", [
"#!/bin/bash -xen",
"yum update aws-cfn-bootstrapn",
"/opt/aws/bin/cfn-signal -e 0 --stack ", { "Ref": "AWS::StackName" },
" --resource AutoScalingGroup ",
" --region ", { "Ref" : "AWS::Region" }, "n"
] ]
}
}
}
}
Wait for completion with
Creation Policy

25. Cloud Formation Functions
"Properties" : {
"MyMyLBDNSName" : {
"Fn::GetAtt" : [ "MyLoadBalancer", "DNSName" ]
}
}
Fn::Base64
Conditional Functions
Fn::And, Fn::Equals, Fn::If, Fn::Not, Fn::Or
Fn::FindInMap
Fn::GetAtt
Fn::GetAZs
Fn::Join
Fn::Select
Ref

26. External Wait Conditions
"myWaitHandle" : {
"Type" : "AWS::CloudFormation::WaitConditionHandle",
"Properties" : {
}
}
"myWaitCondition" : {
"Type" : "AWS::CloudFormation::WaitCondition",
"DependsOn" : "Ec2Instance",
"Properties" : {
"Handle" : { "Ref" : "myWaitHandle" },
"Timeout" : "3600"
}
}

27. External Wait Conditions
"UserData" : {
"Fn::Base64" : {
"Fn::Join" : [ "", ["SignalURL=", { "Ref" : "myWaitHandle" } ] ]
}
}
curl -X PUT -H 'Content-Type:' --data-binary '{"Status" : "SUCCESS","Reason" : "Configuration Complete","
UniqueId" : "ID1234","Data" : "Application has completed configuration."}' "https://cloudformation-
waitcondition-test.s3.amazonaws.com/arn%3Aaws%3Acloudformation%3Aus-east-1%3A034017226601%3Astack%2Fstack-
gosar-20110427004224-test-stack-with-WaitCondition--VEYW%2Fe498ce60-70a1-11e0-81a7-5081d0136786%
2FmyWaitConditionHandle?
Expires=1303976584&AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Signature=ik1twT6hpS4cgNAw7wyOoRejVoo%3D"
SignalURL

28. AWS Cloud Formation CF Stack
B
cfn-signal
WaitCondition
Count : 2
cfn-signal
cfn-signal
A

29. "Parameters" : {
"EnvType" : {
"Description" : "Environment type.",
"Default" : "test",
"Type" : "String",
"AllowedValues" : ["prod", "dev", "test"],
"ConstraintDescription" : "must specify prod, dev, or test."
}
},
Parameters for Dev / Prod / Test

30. "Conditions" : {
"CreateProdResources" : {"Fn::Equals" : [{"Ref" : "EnvType"}, "prod"]},
"CreateDevResources" : {"Fn::Equals" : [{"Ref" : "EnvType"}, "dev"]}
},
"Resources" : {
"EC2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"InstanceType" : { "Fn::If" : [
"CreateProdResources",
"c1.xlarge",
{"Fn::If" : [
"CreateDevResources",
"m1.large",
"m1.small"
]}
]}
}
},
Conditionals

31. AWS Cloud Formation
Template
Prod Stack
Dev, Stage, Prod
Dev Stack
Prod
Dev

32. Customer Stories

33. Management VPC peered to all other VPCs
Per Customer VPC
Need to be able to quickly setup new accounts
Per Environment Application Stack
for each dev, staging, production envs
Services Oriented Architecture (SOA) in the
insurance industry.

35. Cloud Formation
management VPC
Access VPC
VPC for dev, staging, prod
Lambda - CIDR and subnet calculations
Peering of VPCs requires inter stack
communication via outputs

36. "NetworkSubnetFunction": {
"Type": "AWS::Lambda::Function",
"DependsOn": [
"LambdaBasicExecutionRole",
"LambdaBasicExecutionPolicy"
],
"Properties": {
"Code": {
"S3Bucket": "ENTER BUCKER NAME HERE",
"S3Key": "network/get_subnets_from_cidr.zip"
},
"Role": {
"Fn::GetAtt": ["LambdaBasicExecutionRole", "Arn"]
},
"Timeout": 60,
"Handler": "get_subnets_from_cidr.get_subnets_from_cidr",
"Runtime": "python2.7",
"MemorySize": 128
}
},

37. def get_subnets_from_cidr(event, context):
"""
Calculates subnets for a given CIDR and returns the result as a Custom Resource
Response Object to CloudFormation.
Arguments (encapsulated in the 'event' argument by CloudFormation):
cidr - the CIDR from which to extract the subnets in slash notation ('172.16.0.0
/16').
number_of_subnets - the number of subnets to extract. Must be at least 1.
subnet_size - the number of hosts in each extracted subnet. Must be a power of 2.
start_from - return the desired number of subnets starting at this subnet number.
Optional.
Example:
event = {
...
"ResourceProperties": {
"cidr": "172.16.0.0/16",
"number_of_subnets": "4",
"subnet_size": "256",
"start_from": "3"
}
}
result = {
"Id0": "172.16.2.0/24",
"Id1": "172.16.3.0/24",
"Id2": "172.16.4.0/24",

38. "GetVpcSubnetsCustomEvent": {
"Type": "Custom::GetVpcSubnetsCustomEvent",
"Properties": {
"ServiceToken": { "Fn::GetAtt" : ["NetworkSubnetFunction", "Arn"] },
"cidr": {
"Ref": "DevelopmentVPCCIDR"
},
"number_of_subnets": 15,
"subnet_size": 256
}
},

39. "WebDevelopmentSubnetA": {
"Type": "AWS::EC2::Subnet",
"DeletionPolicy": "Delete",
"Properties": {
"VpcId": {
"Ref": "DevelopmentVPC"
},
"MapPublicIpOnLaunch": "true",
"AvailabilityZone": {
"Fn::GetAtt": ["StackOutputInfo", "AZ1"]
},
"CidrBlock": { "Fn::GetAtt" : ["GetVpcSubnetsCustomEvent", "Id0"] },
"Tags": [{
"Key": "Name",
"Value": {
"Fn::Join" : [ "", [ "development-web-", { "Fn::GetAtt": ["StackOutputInfo", "AZ1"] } ] ]
}
}]
}
},

41. What packer fixes?
Images are produced from templates
Images are maintainable
Images are cross cloud provider
Automated, Repeatable and Fast
Can also produce Vagrant Boxes for Dev

42. packer use cases
Continuous Delivery
Dev / Prod Parity
Fast scaling out with non vanilla images
Demo images for multiple cloud providers

43. packer providers
AWS
GCE
Digital Ocean
VMWare
VirtualBox / Vagrant
Docker

44. packer & CloudFormation together
Pre bake images with packer form cmd line
Can be easily initiated from CI server
parameterize AMI ids and give to CF stack run

45. packer & CloudFormation together
AMI is built at pack time, not scale time
Can still use CloudInit to build, just happens at pack-time, not scale
time
No reliance on any external services to bring another server online
Updated AMI is a command-line statement away
Can pack local VMs using identical code

46. Customer Stories

47. Using TerraForm for agile scrum environments
Env is created for scum duration
Env destroyed when done

48. TerraForm
Execution Plans - see what’s going to happen
Relationships & Dependencies - reference other resources
State - get resources to desired state
Multiple Providers - on many clouds

49. TerraForm Basics
provider "aws" {
access_key = "ACCESS_KEY_HERE"
secret_key = "SECRET_KEY_HERE"
region = "us-east-1"
}
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t1.micro"
}

50. TerraForm Basics
resource "aws_elb" "frontend" {
name = "frontend-load-balancer"
listener {
instance_port = 8000
instance_protocol = "http"
lb_port = 80
lb_protocol = "http"
}
instances = ["${aws_instance.app.*.id}"]
}
resource "aws_instance" "app" {
count = 5
ami = "ami-408c7f28"
instance_type = "t1.micro"
}

51. TerraForm Plan
$ terraform plan
...
+ aws_instance.example
ami: "" => "ami-408c7f28"
availability_zone: "" => "<computed>"
instance_type: "" => "t1.micro"
key_name: "" => "<computed>"
private_dns: "" => "<computed>"
private_ip: "" => "<computed>"
public_dns: "" => "<computed>"
public_ip: "" => "<computed>"
security_groups: "" => "<computed>"
subnet_id: "" => "<computed>"

52. Terraform Apply
$ terraform apply
aws_instance.example: Creating...
ami: "" => "ami-408c7f28"
instance_type: "" => "t1.micro"
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
...

53. Terraform Show (State)
$ terraform show
aws_instance.example:
id = i-e60900cd
ami = ami-408c7f28
availability_zone = us-east-1c
instance_type = t1.micro
key_name =
private_dns = domU-12-31-39-12-38-AB.compute-1.internal
private_ip = 10.200.59.89
public_dns = ec2-54-81-21-192.compute-1.amazonaws.com
public_ip = 54.81.21.192
security_groups.# = 1
security_groups.0 = default
subnet_id = ….

54. Key Takeaways
It’s all code - Infra just as app
Automate all the things
Leverage the power of Lambda and CloudFormation
Talk to us, we can help...

55. Thank you