• Save
AWS CloudFormation and Puppet at PuppetConf - Jinesh Varia
Upcoming SlideShare
Loading in...5
×
 

AWS CloudFormation and Puppet at PuppetConf - Jinesh Varia

on

  • 18,140 views

Jinesh varia - AWS CloudFormation Puppet at PuppetConf

Jinesh varia - AWS CloudFormation Puppet at PuppetConf

Statistics

Views

Total Views
18,140
Views on SlideShare
9,932
Embed Views
8,208

Actions

Likes
47
Downloads
0
Comments
2

29 Embeds 8,208

http://understeer.hatenablog.com 4331
http://puppetlabs.com 2047
http://java.dzone.com 1188
http://blog.rajatpandit.com 268
http://lanyrd.com 96
https://puppetlabs.com 69
http://feeds.feedburner.com 68
http://paper.li 43
http://cloud.dzone.com 16
https://confluence.skunk-works.no 14
http://127.0.0.1 11
http://puppetlabs.iron-point.com 9
http://architects.dzone.com 7
http://us-w1.rockmelt.com 5
http://webcache.googleusercontent.com 5
http://col.xtend.int 5
http://twitter.com 4
http://www.techgig.com 3
http://confluence.matomyrnd.com 3
http://a0.twimg.com 3
https://twitter.com 2
http://www.dzone.com 2
http://planet.mysql.com 2
http://translate.googleusercontent.com 2
http://www.puppetlabs.com 1
http://dev.puppetlabs.com 1
http://confluence.skunk-works.no 1
http://www.diffbot.com&_=1360869206159 HTTP 1
http://confluence.xtend.int 1
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • @nilya Thanks
    Are you sure you want to
    Your message goes here
    Processing…
  • Video of this talk http://youtu.be/xpsEKC0tTGk
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • This is highly relevant in the cloud world because innovation is happening at break neck speed…..
  • Lets put everything in a context of a web application
  • See the animation. DirectConnect
  • Autodesk story – Paul Cochrane…
  • The key advance was using our continuous build system to build not only the artifact from source code, but the complete software stack, all the way up to a deployable image in the form of an AMI (Amazon Machine Image for AWS EC2).
  • Cloud-init supports several different mechanisms for passing data to the instance including ways to pass larger, more structured data and a way to provide a script that is executed at instance launch time.
  • Till now people who wanted to evaluate MCollective had to go through a manual process of starting first the ActiveMQ instance, gathering some data and then start a number of other instances supplying user data for the ActiveMQ instance. This was by no means a painful solution but CloudFormation can make this much better.
  • Remember TVs without remote control. We had to walk up to the television set (hardware) to change the channel or the volume. Now we have remote controls, we sit back relax on our couch and control the hardware with our finger tips. Cloud APIs are remote control of the cloud hardware. You don’t have the need to walk up to your hardware, you don’t have to hug your servers anymore. You let go of your (physical) control but retain your ownership
  • Service OptimizationsArchitectural recommendationsHealth ChecksSecurity Audits

AWS CloudFormation and Puppet at PuppetConf - Jinesh Varia AWS CloudFormation and Puppet at PuppetConf - Jinesh Varia Presentation Transcript

  • Architecting for the Cloud: AWS CloudFormation and Puppet
    Jinesh Varia
  • Scale,
    Pace of Innovation,
    Expansion and
    Ecosystem
  • Cloud Customers in 190 Countries
  • » Amazon EC2 with Windows Server
    2008,
    • Spot Instances,
    • Boot from Amazon EBS
    » Amazon CloudFront Streaming
    » Amazon VPC enters Unlimited Beta
    » AWS Region in Northern California
    » International Support for AWS
    Import/Export
    » AWS Multi-Factor Authentication
    » Virtual Private Cloud
    » Lower Reserved Instance Pricing
    » Reserved Instances in EU Region
    » Elastic MapReduce
    » SQS in EU Region
    » Amazon RDS
    » High-Memory Instances
    » Lower EC2 Pricing
    » New SimpleDB Features
    » FPS General Availability
    » Amazon SNS
    » AWS Security Center
    2009
    Jan
    2010
    Jan
    Jul
    Sep
    Oct
    Dec
    Aug
    Nov
    Feb
    Mar
    Apr
    Jun
    May
    Feb
    Mar
    » Amazon EC2 with Windows
    » Amazon EC2 in EU Region
    » AWS Toolkit for Eclipse
    » Amazon EC2 Reserved
    Instances
    » Amazon CloudFront
    Private Content
    » SAS70 Type II Audit
    » AWS SDK for .NET
    » Amazon Elastic MapReduce
    in Europe
    » Amazon EC2 Reserved Instances
    with Windows, Extra Large High
    Memory Instances
    » Amazon S3 Versioning Feature
    » Consolidated Billing for AWS
    » Lower pricing for Outbound Data
    Transfer
    » AWS Import/Export
    » New CloudFront Feature
    » Monitoring, Auto Scaling & Elastic Load Balancing
    » EBS Shared Snapshots
    » SimpleDB in EU Region
    » Monitoring, Auto Scaling &
    Elastic Load Balancing in EU
    » Lower pricing tiers for
    Amazon CloudFront
    » AWS Management Console
    The pace of innovation in 2009
  • » Free Monitoring EC2
    » Amazon Route 53
    » PCI DSS Level 1 Certification
    » Mobile SDKs (Android, iPhone)
    » Large Object S3 Support
    » Florida POP
    » Import/Export APAC
    » Amazon SNS
    » Combined AWS Data Transfer Savings
    » Amazon EMR Bootstrap Actions
    » Amazon ELB Session Stickiness
    » Amazon RDS in EU
    » New Singapore Region
    » RDS Reserved
    » CloudFront Default Root
    » Startup Challenge 2010
    » CloudFront Invalidation
    » AWS Elastic Beanstalk
    » Amazon Simple Email Service
    » Improved AWS Support “Bronze”
    » Amazon CloudWatch Console
    » CloudFront HTTPS
    » NYC Edge Location
    » Lowers Pricing HTTP
    » AWS Import Export GA
    » Amazon SNS
    » Amazon S3 Console
    » Amazon EBS CloudWatch
    » Amazon RDS Read Replicas
    » Suse EC2 Linux
    » Amazon SNS Console
    » Amazon ELB HTTPS
    » AWS Free Tier
    » EMR Resizing Cluster
    » EMR JobFlow Debugging
    » Simple DB Consistent Reads
    » Simple DB Conditional Puts
    » VM Connector
    » Tokyo Region
    » AWS Support JP
    2010
    Jan
    2011
    Jan
    Jul
    Sep
    Oct
    Dec
    Aug
    Nov
    Feb
    Mar
    Apr
    Jun
    May
    Feb
    Mar
    » New VPC
    » Dedicated Instances
    » Windows
    2008 R2
    » Amazon S3 Lowered Pricing
    » CloudFront GA, SLA
    » S3 Multipart
    » GPGPU Instance Types
    » ISO27001/2 Certification
    » Amazon SQS Longer retention, Free Tier
    Amazon S3 Bucket Policies
    » Amazon VPC IP Address
    » Cluster Compute Instances
    » Amazon S3 RRS Notifications
    » AWS Java SDK
    » Windows BYOL
    » Singapore Pop
    » CloudFront Private Streaming
    » Lowered Pricing EC2
    » AWS IAM
    » Amazon VPC Console
    » Micro Instances
    » Amazon Linux AMI
    » Amazon EC2 Tagging, Filtering, Idempotency,
    » Oracle Certified AWS
    » AWS PHP SDK
    » AWS CloudFormation
    » Amazon S3 Static Websites
    » AWS IAM Website Login
    » Paris Edge Location
    » Amazon EC2 Reserved Instances
    with Windows, Extra Large High
    Memory Instances
    » Amazon S3 Versioning Feature
    » Consolidated Billing for AWS
    » Lower pricing for Outbound Data
    Transfer
    » VPC in EU
    » Amazon RDS in US-west
    » Amazon CloudFront Access Logs
    » Amazon RDS Multi-AZ
    » Amazon S3 RRS
    » Amazon RDS Console
    And pace accelerates in 2010….
  • “Every day is a launch day”
    » On-demand Red Hat
    » Stockholm Edge Location
    » AWS Elastic Beanstalk new enhancements
    » New Data Transfer pricing
    » Free Inbound Data Transfer
    » Spot Integration with HPC instances
    » Amazon EMR in APAC
    » AWS Mobile SDKs
    » Live Streaming with CloudFront
    » AWS IAM GA
    » AWS IAM Web Console
    » AWS Import/Export for EBS
    » AWS CloudFormation new features
    » AWS SDK for Ruby
    » Attachment support for Amazon SES
    » AWS Startup Challenge goes global
    » AWS DirectConnect
    » Amazon VPC Everywhere
    » Mulit-AZ VPC
    » AWS IAM Identity Federation
    » AWS toolkit of eclipse 2.0
    » AWS GovCloud US
    » Spot in Amazon EMR
    » Amazon ElastiCache
    » Amazon VM import Win2k3
    » VM Connector
    » Tokyo Region
    » AWS Support JP
    » AWS IAM for CloudFront
    » VPC Virtual Networking
    » VPC Internet Access
    2011
    Jan
    2012
    Jan
    Jul
    Sep
    Oct
    Dec
    Aug
    Nov
    Feb
    Mar
    Apr
    Jun
    May
    Feb
    Mar
    » AWS CloudFormation
    » Amazon S3 Static Websites
    » AWS IAM Website Login
    » Paris Edge Location
    » Amazon Route53
    » New VPC
    » Dedicated Instances in VPC
    » Windows 2008 R2
    » New AZ in JP
    » AWS IAM GA
    » AWS IAM Web Console
    » AWS Beanstalk Tomcat 7 Support
    » Amazon CloudWatch Custom Metrics
    » Amazon CloudWatch lower pricing
    » AWS SAP Certification
    » Amazon RDS for Oracle
    » Amazon ELB ipv6 support, Zone Apex
    » Amazon ELB Security Group integration
    » Amazon Route53 GA, ELB integration
    » Amazon Route 53 Weighted RR
    » New pricing control for Spot
    » AWS CloudFormation new enhancements
    » AWS Mobile SDK GA
    » AWS Toolkit of Visual Studio
    » AWS DirectConnect USWest Location
    » AWS Elastic Beanstalk
    » Amazon Simple Email Service
    » Improved AWS Support “Bronze”
    » Amazon CloudWatch Console
  • Each day, AWS adds the equivalent server capacity to power Amazon when it was a global, $2.76B enterprise (circa 2000)
  • GovCloud-US
    US West
    (Northern California)
    US East
    (Northern Virginia)
    Europe West
    (Dublin)
    Asia Pacific Region
    (Singapore)
    Asia Pacific Region
    (Japan)
    Ashburn, Dallas, Los Angeles, Miami, Newark, Palo Alto, Seattle, St. Louis, Amsterdam, Dublin, Frankfurt, London, Hong Kong, Singapore, Tokyo, New York, Paris
    Amazon CloudFront
    Edge Locations
  • The AWS Cloud
    Your Application
    Tools to access services
    Libraries and SDKs
    .NET/Java etc.
    Web Interface
    Management Console
    Tools
    AWS Toolkit Eclipse, VS
    Command Line Interface
    Cross Service features
    Auth, Authorization, FederationAWS IAM, MFA
    Monitoring
    Amazon CloudWatch
    Deployment and Automation
    AWS Elastic BeanstalkAWS CloudFormation
    High-level building blocks
    Content Delivery
    Amazon CloudFront
    Email
    Amazon SES
    Payments
    Amazon DevPay
    Amazon FPS
    Parallel Processing
    Amazon Elastic MapReduce
    Messaging
    Amazon SNS
    Amazon SQS
    Workforce
    Amazon Mechanical Turk
    Low-level building blocks
    Compute
    Amazon EC2
    Auto Scaling
    Network
    Amazon VPC,
    ELB, DirectConnect
    Amazon Route 53
    Storage
    Amazon S3
    Amazon EBS
    Database
    Amazon RDS
    Amazon SimpleDB
    Amazon ElastiCache
    Amazon Global Physical Infrastructure
    (Geographical Regions, Availability Zones, Edge Locations)
  • www.yourApp.com
    media.yourApp.com
    (Static data)
    Amazon CloudFront
    Amazon Route 53
    Elastic Load
    Balancer
    Amazon
    CloudWatch
    Amazon S3
    Bucket
    Amazon SNS
    Notifications
    Auto Scaling Group
    Amazon SimpleDB
    App Tier
    Email
    ElastiCache Tier
    Amazon RDS
    AZ-1
    AZ-1
    Region
  • Corporate data center
    Availability Zone 1
    DirectConnect
    Location
    10G
    VPC Subnet
    Router
    VPN Gateway
    Customer Gateway
    Corporate Headquarters
    VPC Subnet
    Internet Gateway
    Amazon VPC
    Availability Zone 2
    Branch Offices
    The New Cloud-Ready Enterprise IT
    Amazon S3
    Amazon SES
    Amazon SimpleDB
    Amazon SQS
    AWS Region
  • The “Living” AWS Cloud
    Your Application
    Tools to access services
    Libraries and SDKs
    .NET/Java etc.
    Web Interface
    Management Console
    Tools
    AWS Toolkit Eclipse, VS
    Command Line Interface
    Cross Service features
    Auth, Authorization, FederationAWS IAM, MFA
    Monitoring
    Amazon CloudWatch
    Deployment and Automation
    AWS Elastic BeanstalkAWS CloudFormation
    High-level building blocks
    Content Delivery
    Amazon CloudFront
    Email
    Amazon SES
    Payments
    Amazon DevPay
    Amazon FPS
    Parallel Processing
    Amazon Elastic MapReduce
    Messaging
    Amazon SNS
    Amazon SQS
    Workforce
    Amazon Mechanical Turk
    Low-level building blocks
    Compute
    Amazon EC2
    Auto Scaling
    Network
    Amazon VPC
    Elastic LB
    Amazon Route 53
    Storage
    Amazon S3
    Amazon EBS
    Database
    Amazon RDS
    Amazon SimpleDB
    Amazon ElastiCache
    Amazon Global Physical Infrastructure
    (Geographical Regions, Availability Zones, Edge Locations)
  • The Need for Speed
  • DevOps
  • Ops = businessOps != businessOps ? business
  • Ops Ξ business
  • Elasticity
    is the
    fundamental
    property
    of the cloud
  • Implement Elasticity
    Elasticity during the day
    25% Savings
  • Implement Elasticity
    Elasticity during the year
    50% Savings
  • Implement Elasticity
    Optimize during the month
    75% Savings
  • Mr. Automate
    Development
    Automate
    Using
    Cloud APIs
    Management
    Logistics
    Monitoring
    Deployment
  • The Automation You Always Meant to Build
    Provision and attach 1TB of storage in 2 minutes (from the back of an auto-rickshaw in India).
    10 new Linux servers in 2 minutes (while sitting by the pool on a nice day).
    Monitoring server resources from an iPhone (in a bar).
    Source: Autodesk
  • AWS CloudFormation“Provision your infrastructure stack using one script”
  • www.yourApp.com
    media.yourApp.com
    (Static data)
    Input Parameters
    Resources
    Outputs
    JSON
    Plain Text
    Perfect for Version Control
    Validate-able
    Mappings
    Custom Metadata
    Amazon CloudFront
    Amazon Route 53
    Elastic Load
    Balancer
    Amazon
    CloudWatch
    JSON Template
    AWS CloudFormation
    Service
    Amazon S3
    Bucket
    Amazon SNS
    Notifications
    Auto Scaling Group
    Atomically creates and
    destroys groups of
    AWS Cloud Resources
    Amazon SimpleDB
    App Tier
    Configures the resources
    Multi-Tier or Multi-AZ stacks
    Manages the ordering
    of provisioning
    Email
    ElastiCache Tier
    Amazon RDS
    AZ-1
    Rolls back in case of failure
    Or issues
    AZ-1
    Region
  • AWS CloudFormation “Stacks”
    JSON Template
  • Declarative language
  • {
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Description" : "Create an EC2 instances",
    "Parameters" : {
    "KeyName" : {
    "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
    "Type" : "String"
    }
    },
    "Mappings" : {
    "RegionMap" : {
    "us-east-1" : {
    "AMI" : "ami-76f0061f"
    },
    "us-west-1" : {
    "AMI" : "ami-655a0a20"
    },
    "eu-west-1" : {
    "AMI" : "ami-7fd4e10b"
    },
    "ap-southeast-1" : {
    "AMI" : "ami-72621c20"
    },
    "ap-northeast-1" : {
    "AMI" : "ami-8e08a38f"
    }
    }
    },
    "Resources" : {
    "Ec2Instance" : {
    "Type" : "AWS::EC2::Instance",
    "Properties" : {
    "KeyName" : { "Ref" : "KeyName" },
    "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
    "UserData" : { "Fn::Base64" : "80" }
    }
    }
    },
    "Outputs" : {
    "InstanceId" : {
    "Description" : "InstanceId of the newly created EC2 instance",
    "Value" : { "Ref" : "Ec2Instance" }
    },
    "AZ" : {
    "Description" : "Availability Zone of the newly created EC2 instance",
    "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] }
    },
    "PublicIP" : {
    "Description" : "Public IP address of the newly created EC2 instance",
    "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] }
    }
    }
    }
  • Headers
    {
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Description" : "Create an EC2 instances",
    "Parameters" : {
    "KeyName" : {
    "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
    "Type" : "String"
    }
    },
    "Mappings" : {
    "RegionMap" : {
    "us-east-1" : {
    "AMI" : "ami-76f0061f"
    },
    "us-west-1" : {
    "AMI" : "ami-655a0a20"
    },
    "eu-west-1" : {
    "AMI" : "ami-7fd4e10b"
    },
    "ap-southeast-1" : {
    "AMI" : "ami-72621c20"
    },
    "ap-northeast-1" : {
    "AMI" : "ami-8e08a38f"
    }
    }
    },
    "Resources" : {
    "Ec2Instance" : {
    "Type" : "AWS::EC2::Instance",
    "Properties" : {
    "KeyName" : { "Ref" : "KeyName" },
    "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
    "UserData" : { "Fn::Base64" : "80" }
    }
    }
    },
    "Outputs" : {
    "InstanceId" : {
    "Description" : "InstanceId of the newly created EC2 instance",
    "Value" : { "Ref" : "Ec2Instance" }
    },
    "AZ" : {
    "Description" : "Availability Zone of the newly created EC2 instance",
    "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] }
    },
    "PublicIP" : {
    "Description" : "Public IP address of the newly created EC2 instance",
    "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] }
    }
    }
    }
    Parameters
    Mappings
    Resources
    Outputs
  • Parameters
    Provision-time specification
    Command line options
  • "Parameters" : {
    "KeyName" : {
    "Description" : "Name of an existing
    EC2 KeyPair to enable SSH access to
    the instance",
    "Type" : "String"
    }
    },
  • Mappings
    Conditionals
    Case statements
  • "Mappings" : {
    "RegionMap" : {
    "us-east-1" : {
    "AMI" : "ami-76f0061f"
    },
    "us-west-1" : {
    "AMI" : "ami-655a0a20"
    },
    "eu-west-1" : {
    "AMI" : "ami-7fd4e10b"
    },
    "ap-southeast-1" : {
    "AMI" : "ami-72621c20"
    },
    "ap-northeast-1" : {
    "AMI" : "ami-8e08a38f"
    }
    }
    },
  • "Mappings": {
    "AWSInstanceType2Arch" : {
    "t1.micro" : { "Arch" : "64" },
    "m1.large" : { "Arch" : "64" },
    "m1.xlarge" : { "Arch" : "64" },
    "m2.xlarge" : { "Arch" : "64" },
    "m2.2xlarge" : { "Arch" : "64" },
    "m2.4xlarge" : { "Arch" : "64" },
    "c1.xlarge" : { "Arch" : "64" },
    "cc1.4xlarge" : { "Arch" : "64" }
    },
  • Dereference this mappings
    "ImageId": {
    "Fn::FindInMap": [
    "AWSRegionArch2AMI",
    {
    "Ref": "AWS::Region"
    },
  • Resources
  • "Resources" : {
    "Ec2Instance" : {
    "Type" : "AWS::EC2::Instance",
    "Properties" : {
    "KeyName" : { "Ref" : "KeyName" },
    "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
    "UserData" : { "Fn::Base64" : "80" }
    }
    }
    }
  • "Resources" : {
    "Ec2Instance" : {
    "Type" : "AWS::EC2::Instance",
    "Properties" : {
    "KeyName" : { "Ref" : "KeyName" },
    "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
    "UserData" : { "Fn::Base64" : "80" }
    }
    }
    }
  • "Resources" : {
    "Ec2Instance" : {
    "Type" : "AWS::EC2::Instance",
    "Properties" : {
    "KeyName" : { "Ref" : "KeyName" },
    "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
    "UserData" : { "Fn::Base64" : "80" }
    }
    }
    }
  • "KeyName" : { "Ref" : "KeyName" },
    Parameter reference
  • "ImageId" : {
    "Fn::FindInMap" :
    [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]
    },
  • Map conditional
    "ImageId" : {
    "Fn::FindInMap" :
    [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]
    },
  • "ImageId" : {
    "Fn::FindInMap" :
    [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]
    },
    Name of map
  • "ImageId" : {
    "Fn::FindInMap" :
    [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]
    },
    Intrinsic property reference
  • Outputs
    Returned values
  • "Outputs" : {
    "InstanceId" : {
    "Description" : "InstanceId of the newly created EC2 instance",
    "Value" : { "Ref" : "Ec2Instance" }
    },
    "AZ" : {
    "Description" : "Availability Zone of the newly created EC2 instance",
    "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "AvailabilityZone" ] }
    },
    "PublicIP" : {
    "Description" : "Public IP address of the newly created EC2 instance",
    "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] }
    }
    }
    }
  • "AppDatabase": {"Type": "AWS::CloudFormation::Stack",
    "Metadata": { … },
    "Properties": {
    "TemplateURL": {
    "Fn::Join": [
    "/",
    [
    { … },
    "RDS_MySQL_55.template"
    ]
    ]
    },
    Embedded Stacks
  • Custom Metadata
    "Resources": {"Ec2Instance": {
    "Type": "AWS::EC2::Instance",
    "Metadata": {
    "Comment": "This metadata is available via the cfn-describe-stack-resource command line tool or the DescribeStackResource API call",
    "MyAMI": { "Fn::FindInMap": [ "RegionMap", { "Ref": "AWS::Region“ }, "AMI"]},
    "MyRegion": {"Ref": AWS::Region"},
    "MyStack": {"Ref": "AWS::StackName"}
    },
    "Properties": {
    "ImageId": {"Fn::FindInMap": ["RegionMap",{"Ref": "AWS::Region"}, AMI"]},
    "UserData": { "Fn::Base64": "80"}
    }
    }
    },
  • Standardized Application Stacks
    Apache
    Apache
    IIS
    Apache
    Mongrel
    Tomcat
    ASP.NET
    Mongrel
    Web Server
    Rails
    Struts
    ASP.NET MVC
    Rails
    App Server
    Your Code
    Your Code
    Your Code
    Your Code
    MVC
    logger
    Log4J
    Log4Net
    logger
    Your Code
    RubyGems
    Spring
    Spring.NET
    RubyGems
    Libraries
    memcached
    Hibernate
    nHibernate
    memcached
    Packages
    Ruby Runtime
    JEE
    .NET
    Ruby Runtime
    DB Caching
    Centos
    Linux
    Windows
    Centos
    Framework
    OS
    Java Stack
    .NET Stack
    RoR stack
  • Bootstrapping Applications with AWS CloudFormation
  • Implement Elasticity
    1. Frozen Pizza Model
    IIS
    IIS
    IIS
    IIS
    Apache
    Apache
    IIS
    IIS
    IIS
    IIS
    Tomcat
    Tomcat
    ASP.NET MVC
    ASP.NET MVC
    ASP.NET MVC
    ASP.NET MVC
    Struts
    Struts
    Your Code
    Your Code
    Your Code
    Your Code
    Your Code
    Your Code
    Log4Net
    Log4Net
    Log4Net
    Log4Net
    Log4J
    Log4J
    Spring.NET
    Spring.NET
    Spring.NET
    Spring.NET
    Spring
    Spring
    nHibernate
    nHibernate
    nHibernate
    nHibernate
    Hibernate
    Hibernate
    .NET
    .NET
    .NET
    .NET
    JEE
    JEE
    Amazon EC2
    Windows
    Windows
    Windows
    Windows
    Linux
    Linux
    Java AMI
    Java Stack
  • Build Job does the following:
    build the artifact,
    publish it to Artifactory,
    build the package,
    publish the package to the repo .
    Then there is a follow on job that mounts a base OS image, installs the packages and then creates the final AMI.
    Source: http://techblog.netflix.com/2011/08/building-with-legos.html
  • Implement Elasticity
    2. Take N Bake Pizza Model
    Apache
    Your Code
    Amazon S3
    Tomcat
    Struts
    Log4J
    Spring
    Fetch on boot time
    Apache
    Struts
    Tomcat
    Source Control
    Hibernate
    Your Code
    JEE
    Linux
    Log4J
    Spring
    IIS
    IIS
    IIS
    IIS
    Hibernate
    IIS
    IIS
    IIS
    IIS
    JEE
    .NET
    .NET
    .NET
    .NET
    Linux
    Amazon EC2
    Windows
    Windows
    Windows
    Windows
    Golden AMI
    Java Stack
  • Cloud-Init and EC2 Instance User Data
    Cloud-init supports several different mechanisms for passing data to the instance including ways to pass larger, more structured data and a way to provide a script that is executed at instance launch time.
    • Amazon Linux AMIs
    • Ubuntu Distributions
  • "UserData": {
    "Fn::Base64": {
    "Fn::Join": [
    "",
    [
    "#!/bin/bash -exn",
    "yum -y install git-coren",
    "yum -y install php-pearn",
    "pear install Crypt_HMAC2-1.0.0n",
    "pear install HTTP_Request-1.4.4n",
    "pear channel-discover pear.amazonwebservices.comn",
    "pear install aws/sdkn",
    Bootstrap using User Data
  • Implement Elasticity
    3. Made to Order Pizza Model
    Amazon S3
    Apache
    Apache
    Struts
    Tomcat
    Log4J
    Hibernate
    Your Code
    Spring
    Tomcat
    Struts
    Cookbooks
    Recipes
    Source Control
    Your Code
    PuppetMaster
    Log4J
    Spring
    Hibernate
    JEE
    PuppetClient
    Agent
    Linux
    Linux
    Windows
    Amazon EC2
    AMI (JeOS)
    Java Stack
  • Instances ask you a question “Who am I and what is my role?”
  • Mcollective + CloudFormation
    • Great for small or large clusters of servers
    • Simple naming conventions
    • Parallel job execution
    • Consistent servers
    Great EC2 Demo : http://docs.puppetlabs.com/mcollective/ec2demo.html
  • Best Practices
    Puppet is great for incremental implementation!
    All modules and manifests should be kept under version control.
    Manage users and groups from the outset.
    Puppet Environments are your friend
    Skinny classes, fat modules.
    Use 'notify' for logging. Make it easy to check logs.
    ‘The Trifecta‘- Use the Package, file, service.
  • Implement Elasticity
    3 approaches to designing your AMIs
    Easier to Setup
    Inventory of fully baked AMIs
    (Frozen/Ready made)
    “Golden AMIs” with fetch on boot
    (Take N’ Bake)
    AMIs with JeOSand PuppetMaster (Made to Order)
    More Control
    Easier to maintain
  • More Tools: CloudFormer
    Create a template from the running resources in your account
    Select the resources that should be included
    Customize the logical names
    Define the template output section
    Creates a starting point template for your to edit
    Add parameters
    Abstract properties and flow properties
    One-click launch in your account
    CloudFormer is an appliance that runs in your account
  • One more thing….
  • Optimizing = Cost Savings
    Free Memory
    Free CPU
    Free HDD
    At 1-min intervals
    PUT
    2 weeks
    Alarm
    Amazon CloudWatch
    Instance
    Custom Metrics
    “You could save a bunch of money by switching
    to a small instance, Click on CloudFormation Script to
    Save”
  • In Summary,
    • Bridge the gap : Ops = business
    • Elasticity is the fundamental property of the cloud and implement elasticity
    • AWS CloudFormation gives you an easy way to create the set of cloud resources
    • 3 Pizza Models 
    • Bootstrapping applications using CloudFormation and Puppet removes the muck
  • Thank you!
    Jinesh Varia
    jvaria@amazon.com Twitter:@jinman
  • Still working……
  • Let go of (physical) control
    but retain your ownership
  • Enterprise Security Features
    Amazon VPC
    AWS Identity And Access Management
    User management
    Policy-based granular access control
    Web login to individual users
    Identity Federation (New!)
    Multi-Factor Authentication
    Services Security features
    Amazon S3 ACL and Bucket policies
    Amazon EC2 Security Groups, iptables
    HTTPS API Endpoints
  • SAS 70 Type II Audit
    ISO 27001/2 Certification
    PCI DSS 2.0 Level 1-5
    HIPAA/SOX Compliance
    FISMA A&A Low
    Encrypt data in transit
    Encrypt data at rest
    Protect your AWS Credentials
    Rotate your keys
    Secure your application
    Enforce IAM policies
    Use MFA, VPC, Leverage S3 bucket policies, EC2 Security groups, EFS in EC2 Etc..
    In the Cloud, Security is a Shared Responsibility
    How we secure our
    infrastructure
    How can you secure your application and what is your responsibility?
    What security options and features are available to you?
  • New World
    Old World
    Build security in every layer
  • mcollective