SlideShare a Scribd company logo
1 of 101
Cloud-Powered   Continuous Integration and Deployment Jinesh Varia [email_address] @jinman
About Me Jinesh Varia @jinman [email_address] linkedin/in/jinman http://jinesh.varia.in
“ Cloud Architectures” “ Cloud Best Practices aws” “ Cloud Migration aws”
Customer is the center of our universe
 
Learning from customers Some Learning Lots of Learning
Learning from customers Little or no Learning
Learning from customers Cycle Time
Learning from customers Requirements gathering Release
Reduce the cycle time Requirements gathering Release
Learn faster Requirements gathering Automation + Cloud Release …  while keeping your costs low
Continuous Integration Continuous Deployment Continuous Optimization Cloud-powered Continuous Delivery
Continuous Integration Cloud-powered Continuous Integration Goal: to have a working state of the code at any given time Benefit: Fix bugs earlier when they are cheaper to fix Metric: New guy can check out and compile at first day at job
Poka yo-ke  ( ポカヨケ )
Martin Fowler Paul Duvall Jez Humble David Farley Matt Graham Michael Nygard … .. … ..
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Automate  Everything Hello, I am  Mr. Automate Development And Build Monitoring Staging and  Production Testing and  Deployment
Application Containers -  JBoss ,  Tomcat ,  IIS ,  Mongrel .  NOTE: there are so many app containers, I'm not going to try to list all of them. Build Tools  -  Ant ,  AntContrib ,  NAnt ,  MSBuild ,  Buildr ,  Gant ,  Gradle ,  make ,  Maven ,  Rake Code Review  -  Crucible Code Insight  -  Fisheye     Continuous Integration -  Bamboo ,  Jenkins ,  AntHill Pro ,  Go ,  TeamCity ,  TFS 2010 Database -  Hibernate ,  MySQL ,  Liquibase ,  Oracle ,  PostgreSQL ,  SQL Server ,  SimpleDB ,  SQL Azure ,  Ant ,  MongoDB Database Change Management  -  dbdeploy ,  Liquibase Data Center Configuration Automation  -  Capistrano ,  Cobbler ,  BMC Bladelogic ,  CFEngine ,  IBM Tivoli Provisioning Manager ,  Puppet ,  Chef ,  Bcfg2 ,  AWS Cloud Formation ,  Windows Azure AppFabric   NOTE: There are many names and overlap for this tool "category".  Dependency Management  -  Ivy ,  Archiva ,  Nexus ,  Artifactory ,  Bundler Deployment Automation  -  Java Secure Channel ,  ControlTier ,  Altiris ,  Capistrano ,  Fabric ,  Func Information Sharing  -  Confluence ,  Google Apps Installer  -  InstallShield ,  IzPack Integrated Development Environment (IDE)  -  Eclipse ,  IDEA ,  Visual Studio Issue Tracking  -  Greenhopper ,  JIRA   Multi-Type  -  rPath Passwords  -  PassPack ,  PasswordSafe Protected Configuration  -  ESCAPE ,  ConfigGen Project Management  -  JIRA ,  Pivotal Tracker ,  SmartSheet Provisioning  -  JEOS ,  BoxGrinder ,  CLIP ,  Eucalyptus ,  AppLogic Reporting/Documentation  -  Doxygen ,  Grand ,  GraphViz ,  JavaDoc ,  NDoc ,  SchemaSpy ,  UmlGraph Static Analysis  -  CheckStyle ,  Clover ,  Cobertura ,  FindBugs ,  FxCop ,  JavaNCSS ,  JDepend ,  PMD ,  Sonar ,  Simian Systems Monitoring  -  CloudKick ,  Nagios ,  Zabbix ,  Zenoss Testing   AntUnit ,  Cucumber ,  DbUnit ,  webrat ,  easyb ,  Fitnesse ,  JMeter ,  JUnit ,  NBehave ,  SoapUI ,  Selenium ,  RSpec , SauceLabs Version-Control System  -  SVN/Subversion ,  git ,  Perforce Paul Duvall’s Blog  http://blog.stelligent.com/integrate-button/2011/03/list-of-software-tools-for-continuous-delivery-in-the-cloud.html
 
Discipline
Version Control Developer <code> CI Server Build Slave Build Slave Build Slave Distributed Builds
EBS Root Volume EC2 Instance AMI EBS Snapshot EBS Volume EC2 Instance createImage() RunInstances() StopInstances() StartInstances() TerminateInstances() Stop v/s Terminate
 
 
Amazon S3 Bucket AZ-1 Region Elastic Load Balancer Amazon  CloudWatch Alarms Amazon SNS Notifications www.yourApp.com Amazon SimpleDB Domains Amazon Route 53 Hosted Zone Auto Scaling Group AZ-1 App Tier ElastiCache Tier Amazon SES Email Amazon CloudFront media.yourApp.com (Static data) Amazon  RDS Amazon  EC2 Instances
AWS CloudFormation JSON Template Version Control Update Stack
Version Control CI Server Package Builder Deploy Server Commit to  Git/master Dev Pull Code AMIs Send Build Report to Dev Stop everything if build failed Distributed Builds Run Tests in parallel Staging Env Test Env Code Config Tests Prod Env Push Config Install Create Repo CloudFormation  Templates for Env Generate Cloud Continuous Integration
[object Object],[object Object],[object Object],[object Object],[object Object],Cloud Continuous Integration
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
“ TestOps”
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Automated Testing in Parallel
PROD Very Large Server Storage Bob Test Very Small Server Storage Ted Test Very Small Server Storage Mary Test Very Small Server Storage 7 AM 6 PM RDS Snapshots – Test something quickly
Continuous Integration Continuous Deployment Cloud-powered Continuous Deployment
Cloud-powered Continuous Deployment Goal: 1-click deploy and 1-click rollback Benefit: Release early, release often and iterate quickly Metric: fast feedback of your feature Continuous Deployment
 
Software Inventory is lost revenue
 
Mean Time between deployments (weekday) Max # of deployments in a single hour Mean # of hosts simultaneously receiving a deployment Max # of hosts simultaneously receiving a deployment Amazon May Continuous Deployment Stats (production hosts and environments only) 11.6 Seconds 1079 10000 30000
The need for  speed
[object Object],[object Object]
Break your problem into small batches Small deployments Incremental changes Easy rollbacks
Virtual Images = Real Productivity Gain Java Stack .NET Stack RoR stack Centos Ruby Runtime Your Code logger RubyGems memcached Rails Mongrel Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Windows .NET  Your Code Log4Net Spring.NET nHibernate ASP.NET MVC ASP.NET IIS Centos Ruby Runtime Your Code logger RubyGems memcached Rails Mongrel Apache OS Framework Your Code Libraries Packages DB Caching MVC App Server Web Server
Push to an AMI or Pull from an Instance Inventory of AMIs Golden AMI and  Fetch binaries on boot JeOS AMI and library of recipes (install scripts) Amazon EC2 Amazon EC2 Linux JEE CHEF Puppet Chef/puppet scripts Java AMI Java App Stack Java AMI JeOS AMI Fetch on boot Fetch on boot Fetch on boot Frozen Pizza Model Take N Bake Pizza Model Made to order Pizza Model Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Amazon EC2 Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Your Code Amazon S3 Log4J Spring Struts Linux JEE Hibernate Tomcat Apache Your Code Amazon S3 Hibernate Tomcat Log4J Spring Struts Apache Linux JEE Hibernate Tomcat Apache Linux JEE Hibernate Tomcat Apache Linux JEE Hibernate Tomcat Apache Linux JEE Linux JEE
Cloud Continuous Deployment (Simple) Version Control CI Server Deploy Server Commit to  Git/master Dev Pull Code Send Build Report to Dev Stop everything if build failed Build Run Tests Code Config Tests CloudFormation Templates Package Builder Push Config Repo Base AMIs Instances Scripts Cloud-init
&quot;UserData&quot;: { &quot;Fn::Base64&quot;: { &quot;Fn::Join&quot;: [ &quot;&quot;, [ &quot;#!/bin/bash -ex&quot;, &quot;yum -y install git-core&quot;, &quot;yum -y install php-pear&quot;, &quot;pear install Crypt_HMAC2-1.0.0&quot;, &quot;pear install HTTP_Request-1.4.4&quot;, &quot;pear channel-discover pear.amazonwebservices.com&quot;, &quot;pear install aws/sdk&quot;, Bootstrap using User Data
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Embedded Stacks
Cloud Continuous Deployment Version Control CI Server Deploy Server Commit to  Git/master Dev Pull Code Send Build Report to Dev Stop everything if build failed Build Run Tests Code Config Tests CloudFormation Templates Chef Server (config, package dependencies) Puppet Master  (Manifest, config and mappings) Chef S Puppet M Push Config New Code Reports Base AMIs Managed Instances Scripts recipes
AWS Region Availability Zone 1 Deploy Server Automate deployment to multiple  Availability Zones (Fault Tolerant Zones) Availability Zone 2 Availability Zone 3
Blue Green Deployments
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 Monitoring (CloudWatch) High Error Rate
Auto Scaling Group  (Min, Max # of instances, Availability Zones .. ) Health Check  (Maintain Min # active…) Launch Configuration  (AMIID, Instance type, UserData, Security Groups..) Scaling Trigger  (Metric, Upper Threshold, Lower Threshold, Time interval …) Types of Scaling  (Scale by Schedule, Scale by Policy) Alarm  (Notification  Email, SMS, SQS, HTTP) Availability Zones and Regions Auto Scaling
“ Auto scaling” Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 Auto scaling Max instances Min instances Scaling Trigger Custom Metrics Upper Threshold Lower Threshold Increment by
Dark Launches with feature flags Deploy != Product Launch
v1.1 v1.1 v1.1 v1.1 Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 Happy Path v1.1 New feature Code Path v1.2 Feature = ON request
Dialing up A B Control Treatment
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) 99% 1% … .. A/B Testing  Service v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 90% 10% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 70% 30% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 70% 30% Monitoring (CloudWatch) High Error Rate Rollback v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 90% 10% Rollback Dev, Test v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 70% 30% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 50% 50% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 30% 70% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing  Service 5% 95% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
Split Users by multiple experiments A B Control Treatment 1 C D Treatment 2 Treatment 3
v1.1 v1.1 v1.1 v1.1 Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 90% 5% v1.2.1 v1.2.1 3% v1.2.2 v1.2.2 2%
Blue Green Deployments : RDBMS? ,[object Object],[object Object],[object Object],[object Object]
Timeline V2.1 Example ID NAME ADDRESS ORDERID (Char) 23234 Joe Doe xxx 333424 45322 Rob Smith xxxx 234 2342342 Jane Smith xxxx 23424 2342265 Anne Lee xxxx 2342425
Timeline V2.1 ,[object Object],[object Object],Example DB V1.1 ID NAME ADDRESS ORDERID ORDERID_INT 23234 Joe Doe xxx 333424 45322 Rob Smith xxxx 234 2342342 Jane Smith xxxx 23424 2342265 Anne Lee xxxx 2342425
Deployment Timeline V2.1 ,[object Object],[object Object],[object Object],Example DB V1.1 ID NAME ADDRESS ORDERID ORDERID_INT 23234 Joe Doe xxx 333424 45322 Rob Smith xxxx 234 2342342 Jane Smith xxxx 23424 2342265 Anne Lee xxxx 2342425 3632342 Mark Fox cxxxx 567890 567890
Deployment Timeline V2.1 ,[object Object],[object Object],[object Object],[object Object],Example DB V1.1 ID NAME ADDRESS ORDERID ORDERID_INT 23234 Joe Doe xxx 333424 333424 45322 Rob Smith xxxx 234 234 2342342 Jane Smith xxxx 23424 23424 2342265 Anne Lee xxxx 2342425 2342425 3632342 Mark Fox cxxxx 567890 567890 DB V1.2
Deployment Timeline V2.1 ,[object Object],[object Object],[object Object],[object Object],[object Object],V2.2 Example DB V1.1 ID NAME ADDRESS ORDERID ORDERID_INT 23234 Joe Doe xxx 333424 333424 45322 Rob Smith xxxx 234 234 2342342 Jane Smith xxxx 23424 23424 2342265 Anne Lee xxxx 2342425 2342425 3632342 Mark Fox cxxxx 567890 567890 DB V1.2
Deployment Timeline V2.1 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],V2.2 V2.3 Example DB V1.1 ID NAME ADDRESS ORDERID ORDERID_INT 23234 Joe Doe xxx 333424 333424 45322 Rob Smith xxxx 234 234 2342342 Jane Smith xxxx 23424 23424 2342265 Anne Lee xxxx 2342425 2342425 3632342 Mark Fox cxxxx 567890 567890 433453 Mark Lee cxxxxx 4352342422 DB V1.2
Deployment Timeline V2.1 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],V2.2 V2.3 Example DB V1.1 ID NAME ADDRESS ORDERID_INT 23234 Joe Doe xxx 333424 45322 Rob Smith xxxx 234 2342342 Jane Smith xxxx 23424 2342265 Anne Lee xxxx 2342425 3632342 Mark Fox cxxxx 567890 433453 Mark Lee cxxxxx 4352342422 DB V1.2 DB V1.2
Increasing the speed of Iteration
Break your problem into small batches Stream of small deployments Incremental changes Easy rollbacks
Deploy new software quickly Revert a bad change quickly Cost of  Mistakes
Dynamism of the cloud makes its easy
Small Application Layer Large CPU Week 1 Large CPU Week 1 Week 2 Small Week 1 Week 2 Week 3 Small Week 1 Week 2 Week 3 Week 4 Small Week 1 Week 2 Week 3 Week 4 Week 5 Week 1 Week 2 Week 3 Week4 Week 5 Week 6 Give me Week1 data Writes Reads
Continuous Integration Continuous Deployment Continuous Optimization Cloud-powered Continuous Delivery
Continuous Optimization Cloud-powered Continuous Optimization Goal: Optimize on multiple dimensions : Cost, Performance, HA, Security, Response Time Benefit on optimize for cost - Immediate recurring cost savings Metric: understanding utilization
When you turn off your cloud resources, you actually  stop paying for them
25 % Savings By the time of day
Availability Zone #2 Availability Zone #1 Auto Scaling group : App Tier Auto Scaling group : Web Tier Elastic Load  Balancer www.MyWebSite.com (dynamic data) media.MyWebSite.com  (static data) Amazon Route 53 (DNS) Amazon EC2 Amazon RDS Amazon RDS Amazon S3 Amazon CloudFront
during a year 50 %  Savings
75 %  Savings during a month
Continuous optimization in your architecture and cloud infrastructure results in recurring savings in your  next month’s bill
Instance Amazon CloudWatch Alarm Free Memory Free CPU Free HDD At 1-min intervals Custom Metrics PUT 2 weeks “ You could save a bunch of money by switching  to a  small instance , Click on  CloudFormation Script  to  Save” Cost-aware instances
Choosing the right pricing model On-demand Instances Reserved Instances Spot Instances
Steady State Usage Total Cost for 1 Year-term of 2 application servers Total Cost for 3 Year-term of the same 2 application servers  Usage Fee One-time Fee Total Savings Option 1 On-Demand only $1493 - $1493 - Option 2 On-Demand + Reserved $1008 $227 $1234 ~20% Option 3 All reserved $528 $455 $983 ~35% Usage Fee One-time Fee Total Savings Option 1 On-Demand only $4479 - $4479 - Option 2 On-Demand + Reserved $3024 $350 $3374 ~30% Option 3 All reserved $1584 $700 $2284 ~50%
1-year RI versus On Demand: cost savings realized after first 6 months of usage 3-year RI versus On Demand:  cost savings realized after first 9 months of usage. 3-year RI versus 1-year RI:  Net savings of 3-year RI versus 1-year RI begin by month 13 and continue throughout the RI term (additional 23 months of savings) 1 2 3
Common Pattern: Reserved + On-Demand
Elasticity  is one of the fundamental properties of the cloud that drives many of its economic benefits
Continuous Integration Continuous Deployment Continuous Optimization Cloud-powered Continuous Delivery
Summary  Invest in areas where you get to learn from your customers quickly Automate everything else Cloud Continuous Integration Release early, Release often, Iterate Quickly Get fast feedback  Distributed Builds, Automated Tests in Parallel Cloud Continuous Deployment Reduce the cost of mistakes Increase the speed of iteration Leverage cloud for Blue Green Deployments Cloud Continuous Optimization Keep Optimizing and further reduce costs of  infrastructure
Jinesh Varia [email_address] Twitter: @jinman Credits to Jon Jenkins, Paul Duvall and several engineers at Amazon Thank you!
http://aws.amazon.com
Backup
Optimize by Implementing Elasticity Infrastructure Cost $ time Large Capital Expenditure You just lost customers Predicted Demand Traditional Hardware Actual Demand Cloud Automated Elasticity
Amazon S3 Bucket AZ-1 Region Elastic Load Balancer Amazon  CloudWatch Alarms Amazon SNS Notifications www.yourApp.com Amazon SimpleDB Domains Amazon Route 53 Hosted Zone Auto Scaling Group AZ-1 App Tier ElastiCache Tier Amazon SES Email Amazon CloudFront media.yourApp.com (Static data) Amazon  RDS Amazon  EC2 Instances
Amazon VPC AWS Region VPC Subnet VPC Subnet Corporate data center Corporate Headquarters Availability Zone 1 Availability Zone 2 Branch Offices VPN Gateway Customer Gateway Internet Gateway Router DirectConnect Location Amazon S3 Amazon SimpleDB Amazon SES Amazon SQS The New Cloud-Ready Enterprise IT 10G

More Related Content

What's hot

Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...
Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...
Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...
Amazon Web Services
 

What's hot (20)

AWS 101: Cloud Computing Seminar (2012)
AWS 101: Cloud Computing Seminar (2012)AWS 101: Cloud Computing Seminar (2012)
AWS 101: Cloud Computing Seminar (2012)
 
Moving Enterprise Windows Workloads to AWS
Moving Enterprise Windows Workloads to AWSMoving Enterprise Windows Workloads to AWS
Moving Enterprise Windows Workloads to AWS
 
High Performance Web Applications
High Performance Web ApplicationsHigh Performance Web Applications
High Performance Web Applications
 
Cloud Architecture: Patterns and Best Practices
Cloud Architecture: Patterns and Best PracticesCloud Architecture: Patterns and Best Practices
Cloud Architecture: Patterns and Best Practices
 
AWS Architecting Cloud Apps - Best Practices and Design Patterns By Jinesh Varia
AWS Architecting Cloud Apps - Best Practices and Design Patterns By Jinesh VariaAWS Architecting Cloud Apps - Best Practices and Design Patterns By Jinesh Varia
AWS Architecting Cloud Apps - Best Practices and Design Patterns By Jinesh Varia
 
Managing WorkSpaces at Scale | AWS Public Sector Summit 2016
Managing WorkSpaces at Scale | AWS Public Sector Summit 2016Managing WorkSpaces at Scale | AWS Public Sector Summit 2016
Managing WorkSpaces at Scale | AWS Public Sector Summit 2016
 
VMWare Cloud for the AWS Cloud | AWS Public Sector Summit 2017
VMWare Cloud for the AWS Cloud | AWS Public Sector Summit 2017VMWare Cloud for the AWS Cloud | AWS Public Sector Summit 2017
VMWare Cloud for the AWS Cloud | AWS Public Sector Summit 2017
 
ENT307 VMware and AWS Together - VMware Cloud on AWS
ENT307 VMware and AWS Together - VMware Cloud on AWSENT307 VMware and AWS Together - VMware Cloud on AWS
ENT307 VMware and AWS Together - VMware Cloud on AWS
 
Operational Best Practices in the Cloud
Operational Best Practices in the CloudOperational Best Practices in the Cloud
Operational Best Practices in the Cloud
 
Unique Ways Veritas can Supercharge your AWS Investment - Session Sponsored b...
Unique Ways Veritas can Supercharge your AWS Investment - Session Sponsored b...Unique Ways Veritas can Supercharge your AWS Investment - Session Sponsored b...
Unique Ways Veritas can Supercharge your AWS Investment - Session Sponsored b...
 
Smaller is Better - Exploiting Microservice Architectures on AWS - Technical 201
Smaller is Better - Exploiting Microservice Architectures on AWS - Technical 201Smaller is Better - Exploiting Microservice Architectures on AWS - Technical 201
Smaller is Better - Exploiting Microservice Architectures on AWS - Technical 201
 
Using Amazon RDS to power enterprise applications (Peoplesoft)
Using Amazon RDS to power enterprise applications (Peoplesoft) Using Amazon RDS to power enterprise applications (Peoplesoft)
Using Amazon RDS to power enterprise applications (Peoplesoft)
 
Cloud architecture
Cloud architectureCloud architecture
Cloud architecture
 
Architectures for open and scalable clouds
Architectures for open and scalable cloudsArchitectures for open and scalable clouds
Architectures for open and scalable clouds
 
Distributed Design and Architecture of Cloud Foundry
Distributed Design and Architecture of Cloud FoundryDistributed Design and Architecture of Cloud Foundry
Distributed Design and Architecture of Cloud Foundry
 
AWS Tech Talks: Armazenamento Híbrido na Nuvem
AWS Tech Talks: Armazenamento Híbrido na NuvemAWS Tech Talks: Armazenamento Híbrido na Nuvem
AWS Tech Talks: Armazenamento Híbrido na Nuvem
 
Amazon WorkSpaces - Fully Managed Desktops in the Cloud
Amazon WorkSpaces - Fully Managed Desktops in the CloudAmazon WorkSpaces - Fully Managed Desktops in the Cloud
Amazon WorkSpaces - Fully Managed Desktops in the Cloud
 
Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...
Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...
Disaster Recovery in the AWS Cloud - Red Lion Hotels, Washington Trust Bank, ...
 
Enterprise Cloud Architecture Best Practices
Enterprise Cloud Architecture Best PracticesEnterprise Cloud Architecture Best Practices
Enterprise Cloud Architecture Best Practices
 
JBoss Fuse Service Works
JBoss Fuse Service WorksJBoss Fuse Service Works
JBoss Fuse Service Works
 

Viewers also liked

Zuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That Matter
Zuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That MatterZuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That Matter
Zuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That Matter
Zuora, Inc.
 
Cost Optimisation with Amazon Web Services
 Cost Optimisation with Amazon Web Services Cost Optimisation with Amazon Web Services
Cost Optimisation with Amazon Web Services
Amazon Web Services
 
Challenges in cloud computing to enable future internet of things v0.3
Challenges in cloud computing to enable future internet of things v0.3Challenges in cloud computing to enable future internet of things v0.3
Challenges in cloud computing to enable future internet of things v0.3
Ignacio M. Llorente
 
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
Amazon Web Services
 
AWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS Corp
AWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS CorpAWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS Corp
AWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS Corp
Amazon Web Services
 
Leaders in the Cloud: Identifying Cloud Business Value for Customers
Leaders in the Cloud: Identifying Cloud Business Value for CustomersLeaders in the Cloud: Identifying Cloud Business Value for Customers
Leaders in the Cloud: Identifying Cloud Business Value for Customers
OpSource
 

Viewers also liked (20)

Cloud is such stuff as dreams are made on
Cloud is such stuff as dreams are made onCloud is such stuff as dreams are made on
Cloud is such stuff as dreams are made on
 
Zuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That Matter
Zuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That MatterZuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That Matter
Zuora @ AlwaysOn 2012 - The Only 3 SaaS Metrics That Matter
 
Cloud Computing Technology Overview 2012
Cloud Computing Technology Overview 2012Cloud Computing Technology Overview 2012
Cloud Computing Technology Overview 2012
 
Hadoop and DynamoDB
Hadoop and DynamoDBHadoop and DynamoDB
Hadoop and DynamoDB
 
Getting Started with Amazon CloudSearch
Getting Started with Amazon CloudSearchGetting Started with Amazon CloudSearch
Getting Started with Amazon CloudSearch
 
Cost Optimisation with Amazon Web Services
 Cost Optimisation with Amazon Web Services Cost Optimisation with Amazon Web Services
Cost Optimisation with Amazon Web Services
 
2012 Future of Cloud Computing
2012 Future of Cloud Computing 2012 Future of Cloud Computing
2012 Future of Cloud Computing
 
Big Data & The Cloud
Big Data & The CloudBig Data & The Cloud
Big Data & The Cloud
 
Challenges in cloud computing to enable future internet of things v0.3
Challenges in cloud computing to enable future internet of things v0.3Challenges in cloud computing to enable future internet of things v0.3
Challenges in cloud computing to enable future internet of things v0.3
 
Open APIs: What's Hot, What's Not?
Open APIs: What's Hot, What's Not?Open APIs: What's Hot, What's Not?
Open APIs: What's Hot, What's Not?
 
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
AWS Partner Presentation - PetaByte Scale Computing on Amazon EC2 with BigDat...
 
AWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS Corp
AWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS CorpAWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS Corp
AWS Cloud Use Cases - Ezhil Arasan Babaraj, CSS Corp
 
Cloud Economics: Optimising for Cost
Cloud Economics: Optimising for CostCloud Economics: Optimising for Cost
Cloud Economics: Optimising for Cost
 
Google App Engine for Business 101
Google App Engine for Business 101Google App Engine for Business 101
Google App Engine for Business 101
 
Building the European Cloud Computing Strategy
Building the European Cloud Computing StrategyBuilding the European Cloud Computing Strategy
Building the European Cloud Computing Strategy
 
Leaders in the Cloud: Identifying Cloud Business Value for Customers
Leaders in the Cloud: Identifying Cloud Business Value for CustomersLeaders in the Cloud: Identifying Cloud Business Value for Customers
Leaders in the Cloud: Identifying Cloud Business Value for Customers
 
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
 
AWS Architecting In The Cloud
AWS Architecting In The CloudAWS Architecting In The Cloud
AWS Architecting In The Cloud
 
Cloud Computing Without The Hype An Executive Guide (1.00 Slideshare)
Cloud Computing Without The Hype   An Executive Guide (1.00 Slideshare)Cloud Computing Without The Hype   An Executive Guide (1.00 Slideshare)
Cloud Computing Without The Hype An Executive Guide (1.00 Slideshare)
 
Cloud Computing and Enterprise Architecture
Cloud Computing and Enterprise ArchitectureCloud Computing and Enterprise Architecture
Cloud Computing and Enterprise Architecture
 

Similar to Cloud-powered Continuous Integration and Deployment architectures - Jinesh Varia

Optimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer ToolsOptimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer Tools
Amazon Web Services
 
Continuous Integration using Cruise Control
Continuous Integration using Cruise ControlContinuous Integration using Cruise Control
Continuous Integration using Cruise Control
elliando dias
 
Continuous Delivery in the Cloud
Continuous Delivery in the CloudContinuous Delivery in the Cloud
Continuous Delivery in the Cloud
Fabio Lessa
 

Similar to Cloud-powered Continuous Integration and Deployment architectures - Jinesh Varia (20)

Optimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer ToolsOptimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer Tools
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 
Continuous Integration using Cruise Control
Continuous Integration using Cruise ControlContinuous Integration using Cruise Control
Continuous Integration using Cruise Control
 
AWS Summit Auckland - Application Delivery Patterns for Developers
AWS Summit Auckland - Application Delivery Patterns for DevelopersAWS Summit Auckland - Application Delivery Patterns for Developers
AWS Summit Auckland - Application Delivery Patterns for Developers
 
AWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for GovernmentAWS Webcast - Build Agile Applications in AWS Cloud for Government
AWS Webcast - Build Agile Applications in AWS Cloud for Government
 
Simplified DevOps Bliss -with OpenAI API
Simplified DevOps Bliss -with OpenAI APISimplified DevOps Bliss -with OpenAI API
Simplified DevOps Bliss -with OpenAI API
 
DevOps demystified
DevOps demystifiedDevOps demystified
DevOps demystified
 
AWS Code Services
AWS Code ServicesAWS Code Services
AWS Code Services
 
[RHFSeoul2017]6 Steps to Transform Enterprise Applications
[RHFSeoul2017]6 Steps to Transform Enterprise Applications[RHFSeoul2017]6 Steps to Transform Enterprise Applications
[RHFSeoul2017]6 Steps to Transform Enterprise Applications
 
DevOps, A brief introduction to Vagrant & Ansible
DevOps, A brief introduction to Vagrant & AnsibleDevOps, A brief introduction to Vagrant & Ansible
DevOps, A brief introduction to Vagrant & Ansible
 
Leveraging Elastic Web-Scale Computing with AWS
Leveraging Elastic Web-Scale Computing with AWSLeveraging Elastic Web-Scale Computing with AWS
Leveraging Elastic Web-Scale Computing with AWS
 
(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Scre...
(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Scre...(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Scre...
(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Scre...
 
Continuous Delivery in the Cloud
Continuous Delivery in the CloudContinuous Delivery in the Cloud
Continuous Delivery in the Cloud
 
Continuous Delivery in the AWS Cloud
Continuous Delivery in the AWS CloudContinuous Delivery in the AWS Cloud
Continuous Delivery in the AWS Cloud
 
OSv presentation from Linux Foundation Collaboration Summit
OSv presentation from Linux Foundation Collaboration SummitOSv presentation from Linux Foundation Collaboration Summit
OSv presentation from Linux Foundation Collaboration Summit
 
Remove Undifferentiated Heavy Lifting from Jenkins (DEV201-R1) - AWS re:Inven...
Remove Undifferentiated Heavy Lifting from Jenkins (DEV201-R1) - AWS re:Inven...Remove Undifferentiated Heavy Lifting from Jenkins (DEV201-R1) - AWS re:Inven...
Remove Undifferentiated Heavy Lifting from Jenkins (DEV201-R1) - AWS re:Inven...
 
Scale your Magento app with Elastic Beanstalk
Scale your Magento app with Elastic BeanstalkScale your Magento app with Elastic Beanstalk
Scale your Magento app with Elastic Beanstalk
 
LAST Conference - Dev-Ops and Continuous Delivery
LAST Conference - Dev-Ops and Continuous DeliveryLAST Conference - Dev-Ops and Continuous Delivery
LAST Conference - Dev-Ops and Continuous Delivery
 
DevOps Tooling - Pop-up Loft TLV 2017
DevOps Tooling - Pop-up Loft TLV 2017DevOps Tooling - Pop-up Loft TLV 2017
DevOps Tooling - Pop-up Loft TLV 2017
 
Serverless in production, an experience report (FullStack 2018)
Serverless in production, an experience report (FullStack 2018)Serverless in production, an experience report (FullStack 2018)
Serverless in production, an experience report (FullStack 2018)
 

More from Amazon Web Services

Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
Amazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Recently uploaded

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 

Recently uploaded (20)

The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdf
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistan
 

Cloud-powered Continuous Integration and Deployment architectures - Jinesh Varia

  • 1. Cloud-Powered Continuous Integration and Deployment Jinesh Varia [email_address] @jinman
  • 2. About Me Jinesh Varia @jinman [email_address] linkedin/in/jinman http://jinesh.varia.in
  • 3. “ Cloud Architectures” “ Cloud Best Practices aws” “ Cloud Migration aws”
  • 4. Customer is the center of our universe
  • 5.  
  • 6. Learning from customers Some Learning Lots of Learning
  • 7. Learning from customers Little or no Learning
  • 9. Learning from customers Requirements gathering Release
  • 10. Reduce the cycle time Requirements gathering Release
  • 11. Learn faster Requirements gathering Automation + Cloud Release … while keeping your costs low
  • 12. Continuous Integration Continuous Deployment Continuous Optimization Cloud-powered Continuous Delivery
  • 13. Continuous Integration Cloud-powered Continuous Integration Goal: to have a working state of the code at any given time Benefit: Fix bugs earlier when they are cheaper to fix Metric: New guy can check out and compile at first day at job
  • 14. Poka yo-ke ( ポカヨケ )
  • 15. Martin Fowler Paul Duvall Jez Humble David Farley Matt Graham Michael Nygard … .. … ..
  • 16.
  • 17. Automate Everything Hello, I am Mr. Automate Development And Build Monitoring Staging and Production Testing and Deployment
  • 18. Application Containers -  JBoss ,  Tomcat ,  IIS ,  Mongrel .  NOTE: there are so many app containers, I'm not going to try to list all of them. Build Tools  -  Ant ,  AntContrib ,  NAnt ,  MSBuild ,  Buildr ,  Gant ,  Gradle ,  make ,  Maven ,  Rake Code Review  -  Crucible Code Insight  -  Fisheye     Continuous Integration -  Bamboo ,  Jenkins ,  AntHill Pro ,  Go ,  TeamCity ,  TFS 2010 Database -  Hibernate ,  MySQL ,  Liquibase ,  Oracle ,  PostgreSQL ,  SQL Server ,  SimpleDB ,  SQL Azure ,  Ant ,  MongoDB Database Change Management  -  dbdeploy ,  Liquibase Data Center Configuration Automation  -  Capistrano ,  Cobbler ,  BMC Bladelogic ,  CFEngine ,  IBM Tivoli Provisioning Manager ,  Puppet ,  Chef ,  Bcfg2 ,  AWS Cloud Formation ,  Windows Azure AppFabric   NOTE: There are many names and overlap for this tool &quot;category&quot;.  Dependency Management  -  Ivy ,  Archiva ,  Nexus ,  Artifactory ,  Bundler Deployment Automation  -  Java Secure Channel ,  ControlTier ,  Altiris ,  Capistrano ,  Fabric ,  Func Information Sharing  -  Confluence ,  Google Apps Installer  -  InstallShield ,  IzPack Integrated Development Environment (IDE)  -  Eclipse ,  IDEA ,  Visual Studio Issue Tracking  -  Greenhopper ,  JIRA   Multi-Type  -  rPath Passwords  -  PassPack ,  PasswordSafe Protected Configuration  -  ESCAPE ,  ConfigGen Project Management  -  JIRA ,  Pivotal Tracker ,  SmartSheet Provisioning  -  JEOS ,  BoxGrinder ,  CLIP ,  Eucalyptus ,  AppLogic Reporting/Documentation  -  Doxygen ,  Grand ,  GraphViz ,  JavaDoc ,  NDoc ,  SchemaSpy ,  UmlGraph Static Analysis  -  CheckStyle ,  Clover ,  Cobertura ,  FindBugs ,  FxCop ,  JavaNCSS ,  JDepend ,  PMD ,  Sonar ,  Simian Systems Monitoring  -  CloudKick ,  Nagios ,  Zabbix ,  Zenoss Testing   AntUnit ,  Cucumber ,  DbUnit ,  webrat ,  easyb ,  Fitnesse ,  JMeter ,  JUnit ,  NBehave ,  SoapUI ,  Selenium ,  RSpec , SauceLabs Version-Control System  -  SVN/Subversion ,  git ,  Perforce Paul Duvall’s Blog http://blog.stelligent.com/integrate-button/2011/03/list-of-software-tools-for-continuous-delivery-in-the-cloud.html
  • 19.  
  • 21. Version Control Developer <code> CI Server Build Slave Build Slave Build Slave Distributed Builds
  • 22. EBS Root Volume EC2 Instance AMI EBS Snapshot EBS Volume EC2 Instance createImage() RunInstances() StopInstances() StartInstances() TerminateInstances() Stop v/s Terminate
  • 23.  
  • 24.  
  • 25. Amazon S3 Bucket AZ-1 Region Elastic Load Balancer Amazon CloudWatch Alarms Amazon SNS Notifications www.yourApp.com Amazon SimpleDB Domains Amazon Route 53 Hosted Zone Auto Scaling Group AZ-1 App Tier ElastiCache Tier Amazon SES Email Amazon CloudFront media.yourApp.com (Static data) Amazon RDS Amazon EC2 Instances
  • 26. AWS CloudFormation JSON Template Version Control Update Stack
  • 27. Version Control CI Server Package Builder Deploy Server Commit to Git/master Dev Pull Code AMIs Send Build Report to Dev Stop everything if build failed Distributed Builds Run Tests in parallel Staging Env Test Env Code Config Tests Prod Env Push Config Install Create Repo CloudFormation Templates for Env Generate Cloud Continuous Integration
  • 28.
  • 29.
  • 31.
  • 32. PROD Very Large Server Storage Bob Test Very Small Server Storage Ted Test Very Small Server Storage Mary Test Very Small Server Storage 7 AM 6 PM RDS Snapshots – Test something quickly
  • 33. Continuous Integration Continuous Deployment Cloud-powered Continuous Deployment
  • 34. Cloud-powered Continuous Deployment Goal: 1-click deploy and 1-click rollback Benefit: Release early, release often and iterate quickly Metric: fast feedback of your feature Continuous Deployment
  • 35.  
  • 36. Software Inventory is lost revenue
  • 37.  
  • 38. Mean Time between deployments (weekday) Max # of deployments in a single hour Mean # of hosts simultaneously receiving a deployment Max # of hosts simultaneously receiving a deployment Amazon May Continuous Deployment Stats (production hosts and environments only) 11.6 Seconds 1079 10000 30000
  • 39. The need for speed
  • 40.
  • 41. Break your problem into small batches Small deployments Incremental changes Easy rollbacks
  • 42. Virtual Images = Real Productivity Gain Java Stack .NET Stack RoR stack Centos Ruby Runtime Your Code logger RubyGems memcached Rails Mongrel Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Windows .NET Your Code Log4Net Spring.NET nHibernate ASP.NET MVC ASP.NET IIS Centos Ruby Runtime Your Code logger RubyGems memcached Rails Mongrel Apache OS Framework Your Code Libraries Packages DB Caching MVC App Server Web Server
  • 43. Push to an AMI or Pull from an Instance Inventory of AMIs Golden AMI and Fetch binaries on boot JeOS AMI and library of recipes (install scripts) Amazon EC2 Amazon EC2 Linux JEE CHEF Puppet Chef/puppet scripts Java AMI Java App Stack Java AMI JeOS AMI Fetch on boot Fetch on boot Fetch on boot Frozen Pizza Model Take N Bake Pizza Model Made to order Pizza Model Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Amazon EC2 Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Your Code Amazon S3 Log4J Spring Struts Linux JEE Hibernate Tomcat Apache Your Code Amazon S3 Hibernate Tomcat Log4J Spring Struts Apache Linux JEE Hibernate Tomcat Apache Linux JEE Hibernate Tomcat Apache Linux JEE Hibernate Tomcat Apache Linux JEE Linux JEE
  • 44. Cloud Continuous Deployment (Simple) Version Control CI Server Deploy Server Commit to Git/master Dev Pull Code Send Build Report to Dev Stop everything if build failed Build Run Tests Code Config Tests CloudFormation Templates Package Builder Push Config Repo Base AMIs Instances Scripts Cloud-init
  • 45. &quot;UserData&quot;: { &quot;Fn::Base64&quot;: { &quot;Fn::Join&quot;: [ &quot;&quot;, [ &quot;#!/bin/bash -ex&quot;, &quot;yum -y install git-core&quot;, &quot;yum -y install php-pear&quot;, &quot;pear install Crypt_HMAC2-1.0.0&quot;, &quot;pear install HTTP_Request-1.4.4&quot;, &quot;pear channel-discover pear.amazonwebservices.com&quot;, &quot;pear install aws/sdk&quot;, Bootstrap using User Data
  • 46.
  • 47. Cloud Continuous Deployment Version Control CI Server Deploy Server Commit to Git/master Dev Pull Code Send Build Report to Dev Stop everything if build failed Build Run Tests Code Config Tests CloudFormation Templates Chef Server (config, package dependencies) Puppet Master (Manifest, config and mappings) Chef S Puppet M Push Config New Code Reports Base AMIs Managed Instances Scripts recipes
  • 48. AWS Region Availability Zone 1 Deploy Server Automate deployment to multiple Availability Zones (Fault Tolerant Zones) Availability Zone 2 Availability Zone 3
  • 50. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 Monitoring (CloudWatch) High Error Rate
  • 51. Auto Scaling Group (Min, Max # of instances, Availability Zones .. ) Health Check (Maintain Min # active…) Launch Configuration (AMIID, Instance type, UserData, Security Groups..) Scaling Trigger (Metric, Upper Threshold, Lower Threshold, Time interval …) Types of Scaling (Scale by Schedule, Scale by Policy) Alarm (Notification Email, SMS, SQS, HTTP) Availability Zones and Regions Auto Scaling
  • 52. “ Auto scaling” Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 Auto scaling Max instances Min instances Scaling Trigger Custom Metrics Upper Threshold Lower Threshold Increment by
  • 53. Dark Launches with feature flags Deploy != Product Launch
  • 54. v1.1 v1.1 v1.1 v1.1 Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 Happy Path v1.1 New feature Code Path v1.2 Feature = ON request
  • 55. Dialing up A B Control Treatment
  • 56. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) 99% 1% … .. A/B Testing Service v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2
  • 57. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 90% 10% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2
  • 58. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 70% 30% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
  • 59. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 70% 30% Monitoring (CloudWatch) High Error Rate Rollback v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
  • 60. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 90% 10% Rollback Dev, Test v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
  • 61. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 70% 30% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
  • 62. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 50% 50% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
  • 63. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 30% 70% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
  • 64. Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) … .. A/B Testing Service 5% 95% v1.1 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.2
  • 65. Split Users by multiple experiments A B Control Treatment 1 C D Treatment 2 Treatment 3
  • 66. v1.1 v1.1 v1.1 v1.1 Web Server Fleet (Amazon EC2) Database Fleet (RDS or DB on EC2) Load Balancing (ELB) v1.1 v1.1 v1.1 v1.1 v1.2 v1.2 90% 5% v1.2.1 v1.2.1 3% v1.2.2 v1.2.2 2%
  • 67.
  • 68. Timeline V2.1 Example ID NAME ADDRESS ORDERID (Char) 23234 Joe Doe xxx 333424 45322 Rob Smith xxxx 234 2342342 Jane Smith xxxx 23424 2342265 Anne Lee xxxx 2342425
  • 69.
  • 70.
  • 71.
  • 72.
  • 73.
  • 74.
  • 75. Increasing the speed of Iteration
  • 76. Break your problem into small batches Stream of small deployments Incremental changes Easy rollbacks
  • 77. Deploy new software quickly Revert a bad change quickly Cost of Mistakes
  • 78. Dynamism of the cloud makes its easy
  • 79. Small Application Layer Large CPU Week 1 Large CPU Week 1 Week 2 Small Week 1 Week 2 Week 3 Small Week 1 Week 2 Week 3 Week 4 Small Week 1 Week 2 Week 3 Week 4 Week 5 Week 1 Week 2 Week 3 Week4 Week 5 Week 6 Give me Week1 data Writes Reads
  • 80. Continuous Integration Continuous Deployment Continuous Optimization Cloud-powered Continuous Delivery
  • 81. Continuous Optimization Cloud-powered Continuous Optimization Goal: Optimize on multiple dimensions : Cost, Performance, HA, Security, Response Time Benefit on optimize for cost - Immediate recurring cost savings Metric: understanding utilization
  • 82. When you turn off your cloud resources, you actually stop paying for them
  • 83. 25 % Savings By the time of day
  • 84. Availability Zone #2 Availability Zone #1 Auto Scaling group : App Tier Auto Scaling group : Web Tier Elastic Load Balancer www.MyWebSite.com (dynamic data) media.MyWebSite.com (static data) Amazon Route 53 (DNS) Amazon EC2 Amazon RDS Amazon RDS Amazon S3 Amazon CloudFront
  • 85. during a year 50 % Savings
  • 86. 75 % Savings during a month
  • 87. Continuous optimization in your architecture and cloud infrastructure results in recurring savings in your next month’s bill
  • 88. Instance Amazon CloudWatch Alarm Free Memory Free CPU Free HDD At 1-min intervals Custom Metrics PUT 2 weeks “ You could save a bunch of money by switching to a small instance , Click on CloudFormation Script to Save” Cost-aware instances
  • 89. Choosing the right pricing model On-demand Instances Reserved Instances Spot Instances
  • 90. Steady State Usage Total Cost for 1 Year-term of 2 application servers Total Cost for 3 Year-term of the same 2 application servers Usage Fee One-time Fee Total Savings Option 1 On-Demand only $1493 - $1493 - Option 2 On-Demand + Reserved $1008 $227 $1234 ~20% Option 3 All reserved $528 $455 $983 ~35% Usage Fee One-time Fee Total Savings Option 1 On-Demand only $4479 - $4479 - Option 2 On-Demand + Reserved $3024 $350 $3374 ~30% Option 3 All reserved $1584 $700 $2284 ~50%
  • 91. 1-year RI versus On Demand: cost savings realized after first 6 months of usage 3-year RI versus On Demand: cost savings realized after first 9 months of usage. 3-year RI versus 1-year RI: Net savings of 3-year RI versus 1-year RI begin by month 13 and continue throughout the RI term (additional 23 months of savings) 1 2 3
  • 93. Elasticity is one of the fundamental properties of the cloud that drives many of its economic benefits
  • 94. Continuous Integration Continuous Deployment Continuous Optimization Cloud-powered Continuous Delivery
  • 95. Summary Invest in areas where you get to learn from your customers quickly Automate everything else Cloud Continuous Integration Release early, Release often, Iterate Quickly Get fast feedback Distributed Builds, Automated Tests in Parallel Cloud Continuous Deployment Reduce the cost of mistakes Increase the speed of iteration Leverage cloud for Blue Green Deployments Cloud Continuous Optimization Keep Optimizing and further reduce costs of infrastructure
  • 96. Jinesh Varia [email_address] Twitter: @jinman Credits to Jon Jenkins, Paul Duvall and several engineers at Amazon Thank you!
  • 99. Optimize by Implementing Elasticity Infrastructure Cost $ time Large Capital Expenditure You just lost customers Predicted Demand Traditional Hardware Actual Demand Cloud Automated Elasticity
  • 100. Amazon S3 Bucket AZ-1 Region Elastic Load Balancer Amazon CloudWatch Alarms Amazon SNS Notifications www.yourApp.com Amazon SimpleDB Domains Amazon Route 53 Hosted Zone Auto Scaling Group AZ-1 App Tier ElastiCache Tier Amazon SES Email Amazon CloudFront media.yourApp.com (Static data) Amazon RDS Amazon EC2 Instances
  • 101. Amazon VPC AWS Region VPC Subnet VPC Subnet Corporate data center Corporate Headquarters Availability Zone 1 Availability Zone 2 Branch Offices VPN Gateway Customer Gateway Internet Gateway Router DirectConnect Location Amazon S3 Amazon SimpleDB Amazon SES Amazon SQS The New Cloud-Ready Enterprise IT 10G

Editor's Notes

  1. The presentation will discuss some architectural patterns in continuous integration, deployment and optimization and I will share some of the lessons learned from Amazon.com. The goal of the presentation is to convince you that if you invest your time where you get the maximum learning from your customers, automate everything else in the cloud (CI + CD + CO), you get fast feedback and will be able to release early, release often and recover quickly from your mistakes. Dynamism of the cloud allows you to increase the speed of your iteration and reduce the cost of mistakes so you can continuously innovate while keeping your cost down.
  2. Jinesh Varia – Penn State Data center, FDIC and 5.5 years in AWS.
  3. customers are the center of all equations and discussions. We are continuously listening to customers for feature requests that will make it more easier. At times, where customers don’t know what they want, we rely on statistical significance and confidence intervals to really know the behavior of what impacting the business Hence learning from customers, IMO should be given importance than anything else so that we can get fast feedback, change the course of action, steer to the right direction and align ourselves that makes sense.
  4. Typical workflow from Idea to Release
  5. Focus on learning and Automation with Cloud helps you learn faster, iterate quickly and
  6. There are 3 main ideas
  7. Poka yoke is a design practice originated at Toyota Japan which essentially discusses mistake proofing. Invest in Design systems such that customers don’t make mistakes when they use it. For example, if you were to design an ATM machine and focusing on design how the ATM card will be read, if you design the system that takes the credit card, there is a probabilty that customer might forget it in the machine or the system and mechanics might fail. Instead poka yoke design will be to swiping. That way the customer will never forget as its always in his/her hand and the design is not that complex Continuous integration gives you a poka-yoke design to your build and test systems. Developers get alerted about a bugs ahead of time much before when they are cheaper to fix them.
  8. Everything thing should be in the version control OS, patch levels, OS configuration, your app stack, its configuration, infrastructure config Infrastructure is code, Build scripts is code, test scripts is code
  9. Automate everything People take more time, make errors, they are not auditable Plus its boring and repetitive. They have better things to do, Expensive
  10. Tools soup
  11. Only one way to deploy so you exactly know who, which version, what configuration, easy rollback, which hosts/machines etc.
  12. Leverage the cloud for distributed builds and reduce your build and test run time so you get fast feedback
  13. Stop when you need. Run your builds three times a day and only pay for storage.
  14. Several tools available – most have support for EC2/cloud
  15. you can configure Bamboo to start and shut down elastic instances  automatically , based on build queue demands? Please refer to  Configuring Elastic Bamboo for more information.
  16. Lets put everything in a context of a web application
  17. You can put this all into a CloudFromation script. I recommend that you put that in version control and let it manage the resources for you. CF engine will manage the ordering, rollback, update etc.
  18. Architecture for Cloud Continuous integration Developer commits to git/master CI Server pulls from git/master CI Server runs all regression tests against git/master CI Server returns report to developer and halts if any fail CI Server builds the artifacts CI Server pushes the project manifest to Image manager Package Builder gets configuration information from VSS PB builds packages (gem, .deb, rpms..), stores them in repo PB installs dependencies on an instance/host and creates a virtual Image or multiple images PB generates a CF template from the Config for each environment and stores them in repo CI Server calls deployment server to deploy the image to a given environment (using CF template) Deployment server attempts to deploy the image to instances using cloudformation scripts and alerts the monitoring service
  19. The key advance was using our continuous build system to build not only the artifact from source code, but the complete software stack, all the way up to a deployable image in the form of an AMI (Amazon Machine Image for AWS EC2).
  20. This is where cloud really shines. Where else can you get 100s and 1000s machines to run them in parllel and shut them down when you are done Same goes with benchmarking. Stress test in the Cloud Create AMIs with libraries and dependencies Add “compute power” when needed and turn it off to reduce costs Load test in the Cloud Generate load from one Availability Zone to test on other. Startup a pre-configured TestBox (EC2 instance) in minutes Performance test in the Cloud Test at Global scale - Latency from different parts of the world Store all instrumentation data on S3, SimpleDB. Web App testing Browser based Ajax/Selenium testing from different availability zones (US and EU) Create different deployment environments using scripts Usability Testing On-demand workforce What does Testing in the Cloud mean: Automated, Virtual Test Labs that are live only when you need them Stress test in the Cloud Find the source of latency, Potential Crashes and/or points of Failure. Get Profile information thru logs and instrumentation and measure Load test in the Cloud Generating load from one Availability zone to other “staging” servers on or off 100 concurrent browsing users that randomly click on links. Then, the load can be increased by 100 users every 10 minutes until the total expected user load of 100,000 users is reached. Performance test in the Cloud How fast the page is loading for a given user in given state Usability Testing
  21. Interesting things can be done now. Restore production enviornments and fix bugs at any time.
  22. Software inventory - put code to prod as fast as you can Code that not in production and in the hands of the customer is lost revenue We really want to iterate quickly, deploy often, test, get instant feedback and rollback if necessary but get it out as quickly as possible to the customer
  23. Apollo Environments and Stages Month of May At Amazon, every 11.6 seconds, someone is kicking off a deployment into production. Amazon is doing continous deployment with deployments happening every 11.6 seconds average weekday For the month of may, we kicked off a total of 1079 deployments in a single hour into prod On an average there are around 10,000 hosts receiving a deployment simultanously across a whole bunch of multiple environments For the month of may, we had a peak of around 30,000 hosts simultanously receiving a deployment
  24. speed of iteration matters. not whether you are making the right move but you making the move really really fast. This is great for people like me who often make wrong moves. I can iterate quickly and find the right solution with trial and error. I make this progress quickly through trial and error. failing fast. I dont have a fear of failure   It&apos;s really important to iterate going forward because it is through iteration, incremenet improvement, we will know what works next and make best practices.
  25. We have to be wrong a lot in order to right a lot Cloud really helps you to reduce the cost of failure.
  26. And we can do that by breaking your deployments into stream of small deployments
  27. The first step is create bootstrapped instances so that when they come up they automatically install all the stuff that you need. You can two that in two ways: have a preconfigured AMI or install all the cool stuff on boot time.
  28. Simple architecture that does continuous deployment using Cloud-init Developer commits to git/master CI Server pulls from git/master (every checkin) CI Server runs all regression tests against git/master CI Server returns report to developer and halts if any fail CI Server builds the artifacts and packages stores them in repository (for faster installs) CI Server pushes the project manifest to Image manager Config Server is either Chef Server or Puppet Master CS Manages the chef nodes (or puppet client) CS generates reports provides UI for chef/puppet- managed instances CI Server sends the new code directly to config server (Puppet Master or Chef Server) Deploy server provisions cloud resources DS attempts to deploy the image to instances using cloudformation scripts or EC2 APIs DS leverages the Base AMI with pre-installed client (chef-client or puppetrun) installed
  29. Cloudinit and CF
  30. Simple architecture that does continuous deployment using Chef and/or Puppet Developer commits to git/master CI Server pulls from git/master (every checkin) CI Server runs all regression tests against git/master CI Server returns report to developer and halts if any fail CI Server builds the artifacts and packages stores them in repository (for faster installs) CI Server pushes the project manifest to Image manager Config Server is either Chef Server or Puppet Master CS Manages the chef nodes (or puppet client) CS generates reports provides UI for chef/puppet- managed instances CI Server sends the new code directly to config server (Puppet Master or Chef Server) Deploy server provisions cloud resources DS attempts to deploy the image to instances using cloudformation scripts or EC2 APIs DS leverages the Base AMI with pre-installed client (chef-client or puppetrun) installed
  31. Automate deployment to multiple Azs and increase AZs
  32. So how can you do deployments and quickly deploy without any downtime
  33. One simple technique is to do blue green deloyments. Green is your current deployment, you de
  34. BG + Autoscaling = Cost savings + Scaling up
  35. BG + Darklaunches in which you deploy both the versions side by side but customers does not know about it. Two separate code paths but only one activated. The other one is activated by a feature flag. This can be your beta test where you explicitly turn
  36. Goal is to get instant feedback from your users and how your application behaves when real load. A/B testing is great. Control and Treatment http://20bits.com/articles/an-introduction-to-ab-testing/ Prove success of interface changes Prove interest in new features
  37. What if you have 1000s of nodes. The blue green deployments works well but you may not want to spin up additional 100% capacity. So then you can gradually spin up in small increments of 10% so you only pay for additional 10% of your nodes. Plus you learn more about your customer behavior too. By routing only a small of users to your newly code deployed code base. Start small with 1% - A/B testing is randomizes the traffic and collect feedback on what’s working and what is not working. With a simple configuration change Autoscaling on the other hand scales. You don’t want to send all your traffic to new nodes just yet. Note: you have tested all your application, brought additional capacity before you route real traffic.
  38. Slowly ramp of traffic by simply changing the configuration file in a/b testing and see autoscaling kicking off additional instances as needed. Note here you are not disrupting user behavior because they see only that version that they hit.
  39. Slowly rampup traffic you are know whether the feature is working for your customer and you are learning more about customer.
  40. Pager beeping. Something went wrong. Time to do rollback. Rollbacks are easy…
  41. Change the config file route all the traffic back to your previous hosts…. Test what went wrong…fix….deploy new instances using the same process
  42. Start ramping up again…..
  43. You are gradually deploying new software, using the dynamism of the cloud with zero downtime. And that’s cool.
  44. Autoscaling handles all the capacity planning for you.
  45. Dial up is based on users –user get consistent users experience The beauty of this way is you can run multiple measured Experiments of different features for your customers. Get instant feedback and deploy only if makes sense Based on Statistical significance and confidence intervals, you determine which feature is really causing improvement and creating an impact on the business.
  46. What about datatabases? Those can be tricky and they are not as simple as cutover from green to blue. For all practical purpose, I would recommend using non-relational database. But you can too do blue green style database deployments with zero downtime in several approaches. #3 is the best one because you are leveraging the decoupling of DB and application and breaking your one job into multiple tasks.
  47. Simple Example to change character to Integer database schema change.
  48. Autoscaling in recurring mode
  49. Rollbacks will be easy because you can deploy application and DB combination that worked in the previous iteration.
  50. Again, not all everything can be done when it comes to RDBMS migration. But you can do DB migration and Application in innovative ways to deploy new version in series of steps with zero downtime and easy rollback. speed of iteration matters. not whether you are making the right move but you making the move really really fast. This is great for people like me who often make wrong moves. I can iterate quickly and find the right solution with trial and error. I make this progress quickly through trial and error. failing fast. I dont have a fear of failure   It&apos;s really important to iterate going forward because it is through iteration, incremenet improvement, we will know what works next and make best practices.
  51. The goal is to do it in small batches
  52. By doing this…and continuous deploying new versions you are reducing the cost of doing mistakes.
  53. And you just saw how cloud really helps here
  54. You keep optimizing and you further reduce your cost, impress your boss, individual can make a significant improvement in bottomline and savings. DB with small optimization, reduced the database footprint, you saved company a bunch of money right away. This does not happen in non-cloud area where even if you implement an optimization, you have already paid for the infrastructure. Again, Poka yoke…Cloud really helps you to design right so you work towards efficiency
  55. Only happens in the cloud
  56. Build websites that sleep at night. Build machines only live when you need it
  57. Shrink your server fleet from 6 to 2 at night and bring back
  58. Now this is just the beginning. We really havent spent enough time here. But one developer wrote a script that will dump all the free resources to custom metrics to cloud watch and if they were 2 weeks, it automatically alerts the users to save money. Now that is cool!! This intelligence can now be backed into your platform. This can only happen when you have a great API and you are only paying for your infrastructure
  59. http://calculator.s3.amazonaws.com/calc5.html?key=calc-D192C939-B31D-4F29-B84E-F07F8E6190F9 http://calculator.s3.amazonaws.com/calc5.html?key=calc-B4C1AD60-742A-401E-B99E-C21862C376AC http://calculator.s3.amazonaws.com/calc5.html?key=calc-A66897DE-2773-494D-B046-7C7191BC6336
  60. Without elasticty you will not be able to accelerate
  61. This slide applies to Amazon EC2, but just as easily describes Amazon S3’s value proposition.
  62. Lets put everything in a context of a web application
  63. See the animation. DirectConnect