Published on

Published in: Technology, Business
1 Comment
  • Hello, I found your slide show very interesting. I was just tasked to install couchbase on using EC2 resources and I stumbled upon your slide show. Does AWS or CouchBase have the cloud front template to build out a highly available couchbase cluster or should I build it myself?
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. AWS CloudFormation to Automate Couchbase Membase Clusters Usman Shakeel 7/29/2011
  2. 2. AWS Computing Platform
  3. 3. Couchbase Membase on AWSCloud Formation  Ease of deployment  One template – reuse with multiple configurationsAuto-Scale  Automatically add more nodes to your clusterElastic Beanstalk  Automatically handle the deployment details, capacity provisioning, load-balancing and auto-scaling of your Couchbase/Membase based application )Only pay for the AWSresources used
  4. 4. CloudFormation Automates your Cloud creation and management in a few clicks A wide range of AWS resources supported Declarative, Flexible and Customizable via Parameters Many reusable sample templates available (including WordPress, Tracks, Gollum, Drupal, Joomla, Insoshi, Redmine) Transparent and Open Templates Integrated with Amazon Simple Notification Service (Amazon SNS) No Extra Charge CloudFormer
  5. 5. CloudFormation – Stack Stack is a collection of resources instantiated using a template and parameter values Order of instantiation: determined based on the dependencies specified by the template author Resources are tagged by the stack name A stack is rolled back if it cannot be fully instantiated CloudFormation Stack: API, CLI and Console  CreateStack, DeleteStack  DescribeStackResources, DescribeStackEvents  GetTemplate, ValidateTemplate
  6. 6. CloudFormation – Template A Template is a JSON formatted text file Describes the AWS resources along with the inter-connection Captures the resource relationships Can be parameterized – One template different setups Provides output back to the user Resources are created using logical names to avoid namespace conflicts
  7. 7. CloudFormation Automation with Cloud-init Cloud-init uses the Amazon EC2 UserData field to pass configuration information Stack indicates CREATE_COMPLETE only at the successful installation of packages Easy to upgrade to new OS releases and patches over creating a custom AMI Attribute values dependent on other CloudFormation resources can be passed in to the custom scripts at runtime
  8. 8. CloudFormation Template Anatomy{ "Description" : "A text description for the template usage", "Parameters": { // A set of inputs used to customize the template per deployment }, "Resources" : { // The set of AWS resources and relationships between them }, "Outputs" : { // A set of values to be made visible to the stack creator }, "AWSTemplateFormatVersion" : "2010-09-09"}
  9. 9. CloudFormation Template – Intrinsic Functions "Fn::Base64" : {valueToEncode} "Fn::FindInMap" : ["MapName", "Key", "Value"] "Fn::GetAtt" : ["logicalNameOfResource", "attributeName"] "Fn::GetAZs" : "region" "Fn::Join" : ["delimiter"[comma-delimited list of values]] "Ref" : "logicalName"
  10. 10. Auto Scaling Launch or Terminate EC2 instances based on your policies, schedules or health-checks Scale up to a minimum number of secondary nodes in a Couchbase Membase Cluster Scale based on CPU, network, memory usage or other CloudWatch metrics Scale based on Custom Metrics (data captured from the cluster itself)  CloudWatch can capture custom metrics from you application enabling you to define custom auto scaling policies Auto Scaling groups, policies and configurations can be created in a CloudFormation template
  11. 11. Auto Scaling Example3000 - 3000 CPU’s for one firm’s risk management processesNumber of EC2 Instances 300 CPU’s on weekends300 - Wednesday Thursday Friday Saturday Sunday Monday Tuesday 4/22/2009 4/23/2009 4/24/2009 4/25/2009 4/26/2009 4/27/2009 4/28/2009
  12. 12. DemoCouchbase Membase 6-Pack Cluster creation withCloudFormation Walk through Membase 6-Pack Cluster Template Run CloudFormation Template via the Console Show the newly created Membase clusterAutoscaling Membase 6-Pack Cluster Create Autoscaling Group for the Membase Cluster Create Autoscaling policies Implement Autoscaling Demo the autoscaled Membase Cluster
  13. 13. Membase CloudFormation Template - Parameters
  14. 14. Membase CloudFormation Template - Mappings
  15. 15. Membase CloudFormation Template - Resources Membase Cluster Coordinator Node Use an AMI based on the mapping Create an Instance based on Type parameter Parameter for Membase Package Installation
  16. 16. Membase CloudFormation Template - Resources User Data defines the script that will be run at instance creation Trigger for Wait Handle
  17. 17. Membase CloudFormation Template - Resources Create Auto Scaling group for secondary nodes Secondary node configurations defined separately Wait on Coordinator node creation before creating the Auto Scaling group
  18. 18. Cluster Coordinator NodeSecondary nodes created by Auto Scaling Group
  19. 19. Secondary node 2 Secondary nodesconfigurations defined terminated separately
  20. 20. Auto Scaling starts 2 new nodes to maintain a minimum of 6 nodes
  21. 21. What else? Auto scale Membase Cluster Coordinator Node  Recycle AWS Elastic IP  Membase Cluster end-point stays the same Attach AWS SNS notifications to Auto Scaling events Create Custom Auto Scaling policies  Push custom Membase cluster metrics to Cloud Watch and scale Automate Membase application deployment  Elastic Bean Stalk  Chef and Puppet Fail-over down Membase Cluster nodes appropriately Parameterize the Cluster size and Auto Scale policies
  22. 22. More Information CloudFormation Auto Scaling