CouchConf-SF-AWS-Cloudformation-to-Automate-Couchbase-Membase-Clusters

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Hello, I found your slide show very interesting. I was just tasked to install couchbase on using EC2 resources and I stumbled upon your slide show. Does AWS or CouchBase have the cloud front template to build out a highly available couchbase cluster or should I build it myself?
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
3,235
On Slideshare
3,235
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
43
Comments
1
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. AWS CloudFormation to Automate Couchbase Membase Clusters Usman Shakeel 7/29/2011
  • 2. AWS Computing Platform
  • 3. Couchbase Membase on AWSCloud Formation  Ease of deployment  One template – reuse with multiple configurationsAuto-Scale  Automatically add more nodes to your clusterElastic Beanstalk  Automatically handle the deployment details, capacity provisioning, load-balancing and auto-scaling of your Couchbase/Membase based application )Only pay for the AWSresources used
  • 4. CloudFormation Automates your Cloud creation and management in a few clicks A wide range of AWS resources supported Declarative, Flexible and Customizable via Parameters Many reusable sample templates available (including WordPress, Tracks, Gollum, Drupal, Joomla, Insoshi, Redmine) Transparent and Open Templates Integrated with Amazon Simple Notification Service (Amazon SNS) No Extra Charge CloudFormer
  • 5. CloudFormation – Stack Stack is a collection of resources instantiated using a template and parameter values Order of instantiation: determined based on the dependencies specified by the template author Resources are tagged by the stack name A stack is rolled back if it cannot be fully instantiated CloudFormation Stack: API, CLI and Console  CreateStack, DeleteStack  DescribeStackResources, DescribeStackEvents  GetTemplate, ValidateTemplate
  • 6. CloudFormation – Template A Template is a JSON formatted text file Describes the AWS resources along with the inter-connection Captures the resource relationships Can be parameterized – One template different setups Provides output back to the user Resources are created using logical names to avoid namespace conflicts
  • 7. CloudFormation Automation with Cloud-init Cloud-init uses the Amazon EC2 UserData field to pass configuration information Stack indicates CREATE_COMPLETE only at the successful installation of packages Easy to upgrade to new OS releases and patches over creating a custom AMI Attribute values dependent on other CloudFormation resources can be passed in to the custom scripts at runtime
  • 8. CloudFormation Template Anatomy{ "Description" : "A text description for the template usage", "Parameters": { // A set of inputs used to customize the template per deployment }, "Resources" : { // The set of AWS resources and relationships between them }, "Outputs" : { // A set of values to be made visible to the stack creator }, "AWSTemplateFormatVersion" : "2010-09-09"}
  • 9. CloudFormation Template – Intrinsic Functions "Fn::Base64" : {valueToEncode} "Fn::FindInMap" : ["MapName", "Key", "Value"] "Fn::GetAtt" : ["logicalNameOfResource", "attributeName"] "Fn::GetAZs" : "region" "Fn::Join" : ["delimiter"[comma-delimited list of values]] "Ref" : "logicalName"
  • 10. Auto Scaling Launch or Terminate EC2 instances based on your policies, schedules or health-checks Scale up to a minimum number of secondary nodes in a Couchbase Membase Cluster Scale based on CPU, network, memory usage or other CloudWatch metrics Scale based on Custom Metrics (data captured from the cluster itself)  CloudWatch can capture custom metrics from you application enabling you to define custom auto scaling policies Auto Scaling groups, policies and configurations can be created in a CloudFormation template
  • 11. Auto Scaling Example3000 - 3000 CPU’s for one firm’s risk management processesNumber of EC2 Instances 300 CPU’s on weekends300 - Wednesday Thursday Friday Saturday Sunday Monday Tuesday 4/22/2009 4/23/2009 4/24/2009 4/25/2009 4/26/2009 4/27/2009 4/28/2009
  • 12. DemoCouchbase Membase 6-Pack Cluster creation withCloudFormation Walk through Membase 6-Pack Cluster Template Run CloudFormation Template via the Console Show the newly created Membase clusterAutoscaling Membase 6-Pack Cluster Create Autoscaling Group for the Membase Cluster Create Autoscaling policies Implement Autoscaling Demo the autoscaled Membase Cluster
  • 13. Membase CloudFormation Template - Parameters
  • 14. Membase CloudFormation Template - Mappings
  • 15. Membase CloudFormation Template - Resources Membase Cluster Coordinator Node Use an AMI based on the mapping Create an Instance based on Type parameter Parameter for Membase Package Installation
  • 16. Membase CloudFormation Template - Resources User Data defines the script that will be run at instance creation Trigger for Wait Handle
  • 17. Membase CloudFormation Template - Resources Create Auto Scaling group for secondary nodes Secondary node configurations defined separately Wait on Coordinator node creation before creating the Auto Scaling group
  • 18. Cluster Coordinator NodeSecondary nodes created by Auto Scaling Group
  • 19. Secondary node 2 Secondary nodesconfigurations defined terminated separately
  • 20. Auto Scaling starts 2 new nodes to maintain a minimum of 6 nodes
  • 21. What else? Auto scale Membase Cluster Coordinator Node  Recycle AWS Elastic IP  Membase Cluster end-point stays the same Attach AWS SNS notifications to Auto Scaling events Create Custom Auto Scaling policies  Push custom Membase cluster metrics to Cloud Watch and scale Automate Membase application deployment  Elastic Bean Stalk  Chef and Puppet Fail-over down Membase Cluster nodes appropriately Parameterize the Cluster size and Auto Scale policies
  • 22. More Information CloudFormation http://aws.amazon.com/cloudformation Auto Scaling http://aws.amazon.com/autoscaling