• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Real life aws_user_data_auto_scale
 

Real life aws_user_data_auto_scale

on

  • 1,154 views

Real Life AWS usages of user-data and AutoScale.

Real Life AWS usages of user-data and AutoScale.

Statistics

Views

Total Views
1,154
Views on SlideShare
1,153
Embed Views
1

Actions

Likes
0
Downloads
11
Comments
0

1 Embed 1

http://www.slashdocs.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Real life aws_user_data_auto_scale Real life aws_user_data_auto_scale Presentation Transcript

    • Real Life AWS: user-data & AutoScale Adam Neumann Solutions Architect at IndexMedia [email_address] &
    • Part 1: user-data
        • What is user-data?
        • A problem
        • user-data in the solution
        • Further work
    • What is user-data?
        • user specified instance attribute
        • accessible via HTTP and curl/wget
        • usage wrapped by cloud-init
        • 'runs' the user data once on boot
        • great for configuring instances
    • The concept
        • any script, or init-cloud format
        • #! /bin/bash etc etc
        • cmd line arg or via UI:
    • A Problem
        • too hard to spin up new instances
        • complex, undocumented, manual
        • fab, bash, and misc scripts
        • slow: 30 min per instance
        • first: consolidate, then: user-data
    • IndexMedia example 1
        • small user-data script, bootstrap
        • load big script from S3
        • setup apache, hadoop, virtualenv,...
        • 16 mins, vanilla ubuntu to serving
        • automated: yes, optimal: no
        • could be improved...
    • IndexMedia example 2
        • split script into 2 scripts :)
        • 1: heavy, general, static = 15 min
        • runs on vanilla ubuntu, creates AMI
        • 2: light, instance specific = 90 secs
        • runs on AMI
        • from 30 min manual to 90 sec auto
    • Further work
        • investigate cloud-init format
        • include Puppet/Chef (first, learn Puppet/Chef...)
        • integrate into CI process (environment becomes versioned, deployable, etc)
    • user-data summary
        • great way to configure instances
        • UI and cmd line accessible
        • enables automated+repeatable deployment
        • reduced deployment time, 30 mins to 90 secs
    • Part 2: AutoScale
        • What is AutoScale
        • Our (common) problem
        • Show me the examples!
        • Learnings
    • Part 2: AutoScale
        • cmd line only = no UI = hidden = unknown = scary
        • basically:
        • launching & terminating instances
        • within some bounds
        • based upon metrics
        • a second order system :)
    • AutoScale concepts
      • 1. Launch Configuration
      • individual: instance type, AMI, user-data, etc
      2. AutoScale Group collective: ideal/max/min, AZs 3. Policy how: a reaction, increase, decrease 4. Alerts when: an event, high/low load, CW metrics
    • Our (common) problem
        • maintain consistent experience for users, ie. Pip load time
        • turn on servers when load is high
        • bootstrapped, must turn off servers when load is low!
        • enter AutoScale...
    • Solution = AutoScale
        • servers start/stop based upon current system state
        • spending on IT matches user demand exactly!
        • 4 commands to set it up...
    • 1/4: Launch Configuration
      • as-create-launch-config gorilla_config_7d2
      • --image-id ami-XXXXXXX
      •          --instance-type t1.micro
      • --group "gorilla-live" 
      •          --key indexmedia-live
      • --user-data-file /path/to/bootstrap.sh
    • 2/4: Group
      • as-create-auto-scaling-group as_group
      • --launch-configuration gorilla_config_7d2
      • --availability-zones us-west-1a us-west-1b
      • --min-size 10 --max-size 20
      • --load-balancers my_balancer
      • --health-check-type EC2
      •          --grace-period 120
    • 3/4: Policy
      • as-put-scaling-policy gorilla_scaleup_policy
      • --auto-scaling-group as_group 
      •          --type ChangeInCapacity 
      • --adjustment=1
    • 4/4: Alarm
      • mon-put-metric-alarm "gorilla high load"
      •      --dimensions "AutoScalingGroupName=as_group"
      •      --namespace "AWS/EC2"
      •      --statistic Average --metric-name CPUUtilization
      •      --comparison-operator GreaterThanThreshold
      •      --threshold 80
      •      --evaluation-periods 5 --period 60
      •      --alarm-actions ${POLICY-ARN}
    • ... and done! Learnings
        • 4 steps (well 6...) = elastic system
        • feedback = second order system
        • avoid bad harmonics!
        • shorter instance boot time == quicker system  response time
        • validate alarms & behaviour w/ load tests on AWS
    • AutoScale summary
        • cmd line only; just jump in!
        • 4 distinct concepts, one per cmd
        • quick boot = shorter grace period = quicker scaling up (enter user-data...)
        • key to our multi-AZ, HA elastic cluster, problem solved!
    • Thanks + Questions? Adam Neumann Solutions Architect at IndexMedia [email_address]