• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Decision support for Amazon Spot Instance
 

Decision support for Amazon Spot Instance

on

  • 460 views

 

Statistics

Views

Total Views
460
Views on SlideShare
460
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Decision support for Amazon Spot Instance Decision support for Amazon Spot Instance Presentation Transcript

    • Decision  Support  for  Amazon   EC2  Spot  Instances   Fei  Dong   2011-­‐11-­‐28  11/27/11   1  
    • A  Glimpse  of  Amazon  EC2  •  Reserved  Instance,  On-­‐demand  Instance,  and  SI  •  Different  scenarios:  Cluster  ×  Workload   EC2  Node   CPU   Memory I/O   Per-­‐hour   Type (#EC2  units) Performance Cost m1.small 1 1.7  GB moderate $0.085 m1.large 4 7.5  GB high $0.34 m1.xlarge 8 15  GB high $0.68 c1.medium 5 1.7  GB moderate $0.17 c1.xlarge 20 7  GB high $0.68 cc1.4xlarge 33.5 23  GB very  high $1.60 11/27/11   2  
    • MulU-­‐objecUve  Cluster  Provisioning   1,200Running Time 1,000 800 (min) 600 Actual 400 200 0 m1.small m1.large m1.xlarge c1.medium c1.xlarge 10.00 8.00 Cost ($) 6.00 4.00 Actual 2.00 0.00 m1.small m1.large m1.xlarge c1.medium c1.xlarge EC2 Instance Type for Target Cluster11/27/11   3  
    • Spot  Instance  •  Spot  instances  enable  you  to  bid  for  unused   Amazon  EC2  capacity.  Instances  are  charged   the  Spot  Price  which  is  set  by  Amazon  EC2  and   fluctuates  periodically  depending  on  the   supply  of  and  demand  for  Spot  Instance   capacity.  11/27/11   4  
    • Challenges  &  AssumpUons  •  Challenges:   –  Minimize  monetary  costs  for  a  user  while  meeUng   Service  constrains.   –  Know  nothing  about  Amazon  pricing  strategy  and   other  bid  strategy.  •  AssumpUons:   –  Bid  price  is  fixed.   –  Instance  Type  is  fixed  (no  mix  strategy)   –  Not  consider  the  overhead  to  recover  spot  instances.  11/27/11   5  
    • Pricing  PredicUon  Model  •  Linear  Regression  •  Normal  DistribuUon  •  ExponenUal  DistribuUon   n   ∑ p (1− p )i −1 (i ) H i =111/27/11   6  
    • Predict  Price  Algorithm  1.  Collect  the  prices  over  a  period  of  Ume,  in   order  to  esUmate  mean  and  variance.  2.  Use  the  exponenUal  approximaUon  fidng,   calculate  x  given  the  CDF(X<x)  =  Prob.    3.  Compare  other  models  and  pick  a  maximum   value  as  a  bid  price.  4.  If  the  bid  price  is  smaller  than  the  spot  price,   thus  increase  the  bid  by  33%  for  the  next   interval.  11/27/11   7  
    • Price  PredicUon   eu  linux.m1.small  spot  price  on  11/18/2011   0.35   0.3   0.25   0.2  Price  ($)   Actual   0.15   Predict  Adjust   Predict   0.1   0.05   0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   15   16   17   18   19   20   21   22   23   24   Time  (Hours)   Var(Predict  Adj)  =  0.000769   11/27/11   Var(Predict)                =  0.001786   8  
    • Bid  Strategy   Utility = F(d, b, e, t, n) Deadline d, budget b, Estimated Time e, copt = arg max F(d, b, e, t, n) c∈S Cluster Type t, Number n Min  Time  Mode:  Can  the  job  be  execute  as  soon  as   possible  under  specified  budget  and  deadline   constrains?   Min  Money  Mode:    What  is  the  bid  price  and  instance   type  that  minimize  the  total  monetary  cost?   ExhausUve   Search  11/27/11   9  
    • Experimental  EvaluaUon  •  Choose  5  Spot  Instance  Types   –  M1.small,  m1.large,  m1.xlarge,  c1.medium,   c1.xlarge  •  Run  5  Instances  compared  with  on  demand   instances.  11/27/11   10  
    • Experimental  EvaluaUon  (Ctd.)     1600   1400   Running  Time  (Min)   1200   1000   800   on-­‐demand   600   SI  budget  intensive   400   SI  Ume  intensive   200   0   m1.small   m1.large   m1.xlarge   c1.medium   c1.xlarge   EC2  Instance  Type   14   12   10   Cost  ($)   8   on-­‐demand   6   SI  budget  intensive   4   SI  Ume  intensive   2   0   m1.small   m1.large   m1.xlarge   c1.medium   c1.xlarge   EC2    Instance  Type  11/27/11   11  
    • Case  Analysis   M1.small  Linux  on  11/18/2011   M1.small  spot  instance,  bid  strategy  11/27/11   12  
    • Conclusions  &  Future  Work  •  Conclusions –  More cost-efficient than fixed-size instance choice –  Spot Instances not always provide inexpensive resources for transient workloads•  Future works –  Consider to mix other instance types (e.g. spot instances & reserved instances) –  Disaster Recovery, checking point.11/27/11   13  
    • Reference  •  hjp://aws.amazon.com/ec2/instance-­‐types  •  H.  Herodotou,  F.  Dong,  and  S.  Babu.  No  One  (Cluster)  Size  Fits  All:   AutomaUc  Cluster  Sizing  for  Data-­‐intensive  AnalyUcs.  (Slides)In  Proc.  of   the  ACM  Symposium  on  Cloud  CompuUng  2011  (SOCC  11),  October  2011.  •  D.  Ardagna,  B.  Panicucci  and  M.Passacantando.  A  Game  TheoreUc   FormulaUon  of  the  Service  Provisioning  Problem  in  Cloud  Systems.   WWW2011  Proceedings,  2011  •  N.  Jain,  I.  Menache,  and  O.  Shamir.  On-­‐demand  or  Spot?  Learning-­‐based   Resource  AllocaUon  for  Delay-­‐Tolerant  Batch  CompuUng.      11/27/11   14  
    •   Thank  youJ  11/27/11   15