It's about how we can optimally rent virtual server (i.e., EC2 instances) from Amazon. Firstly presented in IEEE MASCOTS 2011 conference in Raffles Hotel, Singapore.

1. Cost Minimization for Provisioning
Virtual Servers in Amazon Elastic
Compute Cloud
Sivadon Chaisiri1, Rakpong Kaewpuang1, Bu-Sung Lee1,2, and Dusit Niyato1
p g p g g y
1 School of Computer Engineering, Nanyang Technological University, Singapore
2 HP Labs Singapore
In Proc. IEEE Int. Sym. On Modeling, Analysis and Simulation of Computer and
Telecommunication Systems (MASCOTS) 2011, Raffles Hotel Singapore July 25 27
2011 Hotel, Singapore, 25-27
2011
CeMNeT/PDCC Meeting
October 13th, 2011

2. Agenda
• Amazon Elastic Compute Cloud (EC2)
• Research Challenges
• System Model & A
S t M d l Assumptions
ti
• Proposed Algorithms
• Performance Evaluation
• Summary
2

3. Amazon Elastic Compute Cloud
• EC2 = virtual server hosting service
– Dynamic & scalable provision of EC2 instances
– Several types of servers (i e instance types)
(i.e.,
– Pay-per-use basis
– Selectable hosting zones (i e Regions and Zones)
(i.e.,
– Integrated services e.g., EBS, S3, and SimpleDB
• Ser er
Server pro isioning
provisioning options
options: on-demand,
d d
reserved, and spot options
3

4. Amazon Elastic Compute Cloud (cont..)
• Prices of Linux instance in Northern Virginia
Instance Type On-demand Reserved* Reserved* Spot**
1-Year 3-Year
t1.micro ¢2 / hour ¢0.7 / hour, ¢0.7 / hour, ¢0.9 / hour
fee $54 fee $82
m1.small ¢8.5 / hour ¢3 / hour, ¢3 / hour, ¢2.9 / hour
fee $227.50 fee $350
c1.xlarge ¢68 / hour ¢24 / hour, ¢24 / hour, ¢24 / hour
fee $1 820
$1,820 fee $2 800
$2,800
* The one-time fee is paid in advance. The usage price is discounted.
** A spot price on December 11 2010
11,
4

5. Amazon Elastic Compute Cloud (cont..)
• Spot option
– Auction-based provisioning
– User submits a request & bid
price of spot instances to EC2
– Spot price set by Amazon is
fluctuated based on supply-
and-demand
– Amazon makes a decision: IF
bid price > spot price THEN the
spot instances can run
– Every running spot instance is
charged with the same spot
price
– Running spot instances can be
terminated without warning!
5

6. Amazon Elastic Compute Cloud (cont )
(cont..)
• Server provision of Linux c1.xlarge instance
Option Price Cost Save
On-demand (1 Year) ¢68 / hour $5,956.80 Based
Reserved (1 Year) ¢24 / hour, fee $1 820
hour $1,820 $3,922.40
$3 922 40 34.15%
34 15%
Spot* (1 Year) ¢24 / hour $2,102.40 64.71%
Option Price Cost Save
On-demand (3 Years) ¢68 / hour $17,870.40 Based
Reserved (3 Years) ¢24 / hour, fee $2,800 $9,107.20 49.04%
Spot* (3 Years) ¢24 / hour $6,307.20 64.71%
* Spot price is assumed to be fixed
6

7. Research Challenges
• EC2 instances provisioning Only 1 server is required
with prob 0.6
– How many instances ? and which
instance types to be provisioned ?
– How to achieve the optimal number of
instances under uncertainties ? Cost
minimization of server provision Demand uncertainty
• Demand uncertainty of an application
f li ti
– Amount of server-hours required to
Spot price > on-demand with prob 0.37
execute certain application
pp
Reservation price ($0.007)
• Price uncertainty of spot instances
On-demand price ($0.02)
– Spot price < reservation, >= reservation,
< on-demand, >= on demand
on demand on-demand
• Availability of spot instances
Price uncertainty
– Described by Bernoulli distribution of t1 micro
t1.micro
(% success bid or % failure bid)
7

8. System Model & Assumptions
• Two algorithms based on stochastic programming
– Long-term server provisioning algorithm considers reserved and on-demand
– Short-term server provisioning algorithm considers spot and on-demand
p g g p
• Algorithm solution: the optimal number of instances of each instance
type and provisioning option provisioned to each application
• A EC2 i t
An instance i provisioned t a certain application
is i i d to t i li ti
• Multiple instances can be grouped to serve the same
application
8

9. System Model & Assumptions (cont..)
• Performance factor
– : performance of the based server
– : performance of instance type
f fi t t
• Not consider how a bid price is defined for spot instances
• Neglect data transfer costs
• Additional product/service costs are neglected e.g., S3 and EBS
• Checkpointing mechanism is available
• Uncertainty
– Uncertain parameter is described by scenarios associating with a
probability distribution (a scenario occurs with a probability)
– The set of all possible scenarios is obtained as follows:
9

10. Proposed Algorithms
• Stochastic programming model of long term provisioning
long-term
• Robust optimization model
• “Robust”: less sensitive to any occurrence of uncertainty
• Solution-robustness : close to optimal solution
• Model-robustness : avoidable on-demand & oversubscribed costs
Expectation Variance Penalty
10

11. Proposed Algorithms (cont..)
• St h ti programming model of short-term provisioning
Stochastic i d l f h tt i i i
where denotes the total provisioning cost
11

12. Proposed Algorithms (cont..)
• S
Sample-average approximation (SAA) i applied t short-
l i ti is li d to h t
term planning
– Computational complexity reduction i.e., scenario set is too big
p p y , g
– Sampling technique e.g., Monte Carlo or Latin Hypercube
– Scenario size is reduced while (near-) optimal solution is achieved
• Solution of SAA (detailed in Section IV B)
IV-B)
– Lower bound estimates
where and
– Upper bound estimates
where and
12

13. Performance Evaluation
• Numerical studies are performed to evaluate long-and short-
term provisioning algorithms
p g g
• The algorithms are implemented and solved by GAMS/CPLEX
• Only single one application is considered
• Obtain performance factors (λ) by HPL and GotoBLAS2
• Real data from HPCC@NTU and Amazon EC2
• E l t di t
Evaluated instance ttypes
Instance Type Price of Price of GFLOPS Performance
reserved On-demand factor, λ
c1.xlarge ¢24 / hour, fee $1,820 ¢68 / hour 57.20 1/2
m1.xlarge ¢24 / hour, fee $1,820 ¢68 / hour 24.44 1/3
t1.micro ¢0.7 / hour, fee $54 ¢2 / hour 4.96 1/15
13

14. Performance Evaluation (cont..)
Evaluation of long term provisioning algorithm (1 year planning)
long-term
• Demand: discrete normal dist. with 16 scenarios, mean 8.5, var 2
• Notations
– : optimal solution of deterministic optimization problem
– : solution of on-demand provisioning
– : optimal solution of stochastic programming problem
– : optimal solution of robust optimization problem
– Optimality closeness, , for (od) is always 1.5
– odc : on-demand cost
odc.
– osc. : oversubscribed cost
14

15. Performance Evaluation (cont..)
Evaluation of short-term provisioning algorithm
p g g
• Uncertainty parameters
• Bid prices for c1.xlarge and m1.xlarge: $0.264 and $0.240
• Probability of success bid for c1.xlarge and m1.xlarge: 0.899 and 0.467
• Results: provision 100 server-hours for both instance types, but zero
server-hour for t1.micro
• Other experiment: spot p
p p prices p1 and p2 occur with prob 0.95 and 0.05
p
15

16. Performance Evaluation (cont..)
Evaluation of short-term provisioning algorithm
short term
• Evaluation of different sampling techniques using SAA approach
Monte Carlo Latin Hypercube
16

17. Summary
• Long- and short-term server provisioning algorithms
based on stochastic programming have been proposed
• The algorithms can minimize the total provisioning
costs under demand and price uncertainty
• The algorithms can be applied to other cloud providers
(e.g., GoGrid and SpotCloud Market)
• Future work: efficient strategies for bidding spot
instances under price and demand fluctuation
17

18. THANK YOU
18

19. System Model & Assumptions (cont..)
• Discrete probability distributions
– Distribution of spot prices, EC2 publicly provides historical spot prices
– Distribution of demand e.g., log files of server access
demand, e g
• How to achieve the amount of server-hours
Number of business transaction
BLACK BOX
/ computational jobs
Amount of server-hours
server hours
19