http://www.cs.virginia.edu/~mm5bw/papers/Cloud%20VM%20Startup%20Performance%20Study.pdf
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6253534
www.mingmao.org
Amazon Web Service, EC2, Windows Azure, Rackspace, Spot Instances, VM Image, OS,
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
A performance study on the vm startup time in the cloud
1. 1
A Performance Study on the VM
Startup Time in the Cloud
Cloud 2012
(June 25, Honolulu, Hawaii)
Ming Mao, Marty Humphrey
CS Department, University of Virginia
2. Introduction
2
One great advantage of the cloud is dynamic scalability,
BUT…
Only meaningful when the compute resource is available in time
The fact is that acquired instances are NOT ready immediately
Why study the cloud VM startup time
Important to time sensitive applications
Important to resource auto-scaling mechanisms
Inconsistent definitions across cloud providers
To set up a reference for the community
To know the difference from two years ago
3. Experiment setup
3
Type OS Image Location AWS, Windows Azure,
Amazon EC2 Rackspace
m1.small Linux(Fedora) ami-48aa4921 us-east-1a
m1.small Windows (Win Server 2008) us-east-1a Oct 15th, 2011 – Feb 15th 2012
ami-fbf93092
Windows Azure A client periodically collects the
Small WebRole South instance startup information
default WebRole app in Azure SDK Central US
Small WorkerRole South By different factors – time,
default WorkerRole app in Azure SDK Central US
Small VMRole South image size, machine type,
Win Server 2008R2 Central US
Rackspace
location, etc.
Type IV Linux (Fedora) flavor 71 N/A
Type IV Windows (Win Server 2008R2) flavor 28 N/A
VM Definition of VM startup time
Linux VMs The first successful ssh login
Windows VMs The first successful remote desktop connection
WebRoles The first successful http request
WorkerRoles The first successful logging
4. Results (1) – by time
4
Cloud OS Average VM startup time
EC2 Linux 96.9 seconds
EC2 Windows 810.2 seconds
Azure WebRole 374.8 seconds
Azure WorkerRole 406.2 seconds
Azure VMRole 356.6 seconds
Rackspace Linux 44.2 seconds
Rackspace Windows 429.2 seconds
5. Results (2) – by image size
5
Cloud Average Data Transfer Rate
between VM and Image Store
EC2 10.9 MB/s
Azure 1.1MB/s
Rackspace 22.5 MB/s
6. Results (3) – by machine type
6
Azure, Rackspace – the VM startup time goes longer as the
instance type goes larger
EC2 – no significant differences among different instance
types, except the micro ones
7. Results (4) – by location
7
No significant differences among locations
EC2 - The newly established data center has a slightly slower
startup performance and greater variance
8. Results (5) – multiple instances
8
EC2 – VM startup time is relatively constant.
Azure – the last VM instance sometimes took significantly
longer than the first one.
RS – does not support instance acquisition in batch
9. Results (6 - 1) – spot instances, by time
9
Spot instances have a longer waiting time but similar VM
startup performance compared to on-demand instances
10. Results (6 - 2) – spot instances, by vm type
10
Similar to on-demand instances, the VM startup time shows
no significant differences across different machine types
11. Results (6 - 3) – spot instances, by location
11
The US east region shows longer spot instance acquisition
time
12. Results (6 - 4) – spot instances, multiple instances
12
Similar to on-demand instances, spot instances show
constant performance across all the 16 machines acquired at
the same time
13. Results (6 - 5) – spot instances, by real-time price
13
No significant relations between the VM startup time and the
real-time price
14. Result (7) – other facts
14
Spot instances are not
always cheaper than on-
demand VMs
Acquisition requests are
not always successfully
served (unusable
machines).
Cloud Success Rate VM Release Time
EC2 99.2% 3-8 seconds
Azure 99.6% 8-21 seconds
Rackspace 92.0% 3-8 seconds
15. Summary
15
Summary
VM startup time is independent of time of the day.
EC2, Rackspace - Windows instances take 9 times longer
than Linux instances. Azure - all three Role instances show
similar performances.
Both the size of the OS image and the instance type can
largely affect the VM startup time.
spot instances - longer VM startup time and greater variance
RS had a higher instance acquisition failure rate (8%) > EC2
(0.8%) & Azure (0.4%).
Azure – 200-second improvement and smaller variances;
EC2 – performance does not change.