Your SlideShare is downloading. ×
0
Jesse AndersonEC2 PERFORMANCE, SPOT INSTANCE ROI ANDEMR SCALABILITY
AMAZON WEB SERVICES (AWS)   Elastic Cloud Compute (EC2)     Virtual   Machine in Cloud   Simple Storage Service (S3)   ...
EC2 PRICE TYPES   Spot Instances     Systemfor bidding on unused instances     Same Performance     Go away (abruptly)...
SPOT INSTANCE SAVINGS
MILLION MONKEYS PROJECT Randomly recreated Shakespeare Open source Good metric for CPU and memory
EC2 SPECIFICATIONSInstance Name    Memory   EC2 Compute         Platform I/O                          Units/Cores         ...
EC2 PERFORMANCE   My Core 2 Duo 2.66 GHZ did 50,000,000,000 character groups
EC2 COST PER HOUR ON DEMAND/SPOT
PRICE PER UNIT
EMR (HADOOP) CLUSTERING Tests of 1, 2, 3, 4, 5, 10, 20 node clusters Price Scalability
EMR COST
PRICE PER UNIT IN A CLUSTER
CLUSTERED CHARACTER GROUPS
EMR/HADOOP SCALABILITY PERCENTAGE
EMR/HADOOP SCALABILITY ABSOLUTE
BREAKDOWNS   Original project would have run in 3 days 9    hours     Took   1.5 months before 20 node cluster costs $4...
ENGINEERING FOR THE CLOUD Establish if a good fit Test the EC2 performance Figure out a unit or widget Find the most c...
CONCLUSIONS   Spot Instance Saves     From $2.20 to $1.30 per hour     Saved $1,000 in one run   Hadoop/EMR Scalabilit...
MORE INFORMATION http://www.jesse-anderson.com/2012/02/ec2-  performance-spot-instance-roi-and-emr-  scalability/ @jesse...
Upcoming SlideShare
Loading in...5
×

EC2 Performance, Spot Instance ROI and EMR Scalability

1,938

Published on

The presentation accompanying my research into Amazon Web Services EC2 performance, Spot instance ROI and EMR (Hadoop) scalability.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,938
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "EC2 Performance, Spot Instance ROI and EMR Scalability"

  1. 1. Jesse AndersonEC2 PERFORMANCE, SPOT INSTANCE ROI ANDEMR SCALABILITY
  2. 2. AMAZON WEB SERVICES (AWS) Elastic Cloud Compute (EC2)  Virtual Machine in Cloud Simple Storage Service (S3)  Network Share in Cloud Elastic MapReduce (EMR)  Cluster of EC2 instances for Hadoop cluster
  3. 3. EC2 PRICE TYPES Spot Instances  Systemfor bidding on unused instances  Same Performance  Go away (abruptly) if outbid On Demand  Ad Hoc starting Reserved  Not Covered
  4. 4. SPOT INSTANCE SAVINGS
  5. 5. MILLION MONKEYS PROJECT Randomly recreated Shakespeare Open source Good metric for CPU and memory
  6. 6. EC2 SPECIFICATIONSInstance Name Memory EC2 Compute Platform I/O Units/Cores PerformanceSmall 1.7 GB 1 EC2 on 1 Core 32-bit ModerateLarge 7.5 GB 4 EC2 on 2 Cores 64-bit HighExtra Large 15 GB 8 EC2 on 8 Cores 64-bit HighHigh-CPU 1.7 GB 5 EC2 on 2 Cores 32-bit ModerateMediumHigh-CPU Large 7 GB 20 EC2 on 8 Cores 64-bit HighQuad XL 23 GB 33.5 on 8 Cores 64-bit Very High EC2 Compute Unit (ECU) – One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.
  7. 7. EC2 PERFORMANCE My Core 2 Duo 2.66 GHZ did 50,000,000,000 character groups
  8. 8. EC2 COST PER HOUR ON DEMAND/SPOT
  9. 9. PRICE PER UNIT
  10. 10. EMR (HADOOP) CLUSTERING Tests of 1, 2, 3, 4, 5, 10, 20 node clusters Price Scalability
  11. 11. EMR COST
  12. 12. PRICE PER UNIT IN A CLUSTER
  13. 13. CLUSTERED CHARACTER GROUPS
  14. 14. EMR/HADOOP SCALABILITY PERCENTAGE
  15. 15. EMR/HADOOP SCALABILITY ABSOLUTE
  16. 16. BREAKDOWNS Original project would have run in 3 days 9 hours  Took 1.5 months before 20 node cluster costs $45.44 per day 5 day run cost $317 11 day run cost $528
  17. 17. ENGINEERING FOR THE CLOUD Establish if a good fit Test the EC2 performance Figure out a unit or widget Find the most cost efficient EC2 performer with price per unit/widget Engineer with Spot Instances in mind
  18. 18. CONCLUSIONS Spot Instance Saves  From $2.20 to $1.30 per hour  Saved $1,000 in one run Hadoop/EMR Scalability  95% efficiency at 2-5 nodes  87% efficiency at 10 nodes  84% efficiency at 20 nodes
  19. 19. MORE INFORMATION http://www.jesse-anderson.com/2012/02/ec2- performance-spot-instance-roi-and-emr- scalability/ @jessetanderson
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×