Amazon’s Cloud Computing Efforts: S3 and EC2 By: Chris Sosa

Materials Provided By “ An Evaluation of Amazon’s Grid Computing Services: EC2, S3 and SQS” by Garfinkel When I refer to tests, I mean from this paper Blogs Miscellaneous articles Amazon’s Web Services website

“ An Evaluation of Amazon’s Grid Computing Services: EC2, S3 and SQS” by Garfinkel

When I refer to tests, I mean from this paper

Blogs

Miscellaneous articles

Amazon’s Web Services website

Motivation Producing a large accessible computing and storage infrastructure is expensive Physical Space, cooling, power, management Money Experience Large maintenance costs Never sure how much we actually need

Producing a large accessible computing and storage infrastructure is expensive

Physical Space, cooling, power, management

Money

Experience

Large maintenance costs

Never sure how much we actually need

Utility Computing Basically outsourcing infrastructure to the big guys It’s all about economy of scale Goal to make web-scale computing easier … how many buzz words does it take to fill a gallon jar?

Basically outsourcing infrastructure to the big guys

It’s all about economy of scale

Goal to make web-scale computing easier

… how many buzz words does it take to fill a gallon jar?

Amazon’s Cloud Computing Overview Amazon’s Elastic Compute Cloud (EC2) Provides resizable / on-demand computing capacity Amazon’s Simple Storage Service (S3) Storage for the Internet Provides web services interface to store and retrieve any amount of data at any time and from anywhere (for a price)

Amazon’s Elastic Compute Cloud (EC2)

Provides resizable / on-demand computing capacity

Amazon’s Simple Storage Service (S3)

Storage for the Internet

Provides web services interface to store and retrieve any amount of data at any time and from anywhere (for a price)

Amazon’s Web Service Model Interface Web-based dashboard REST API – standard HTTP(/s) GET, PUT, DELETE commands Remote procedure calls through SOAP Variety of tools that perform basic operations Security Each request is signed 40-character secret key with HMAC X.509 certificates either given to or created by Amazon Signed timestamps to prevent replay attacks Can use TLS / SSL

Interface

Web-based dashboard

REST API – standard HTTP(/s) GET, PUT, DELETE commands

Remote procedure calls through SOAP

Variety of tools that perform basic operations

Security

Each request is signed

40-character secret key with HMAC

X.509 certificates either given to or created by Amazon

Signed timestamps to prevent replay attacks

Can use TLS / SSL

First Up – EC2 Amazon uses Linux-based Virtual Machines running on Xen Actual resources are virtualized So much so that they are not even backed by real hardware “ Predictably provided” Two security mechanisms Write ssh public key into the VM image Firewall at the server side

Amazon uses Linux-based Virtual Machines running on Xen

Actual resources are virtualized

So much so that they are not even backed by real hardware

“ Predictably provided”

Two security mechanisms

Write ssh public key into the VM image

Firewall at the server side

EC2 Cost Model Instances $0.10 - Small Instance (per hour) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core - 1.7 GHz processor), 160 GB of instance storage, 32-bit platform $0.40 - Large Instance (per hour) 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform $0.80 - Extra Large Instance (per hour) 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform Data Transfer $0.10 per GB - all data transfer in $0.18 per GB - first 10 TB / month data transfer out $0.16 per GB - next 40 TB / month data transfer out $0.13 per GB - data transfer out / month over 50 TB

Instances

$0.10 - Small Instance (per hour)

1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core - 1.7 GHz processor), 160 GB of instance storage, 32-bit platform

$0.40 - Large Instance (per hour)

7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform

$0.80 - Extra Large Instance (per hour)

15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

Data Transfer

$0.10 per GB - all data transfer in

$0.18 per GB - first 10 TB / month data transfer out

$0.16 per GB - next 40 TB / month data transfer out

$0.13 per GB - data transfer out / month over 50 TB

The Good Only pay for what you need “Guaranteed” to be 99.9% reliable (good enough?) (targets 99.99%) Fast, responsive and reliable Only takes a couple minutes to start a new instance Data transfer free between S3 and EC2

Only pay for what you need

“Guaranteed” to be 99.9% reliable (good enough?) (targets 99.99%)

Fast, responsive and reliable

Only takes a couple minutes to start a new instance

Data transfer free between S3 and EC2

The Bad Still pay for computing you aren’t using but still running No persistent storage Can use S3 Writing to S3 slower and literally more expensive Poor termination clause Only given 60 day notice

Still pay for computing you aren’t using but still running

No persistent storage

Can use S3

Writing to S3 slower and literally more expensive

Poor termination clause

Only given 60 day notice

Second – S3 250 Mb/second network link Objects stored in buckets Buckets have own namespace Up to 100 buckets per account Unlimited objects per bucket 5 GB limit on size of objects Computes MD5 checksum’s to check if transfers are correct Up to application to retry ACL’s by accounts / certificates allowed per object and per bucket (two-level) Basic interface to read/write/create/delete/chmod etc

250 Mb/second network link

Objects stored in buckets

Buckets have own namespace

Up to 100 buckets per account

Unlimited objects per bucket

5 GB limit on size of objects

Computes MD5 checksum’s to check if transfers are correct

Up to application to retry

ACL’s by accounts / certificates allowed per object and per bucket (two-level)

Basic interface to read/write/create/delete/chmod etc

S3 Pricing Storage $0.15 per GB-Month of storage used Data Transfer $0.10 per GB - all data transfer in $0.18 per GB - first 10 TB / month data transfer out $0.16 per GB - next 40 TB / month data transfer out $0.13 per GB - data transfer out / month over 50 TB Requests $0.01 per 1,000 PUT or LIST requests $0.01 per 10,000 GET and all other requests* * No charge for delete requests

Storage

$0.15 per GB-Month of storage used

Data Transfer

$0.10 per GB - all data transfer in

$0.18 per GB - first 10 TB / month data transfer out

$0.16 per GB - next 40 TB / month data transfer out

$0.13 per GB - data transfer out / month over 50 TB

Requests

$0.01 per 1,000 PUT or LIST requests

$0.01 per 10,000 GET and all other requests* * No charge for delete requests

The Pretty Available during 100% of tests Performance Works well for large file sizes Approaches maximum of 30 Mb/s with multiple threads (6) for test in paper Performs at about 10 MB /s with one thread doing one large data push

Available during 100% of tests

Performance

Works well for large file sizes

Approaches maximum of 30 Mb/s with multiple threads (6) for test in paper

Performs at about 10 MB /s with one thread doing one large data push

The Pretty Ugly No guarantee of persistence in data Will real companies buy into this? Claimed throughput only achievable with data transfers of 16MB or larger High transaction overhead Amazon has declined to explain why! Same Service Level Agreement terms as EC2

No guarantee of persistence in data

Will real companies buy into this?

Claimed throughput only achievable with data transfers of 16MB or larger

High transaction overhead

Amazon has declined to explain why!

Same Service Level Agreement terms as EC2

Conclusion EC2 and S3 provide applications with the ability to allocate computing and storage resources on demand This IS useful Cheaper than the alternative for reasonably sized companies Most claims of service have been validated Issues with bandwidth claims for S3 Issues with its Service Level Terms may plague EC2 and S3 from becoming more utilized

EC2 and S3 provide applications with the ability to allocate computing and storage resources on demand

This IS useful

Cheaper than the alternative for reasonably sized companies

Most claims of service have been validated

Issues with bandwidth claims for S3

Issues with its Service Level Terms may plague EC2 and S3 from becoming more utilized

Questions