Amazon WS Overview

8,276 views
8,136 views

Published on

A compact overview of Amazon's WebServices offerings.

Published in: Business, Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,276
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
185
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Amazon WS Overview

  1. 1. Amazon Web Services – Introduction & Discussion
  2. 2. Amazon Web Services – <ul><li>Overview of Amazon's Services </li></ul><ul><ul><li>Explore each type of service </li></ul></ul><ul><li>Not implementation specific [1] </li></ul><ul><li>Open Discussion </li></ul>[1] http://highscalability.com/amazon-architecture
  3. 3. Amazon Web Services – The Picture Associates Alexa Marketplace DevPay WebStore Prime Grocery Historical Pricing [1] http://highscalability.com/paper-dynamo-amazon-s-highly-available-key-value-store
  4. 4. <ul><li>Small Pieces Loosely Joined – </li></ul><ul><li>“ ...There are intriguing assertions: that the Web is &quot;broken on purpose&quot; and that its many pockets of erroneous information and its available forums [...] let people feel more comfortable with their own inherent imperfections . [...] Weinberger's analysis, though occasionally facile and too relentlessly optimistic and overstated , is surely destined to be the subject of furious debate in chat rooms the cyber-world over. --H. O'Billovich” [1] </li></ul>[1] http://www.amazon.com/Small-Pieces-Loosely-Joined-Unified/dp/B00008NRGI/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1204731761&sr=8-1 Amazon Web Services – “Defined”
  5. 5. <ul><li>Small Pieces Loosely Joined </li></ul><ul><li>Think Unix Model – </li></ul><ul><ul><li>“ Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.” -- Doug McIlroy [1] </li></ul></ul>[1] [Salus] Peter H. Salus. A Quarter-Century of Unix. Addison-Wesley. 1994. ISBN 0-201-54777-5 Amazon Web Services – “Defined”
  6. 6. <ul><li>Small Pieces Loosely Joined </li></ul><ul><li>Think Unix Model </li></ul><ul><li>“ Single” Deployment </li></ul><ul><li>Embrace the Power of Simplicity </li></ul><ul><li>Open Collaboration </li></ul><ul><li>Leverages “Momentum Makers” </li></ul><ul><li>“ Force Multiplication” </li></ul><ul><ul><li>Estimates of 40% of sales from affiliates [1] </li></ul></ul>Amazon Web Services – “Defined” [1] http://en.wikipedia.org/wiki/Amazon.com
  7. 7. Amazon Web Services – From the Ground Up <ul><li>Provides a Storage Service – Since April 2006 [1] </li></ul><ul><li>Geographically distributed content network </li></ul><ul><ul><li>U.S. and European Union </li></ul></ul><ul><ul><li>REST and SOAP interfaces over HTTP </li></ul></ul><ul><ul><li>A BitTorrent(TM) protocol interface is provided </li></ul></ul><ul><li>Does have a Service Level Agreement [2] </li></ul><ul><ul><li>25% for < 99% </li></ul></ul><ul><ul><li>10% for > 99% and < 99.9% </li></ul></ul><ul><ul><li>“ 3 nines” implies <= 8 hours 46 minutes down time </li></ul></ul>[1] http://en.wikipedia.org/wiki/Amazon_S3 [2] http://www.amazon.com/gp/browse.html%3fnode%3d379654011&token=3A0F170E7CEFE27BDC730D3D7344512BC1296B83
  8. 8. <ul><li>Uses – Extensive use internally </li></ul><ul><ul><li>SmugMug [1] </li></ul></ul><ul><ul><li>Many independent blogs & “startups” [2] </li></ul></ul><ul><ul><li>Even Firefox extensions as interfaces ! </li></ul></ul><ul><li>Cost – Less then you'd think [3] </li></ul><ul><ul><li>U.S. – $0.15 per GB/month </li></ul></ul><ul><ul><ul><li>In: $0.10 /GB, Out: graduated $0.13 - 0.18 GB/month </li></ul></ul></ul><ul><ul><ul><ul><li>$.01 per 1,000 PUT or 10,000 GET requests </li></ul></ul></ul></ul><ul><ul><ul><li>20GB & 2GB is ~$40 per year [4] </li></ul></ul></ul><ul><ul><li>Webmail.us reportedly saved 75% with S3 </li></ul></ul><ul><ul><li>No cost to transfer between Amazon Services ! </li></ul></ul>[1] http://blogs.smugmug.com/don/2006/11/10/amazon-s3-show-me-the-money/ [2] http://calculator.s3.amazonaws.com/calc5.html [3] http://www.codinghorror.com/blog/archives/000808.html [4] http://www.jungledisk.com/ Amazon Web Services – From the Ground Up
  9. 9. Amazon Web Services – From the Ground Up <ul><li>Files (objects) are placed in “buckets” </li></ul><ul><ul><li>Unlimited objects in a bucket </li></ul></ul><ul><ul><ul><li>Buckets host objects between 1byte to 5GB </li></ul></ul></ul><ul><ul><ul><li>Objects can be public, private or specific rights </li></ul></ul></ul><ul><ul><li>Retrieved via a unique key – global address space </li></ul></ul><ul><ul><ul><li>Typically linked with an MD5sum </li></ul></ul></ul><ul><ul><li>Subdirectories are done via strategic naming </li></ul></ul><ul><ul><ul><li>“ jay” and “jay/photos” are logically independent but might be related at an application level </li></ul></ul></ul><ul><ul><ul><li>Another interesting solution involves loopback mounts [1] </li></ul></ul></ul>[1] http://developer.amazonwebservices.com/connect/thread.jspa?threadID=10271&tstart=30
  10. 10. <ul><li>Examples: </li></ul><ul><ul><li>CLI [1] </li></ul></ul><ul><ul><ul><li>./s3-get -k MyAWSAccessKeyId -s AWSSecretAccessKeyIdFile /bucket/pathtoobject </li></ul></ul></ul><ul><ul><li>Python [2] </li></ul></ul><ul><ul><ul><li>>>> from boto.s3.connection import S3Connection, Key </li></ul></ul></ul><ul><ul><ul><li>>>> conn = S3Connection('<aws access key>', '<aws secret key>') </li></ul></ul></ul><ul><ul><ul><li>>>> bucket = conn.create_bucket('garnaat_fileit') </li></ul></ul></ul><ul><ul><ul><li>>>> k = Key(bucket) </li></ul></ul></ul><ul><ul><ul><li>>>> k.key = 'foobar' </li></ul></ul></ul><ul><ul><ul><li>>>> k.set_contents_from_string('This is a test of S3') </li></ul></ul></ul><ul><ul><ul><li>>>> k.ghet_contents_as_string() </li></ul></ul></ul><ul><ul><ul><li>>>> k.key = 'myfile' </li></ul></ul></ul><ul><ul><ul><li>>>> k.set_contents_from_filename('foo.jpg') </li></ul></ul></ul><ul><ul><ul><li>>>> k.get_contents_to_filename('bar.jpg') </li></ul></ul></ul>[1] http://code.google.com/p/s3-bash/ [2] http://boto.googlecode.com/svn/trunk/doc/s3_tut.txt Amazon Web Services – From the Ground Up
  11. 11. Amazon Web Services – Redefining the Dictionary <ul><li>... if databases were straightforward and easy </li></ul><ul><li>Web Services interface – Queries and Submissions </li></ul><ul><li>No schema – </li></ul><ul><ul><li>Automatic indexes </li></ul></ul><ul><ul><li>Loose enforcement of structure </li></ul></ul><ul><ul><li>Consistency is “eventual” (MVCC) [1] </li></ul></ul><ul><li>Automatic scaling & caching </li></ul><ul><ul><li>No configuration & management required </li></ul></ul><ul><ul><li>For “real-time data” applications; e.g. shopping carts </li></ul></ul><ul><ul><li>Heavily reliant on S3 </li></ul></ul>[1] http://en.wikipedia.org/wiki/Multiversion_concurrency_control http://www.sriramkrishnan.com/blog/2007/12/amazon-simpledb-technical-overview.html
  12. 12. <ul><li>CREATE a domain to store data </li></ul><ul><ul><li>Beta constraint – 100 domains max 10 GB each </li></ul></ul><ul><li>GET, PUT or DELETE items </li></ul><ul><ul><li>Items have attribute-value pairs </li></ul></ul><ul><ul><ul><li>“ size = large” </li></ul></ul></ul><ul><ul><ul><li>“ color = blue, red” </li></ul></ul></ul><ul><ul><li>Each item can have up to 256 attribute values. </li></ul></ul><ul><ul><ul><li>Each attribute value can range from 1 to 1,024 bytes. </li></ul></ul></ul><ul><li>QUERY – max 5 seconds </li></ul><ul><ul><li>Operators: =, !=, <, > <=, >=, STARTS-WITH, AND, OR, NOT, INTERSECTION AND UNION. </li></ul></ul>http://www.amazon.com/b/ref=sc_fe_l_2?ie=UTF8&node=342335011&no=3435361&me=A36L942TSJ2AJA Amazon Web Services – Redefining the Dictionary
  13. 13. <ul><li>Provides a Storage Queue – Simple! </li></ul><ul><ul><li>For $1 a user can transmit over 500,000 messages [1] </li></ul></ul><ul><ul><li>$0.01 per 10,000 Amazon SQS Requests ($0.000001 per Request) </li></ul></ul><ul><ul><li>Other bandwidth fees still apply (similar to S3 #'s) </li></ul></ul><ul><li>Message Characteristics </li></ul><ul><ul><li>Max messages size - 8KB </li></ul></ul><ul><ul><li>Latency should be anticipated [2] </li></ul></ul><ul><ul><li>Transmit and Receive order not guaranteed </li></ul></ul><ul><ul><li>Messages held for 4 days max </li></ul></ul><ul><li>Data Flow – </li></ul><ul><ul><li>Find an Amazon SQS queue </li></ul></ul><ul><ul><li>SendMessage to the queue </li></ul></ul><ul><ul><li>ReceiveMessage is called </li></ul></ul><ul><ul><li>ReceiveMessage returns a message </li></ul></ul><ul><ul><ul><li>Locks that message until the visibility timeout has passed. </li></ul></ul></ul><ul><ul><li>DeleteMessage removes the message from the queue </li></ul></ul>[1] http://www.amazon.com/Simple-Queue-Service-home-page/b?ie=UTF8&node=13584001 [2] http://en.wikipedia.org/wiki/Amazon_SQS Amazon Web Services – “Ground Control to Major Tom...”
  14. 14. <ul><li>Example: </li></ul><ul><ul><li>Python [1] </li></ul></ul><ul><ul><ul><li>>>> from boto.sqs.connection import SQSConnection, Message </li></ul></ul></ul><ul><ul><ul><li>>>> conn = SQSConnection('<aws access key>', '<aws secret key>') </li></ul></ul></ul><ul><ul><ul><li>>>> q = conn.create_queue('myqueue', 60) </li></ul></ul></ul><ul><ul><ul><li>>>> m = Message() </li></ul></ul></ul><ul><ul><ul><li>>>> m.set_body('This is my first message.') </li></ul></ul></ul><ul><ul><ul><li>>>> rs = q.write(m) </li></ul></ul></ul><ul><ul><ul><li>>>> rs = q.get_messages() </li></ul></ul></ul><ul><ul><ul><li>>>> rs[0].get_body() </li></ul></ul></ul><ul><ul><ul><li>>>> q.delete_message(rs[0]) </li></ul></ul></ul>[1] http://boto.googlecode.com/svn/trunk/doc/sqs_tut.txt [2] http://aws.typepad.com/aws/2007/05/sqs_super_queue.html Amazon Web Services – “Ground Control to Major Tom...”
  15. 15. Amazon Web Services – “Show me the Money” <ul><li>Supports multiple forms of payment </li></ul><ul><ul><li>Permits true micro-payment [1] </li></ul></ul><ul><ul><li>Costs vary based on Amazon's costs </li></ul></ul><ul><li>Amazon manages security and fraud responsibility </li></ul><ul><li>“ GateKeeper” enforces policies for senders and receivers </li></ul><ul><li>Explicit “Payment Instructions” language </li></ul><ul><ul><li>“ only five transactions or $50 per month&quot; or &quot;refuse payment after August 1&quot; </li></ul></ul><ul><ul><li>Three token exchange </li></ul></ul><ul><li>Integrates with Amazon's existing customer experience </li></ul><ul><ul><li>“ Tens of Millions of existing customers” </li></ul></ul><ul><ul><li>“ Pay Now” Widget </li></ul></ul><ul><ul><li>Promotes “market places” for intermediaries </li></ul></ul>[1] http://paulstamatiou.com/2007/08/04/why-you-shouldnt-ignore-amazons-new-fps [2] http://www.amazon.com/gp/browse.html?node=342430011 [3] http://docs.amazonwebservices.com/AmazonFPS/2007-01-08/FPSGettingStartedGuide/
  16. 16. Amazon Web Services – The Sky's the Limit [1] http://www.amazon.com/b/?node=201590011 <ul><li>Provides programmatic compute capacity </li></ul><ul><ul><li>Including pre-configured appliance images </li></ul></ul><ul><li>“ ... allows you to obtain and configure capacity with minimal friction” [1] </li></ul><ul><li>The Basic Process </li></ul><ul><ul><li>Build or Identify an appropriate AMI </li></ul></ul><ul><ul><li>Configure Security and Access </li></ul></ul><ul><ul><li>Start, Monitor and Stop Instances </li></ul></ul><ul><ul><li>Pay </li></ul></ul>
  17. 17. Amazon Web Services – The Sky's the Limit <ul><li>What's an AMI ? - Amazon Machine Instance </li></ul><ul><ul><li>“ Golden Master” of the image you want to run </li></ul></ul><ul><ul><li>Linux only, Windows is a hack [1] </li></ul></ul><ul><ul><li>AMI's can be private [2] – Vendor tools like rPath [3] </li></ul></ul><ul><ul><li>Leverage an Open “Operating Systems” bazaar </li></ul></ul><ul><ul><ul><li>Some AMI's can have “associate” costs per instance </li></ul></ul></ul><ul><li>Multiple Instances can run from a single AMI </li></ul><ul><ul><li>By default, limited to 20, simple request for more </li></ul></ul><ul><li>Getting started with EC2 [4] </li></ul><ul><li>Mike Culver - Introductory video [5] </li></ul>[1] http://www.enomaly.net/wiki.1137+M5c1693b2f05.0.html [2]http://docs.amazonwebservices.com/AmazonEC2/dg/2006-06-26/creating-an-ami.html [3] http://highscalability.com/product-rpath-creating-and-managing-virtual-appliances [4] http://docs.amazonwebservices.com/AWSEC2/2007-08-29/DeveloperGuide/ [5] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=583&categoryID=100
  18. 18. Amazon Web Services – The Sky's the Limit <ul><li>Example: </li></ul><ul><ul><li>> ec2-describe-images -x all </li></ul></ul><ul><ul><ul><li>IMAGE ami-23b6534a ec2-public-images/fedora-7-apache-manafest.xml </li></ul></ul></ul><ul><ul><li>> ec2-add-keypair j-keypair </li></ul></ul><ul><ul><ul><li>(generates ssh keypairs) </li></ul></ul></ul><ul><ul><li>> ec2-run-instance ami-23b6534a -k j-keypair </li></ul></ul><ul><ul><ul><li>(could take up to a few minutes) </li></ul></ul></ul><ul><ul><li>> ec2-authorize default -p 22 </li></ul></ul><ul><ul><ul><li>(Authorize port 22 for the default security group) </li></ul></ul></ul><ul><ul><li>> ec2-describe-instances </li></ul></ul><ul><ul><ul><li>(Shows instance info including DNS name) </li></ul></ul></ul><ul><ul><li>> ec2-terminate-instances <image name> </li></ul></ul><ul><ul><li>> ec2run ami-e3a5408a -n 20 -g appserver </li></ul></ul>
  19. 19. Amazon Web Services – The Sky's the Limit [1] http://www.amazon.com/b/?node=201590011 [2] http://www.amazon.com/b/?node=201590011#measure [3] http://www.amazon.com/b/ref=sc_fe_c_0_201590011_2?ie=UTF8&node=370375011&no=201590011&me=A36L942TSJ2AJA <ul><li>Small Instance - $.10 / hour </li></ul><ul><ul><li>1.7 GB of memory, </li></ul></ul><ul><ul><li>1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit) </li></ul></ul><ul><ul><li>160 GB of instance storage, 32-bit platform </li></ul></ul><ul><li>Large Instance - $0.40 / hour </li></ul><ul><ul><li>7.5 GB of memory </li></ul></ul><ul><ul><li>4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each) </li></ul></ul><ul><ul><li>850 GB of instance storage, 64-bit platform </li></ul></ul><ul><li>Extra Large Instance - $0.80 / hour </li></ul><ul><ul><li>15 GB of memory </li></ul></ul><ul><ul><li>8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each) </li></ul></ul><ul><ul><li>1690 GB of instance storage, 64-bit platform </li></ul></ul><ul><li>1 EC2 Compute Unit is ~ 1.0-1.2 GHz 2007 Opteron or 2007 Xeon </li></ul><ul><li>Bandwidth costs of still apply at 256Mb / sec </li></ul>
  20. 20. Amazon Web Services – The Sky's the Limit <ul><li>Uses x509 certificates for Developer Auth. </li></ul><ul><ul><li>SSH keypairs for instance authenticaion </li></ul></ul><ul><li>Instances can belong to “security groups” </li></ul><ul><li>Network is dynamic via NAT </li></ul><ul><ul><li>IP's and hostnames change </li></ul></ul><ul><li>Momentum </li></ul><ul><ul><li>RedHat is deploying a “cloud” on EC2 [1] </li></ul></ul><ul><ul><li>Facebook [2] </li></ul></ul><ul><ul><li>Hadoop [3] </li></ul></ul><ul><ul><li>EC2 for Python Programmers [4] </li></ul></ul>[1] http://www.redhat.com/solutions/cloud/ [2] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1044&ref=featured http://www.amazon.com/gp/browse.html?node=391557011 [3] http://blog.awswebshop.com/2008/02/27/taking-massive-distributed-computing-to-the-common-man-hadoop-on-amazon-ec2s3/ [4] http://jimmyg.org/2007/09/01/amazon-ec2-for-people-who-prefer-debian-and-python-over-fedora-and-java/
  21. 21. Amazon Web Services – From the Ground Up <ul><li>Amazon Mechanical Turk </li></ul><ul><ul><li>Launched in late 2005 [1] </li></ul></ul>[1] http://www.egge.net/~savory/blog_mar_04.htm
  22. 22. Amazon Web Services – From the Ground Up <ul><li>Amazon Mechanical Turk </li></ul><ul><ul><li>Launched in late 2005 [1] </li></ul></ul><ul><li>“ Artificial Artificial Intelligence” - Jeff Bezos </li></ul>[1] http://en.wikipedia.org/wiki/Amazon.com
  23. 23. Amazon Web Services – From the Ground Up [1] http://www.amazon.com/gp/browse.html?node=15879911 [2] http://www.techcrunch.com/2007/09/08/search-for-steve-fossett-expands-to-amazons-mechanical-turk/ <ul><li>Humans augment computational requirements </li></ul><ul><li>Let someone else solve complex AI problems </li></ul><ul><ul><li>Get along with the business of making money </li></ul></ul><ul><li>Built around Human Intelligence Tasks (HITs) </li></ul><ul><li>Examples; </li></ul><ul><ul><li>“ Transcribe audio data” </li></ul></ul><ul><ul><li>“ Are these paragraphs synonymous?” </li></ul></ul><ul><ul><li>“ Find the business in this photo” </li></ul></ul><ul><ul><li>“ Write a review...” </li></ul></ul><ul><ul><li>“ Search for survivors” </li></ul></ul><ul><ul><ul><li>“ So far, searchers have found half a dozen previously unknown crash sites, but no sign of Fossett’s plane.” [2] </li></ul></ul></ul>
  24. 24. Amazon Web Services – The Example [1] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=691&categoryID=100 <ul><li>Mitch Garnaat </li></ul><ul><ul><li>Author of Bono </li></ul></ul><ul><li>“ DVS” transcoding </li></ul><ul><li>Tutorial [1] </li></ul>
  25. 25. <ul><li>Read a message from the input queue - SQS </li></ul><ul><li>Based on message, retrieve an input file - S3 </li></ul><ul><li>Perform our video conversion processing - EC2 </li></ul><ul><li>Store the generated files - S3 </li></ul><ul><li>Write a message the the output queue - SQS </li></ul><ul><li>Delete the input message from the input queue - SQS </li></ul>Amazon Web Services – The Example [1] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=691&categoryID=100
  26. 26. Amazon Web Services – The Example [1] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=691&categoryID=100 <ul><li>Input Message </li></ul><ul><ul><li>Bucket: garnaat_fileit </li></ul></ul><ul><ul><li>InputKey: f84e4a20b571abc69baf2277d193e596 </li></ul></ul><ul><ul><li>Date: Tue, 20 Feb 2007 17:21:21 GMT </li></ul></ul><ul><ul><li>OriginalFileName: MVI_3113.AVI </li></ul></ul><ul><ul><li>Size: 1126472 </li></ul></ul><ul><li>Output Message </li></ul><ul><ul><li>Bucket: garnaat_fileit </li></ul></ul><ul><ul><li>InputKey: f84e4a20b571abc69baf2277d193e596 </li></ul></ul><ul><ul><li>Date: Tue, 20 Feb 2007 17:21:21 GMT </li></ul></ul><ul><ul><li>OriginalFileName: MVI_3113.AVI </li></ul></ul><ul><ul><li>Size: 1126472 </li></ul></ul><ul><ul><li>OutputKey: e69e376be5af6f88f81d3e31adf27988;type=video/quicktime </li></ul></ul><ul><ul><li>Server: ConvertVideo </li></ul></ul><ul><ul><li>Host: domU-12-31-34-00-02-82 </li></ul></ul><ul><ul><li>Service-Read: Wed, 21 Feb 2007 01:28:14 GMT </li></ul></ul><ul><ul><li>Service-Write: Wed, 21 Feb 2007 01:28:27 GMT </li></ul></ul>
  27. 27. Amazon Web Services – The Example [1] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=691&categoryID=100 <ul><li>Define the Service Instance </li></ul><ul><ul><ul><li>from boto.services.service import Service </li></ul></ul></ul><ul><ul><ul><li>import os </li></ul></ul></ul><ul><ul><ul><li>class ConvertVideo(Service): </li></ul></ul></ul><ul><ul><ul><li>ProcessingTime = 30 </li></ul></ul></ul><ul><ul><ul><li>Command = &quot;&quot;&quot;ffmpeg -y -i %s -f mov -r 29.97 -b 1200kb -mbd 2 -flags </li></ul></ul></ul><ul><ul><ul><li>+4mv+trell -aic 2 -cmp 2 -subcmp 2 -ar 48000 -ab 192 -s 320x240 </li></ul></ul></ul><ul><ul><ul><li>-vcodec mpeg4 -acodec aac %s&quot;&quot;&quot; </li></ul></ul></ul><ul><ul><ul><li>def process_file(self, in_file_name, msg): </li></ul></ul></ul><ul><ul><ul><li>out_file_name = os.path.join(self.working_dir, 'out.mov') </li></ul></ul></ul><ul><ul><ul><li>command = self.Command % (in_file_name, out_file_name) </li></ul></ul></ul><ul><ul><ul><li>os.system(command) </li></ul></ul></ul><ul><ul><ul><li>return [(out_file_name, 'video/quicktime')] </li></ul></ul></ul>
  28. 28. Amazon Web Services – The Example [1] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=691&categoryID=100 <ul><li>$ boto/services/submit_files.py -b myvideos -q vc-input ~/movies </li></ul><ul><li>... </li></ul><ul><li>50 files successfully submitted. </li></ul><ul><li>$ boto/services/start_service.py -r -m boto.services.convertvideo </li></ul><ul><li>-c ConvertVideo -a ami-2eba5f47 -i vc-input -o vc-output </li></ul><ul><li>-e mitch@garnaat.com -n 1 </li></ul><ul><li>python boto/services/get_results.py -q vc-status ~/movies </li></ul><ul><li>retrieving file: MVI_3110.mov </li></ul><ul><li>... </li></ul><ul><li>50 results successfully retrieved. </li></ul><ul><li>Minimum Processing Time: 2 </li></ul><ul><li>Maximum Processing Time: 58 </li></ul><ul><li>Average Processing Time: 17.820000 </li></ul><ul><li>Elapsed Time: 896 </li></ul><ul><li>Throughput: 3.348214 transactions / minute </li></ul><ul><li>$ python boto/services/get_results.py -q test-status ~/movies </li></ul><ul><li>retrieving file: MVI_3110.mov </li></ul><ul><li>500 results successfully retrieved. </li></ul><ul><li>... </li></ul><ul><li>Minimum Processing Time: 2 </li></ul><ul><li>Maximum Processing Time: 60 </li></ul><ul><li>Average Processing Time: 17.794000 </li></ul><ul><li>Elapsed Time: 928 </li></ul><ul><li>Throughput: 32.327586 transactions / minute </li></ul>Submit Files and Setup Queue Startup EC2 worker(s) 1 Instance 10 Instances, 500 files
  29. 29. Amazon Web Services – The Example [1] http://developer.amazonwebservices.com/connect/entry.jspa?externalID=691&categoryID=100 <ul><li>Okay... so what ? </li></ul><ul><li>$.004 per movie, $8k server => 2,000,000 movies </li></ul><ul><li>24 hrs * 60 min = 1440 movies / day * 365 days </li></ul><ul><li>525600 movies / year * 4 cameras = 2.1M </li></ul>
  30. 30. <ul><li>This matters to Amazon ! </li></ul><ul><ul><li>For the period of 4Q07, bandwidth utilized for S3 & EC2 was greater than the combined bandwidth of Amazon.com’s global websites. [1] </li></ul></ul><ul><ul><ul><li>In Dec '07 Amazon was the #7 website in the US [2] </li></ul></ul></ul><ul><ul><li>Same period 330,000+ new developers registered </li></ul></ul><ul><ul><ul><li>Up 30,000+ from the previous last quarter. </li></ul></ul></ul><ul><li>Success Stories [3] </li></ul><ul><li>Startup Challenge [4] </li></ul>Amazon Web Services – “So Who, What, Why” [1] http://biz.yahoo.com/bw/080130/20080130006013.html [2] http://www.comscore.com/press/release.asp?press=2000 [3] http://www.amazon.com/gp/browse.html?node=182241011 [4] http://www.amazon.com/gp/browse.html?node=377634011
  31. 31. <ul><li>It's not just Amazon </li></ul><ul><ul><li>Google [1] </li></ul></ul><ul><ul><li>Microsoft! [2] </li></ul></ul><ul><ul><li>Yahoo </li></ul></ul><ul><li>IBM [3][4][5][6][7] </li></ul><ul><li>Additional Examples </li></ul><ul><ul><li>BigTable, GFS, MapReduce </li></ul></ul><ul><ul><li>CouchDB </li></ul></ul><ul><ul><li>Hadoop </li></ul></ul>Amazon Web Services – “So Who, What, Why” [1] http://feedblog.org/2007/12/14/google-vs-amazon-in-open-infrastructure/ [2] http://www.microsoft.com/sql/dataservices/default.mspx [3] http://redmonk.com/jgovernor/2007/04/19/why-ibm-should-acquire-amazon/ [4] http://www.ogf.org/OGF22/materials/1137/Irving+Wladawsky-Berger+Keynote.pdf [5] http://www.news.com/8301-10784_3-9817905-7.html [6] http://www-03.ibm.com/press/us/en/pressrelease/22613.wss [7] http://blogs.zdnet.com/SAAS/?p=415
  32. 32. Amazon Web Services – Take Away <ul><li>Customers can leverage the “cloud” ! </li></ul><ul><ul><li>Energy getting invested “up the stack” </li></ul></ul><ul><li>“ The Industry” is embracing “simplicity” </li></ul><ul><ul><li>“ Functional programming” renaissance </li></ul></ul><ul><ul><ul><li>Analogs to the resurgence of the mainframe </li></ul></ul></ul><ul><ul><li>Application Trend - trust (but verify) </li></ul></ul><ul><ul><ul><li>Intuition over specificity </li></ul></ul></ul><ul><li>Amazon doesn't re-instantiate components </li></ul><ul><ul><li>Insanity - “doing the same thing over and over and expecting different results.” </li></ul></ul><ul><li>Define Commodity - ... </li></ul>
  33. 33. Amazon Web Services – Thinkovers [1] http://www.kitchensoap.com/2008/02/27/when-do-you-get-too-big-to-use-cloudy-stuff/ [2] http://www.readwriteweb.com/archives/amazon_haas_hardware_as_a_service.php <ul><li>[1] </li></ul><ul><li>[2] </li></ul>

×