Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr S...
Today <ul><li>Introduction </li></ul><ul><li>Amazon Web Services overview </li></ul><ul><li>Amazon EC2 </li></ul><ul><li>A...
Who am I? <ul><li>Software development background </li></ul><ul><li>5 Years with Amazon: </li></ul><ul><ul><ul><li>Senior ...
Amazon? <ul><li>Consumers (Buy) </li></ul><ul><li>Associates, Sellers & Merchants (Sell) </li></ul><ul><li>Developers (Inn...
Issues Facing Developers <ul><li>70% of Web Development Effort is “Muck”: </li></ul><ul><ul><li>Data Centers </li></ul></u...
Dream or Nightmare? <ul><li>Slashdot/Digg/TechCrunch Effect </li></ul><ul><li>Rapid, unexpected customer demand/growth </l...
Seasonal Spikes
More Seasonal Spikes <ul><li>“ Every year, we take the busiest minute of the busiest hour of the busiest day and build cap...
Solution: Web-Scale Computing <ul><li>Scale capacity on demand </li></ul><ul><li>Turn fixed costs into variable costs </li...
Amazon Web Services Are... <ul><li>A set of APIs and business models which give developer-level access to Amazon’s infrast...
The Commercial Side <ul><li>Standard licensing terms </li></ul><ul><li>Commercially usable </li></ul><ul><li>Aggressive pr...
Web Services Billing
Infrastructure Services Compute Store Message Elastic Compute Cloud Simple Storage  Service Simple Queue Service
Amazon Simple Storage Service S3
Amazon Simple Storage Service $.15 per GB per month storage <ul><li>Object-Based Storage </li></ul><ul><li>1 B – 5 GB / ob...
Amazon S3 Concepts <ul><li>Objects: </li></ul><ul><ul><li>Opaque data to be stored (1 byte … 5 Gigabytes) </li></ul></ul><...
S3 SOAP/Query API <ul><li>Service: </li></ul><ul><ul><li>ListAllMyBuckets </li></ul></ul><ul><li>Buckets: </li></ul><ul><u...
 
 
Amazon Simple Queue Service SQS
Amazon Simple Queue Service $.10 per 1000 messages <ul><li>Scalable Queuing </li></ul><ul><li>Elastic Capacity </li></ul><...
Amazon SQS Concepts <ul><li>Queues: </li></ul><ul><ul><li>Named message container </li></ul></ul><ul><ul><li>Persistent </...
Amazon SQS At Work
SQS SOAP/Query API <ul><li>Queues: </li></ul><ul><ul><li>ListQueues </li></ul></ul><ul><ul><li>DeleteQueue </li></ul></ul>...
Amazon Elastic Compute Cloud EC2
Amazon Elastic Compute Cloud $.10 per server hour <ul><li>Virtual Compute Cloud </li></ul><ul><li>Elastic Capacity </li></...
Amazon EC2 Concepts <ul><li>Amazon Machine Image (AMI): </li></ul><ul><ul><li>Bootable root disk </li></ul></ul><ul><ul><l...
Root-level access
Amazon EC2 At Work <ul><li>Startups </li></ul><ul><ul><li>Cruxy – Media transcoding </li></ul></ul><ul><ul><li>GigaVox Med...
EC2 SOAP/Query API <ul><li>Images: </li></ul><ul><ul><li>RegisterImage </li></ul></ul><ul><ul><li>DescribeImages </li></ul...
 
Sample Web-Scale Architecture GigaVox Media
Web-Scale Architecture
GigaVox Economics <ul><li>Implemented Amazon S3, Amazon EC2 and Amazon SQS in November 2006 </li></ul><ul><li>Created an i...
<ul><li>Q & A </li></ul>
Thank You! <ul><li>Pointers </li></ul><ul><li>Portal </li></ul><ul><li>http://aws.amazon.com </li></ul><ul><li>Blog </li><...
Upcoming SlideShare
Loading in …5
×

Amazon

879 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
879
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 08/06/10 02:45 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. Reducing the barriers of entry to building web-friendly applications and businesses. We have looked at the investment we have made in technology over the last 10 years and Amazon Web Services strategy is to see what parts of our technology is exposed to developers outside of Amazon.com
  • Amazon

    1. 1. Amazon Web Services: Building Highly Scalable Web Applications Institutional Web Management Workshop July 2007 Jeff Barr Senior Web Services Evangelist [email_address]
    2. 2. Today <ul><li>Introduction </li></ul><ul><li>Amazon Web Services overview </li></ul><ul><li>Amazon EC2 </li></ul><ul><li>Amazon S3 </li></ul><ul><li>Amazon SQS </li></ul><ul><li>Application Architecture </li></ul><ul><li>Questions </li></ul>
    3. 3. Who am I? <ul><li>Software development background </li></ul><ul><li>5 Years with Amazon: </li></ul><ul><ul><ul><li>Senior Developer </li></ul></ul></ul><ul><ul><ul><li>Web Services Evangelist </li></ul></ul></ul><ul><ul><ul><li>Senior Web Services Evangelist </li></ul></ul></ul><ul><li>Veteran of several startups </li></ul><ul><li>Microsoft .Net Team </li></ul><ul><li>Visix Software co-founder </li></ul>
    4. 4. Amazon? <ul><li>Consumers (Buy) </li></ul><ul><li>Associates, Sellers & Merchants (Sell) </li></ul><ul><li>Developers (Innovate) </li></ul>
    5. 5. Issues Facing Developers <ul><li>70% of Web Development Effort is “Muck”: </li></ul><ul><ul><li>Data Centers </li></ul></ul><ul><ul><li>Bandwidth / Power / Cooling </li></ul></ul><ul><ul><li>Operations </li></ul></ul><ul><ul><li>Staffing </li></ul></ul><ul><li>Scaling is Difficult and Expensive: </li></ul><ul><ul><li>Large Up-Front Investment </li></ul></ul><ul><ul><li>Invest Ahead of Demand </li></ul></ul><ul><ul><li>Load is Unpredictable </li></ul></ul>
    6. 6. Dream or Nightmare? <ul><li>Slashdot/Digg/TechCrunch Effect </li></ul><ul><li>Rapid, unexpected customer demand/growth </li></ul>
    7. 7. Seasonal Spikes
    8. 8. More Seasonal Spikes <ul><li>“ Every year, we take the busiest minute of the busiest hour of the busiest day and build capacity on that, we built our systems to (handle that load) and we went above and beyond that.” * </li></ul><ul><li>-- Scott Gulbransen </li></ul><ul><li>Intuit Spokesman </li></ul>* http://news.com.com/2100-1038_3-6177341.html
    9. 9. Solution: Web-Scale Computing <ul><li>Scale capacity on demand </li></ul><ul><li>Turn fixed costs into variable costs </li></ul><ul><li>Always available </li></ul><ul><li>Rock-solid reliability </li></ul><ul><li>Simple APIs and conceptual models </li></ul><ul><li>Cost-effective </li></ul><ul><li>Reduced time to market </li></ul><ul><li>Focus on product & core competencies </li></ul>
    10. 10. Amazon Web Services Are... <ul><li>A set of APIs and business models which give developer-level access to Amazon’s infrastructure and content: </li></ul><ul><li>Data As A Service </li></ul><ul><ul><li>Amazon E-Commerce Service </li></ul></ul><ul><ul><li>Amazon Historical Pricing </li></ul></ul><ul><li>Search As A Service </li></ul><ul><ul><li>Alexa Web Information Service </li></ul></ul><ul><ul><li>Alexa Top Sites </li></ul></ul><ul><ul><li>Alexa Site Thumbnail </li></ul></ul><ul><ul><li>Alexa Web Search Platform </li></ul></ul><ul><li>Infrastructure As A Service </li></ul><ul><ul><li>Amazon Simple Queue Service </li></ul></ul><ul><ul><li>Amazon Simple Storage Service </li></ul></ul><ul><ul><li>Amazon Elastic Compute Cloud </li></ul></ul><ul><li>People As A Service </li></ul><ul><ul><li>Amazon Mechanical Turk </li></ul></ul>
    11. 11. The Commercial Side <ul><li>Standard licensing terms </li></ul><ul><li>Commercially usable </li></ul><ul><li>Aggressive pricing </li></ul><ul><li>Monthly credit card billing </li></ul><ul><li>Self-serve model: </li></ul><ul><ul><li>Sign up as developer </li></ul></ul><ul><ul><li>Choose services </li></ul></ul><ul><ul><li>Agree to service licenses </li></ul></ul><ul><ul><li>Enter payment info </li></ul></ul><ul><ul><li>Start coding </li></ul></ul>
    12. 12. Web Services Billing
    13. 13. Infrastructure Services Compute Store Message Elastic Compute Cloud Simple Storage Service Simple Queue Service
    14. 14. Amazon Simple Storage Service S3
    15. 15. Amazon Simple Storage Service $.15 per GB per month storage <ul><li>Object-Based Storage </li></ul><ul><li>1 B – 5 GB / object </li></ul><ul><li>Fast, Reliable, Scalable </li></ul><ul><li>Redundant, Dispersed </li></ul><ul><li>99.99% Availability Goal </li></ul><ul><li>Private or Public </li></ul><ul><li>Per-object URLs & ACLs </li></ul><ul><li>BitTorrent Support </li></ul>$.10 - $.18 per GB data transfer $.01 for 1000 to 10000 requests
    16. 16. Amazon S3 Concepts <ul><li>Objects: </li></ul><ul><ul><li>Opaque data to be stored (1 byte … 5 Gigabytes) </li></ul></ul><ul><ul><li>Authentication and access controls </li></ul></ul><ul><li>Buckets: </li></ul><ul><ul><li>Object container – any number of objects </li></ul></ul><ul><ul><li>100 buckets per account / buckets are “owned” </li></ul></ul><ul><li>Keys: </li></ul><ul><ul><li>Unique object identifier within bucket </li></ul></ul><ul><ul><li>Up to 1024 bytes long </li></ul></ul><ul><ul><li>Flat object storage model </li></ul></ul><ul><li>Standards-Based Interfaces: </li></ul><ul><ul><li>REST and SOAP </li></ul></ul><ul><ul><li>URL-Addressability – every object has a URL </li></ul></ul>
    17. 17. S3 SOAP/Query API <ul><li>Service: </li></ul><ul><ul><li>ListAllMyBuckets </li></ul></ul><ul><li>Buckets: </li></ul><ul><ul><li>CreateBucket </li></ul></ul><ul><ul><li>DeleteBucket </li></ul></ul><ul><ul><li>ListBucket </li></ul></ul><ul><ul><li>GetBucketAccessControlPolicy </li></ul></ul><ul><ul><li>SetBucketAccessControlPolicy </li></ul></ul><ul><ul><li>GetBucketLoggingStatus </li></ul></ul><ul><ul><li>SetBucketLoggingStatus </li></ul></ul><ul><li>Objects: </li></ul><ul><ul><li>PutObject </li></ul></ul><ul><ul><li>PutObjectInline </li></ul></ul><ul><ul><li>GetObject </li></ul></ul><ul><ul><li>GetObjectExtended </li></ul></ul><ul><ul><li>DeleteObject </li></ul></ul><ul><ul><li>GetObjectAccessControlPolicy </li></ul></ul><ul><ul><li>SetObjectAccessControlPolicy </li></ul></ul>
    18. 20. Amazon Simple Queue Service SQS
    19. 21. Amazon Simple Queue Service $.10 per 1000 messages <ul><li>Scalable Queuing </li></ul><ul><li>Elastic Capacity </li></ul><ul><li>Reliable, Simple, Secure </li></ul><ul><li>Inter-process messaging, data buffering, architecture component </li></ul>$.10 - $.18 per GB data transfer
    20. 22. Amazon SQS Concepts <ul><li>Queues: </li></ul><ul><ul><li>Named message container </li></ul></ul><ul><ul><li>Persistent </li></ul></ul><ul><li>Messages: </li></ul><ul><ul><li>Up to 256KB of data per message </li></ul></ul><ul><ul><li>Peek / Lock access model </li></ul></ul><ul><li>Scalable: </li></ul><ul><ul><li>Unlimited number of queues per account </li></ul></ul><ul><ul><li>Unlimited number of messages per queue </li></ul></ul>
    21. 23. Amazon SQS At Work
    22. 24. SQS SOAP/Query API <ul><li>Queues: </li></ul><ul><ul><li>ListQueues </li></ul></ul><ul><ul><li>DeleteQueue </li></ul></ul><ul><ul><li>SetVisibilityTimeout </li></ul></ul><ul><ul><li>GetVisibilityTimeout </li></ul></ul><ul><li>Messages: </li></ul><ul><ul><li>SendMessage </li></ul></ul><ul><ul><li>ReceiveMessage </li></ul></ul><ul><ul><li>DeleteMessage </li></ul></ul><ul><ul><li>PeekMessage </li></ul></ul><ul><li>Security: </li></ul><ul><ul><li>AddGrant </li></ul></ul><ul><ul><li>ListGrants </li></ul></ul><ul><ul><li>RemoveGrant </li></ul></ul>
    23. 25. Amazon Elastic Compute Cloud EC2
    24. 26. Amazon Elastic Compute Cloud $.10 per server hour <ul><li>Virtual Compute Cloud </li></ul><ul><li>Elastic Capacity </li></ul><ul><li>1.7 GHz x86 </li></ul><ul><li>1.7 GB RAM </li></ul><ul><li>160 GB Disk </li></ul><ul><li>250 MB/Second Network </li></ul><ul><li>Network Security Model </li></ul><ul><li>Time or Traffic-based Scaling, Load testing, Simulation and Analysis, Rendering, Software as a Service Platform, Hosting </li></ul>$.10 - $.18 per GB data transfer
    25. 27. Amazon EC2 Concepts <ul><li>Amazon Machine Image (AMI): </li></ul><ul><ul><li>Bootable root disk </li></ul></ul><ul><ul><li>Pre-defined or user-built </li></ul></ul><ul><ul><li>Catalog of user-built AMIs </li></ul></ul><ul><ul><li>OS: Fedora, Centos, Gentoo, Debian, Ubuntu, Windows Server </li></ul></ul><ul><ul><li>App Stack: LAMP, mpiBLAST, Hadoop </li></ul></ul><ul><li>Instance: </li></ul><ul><ul><li>Running copy of an AMI </li></ul></ul><ul><ul><li>Launch in less than 2 minutes </li></ul></ul><ul><ul><li>Start/stop programmatically </li></ul></ul><ul><li>Network Security Model: </li></ul><ul><ul><li>Explicit access control </li></ul></ul><ul><ul><li>Security groups </li></ul></ul><ul><li>Inter-service bandwidth is free </li></ul>
    26. 28. Root-level access
    27. 29. Amazon EC2 At Work <ul><li>Startups </li></ul><ul><ul><li>Cruxy – Media transcoding </li></ul></ul><ul><ul><li>GigaVox Media – Podcast Management </li></ul></ul><ul><li>Fortune 500 clients: </li></ul><ul><ul><li>High-Impact, S hort-Term Projects </li></ul></ul><ul><ul><li>Development Host </li></ul></ul><ul><li>Science / Research: </li></ul><ul><ul><li>Hadoop / MapReduce </li></ul></ul><ul><ul><li>mpiBLAST </li></ul></ul><ul><li>Load-Management and Load Balancing Tools: </li></ul><ul><ul><li>Pound </li></ul></ul><ul><ul><li>Weogeo </li></ul></ul><ul><ul><li>Rightscale </li></ul></ul>
    28. 30. EC2 SOAP/Query API <ul><li>Images: </li></ul><ul><ul><li>RegisterImage </li></ul></ul><ul><ul><li>DescribeImages </li></ul></ul><ul><ul><li>DeregisterImage </li></ul></ul><ul><li>Instances: </li></ul><ul><ul><li>RunInstances </li></ul></ul><ul><ul><li>DescribeInstances </li></ul></ul><ul><ul><li>TerminateInstances </li></ul></ul><ul><ul><li>GetConsoleOutput </li></ul></ul><ul><ul><li>RebootInstances </li></ul></ul><ul><li>Keypairs: </li></ul><ul><ul><li>CreateKeyPair </li></ul></ul><ul><ul><li>DescribeKeyPairs </li></ul></ul><ul><ul><li>DeleteKeyPair </li></ul></ul><ul><li>Image Attributes: </li></ul><ul><ul><li>ModifyImageAttribute </li></ul></ul><ul><ul><li>DescribeImageAttribute </li></ul></ul><ul><ul><li>ResetImageAttribute </li></ul></ul><ul><li>Security Groups: </li></ul><ul><ul><li>CreateSecurityGroup </li></ul></ul><ul><ul><li>DescribeSecurityGroups </li></ul></ul><ul><ul><li>DeleteSecurityGroup </li></ul></ul><ul><ul><li>AuthorizeSecurityGroupIngress </li></ul></ul><ul><ul><li>RevokeSecurityGroupIngress </li></ul></ul>
    29. 32. Sample Web-Scale Architecture GigaVox Media
    30. 33. Web-Scale Architecture
    31. 34. GigaVox Economics <ul><li>Implemented Amazon S3, Amazon EC2 and Amazon SQS in November 2006 </li></ul><ul><li>Created an infinitely scalable infrastructure for less than $100 - building the same infrastructure themselves would have cost thousands of dollars </li></ul><ul><li>Reduced staffing requirements - far less responsibility for 24x7 operations </li></ul>
    32. 35. <ul><li>Q & A </li></ul>
    33. 36. Thank You! <ul><li>Pointers </li></ul><ul><li>Portal </li></ul><ul><li>http://aws.amazon.com </li></ul><ul><li>Blog </li></ul><ul><li>http:// aws.typepad.com </li></ul><ul><li>EC2 </li></ul><ul><li>http://aws.amazon.com/ec2 </li></ul><ul><li>S3 </li></ul><ul><li>http://aws.amazon.com/s3 </li></ul><ul><li>Resource Center </li></ul><ul><li>http://aws.amazon.com/resources </li></ul><ul><li>Forums </li></ul><ul><li>http://aws.amazon.com/forums </li></ul>Jeff Barr Senior Web Services Evangelist [email_address]

    ×