Web Scale Computing Mike Culver Amazon Web Services
Agenda Web Scale Computing Overview Code
The Obligatory Timeline Slide 1959 1969 1982 1996 2004 2006 Darkness Web as a Platform Web Services, Resources Eliminated Web Awareness Dot-Com Bubble Web 2.0 Web Scale Computing 2001 1997 COBOL, Edsel Amazon.com Internet ARPANET
 
Being Relevant Counts
 
It’s All About Ideas
Ideas Cost Money a + bc
But How Much Money? a + bc Our favorite numbers: 0.10, 0.15, and 0.20 x
Amazon Web Services Are... A set of APIs and business models which give developers access to Amazon technology and content Data As a Service Amazon E-Commerce Service Amazon Historical Pricing Infrastructure As a Service Amazon Simple Queue Service Amazon Simple Storage Service Amazon Elastic Compute Cloud Search As a Service Alexa Web Information Service Alexa Top Sites Alexa Site Thumbnail Alexa Web Search Platform People As a Service Amazon Mechanical Turk
Amazon Web Scale Infrastructure: We Do the Muck Leverage Amazon’s Core Expertise Scales to the Enterprise Priced for People Reliable Simple to Use Highly Available Using Industry Standards HTTP, XML, REST, SOAP, XSLT With World-Class Developer Support
What do these companies have in common?
Amazon E-Commerce Service
Amazon E-Commerce Service Exposes Amazon's product data plus a shopping cart  In 4 th  major release Millions of products Rich data schema Product images Reviews Wishlists Commissions via Amazon Associates program
Amazon.com With Your Look
Amazon ECS Cost  $0.00
Amazon Simple Storage Service
Amazon S3 is… Simple Data Storage in Amazon Data Center Web Service interface No set-up fee, No monthly minimum Storage: $0.15 per GB/Month Data Transfer: $0.20/GB to transfer data Private and  public storage Each object up to 5GB in size
Many ISV Implementations S3 Explorer filicio.us Jungle Disk S3 Firefox Organizer MyOwnDB
Amazon Elastic Compute Cloud
Amazon EC2 Is… A Web service that provides resizable compute capacity in the cloud.  Designed to make Web-scale computing easier for developers.  A simple Web service interface that provides complete control of your computing resources
Amazon EC2 Benefits You Because It Reduces the time required to obtain and boot new server instances to minutes Quickly scales capacity, both up and down, as your computing requirements change  Changes the economics of computing: Pay only for capacity that you actually use a + bc  becomes just  bc
Amazon EC2 Changes Computing Economics No start-up, monthly, or fixed costs $0.10 per CPU hour $0.20 per GB transferred across Net No cost to transfer data between Amazon S3 and Amazon EC2
Amazon EC2 Profile 1.7 Ghz x86 processor 1.7 GB of RAM 160 GB disk 250 Mb/second network
Web-Scale for Mobile Services Used Amazon S3 Amazon EC2 Estimated Savings $650,000 Gumiyo.com
Amazon Mechanical Turk
Amazon Mechanical Turk Is… A web service that exposes an on-demand global workforce ready to complete small tasks in exchange for micro-payments Frictionless. Outsourcing per-se is irrelevant. A web services API
Identify Road Markings
Are These the Same MP3 Players?
How It Works www.mturk.com Workers Artificial, Artificially  Intelligent Software Requester (Developer) Human  Intelligence  Tasks (HITs) Completed  HITs Worker  Qualifications
Amazon Mechanical Turk is Relevant to Everyone Software Developers Solves problem of building applications that until now have not worked well without human intelligence Businesses Efficiently, effectively and inexpensively complete millions of tasks that require human intelligence Anyone Make money using the skills you already have whenever your schedule allows. Previously unimaginable business ideas are now feasible…
Man Versus Machine
But Not Really
Podcast transcription service provider  Transcribes audio into high-quality text  Amazon Mechanical Turk: Workers transcribe podcasts and index text within search engine  Amazon Simple Storage: Stores the podcasts and related files
Amazon Simple Queue
Amazon Simple Queue Is A reliable, highly scalable hosted distributed queue for storing messages as they travel between computers.  Amazon's web-scale messaging infrastructure as a web service Platform-agnostic, allowing any computer on the Internet to add or read messages without any installed software or special firewall configurations
Amazon SQS Highlights Reliable Runs within Amazon's high-availability data centers.  Messages are stored redundantly across multiple servers and data centers.  Simple: Only Six Methods Scalable  Secure: Strong access controls
Amazon SQS Pricing Pay only for what you use No minimum fee No start-up cost $0.10 per 1,000 messages sent ($0.0001 per message sent)  $0.20 per GB of data transferred
Alexa Web Services
Four Alexa Web Services: One Vast Web Database Alexa Web Information Service:  Web search engine, traffic information for web sites, a web site directory, and more. Alexa Top Sites:  Programmatic access top sites data is available for the web as a whole, or for individual countries
Four Alexa Web Services: One Vast Web Database Alexa Site Thumbnail: Programmatic access to thumbnail images for the home pages of web sites.  Alexa Web Search Platform: Directly analyze Alexa's Web archive to create and publish new search services
Amazon Historical Pricing
Amazon Historical Pricing Highlights Access to 3 years of actual sales history from books, music, videos, and DVDs sold on Amazon.com  Returns the average, minimum, maximum, and median price for items Up to ten items per request Pricing: $249/mo. for up to 60,000 requests/mo.
Learn More About AWS Explore  http://aws.amazon.com Read Our Blog at  http://aws.typepad.com Amazon’s hiring  www.amazon.com/jobs
 
Code!!!

Amazon web services

  • 1.
    Web Scale ComputingMike Culver Amazon Web Services
  • 2.
    Agenda Web ScaleComputing Overview Code
  • 3.
    The Obligatory TimelineSlide 1959 1969 1982 1996 2004 2006 Darkness Web as a Platform Web Services, Resources Eliminated Web Awareness Dot-Com Bubble Web 2.0 Web Scale Computing 2001 1997 COBOL, Edsel Amazon.com Internet ARPANET
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
    But How MuchMoney? a + bc Our favorite numbers: 0.10, 0.15, and 0.20 x
  • 10.
    Amazon Web ServicesAre... A set of APIs and business models which give developers access to Amazon technology and content Data As a Service Amazon E-Commerce Service Amazon Historical Pricing Infrastructure As a Service Amazon Simple Queue Service Amazon Simple Storage Service Amazon Elastic Compute Cloud Search As a Service Alexa Web Information Service Alexa Top Sites Alexa Site Thumbnail Alexa Web Search Platform People As a Service Amazon Mechanical Turk
  • 11.
    Amazon Web ScaleInfrastructure: We Do the Muck Leverage Amazon’s Core Expertise Scales to the Enterprise Priced for People Reliable Simple to Use Highly Available Using Industry Standards HTTP, XML, REST, SOAP, XSLT With World-Class Developer Support
  • 12.
    What do thesecompanies have in common?
  • 13.
  • 14.
    Amazon E-Commerce ServiceExposes Amazon's product data plus a shopping cart In 4 th major release Millions of products Rich data schema Product images Reviews Wishlists Commissions via Amazon Associates program
  • 15.
  • 16.
  • 17.
  • 18.
    Amazon S3 is…Simple Data Storage in Amazon Data Center Web Service interface No set-up fee, No monthly minimum Storage: $0.15 per GB/Month Data Transfer: $0.20/GB to transfer data Private and public storage Each object up to 5GB in size
  • 19.
    Many ISV ImplementationsS3 Explorer filicio.us Jungle Disk S3 Firefox Organizer MyOwnDB
  • 20.
  • 21.
    Amazon EC2 Is…A Web service that provides resizable compute capacity in the cloud. Designed to make Web-scale computing easier for developers. A simple Web service interface that provides complete control of your computing resources
  • 22.
    Amazon EC2 BenefitsYou Because It Reduces the time required to obtain and boot new server instances to minutes Quickly scales capacity, both up and down, as your computing requirements change Changes the economics of computing: Pay only for capacity that you actually use a + bc becomes just bc
  • 23.
    Amazon EC2 ChangesComputing Economics No start-up, monthly, or fixed costs $0.10 per CPU hour $0.20 per GB transferred across Net No cost to transfer data between Amazon S3 and Amazon EC2
  • 24.
    Amazon EC2 Profile1.7 Ghz x86 processor 1.7 GB of RAM 160 GB disk 250 Mb/second network
  • 25.
    Web-Scale for MobileServices Used Amazon S3 Amazon EC2 Estimated Savings $650,000 Gumiyo.com
  • 26.
  • 27.
    Amazon Mechanical TurkIs… A web service that exposes an on-demand global workforce ready to complete small tasks in exchange for micro-payments Frictionless. Outsourcing per-se is irrelevant. A web services API
  • 28.
  • 29.
    Are These theSame MP3 Players?
  • 30.
    How It Workswww.mturk.com Workers Artificial, Artificially Intelligent Software Requester (Developer) Human Intelligence Tasks (HITs) Completed HITs Worker Qualifications
  • 31.
    Amazon Mechanical Turkis Relevant to Everyone Software Developers Solves problem of building applications that until now have not worked well without human intelligence Businesses Efficiently, effectively and inexpensively complete millions of tasks that require human intelligence Anyone Make money using the skills you already have whenever your schedule allows. Previously unimaginable business ideas are now feasible…
  • 32.
  • 33.
  • 34.
    Podcast transcription serviceprovider Transcribes audio into high-quality text Amazon Mechanical Turk: Workers transcribe podcasts and index text within search engine Amazon Simple Storage: Stores the podcasts and related files
  • 35.
  • 36.
    Amazon Simple QueueIs A reliable, highly scalable hosted distributed queue for storing messages as they travel between computers. Amazon's web-scale messaging infrastructure as a web service Platform-agnostic, allowing any computer on the Internet to add or read messages without any installed software or special firewall configurations
  • 37.
    Amazon SQS HighlightsReliable Runs within Amazon's high-availability data centers. Messages are stored redundantly across multiple servers and data centers. Simple: Only Six Methods Scalable Secure: Strong access controls
  • 38.
    Amazon SQS PricingPay only for what you use No minimum fee No start-up cost $0.10 per 1,000 messages sent ($0.0001 per message sent) $0.20 per GB of data transferred
  • 39.
  • 40.
    Four Alexa WebServices: One Vast Web Database Alexa Web Information Service: Web search engine, traffic information for web sites, a web site directory, and more. Alexa Top Sites: Programmatic access top sites data is available for the web as a whole, or for individual countries
  • 41.
    Four Alexa WebServices: One Vast Web Database Alexa Site Thumbnail: Programmatic access to thumbnail images for the home pages of web sites. Alexa Web Search Platform: Directly analyze Alexa's Web archive to create and publish new search services
  • 42.
  • 43.
    Amazon Historical PricingHighlights Access to 3 years of actual sales history from books, music, videos, and DVDs sold on Amazon.com Returns the average, minimum, maximum, and median price for items Up to ten items per request Pricing: $249/mo. for up to 60,000 requests/mo.
  • 44.
    Learn More AboutAWS Explore http://aws.amazon.com Read Our Blog at http://aws.typepad.com Amazon’s hiring www.amazon.com/jobs
  • 45.
  • 46.

Editor's Notes

  • #11 01/28/12 12:47 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. Reducing the barriers of entry to building web-friendly applications and businesses. We have looked at the investment we have made in technology over the last 10 years and Amazon Web Services strategy is to see what parts of our technology is exposed to developers outside of Amazon.com
  • #13 And here is a number of other companies that have built exciting applications on the web….
  • #31 01/28/12 12:47 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. Here is how it works: Let’s say you are a developer with an amazing new idea for a business that before now has not been feasible because of the resources necessary to access a network of humans. In the context of AMT, we call you a Requester. You use the API to submit submit tasks that require human intelligence – what we call HITs – to the Amazon Mechanical Turk web site where a vast network of people – what we call Workers – come to make money by using the skills their innate, and still extremely valuable, common human sense. They complete these tasks and submit them for approval back to the Requester or the software they wrote. Once the HIT is verified as accurate, the system automatically pays the Workers. Developers can program this entire process to happen automatically and be integrated into the processing of their software. What does this mean for developers?
  • #32 For software developers, the Amazon Mechanical Turk web service solves the problem of building applications that until now have not worked well because they lack human intelligence. Humans are much more effective than computers at solving some types of problems, like finding specific objects in pictures, evaluating beauty, or translating text. The Amazon Mechanical Turk web service gives developers a programmable interface to a network of humans to solve these kinds of problems and incorporate this human intelligence into their applications. For businesses and entrepreneurs who want tasks completed, the Amazon Mechanical Turk web service solves the problem of getting work done in a cost-effective manner by people who have the skill to do the work. The service provides access to a vast network of human intelligence with the efficiencies and cost-effectiveness of computers. Oftentimes, the cost of establishing a network of skilled people to do the work outweighs the value of completing it. By turning the fixed costs into variable costs that scale with business needs, the Amazon Mechanical Turk web service eliminates this barrier and allows work to be completed that before was not economical. For people who want to earn money in their spare time, the Amazon Mechanical Turk web site solves the problem of finding work that they can do wherever and whenever they want. 01/28/12 12:47 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
  • #33 01/28/12 12:47 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. - Let’s talk about where it came from - In 1769, Hungarian nobleman Wolfgang von Kempelen astonished Europe by building a mechanical chess-playing automaton that defeated nearly every opponent it faced. A life-sized wooden mannequin, adorned with a fur-trimmed robe and a turban, Kempelen’s "Turk" was seated behind a cabinet and toured Europe confounding such brilliant challengers as Benjamin Franklin and Napoleon Bonaparte. To persuade skeptical audiences, Kempelen would slide open the cabinet’s doors to reveal the intricate set of gears, cogs and springs that powered his invention. He convinced them that he had built a machine that made decisions using artificial intelligence.
  • #34 01/28/12 12:47 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. What they did not know was the secret behind the Mechanical Turk: a human chess master cleverly concealed inside.