Web Scale Computing



 Mike Culver
 Amazon Web Services
Agenda

   Web Scale Computing Overview
   Code
The Obligatory Timeline Slide



  COBOL,                                                Amazon.com
   Edsel      ARPANET                          Internet

                                  Web         Web as a           Web Services,
           Darkness
                                Awareness     Platform        Resources Eliminated
                    9       2       6 7                1    4             6
     9           96      98       99 199             00 200            00
 195            1       1        1                  2                 2
                                       Dot-Com Bubble       Web 2.0           Web Scale
                                                                              Computing
Being Relevant Counts
It’s All About Ideas
Ideas Cost Money



           a + bc
But How Much Money?




              x + bc
              a
       Our favorite numbers: 0.10, 0.15, and 0.20
Amazon Web Services Are...

   A set of APIs and business models which give
   developers access to Amazon technology and
   content
    Data As a Service                  Search As a Service
       Amazon E-Commerce Service          Alexa Web Information Service
       Amazon Historical Pricing          Alexa Top Sites
                                          Alexa Site Thumbnail
                                          Alexa Web Search Platform

    Infrastructure As a Service
       Amazon Simple Queue Service
       Amazon Simple Storage Service   People As a Service
       Amazon Elastic Compute Cloud       Amazon Mechanical Turk



                                                                      10
Amazon Web Scale Infrastructure:
We Do the Muck
    Leverage Amazon’s Core Expertise
      Scales to the Enterprise
      Priced for People
      Reliable
      Simple to Use
      Highly Available
    Using Industry Standards
      HTTP, XML, REST, SOAP, XSLT
    With World-Class Developer Support
What do these companies have in
 common?
Amazon E-Commerce Service
Amazon E-Commerce Service

   Exposes Amazon's product data plus a
   shopping cart
   In 4th major release
      Millions of products
      Rich data schema
      Product images
      Reviews
      Wishlists
   Commissions via Amazon Associates
   program
Amazon.com With Your Look
Amazon ECS Cost




       $0.00
Amazon Simple Storage Service
Amazon S3 is… Simple

   Data Storage in Amazon Data Center
   Web Service interface
   No set-up fee, No monthly minimum
   Storage: $0.15 per GB/Month
   Data Transfer: $0.20/GB to transfer data
   Private and public storage
   Each object up to 5GB in size
Many ISV Implementations

  S3 Explorer            filicio.us        Jungle Disk




  S3 Firefox Organizer                MyOwnDB
Amazon Elastic Compute Cloud
Amazon EC2 Is…

   A Web service that provides resizable
   compute capacity in the cloud.
   Designed to make Web-scale computing
   easier for developers.
   A simple Web service interface that provides
   complete control of your computing
   resources
Amazon EC2 Benefits You
Because It
   Reduces the time required to obtain and boot
   new server instances to minutes
   Quickly scales capacity, both up and down,
   as your computing requirements change
   Changes the economics of computing:
     Pay only for capacity that you actually use
     a + bc becomes just bc
Amazon EC2 Changes
Computing Economics
   No start-up, monthly, or fixed costs
   $0.10 per CPU hour
   $0.20 per GB transferred across Net
   No cost to transfer data between Amazon S3
   and Amazon EC2
Amazon EC2 Profile

   1.7 Ghz x86 processor
   1.7 GB of RAM
   160 GB disk
   250 Mb/second network
Web-Scale for Mobile
Gumiyo.com             Services Used
                       •Amazon S3
                       •Amazon EC2


                       Estimated Savings
                       $650,000
Amazon Mechanical Turk
Amazon Mechanical Turk Is…

   A web service that exposes an on-demand
   global workforce ready to complete small
   tasks in exchange for micro-payments
   Frictionless. Outsourcing per-se is irrelevant.
   A web services API
Identify Road Markings
Are These the Same MP3 Players?
How It Works
                                             www.mturk.com


     Requester                 Human
    (Developer)              Intelligence
                             Tasks (HITs)




                              Worker
                            Qualifications



                            Completed
 Artificial, Artificially      HITs

 Intelligent Software
                                                  Workers
                                                            30
Man Versus Machine




                     32
But Not Really




                 33
Podcast transcription service provider
Transcribes audio into high-quality text
Amazon Mechanical Turk: Workers
transcribe podcasts and index text within
search engine
Amazon Simple Storage: Stores the
podcasts and related files
Amazon Simple Queue
Amazon Simple Queue Is

   A reliable, highly scalable hosted distributed
   queue for storing messages as they travel
   between computers.
   Amazon's web-scale messaging
   infrastructure as a web service
   Platform-agnostic, allowing any computer on
   the Internet to add or read messages without
   any installed software or special firewall
   configurations
Amazon SQS Highlights

   Reliable
     Runs within Amazon's high-availability
     data centers.
     Messages are stored redundantly across
     multiple servers and data centers.
   Simple: Only Six Methods
   Scalable
   Secure: Strong access controls
Amazon SQS Pricing

   Pay only for what you use
     No minimum fee
     No start-up cost
   $0.10 per 1,000 messages sent ($0.0001 per
   message sent)
   $0.20 per GB of data transferred
Alexa Web Services
Four Alexa Web Services:
One Vast Web Database
    Alexa Web Information Service:
    Web search engine, traffic information for
    web sites, a web site directory, and more.
    Alexa Top Sites:
    Programmatic access top sites data is
    available for the web as a whole, or for
    individual countries
Four Alexa Web Services:
One Vast Web Database
    Alexa Site Thumbnail:
    Programmatic access to thumbnail images
    for the home pages of web sites.
    Alexa Web Search Platform:
    Directly analyze Alexa's Web archive to
    create and publish new search services
Amazon Historical Pricing
Amazon Historical Pricing

    Highlights
       Access to 3 years of actual sales history
       from books, music, videos, and DVDs sold
       on Amazon.com
       Returns the average, minimum, maximum,
       and median price for items
       Up to ten items per request
    Pricing:
    $249/mo. for up to 60,000 requests/mo.
Learn More About AWS

   Explore http://aws.amazon.com
   Read Our Blog at http://aws.typepad.com
   Amazon’s hiring www.amazon.com/jobs
Code!!!

Amazon Web Services

  • 1.
    Web Scale Computing Mike Culver Amazon Web Services
  • 2.
    Agenda Web Scale Computing Overview Code
  • 3.
    The Obligatory TimelineSlide COBOL, Amazon.com Edsel ARPANET Internet Web Web as a Web Services, Darkness Awareness Platform Resources Eliminated 9 2 6 7 1 4 6 9 96 98 99 199 00 200 00 195 1 1 1 2 2 Dot-Com Bubble Web 2.0 Web Scale Computing
  • 5.
  • 7.
  • 8.
  • 9.
    But How MuchMoney? x + bc a Our favorite numbers: 0.10, 0.15, and 0.20
  • 10.
    Amazon Web ServicesAre... A set of APIs and business models which give developers access to Amazon technology and content Data As a Service Search As a Service Amazon E-Commerce Service Alexa Web Information Service Amazon Historical Pricing Alexa Top Sites Alexa Site Thumbnail Alexa Web Search Platform Infrastructure As a Service Amazon Simple Queue Service Amazon Simple Storage Service People As a Service Amazon Elastic Compute Cloud Amazon Mechanical Turk 10
  • 11.
    Amazon Web ScaleInfrastructure: We Do the Muck Leverage Amazon’s Core Expertise Scales to the Enterprise Priced for People Reliable Simple to Use Highly Available Using Industry Standards HTTP, XML, REST, SOAP, XSLT With World-Class Developer Support
  • 12.
    What do thesecompanies have in common?
  • 13.
  • 14.
    Amazon E-Commerce Service Exposes Amazon's product data plus a shopping cart In 4th major release Millions of products Rich data schema Product images Reviews Wishlists Commissions via Amazon Associates program
  • 15.
  • 16.
  • 17.
  • 18.
    Amazon S3 is…Simple Data Storage in Amazon Data Center Web Service interface No set-up fee, No monthly minimum Storage: $0.15 per GB/Month Data Transfer: $0.20/GB to transfer data Private and public storage Each object up to 5GB in size
  • 19.
    Many ISV Implementations S3 Explorer filicio.us Jungle Disk S3 Firefox Organizer MyOwnDB
  • 20.
  • 21.
    Amazon EC2 Is… A Web service that provides resizable compute capacity in the cloud. Designed to make Web-scale computing easier for developers. A simple Web service interface that provides complete control of your computing resources
  • 22.
    Amazon EC2 BenefitsYou Because It Reduces the time required to obtain and boot new server instances to minutes Quickly scales capacity, both up and down, as your computing requirements change Changes the economics of computing: Pay only for capacity that you actually use a + bc becomes just bc
  • 23.
    Amazon EC2 Changes ComputingEconomics No start-up, monthly, or fixed costs $0.10 per CPU hour $0.20 per GB transferred across Net No cost to transfer data between Amazon S3 and Amazon EC2
  • 24.
    Amazon EC2 Profile 1.7 Ghz x86 processor 1.7 GB of RAM 160 GB disk 250 Mb/second network
  • 25.
    Web-Scale for Mobile Gumiyo.com Services Used •Amazon S3 •Amazon EC2 Estimated Savings $650,000
  • 26.
  • 27.
    Amazon Mechanical TurkIs… A web service that exposes an on-demand global workforce ready to complete small tasks in exchange for micro-payments Frictionless. Outsourcing per-se is irrelevant. A web services API
  • 28.
  • 29.
    Are These theSame MP3 Players?
  • 30.
    How It Works www.mturk.com Requester Human (Developer) Intelligence Tasks (HITs) Worker Qualifications Completed Artificial, Artificially HITs Intelligent Software Workers 30
  • 31.
  • 32.
  • 33.
    Podcast transcription serviceprovider Transcribes audio into high-quality text Amazon Mechanical Turk: Workers transcribe podcasts and index text within search engine Amazon Simple Storage: Stores the podcasts and related files
  • 34.
  • 35.
    Amazon Simple QueueIs A reliable, highly scalable hosted distributed queue for storing messages as they travel between computers. Amazon's web-scale messaging infrastructure as a web service Platform-agnostic, allowing any computer on the Internet to add or read messages without any installed software or special firewall configurations
  • 36.
    Amazon SQS Highlights Reliable Runs within Amazon's high-availability data centers. Messages are stored redundantly across multiple servers and data centers. Simple: Only Six Methods Scalable Secure: Strong access controls
  • 37.
    Amazon SQS Pricing Pay only for what you use No minimum fee No start-up cost $0.10 per 1,000 messages sent ($0.0001 per message sent) $0.20 per GB of data transferred
  • 38.
  • 39.
    Four Alexa WebServices: One Vast Web Database Alexa Web Information Service: Web search engine, traffic information for web sites, a web site directory, and more. Alexa Top Sites: Programmatic access top sites data is available for the web as a whole, or for individual countries
  • 40.
    Four Alexa WebServices: One Vast Web Database Alexa Site Thumbnail: Programmatic access to thumbnail images for the home pages of web sites. Alexa Web Search Platform: Directly analyze Alexa's Web archive to create and publish new search services
  • 41.
  • 42.
    Amazon Historical Pricing Highlights Access to 3 years of actual sales history from books, music, videos, and DVDs sold on Amazon.com Returns the average, minimum, maximum, and median price for items Up to ten items per request Pricing: $249/mo. for up to 60,000 requests/mo.
  • 43.
    Learn More AboutAWS Explore http://aws.amazon.com Read Our Blog at http://aws.typepad.com Amazon’s hiring www.amazon.com/jobs
  • 45.

Editor's Notes

  • #11 03/28/13 10:57 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. Reducing the barriers of entry to building web-friendly applications and businesses. We have looked at the investment we have made in technology over the last 10 years and Amazon Web Services strategy is to see what parts of our technology is exposed to developers outside of Amazon.com
  • #13 And here is a number of other companies that have built exciting applications on the web….
  • #31 03/28/13 10:57 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. Here is how it works: Let’s say you are a developer with an amazing new idea for a business that before now has not been feasible because of the resources necessary to access a network of humans. In the context of AMT, we call you a Requester. You use the API to submit submit tasks that require human intelligence – what we call HITs – to the Amazon Mechanical Turk web site where a vast network of people – what we call Workers – come to make money by using the skills their innate, and still extremely valuable, common human sense. They complete these tasks and submit them for approval back to the Requester or the software they wrote. Once the HIT is verified as accurate, the system automatically pays the Workers. Developers can program this entire process to happen automatically and be integrated into the processing of their software. What does this mean for developers?
  • #32 For software developers, the Amazon Mechanical Turk web service solves the problem of building applications that until now have not worked well because they lack human intelligence. Humans are much more effective than computers at solving some types of problems, like finding specific objects in pictures, evaluating beauty, or translating text. The Amazon Mechanical Turk web service gives developers a programmable interface to a network of humans to solve these kinds of problems and incorporate this human intelligence into their applications. For businesses and entrepreneurs who want tasks completed, the Amazon Mechanical Turk web service solves the problem of getting work done in a cost-effective manner by people who have the skill to do the work. The service provides access to a vast network of human intelligence with the efficiencies and cost-effectiveness of computers. Oftentimes, the cost of establishing a network of skilled people to do the work outweighs the value of completing it. By turning the fixed costs into variable costs that scale with business needs, the Amazon Mechanical Turk web service eliminates this barrier and allows work to be completed that before was not economical. For people who want to earn money in their spare time, the Amazon Mechanical Turk web site solves the problem of finding work that they can do wherever and whenever they want. 03/28/13 10:57 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
  • #33 03/28/13 10:57 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. - Let’s talk about where it came from - In 1769, Hungarian nobleman Wolfgang von Kempelen astonished Europe by building a mechanical chess-playing automaton that defeated nearly every opponent it faced. A life-sized wooden mannequin, adorned with a fur-trimmed robe and a turban, Kempelen’s "Turk" was seated behind a cabinet and toured Europe confounding such brilliant challengers as Benjamin Franklin and Napoleon Bonaparte. To persuade skeptical audiences, Kempelen would slide open the cabinet’s doors to reveal the intricate set of gears, cogs and springs that powered his invention. He convinced them that he had built a machine that made decisions using artificial intelligence.
  • #34 03/28/13 10:57 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary. What they did not know was the secret behind the Mechanical Turk: a human chess master cleverly concealed inside.