Werner Vogels @ FOWA Feb 07

Web Scale Computing: Compete on Ideas, Not Resources Werner Vogels Fast Forward to 33 minutes CTO – Amazon.com 2 3 4 What if… What if… • Launching a new business on the web was simple? • Launching a new business on the web was simple? • You only had to focus on “the business”? • You only had to focus on “the business”? • You could manage growth more easily? • You could manage growth more easily? What if you only had to compete on ideas? 5 6 1

PUSH VS PULL • Demand is anticipated • Demand is uncertain • Top down design • Emergent design • Centralized design • Decentralized • Procedural • Modular • Tightly coupled • Loosely coupled • Resource centric • People centric • Restricted participation • Open participation • Limited re-engineering • Rapid incremental innovation 7 8 Forces Driving Alternative Resource Models DESIGN • Increasing Uncertainty l • Growing Abundance de • Intensifying Competition o M REFINE DEPLOY • Growing Power of Customers h • Greater Focus on Learning and Improvisation us P EXECUTE & MONITOR 9 10 Resource Management in an Uncertain World FIND • Acquire resources on demand l • Release resources when no longer needed de • Pay for what you use o CONNECT REFLECT • Leverage other’s core competencies lM l Pu INNOVATE 11 12 2

A New Web Business using the Push Model The 70 / 30 Switch Undifferentiated Successful Your Idea “Heavy Lifting” Product 30 % of time, energy, and dollars on differentiated value creation Hardware Costs Software Costs Maintenance Expertise Load Balancing Scaling and Physical Growth 70 % of time, energy, and dollars Costs to run idle servers on undifferentiated heavy lifting Bandwidth Management Server Hosting 13 14 How can you ever build an application that needs to • scale to 100% - 1000% increase popularity? • provide 4 nines uptime? • survive a complete datacenter failure? • survive a network partition • While keeping cost low at the same time? 15 16 The Reliability of Hard Disks Failure Trends in a Large Disk Drive Population Chapter 4: Priorities – Scale Later – It is too hard to get right Eduardo Pinheiro, Wolf-Dietrich Weber and Luiz André Barroso 17 18 3

Challenges to just get the infrastructure right Resources in the Pull Model: Web-Scale Computing • Networks, hardware, operating systems, databases, Scalable Infrastructure that allow your applications middleware all fail all the time to meet infinite demand, cheaply and reliably • How to beat the CAP theorem • Turn huge fixed costs into variable • Datacenters also fail (tornados, heat waves) • Scale up and down as your business does • To use scale as a cost-effective advantage you need really large numbers • Pay as you go • Operations is just plain hard, large & small • Leverage other’s core competencies • Invest intellectual bandwidth to build the infrastructure • Focus on your Idea • Invest funds for something that may not happen (yet) 19 20 Amazon S3, EC2 & SQS Amazon Simple Storage Service (S3) Scalable – Increase or decrease capacity in minutes Storage for the internet via web service calls • Cost-Effective – Low rate, pay-as-you-go Private and public storage options • Reliable – Runs on Amazon’s proven infrastructure Simple – SOAP- and REST-based Web Service Calls $.15 per GB/ per Month to store data • Compatible – Use Amazon EC2 and S3 together to receive free data $.20 per GB/ per Month to transfer data • transmission between services, decreased latency, and consistency. Use Cases: Unlimited Data Storage, Media Sharing/ • Distribution, Archiving, Server Back-ups 21 22 Smugmug’s growth 140 120 millions of 100 photos 80 60 40 20 0 2002 2003 2004 2005 2006 23 24 4

Smugmug & Amazon S3 • Saved $474,000 in first 7 months of operation • Pre-S3: – Buying $40K/month of new hardware – Without S3 this would now be $80K/month • Post-S3: – Selling excess hardware on eBay – Costs even lower than projected • Expecting savings of $1M to $2M in 2007 25 26 27 28 S3 Explorer Jungle Disk Amazon Elastic Compute Cloud (EC2) filicio.us Virtual Computing Environment Scalable Capacity – Pay as You Go $.10 per Server Hour $.20 per GB of data transfer Use cases: Load testing, Time or Traffic-based S3 Firefox Organizer MyOwnDB Scaling, Simulation and Analysis, Rendering, SaaS Platform 29 30 5

31 32 AWS Product Family Infrastructure: Web Search: Amazon Simple Queue Service Alexa Web Search Amazon Simple Storage Service Alexa Web Information Service Amazon Elastic Compute Cloud Alexa Top Sites Alexa Site Thumbnail E-Commerce: Workforce / Workflow: Amazon E-Commerce Service Amazon Historical Pricing Amazon Mechanical Turk 33 34 http:/ /aws.amazon.com 35 6

Werner Vogels @ FOWA Feb 07

by carsonsystems

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 2

Statistics

Werner Vogels @ FOWA Feb 07 Presentation Transcript