This document discusses the concept of web scale computing and how it allows businesses to focus on their core ideas rather than infrastructure challenges. It outlines how web scale computing provides scalable infrastructure as a service, allowing applications to meet infinite demand at low cost through services like Amazon S3, EC2, and SQS. This approach transforms huge fixed infrastructure costs into variable pay-as-you-go costs and allows businesses to scale up and down as needed.

1. Web Scale Computing:
Compete on Ideas, Not Resources
Werner Vogels
Fast Forward to 33 minutes
CTO – Amazon.com
2
3 4
What if… What if…
• Launching a new business on the web was simple?
• Launching a new business on the web was simple?
• You only had to focus on “the business”?
• You only had to focus on “the business”?
• You could manage growth more easily?
• You could manage growth more easily?
What if you only had to compete
on ideas?
5 6
1

2. PUSH VS PULL
• Demand is anticipated • Demand is uncertain
• Top down design • Emergent design
• Centralized design • Decentralized
• Procedural • Modular
• Tightly coupled • Loosely coupled
• Resource centric • People centric
• Restricted participation • Open participation
• Limited re-engineering • Rapid incremental
innovation
7 8
Forces Driving Alternative Resource Models
DESIGN
• Increasing Uncertainty
l
• Growing Abundance
de
• Intensifying Competition
o
M
REFINE DEPLOY
• Growing Power of Customers
h
• Greater Focus on Learning and Improvisation
us
P
EXECUTE
& MONITOR
9 10
Resource Management in an Uncertain World
FIND
• Acquire resources on demand
l • Release resources when no longer needed
de • Pay for what you use
o CONNECT
REFLECT • Leverage other’s core competencies
lM
l
Pu
INNOVATE
11 12
2

3. A New Web Business using the Push Model The 70 / 30 Switch
Undifferentiated Successful
Your Idea
“Heavy Lifting” Product 30 % of time, energy, and dollars
on differentiated value creation
Hardware Costs
Software Costs
Maintenance
Expertise
Load Balancing
Scaling and Physical Growth
70 % of time, energy, and dollars
Costs to run idle servers
on undifferentiated heavy lifting
Bandwidth Management
Server Hosting
13 14
How can you ever build an application that needs to
• scale to 100% - 1000% increase popularity?
• provide 4 nines uptime?
• survive a complete datacenter failure?
• survive a network partition
• While keeping cost low at the same time?
15 16
The Reliability of Hard Disks
Failure Trends in a Large Disk Drive Population
Chapter 4: Priorities – Scale Later – It is too hard to get right
Eduardo Pinheiro, Wolf-Dietrich Weber and Luiz André Barroso
17 18
3

4. Challenges to just get the infrastructure right Resources in the Pull Model: Web-Scale Computing
• Networks, hardware, operating systems, databases, Scalable Infrastructure that allow your applications
middleware all fail all the time to meet infinite demand, cheaply and reliably
• How to beat the CAP theorem
• Turn huge fixed costs into variable
• Datacenters also fail (tornados, heat waves)
• Scale up and down as your business does
• To use scale as a cost-effective advantage you need
really large numbers • Pay as you go
• Operations is just plain hard, large & small • Leverage other’s core competencies
• Invest intellectual bandwidth to build the infrastructure • Focus on your Idea
• Invest funds for something that may not happen (yet)
19 20
Amazon S3, EC2 & SQS Amazon Simple Storage Service (S3)
Scalable – Increase or decrease capacity in minutes Storage for the internet via web service calls
•
Cost-Effective – Low rate, pay-as-you-go Private and public storage options
•
Reliable – Runs on Amazon’s proven infrastructure
Simple – SOAP- and REST-based Web Service Calls
$.15 per GB/ per Month to store data
•
Compatible – Use Amazon EC2 and S3 together to receive free data
$.20 per GB/ per Month to transfer data
•
transmission between services, decreased latency, and consistency.
Use Cases: Unlimited Data Storage, Media Sharing/
•
Distribution, Archiving, Server Back-ups
21 22
Smugmug’s growth
140
120 millions of
100 photos
80
60
40
20
0
2002 2003 2004 2005 2006
23 24
4

5. Smugmug & Amazon S3
• Saved $474,000 in first 7 months of operation
• Pre-S3:
– Buying $40K/month of new hardware
– Without S3 this would now be $80K/month
• Post-S3:
– Selling excess hardware on eBay
– Costs even lower than projected
• Expecting savings of $1M to $2M in 2007
25 26
27 28
S3 Explorer Jungle Disk
Amazon Elastic Compute Cloud (EC2)
filicio.us
Virtual Computing Environment
Scalable Capacity – Pay as You Go
$.10 per Server Hour
$.20 per GB of data transfer
Use cases: Load testing, Time or Traffic-based
S3 Firefox Organizer
MyOwnDB Scaling, Simulation and Analysis, Rendering, SaaS
Platform
29 30
5

6. 31 32
AWS Product Family
Infrastructure: Web Search:
Amazon Simple Queue Service Alexa Web Search
Amazon Simple Storage Service Alexa Web Information Service
Amazon Elastic Compute Cloud Alexa Top Sites
Alexa Site Thumbnail
E-Commerce:
Workforce / Workflow:
Amazon E-Commerce Service
Amazon Historical Pricing Amazon Mechanical Turk
33 34
http:/ /aws.amazon.com
35
6