Chuck Neerdaels VP of Storage and Edge Services Y! Scalable Storage and Delivery Service

Cloud Storage & Edge Services Overview Bulk Storage (Video, Mail attachments, Images, …) : MObStor DORA (Clustered Object Store) Structured Storage (key-value and user data): Sherpa/PNUTs User Database (UDB) UDS/UPS

Bulk Storage (Video, Mail attachments, Images, …) :

MObStor

DORA (Clustered Object Store)

Structured Storage (key-value and user data):

Sherpa/PNUTs

User Database (UDB)

UDS/UPS

CS&E cont. Messaging and Replication: Many variations Edge: ySquid Traffic Server Core Proxy Infrastructure Multi-vendor CDN Global DNS & local load balancing technology

Messaging and Replication:

Many variations

Edge:

ySquid

Traffic Server

Core Proxy Infrastructure

Multi-vendor CDN

Global DNS & local load balancing technology

Typical Use Cases User Data and Preferences Image/Video Storage & Delivery RSS Feeds Caching, Load Balancing

A Quick Demo…

more cute puppy photos

Want a real demo? Sports.yahoo.com Travel.yahoo.com Finance.yahoo.com Mail.yahoo.com News.yahoo.com *.yahoo.com

Infrastructure and Platform Service Reality Demos are only notable when they fail On-line 7x24x365 Through upgrades, attacks, outages, human errors, … Legitimate use cases can be as challenging as abuse It’s all assumed to, “just work” But, what if they didn’t…

Demos are only notable when they fail

On-line 7x24x365

Through upgrades, attacks, outages, human errors, …

Legitimate use cases can be as challenging as abuse

It’s all assumed to, “just work”

But, what if they didn’t…

But what if they didn’t… Globally: Sites offline Load balancing fails Images, CSS, and JavaScript not served No logins and personalization Normalized RSS feeds go stale Site thumbnails fail Server-to-server communication ceases Ads Ads not served Ad pipelines halt Flickr: No more video or images Mail: Webmail slow of offline, attachments missing Mobile & Social: Features go offline Sports Overwhelmed every Sunday Fantasy Sports draft data gets stale Yahoo! sites would become stale, boring, slow, non-profitable, or non-existent

Globally:

Sites offline

Load balancing fails

Images, CSS, and JavaScript not served

No logins and personalization

Normalized RSS feeds go stale

Site thumbnails fail

Server-to-server communication ceases

Ads

Ads not served

Ad pipelines halt

Flickr:

No more video or images

Mail:

Webmail slow of offline, attachments missing

Mobile & Social:

Features go offline

Sports

Overwhelmed every Sunday

Fantasy Sports draft data gets stale

What Makes an On-line Cloud Service at Y!… Global

Internal Expectations Global Solution, Including Emerging Markets Fully Managed Multi-tenant (Finance) Single tenant (Customers) Elastic Predictable SLA’s Simple to provision and manage Providing the illusion of infinite bandwidth, capacity, (and single tenancy)…

Global Solution, Including Emerging Markets

Fully Managed

Multi-tenant (Finance)

Single tenant (Customers)

Elastic

Predictable SLA’s

Simple to provision and manage

Providing the illusion of infinite bandwidth, capacity, (and single tenancy)…

Common Challenges Speed of light Huge traffic spike potential Cost Partitioned network failures are assumed Consumer/User intuition

Speed of light

Huge traffic spike potential

Cost

Partitioned network failures are assumed

Consumer/User intuition

A Little More Detail: MObStor & Sherpa

MObStor: Blowing Through the Cloudiness Global Servers Caching Protocol Termination Authentication Content Routing Local Servers Auto Expiration De-duplication Object Placement Re-replication

MObStor: Trying to Increase 9’s DNS GSLB Hardware VIP Scalable Session Manager Internal DNS with Load Feedback Hardware VIP Local Replication Global Replication Separate Metadata Replication

Sherpa/PNUTs

Sherpa/PNUTs DNS GSLB Hardware VIP Scalable Router & Session Manager Consistency vs. Availability Knobs Local Replication Global Replication Ordered Messaging

Physics and Economics for a Global Cloud…

What’s your target SLA? 0 Distance between producer/consumer *Illustrative, and not intended to portray reality Latency Speed of light + network degradation

What’s your budget? Latency Replication Costs *Illustrative, and not intended to portray reality $$$ $

Lessons Learned Intuition is often wrong – let data drive your data Provide hooks, experimental feedback, and mobility N-way global replication is really expensive Customers don’t understand 95/5 billing Customers don’t understand CAP theorem Verify all provisioning Strive for quality, but plan for failure

Intuition is often wrong – let data drive your data

Provide hooks, experimental feedback, and mobility

N-way global replication is really expensive

Customers don’t understand 95/5 billing

Customers don’t understand CAP theorem

Verify all provisioning

Strive for quality, but plan for failure

Why would any of you care about a Y! proprietary cloud?

Committed to Open Source Component approach – current pipeline: ZooKeeper (already available) Traffic Server (just released!) More to come

Component approach – current pipeline:

ZooKeeper (already available)

Traffic Server (just released!)

More to come

YAHOO! IS OPEN SOURCING Cloud building block Processes up to 35K req/second per box Extensible Framework Edge Caching, Load Balancing, Proxying Leveraged in Edge, Storage, Serving services Supports more than half of all Yahoo! traffic Available via the Apache Software Foundation

YAHOO! IS OPEN SOURCING

Cloud building block

Processes up to 35K req/second per box

Extensible Framework

Edge Caching, Load Balancing, Proxying

Leveraged in Edge, Storage, Serving services

Supports more than half of all Yahoo! traffic

Available via the Apache Software Foundation

Questions? Y! Scalable Storage and Delivery Service