Accelerating Innovation with Cloud Computing

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

more share options
Embed For WordPress.com

Without related presentations

Notes on slide 1

Introduction self Good morning! Thank you for joining me at such an early hour. I’m Shelton Shugar, lead cloud computing at Yahoo!. Before I talk about Cloud Computing at Y!, a disclaimer!

_____ Before I begin telling you about Y! Cloud Computing, a disclaimer I’m not trying to sell you anything. Y! is not selling a service, software, consulting, or hardware. I’m here to share our ideas about Cloud Computing, to talk about our Yahoo!’s Cloud Computing initiative, and to discuss our views on Open Source and our contributions. Vision of Cloud Cloud Services Usage of Cloud Benefits of Cloud Open Source initiatives _____ At Yahoo!, Cloud Computing is NOT about saving money. Improve infrastructure utilization Simplify operating environment Expect Cloud to reduce cost But saving money is not the driver _____

_____ We believe, Cloud computing is about Driving Innovation. At Yahoo!, Cloud computing, Cloud Computing Get products to market faster Add features faster Analyze enormous amounts to improve consumer experience Thousands of develops focus on products, features, research instead of infrastructure With Cloud, we become more agile, but on a stable base

Hundreds of properties/products acquisitions, legacy – all have own infrastructure requires large numbers of resources to support Overlapping needs Cloud horizontal services provides a shared environment and frees developers to focus on products and research Y! Scale influences selection of Cloud Services and architecture Enormous scale causes us to focus on scale first, features second Enormous Scale 500M+ unique visitors monthly 300M+ Mail users 100’s of petabytes of storage 100’s of billions of objects stored Billions of daily requests Global footprint

List all service buckets Edge, Serving, Storage, Batch, Data Collection Connect to non-cloud services Top to bottom for each service Function Systems Capabilities Usage Maturity

Use case of Cloud at Y! starting with our Home page Walk thru…referring to the function and system Benefits to consumer

With the cloud and Yahoo! Traffic Server, Yahoo! Sports was used Cloud to dramatically speed dynamic content updates and access during the NBA draft. This is a 20X improvement in refresh rate. The result: Basketball fans could follow the draft online in almost real-time. Solution is general – can be applied to virtually all properties (end)

Before the cloud, spam filtering was serial and singular. With the cloud and Hadoop, spam is now examined for co-occurrence and machine learning is applied to more parts of the message. Less spam in Yahoo! Mail inboxes! [need talking points] (end)

Before the cloud, Yahoo! was running ad optimization algorithms every 6 hours, with servers considerable idle time between runs. With the cloud, ad optimization algorithms run every 30 minutes and idle time is effectively zero – without any increase in hardware. [need talking points] (end)

I'd define YQL as a common interface for storing, retrieving, and querying web services (both internal Y! and external).

For other applications that depend on global availably of synchronized data, for example: Abuse Detection URL Reputations Social Applications Cloud storage services that support n-way, global replication enable developers to focus on the task at hand (so some wording). Faster time to market shows up mostly around things like YQL – a small team can hammer out a new use case in hours or days. Both MObStor and Sherpa have essentially made ti so new projects like YQL are a blip – after a little provisioning, they just work. The biggest opportunities with folks like Mail or Registration (abuse detection or URL reputations for example) aren’t as much about time to market as allowing for something they just wouldn’t have been possible pre-Sherpa. The same could be said for Digu (Guru) and MObStor – there wasn’t any other solution that could give them n-way global replication for stored objects. The alternative was a filer in one colo with no BCP. But projects this size still have big release cycles, so time to market is a deceptive metric.

Describe each contribution Benefits of open source Plan to open source cloud technology as it matures and doesn’t include Y! specific stuff Developing Hadoop and other Open Source solutions. Aggressively Open Sourcing Yahoo!’s large-scale cloud solutions. Participating in research projects including Open Cirrus, M45

In June, we released a preview version of Yahoo’s Hadoop distro. Now, we’re providing the same production version (RC7) that runs on Yahoo!’s own cloud. (end)

Yahoo! Traffic Server is an extensively updated version of the commercial product we acquired from Inktomi – now available free and open to the world! Yahoo! Traffic Server is up to 100 times faster than existing open-source caching software. (end)

Moving to cloud requires commitment…it doesn’t happen by accident, it requires senior management support, it requires funding to develop and the integrate/change existing applications

4 Favorites

Jagat Kothari, Senior Software Engineer at Home, favorited this 1 week ago
Tags yahoo
F Nagai favorited this 2 weeks ago
Fenng Feng, Evangelist at Alipay.com, favorited this 2 weeks ago
Tags arch
yarapavan favorited this 2 weeks ago
Tags 2009 yahoo cloud cloudexpo innovation

Accelerating Innovation with Cloud Computing - Presentation Transcript

Shelton Shugar Cloud Computing Conference & Expo November 3, 2009 Accelerating Innovation with Cloud Computing
I’m not selling anything
Cloud Computing is NOT about saving money
Yahoo! is Perfect for Cloud Computing HUNDREDS OF PROPERTIES / PRODUCTS 600M UNIQUE USERS / MONTH 300M+ YAHOO! MAIL USERS / MONTH HUNDREDS OF PETABYTES OF STORAGE BILLIONS OF OBJECTS STORED PETABYTES OF TRAFFIC DAILY
Yahoo! Cloud Strategy
- Creating a private Cloud for Yahoo!
- Optimizing for global Yahoo! properties
- Data processing and serving environments
- Multi-year effort
- Open Source
Inside Yahoo!’s Cloud
Yahoo!’s Cloud Use Case Advertising Optimization & Delivery Content Optimization Search Index Image/Video Storage & Delivery RSS Feeds Caching, Load Balancing Machine Learning (e.g. Spam filters)
Cloud improves dynamic content refresh rates and consumer access speed
Cloud abstracts away scale for processing enormous data sets
Cloud speeds advertising optimization by improving infrastructure utilization
Cloud Speeds Time To Market
- YQL
- YQL Open Data Tables built on Cloud storage
Cloud Solving Industry-wide Problems
- Mail abuse detection
- Cloud storage
- Developers focus on task at hand
Yahoo!’s Open Source for Cloud
- YAHOO! IS OPEN SOURCING
- Cloud building block
- Processes up to 35K req/second per box
- Extensible Framework
- Edge Caching, Load Balancing, Proxying
- Leveraged in Edge, Storage, Serving services
- Supports more than half of all Yahoo! traffic
- Available via the Apache Software Foundation
- Current version of Hadoop running @ Yahoo!
- Includes improvements to:
- Robustness, Security, Performance, Operability
- Organizational commitment
- Investment
- Time
Cloud Computing is worth it!