Your SlideShare is downloading. ×
0
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Walking Through Cloud Serving at Yahoo!
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Walking Through Cloud Serving at Yahoo!

4,667

Published on

From Surendra Reddy's presentation "Walking Through Cloud Serving at Yahoo!" at the 2009 Cloud Computing Expo in Santa Clara, CA, USA. Here's the talk description on the Expo's site: …

From Surendra Reddy's presentation "Walking Through Cloud Serving at Yahoo!" at the 2009 Cloud Computing Expo in Santa Clara, CA, USA. Here's the talk description on the Expo's site: http://cloudcomputingexpo.com/event/session/508

Published in: Technology
0 Comments
18 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,667
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
18
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • _____We believe, Cloud computing is about Driving Innovation.At Yahoo!, we are developing and deploying Cloud Services to help us drive innovationBy providing a set of horizontal Cloud services that are broadly applicable to building many Yahoo! products We are able to reallocate many developers to building products and features from building product-centric infrastructure. We are providing a stable base of Cloud Services that enable product developers to iterate quickly and innovate without risking product quality We are providing the ability to process and analyze enormous amounts of data easily and quickly for purposes of improving consumer experience with our products.
  • Hundreds of properties/productsEnormous Scale 500M+ unique visitors monthly 300M+ Mail users 100’s of petabytes of storage 100’s of billions of objects stored Billions of daily requestsGlobal footprint
  • Transcript

    • 1. Yahoo Cloud Serving<br />Surendra Reddy<br />Cloud Computing Conference &amp; ExpoNovember 3, 2009<br /> <br />
    • 2. Imagining the Future of Cloud!<br />
    • 3. Cloud is a Promise, But …<br />
    • 4. Reference:<br /> Tussle in Cyberspace: Defining Tomorrow’s Internet<br /> http://conferences.sigcomm.org/sigcomm/2002/papers/tussle.pdf<br />
    • 5. Customer Tussles:<br /><ul><li>Outages: 76% outages caused by change management
    • 6. Unmet Needs: Rapidly changing needs</li></li></ul><li>Management Tussles:<br /><ul><li>Growing Costs and Complexity
    • 7. Long Deployment Cycles: Inflexible, peak-provisioned infrastructure</li></li></ul><li>Operations Tussles:<br /><ul><li>Entangled in many day-to-day failures & troubleshooting
    • 8. Crippled Infrastructure
    • 9. Building on weak foundation</li></li></ul><li>Developers Tussles:<br /><ul><li>Bloated Operating System, App Stacks
    • 10. Staggering Complexities of Dependencies
    • 11. No time for QA – Bugs Creeping in all over</li></li></ul><li>Security Tussles:<br /><ul><li>Whose problem is it?
    • 12. Auditability and Assurance
    • 13. Governance
    • 14. Compliance</li></li></ul><li>
    • 15. As a Result: <br /><ul><li>More Friction, Waste, and Frustration
    • 16. Less Innovation </li></li></ul><li>So, What is Cloud?<br /><ul><li>C – Customer Driven
    • 17. L – Lean
    • 18. O – Optimized
    • 19. U – Utility
    • 20. D – Demand-Driven</li></li></ul><li><ul><li>Cloudis NOT about Cost cutting
    • 21. It is about Value Creation enabling Innovation</li></li></ul><li><ul><li>Cloud is an opportunity to clean up the Infrastructure mess!</li></li></ul><li>
    • 22. Yahoo! is Perfect for Cloud Computing<br />600M<br />UNIQUE USERS / MONTH<br />300M+<br />YAHOO! MAIL USERS / MONTH<br />HUNDREDS<br />OF PROPERTIES / PRODUCTS<br />BILLIONS<br />OF OBJECTS STORED<br />HUNDREDS<br />OF PETABYTES OF STORAGE<br />PETABYTES<br />OF TRAFFIC DAILY<br />
    • 23. <ul><li>Declarative language for specifying the full, end-to-end structure of a service
    • 24. Includes multiple environments: Dev, QA, Staging, and production
    • 25. Extreme Automation</li></li></ul><li>Yahoo! Cloud Serving - Principles<br /><ul><li>Metastructure: Abstracts concerns of the underlying infrastructure and the network communication
    • 26. Declarative application structure
    • 27. Continuous Integration, Continuous Deployment, & Continuous Compliance*
    • 28. End-to-End Security</li></ul>* Rich Miller @RHM2K first coined “Continuous Compliance” <br />
    • 29. Yahoo! Cloud Serving - Principles<br /><ul><li>Virtualized hardware
    • 30. Standardized software stacks and packaging
    • 31. Integrated service management
    • 32. Containers vs. Machines</li></li></ul><li>Yahoo! Cloud Serving - Principles<br /><ul><li>Elastic serving of changing workloads
    • 33. Controlled/Intelligent traffic direction
    • 34. Controlled execution environment
    • 35. Managed Communication
    • 36. Service Associations, Bindings, and Access Controls</li></li></ul><li>Yahoo! Cloud: Tiers without Tears<br />Entrypoint<br />Naming<br />Presentation<br />Tier<br />Application Logic<br />Tier<br />Bindings<br />Connections<br />Database Tier<br />
    • 37. Yahoo! Cloud: Tiers without Tears<br />Entrypoint<br />Naming<br />Load<br />Balancer<br />Presentation<br />Tier<br />Application Logic<br />Tier<br />node<br />node<br />node<br />Bindings<br />Connections<br />Database Tier<br />
    • 38. Yahoo! Cloud: Extreme Automation<br />Abstracting Services<br />Component AnswersCore {<br /> Entry root;<br />Tier pres {<br /> app = “answers_web_tier”;<br /> Exit applogic;<br /> nsize = small;<br /> nnodes = 3;<br /> };<br />Tier applogic {<br /> app = “answers_al_tier”;<br /> Exit data;<br /> nsize = medium;<br /> nnodes = 3;<br /> };<br />Tier data;<br /> Connections {<br /> root -&amp;gt; pres;<br /> pres.applogic -&amp;gt; applogic;<br /> applogic.data -&amp;gt; data;<br /> }<br />} <br />Presentation<br />Tier<br />Application Logic<br />Tier<br />Database Tier<br />
    • 39. Yahoo! Cloud: Extreme Automation<br />QA Environment<br />Environment qa(ne1) {<br /> TrafficDirector root {<br /> name = “qa.answers.yahoo.com”;<br /> access = testing;<br /> };<br /> AnswersCore core;<br /> Connections {<br /> root -&amp;gt; core;<br /> }<br />} <br />Presentation<br />Tier<br />Presentation<br />Tier<br />Application Logic<br />Tier<br />Application Logic<br />Tier<br />Testing Testing<br />Environment production(ac4,sk1) {<br /> TeeTrafficDirector root {<br /> name = “answers.yahoo.com”;<br /> access = public;<br /> split_percent = 10;<br /> split_destination = qa.root;<br /> };<br /> AnswersCore core;<br /> Connections {<br /> root -&amp;gt; core;<br /> }<br />} <br />Database Tier<br />Database Tier<br />
    • 40. Yahoo! Cloud: Extreme Automation<br />Apps written against rich stacks<br />Includes core infra and platforms<br /><ul><li>Infrastructure: Security, monitoring, metering, abuse prevention
    • 41. Platforms: data, user info, ad systems, configuration bundles
    • 42. All “auto configured”</li></li></ul><li>Serving Cloud Serving for Y!’s Cloud<br />
    • 43. <ul><li>Infrastructure to Metastructure
    • 44. Machines to Containers
    • 45. Friction to Innovation
    • 46. Tears to Tiers
    • 47. Complexity to Simplexity
    • 48. Costs to Revenue Streams</li></li></ul><li>Cloud is a Promise, Service is the fulfillment!<br />
    • 49. OPEN CLOUD ACCESS PROTOCOL<br />
    • 50. OPEN CLOUD ACCESS PROTOCOL<br /><ul><li>Definition, Deployment, and Life Cycle Management of Cloud Resources
    • 51. Allocation, Provisioning, and Metering of Cloud Resources
    • 52. Metadata/Registry for Cloud Resources
    • 53. Virtual Infrastructure Management</li></li></ul><li>Yahoo! Submitted OpenCAP to IETF<br /><ul><li>Why IETF?
    • 54. Specification behind Yahoo Implementation
    • 55. Plan to open source implementation</li></li></ul><li>Open Cloud Access Protocol<br />Resource Model<br /><ul><li>All Infrastructure and Functional Components are modeled as Resources
    • 56. Nodes, Networks, Storage Devices, Databases, Load Balancers, Identities, Key Value Store, Object Stores, Caching Services</li></ul>Resource Properties<br /><ul><li>Modeled as JSON objects
    • 57. Standard Catalog of attributes, extensible</li></ul>Resource Operations<br /><ul><li>Operation, Control, and Notification</li></li></ul><li>Open Cloud Access Protocol<br />Management Operation Services<br /><ul><li>CREATE - Create an instance of a managed resource
    • 58. DELETE - Delete an instance of a managed resource
    • 59. GET - Request managed resource properties (for one resource or a set of resource)
    • 60. CANCEL - Cancel an outstanding request
    • 61. SET - Set managed resource properties
    • 62. ACTION - Request an action to be performed on a managed resource</li></li></ul><li>Open Cloud Access Protocol<br />Management Notification Services<br /><ul><li>EVENT- Send events occurring on managed resources (VM events, VM stats etc, SNMP events etc)
    • 63. ALERT- Send alerts for resource failures and exceptions</li></ul>Life Cycle Management Services<br /><ul><li>START- Starts the resource
    • 64. TERMINATE – Stop the resource
    • 65. SUSPEND – Suspend the resource
    • 66. RESUME – Resume the resource
    • 67. ARCHIVE – Archive the resource</li></li></ul><li>Yahoo! Open Source &amp; Research<br />Open Source<br /><ul><li>Cloud Serving Platform, Yahoo Implementation of OpenCAP
    • 68. Traffic Server</li></ul>Research Activities<br /><ul><li>Open Cloud Consortium
    • 69. OpenCirrus & M45</li></li></ul><li>THANK YOU<br />Surendra Reddy<br />@sureddy twitter<br />sureddy@yahoo-inc.com email<br />yCloudBlog.com blog<br />
    • 70. TUESDAY, 11/3<br />4:50pm – 5:35 pm<br />Hadoop @ Yahoo! – Internet<br />Scale Data Processing<br />Eric Baldeschwieler<br />VP, Hadoop Software Development<br />WEDNESDAY, 11/4<br />9:10 am - 9:55 am<br />Yahoo! Scalable Storage and<br />Delivery Services<br />Chuck Neerdaels<br />VP, Storage and Edge Services<br />VISIT BOOTH #103 <br />TO TALK WITH YAHOO! ENGINEERS AND LEARN MORE ABOUT YAHOO!’S VISION FOR CLOUD COMPUTING.<br />

    ×