Walking Through Cloud Serving at Yahoo!

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    _____We believe, Cloud computing is about Driving Innovation.At Yahoo!, we are developing and deploying Cloud Services to help us drive innovationBy providing a set of horizontal Cloud services that are broadly applicable to building many Yahoo! products We are able to reallocate many developers to building products and features from building product-centric infrastructure. We are providing a stable base of Cloud Services that enable product developers to iterate quickly and innovate without risking product quality We are providing the ability to process and analyze enormous amounts of data easily and quickly for purposes of improving consumer experience with our products.

    Hundreds of properties/productsEnormous Scale 500M+ unique visitors monthly 300M+ Mail users 100’s of petabytes of storage 100’s of billions of objects stored Billions of daily requestsGlobal footprint

    12 Favorites

    Walking Through Cloud Serving at Yahoo! - Presentation Transcript

    1. Yahoo Cloud Serving
      Surendra Reddy
      Cloud Computing Conference & ExpoNovember 3, 2009
       
    2. Imagining the Future of Cloud!
    3. Cloud is a Promise, But …
    4. Reference:
      Tussle in Cyberspace: Defining Tomorrow’s Internet
      http://conferences.sigcomm.org/sigcomm/2002/papers/tussle.pdf
    5. Customer Tussles:
      • Outages: 76% outages caused by change management
      • Unmet Needs: Rapidly changing needs
    6. Management Tussles:
      • Growing Costs and Complexity
      • Long Deployment Cycles: Inflexible, peak-provisioned infrastructure
    7. Operations Tussles:
      • Entangled in many day-to-day failures & troubleshooting
      • Crippled Infrastructure
      • Building on weak foundation
    8. Developers Tussles:
      • Bloated Operating System, App Stacks
      • Staggering Complexities of Dependencies
      • No time for QA – Bugs Creeping in all over
    9. Security Tussles:
      • Whose problem is it?
      • Auditability and Assurance
      • Governance
      • Compliance
    10. As a Result:
      • More Friction, Waste, and Frustration
      • Less Innovation
    11. So, What is Cloud?
      • C – Customer Driven
      • L – Lean
      • O – Optimized
      • U – Utility
      • D – Demand-Driven
      • Cloudis NOT about Cost cutting
      • It is about Value Creation enabling Innovation
      • Cloud is an opportunity to clean up the Infrastructure mess!
    12. Yahoo! is Perfect for Cloud Computing
      600M
      UNIQUE USERS / MONTH
      300M+
      YAHOO! MAIL USERS / MONTH
      HUNDREDS
      OF PROPERTIES / PRODUCTS
      BILLIONS
      OF OBJECTS STORED
      HUNDREDS
      OF PETABYTES OF STORAGE
      PETABYTES
      OF TRAFFIC DAILY
      • Declarative language for specifying the full, end-to-end structure of a service
      • Includes multiple environments: Dev, QA, Staging, and production
      • Extreme Automation
    13. Yahoo! Cloud Serving - Principles
      • Metastructure: Abstracts concerns of the underlying infrastructure and the network communication
      • Declarative application structure
      • Continuous Integration, Continuous Deployment, & Continuous Compliance*
      • End-to-End Security
      * Rich Miller @RHM2K first coined “Continuous Compliance”
    14. Yahoo! Cloud Serving - Principles
      • Virtualized hardware
      • Standardized software stacks and packaging
      • Integrated service management
      • Containers vs. Machines
    15. Yahoo! Cloud Serving - Principles
      • Elastic serving of changing workloads
      • Controlled/Intelligent traffic direction
      • Controlled execution environment
      • Managed Communication
      • Service Associations, Bindings, and Access Controls
    16. Yahoo! Cloud: Tiers without Tears
      Entrypoint
      Naming
      Presentation
      Tier
      Application Logic
      Tier
      Bindings
      Connections
      Database Tier
    17. Yahoo! Cloud: Tiers without Tears
      Entrypoint
      Naming
      Load
      Balancer
      Presentation
      Tier
      Application Logic
      Tier
      node
      node
      node
      Bindings
      Connections
      Database Tier
    18. Yahoo! Cloud: Extreme Automation
      Abstracting Services
      Component AnswersCore {
      Entry root;
      Tier pres {
      app = “answers_web_tier”;
      Exit applogic;
      nsize = small;
      nnodes = 3;
      };
      Tier applogic {
      app = “answers_al_tier”;
      Exit data;
      nsize = medium;
      nnodes = 3;
      };
      Tier data;
      Connections {
      root -> pres;
      pres.applogic -> applogic;
      applogic.data -> data;
      }
      }
      Presentation
      Tier
      Application Logic
      Tier
      Database Tier
    19. Yahoo! Cloud: Extreme Automation
      QA Environment
      Environment qa(ne1) {
      TrafficDirector root {
      name = “qa.answers.yahoo.com”;
      access = testing;
      };
      AnswersCore core;
      Connections {
      root -> core;
      }
      }
      Presentation
      Tier
      Presentation
      Tier
      Application Logic
      Tier
      Application Logic
      Tier
      Testing Testing
      Environment production(ac4,sk1) {
      TeeTrafficDirector root {
      name = “answers.yahoo.com”;
      access = public;
      split_percent = 10;
      split_destination = qa.root;
      };
      AnswersCore core;
      Connections {
      root -> core;
      }
      }
      Database Tier
      Database Tier
    20. Yahoo! Cloud: Extreme Automation
      Apps written against rich stacks
      Includes core infra and platforms
      • Infrastructure: Security, monitoring, metering, abuse prevention
      • Platforms: data, user info, ad systems, configuration bundles
      • All “auto configured”
    21. Serving Cloud Serving for Y!’s Cloud
      • Infrastructure to Metastructure
      • Machines to Containers
      • Friction to Innovation
      • Tears to Tiers
      • Complexity to Simplexity
      • Costs to Revenue Streams
    22. Cloud is a Promise, Service is the fulfillment!
    23. OPEN CLOUD ACCESS PROTOCOL
    24. OPEN CLOUD ACCESS PROTOCOL
      • Definition, Deployment, and Life Cycle Management of Cloud Resources
      • Allocation, Provisioning, and Metering of Cloud Resources
      • Metadata/Registry for Cloud Resources
      • Virtual Infrastructure Management
    25. Yahoo! Submitted OpenCAP to IETF
      • Why IETF?
      • Specification behind Yahoo Implementation
      • Plan to open source implementation
    26. Open Cloud Access Protocol
      Resource Model
      • All Infrastructure and Functional Components are modeled as Resources
      • Nodes, Networks, Storage Devices, Databases, Load Balancers, Identities, Key Value Store, Object Stores, Caching Services
      Resource Properties
      • Modeled as JSON objects
      • Standard Catalog of attributes, extensible
      Resource Operations
      • Operation, Control, and Notification
    27. Open Cloud Access Protocol
      Management Operation Services
      • CREATE - Create an instance of a managed resource
      • DELETE - Delete an instance of a managed resource
      • GET - Request managed resource properties (for one resource or a set of resource)
      • CANCEL - Cancel an outstanding request
      • SET - Set managed resource properties
      • ACTION - Request an action to be performed on a managed resource
    28. Open Cloud Access Protocol
      Management Notification Services
      • EVENT- Send events occurring on managed resources (VM events, VM stats etc, SNMP events etc)
      • ALERT- Send alerts for resource failures and exceptions
      Life Cycle Management Services
      • START- Starts the resource
      • TERMINATE – Stop the resource
      • SUSPEND – Suspend the resource
      • RESUME – Resume the resource
      • ARCHIVE – Archive the resource
    29. Yahoo! Open Source & Research
      Open Source
      • Cloud Serving Platform, Yahoo Implementation of OpenCAP
      • Traffic Server
      Research Activities
      • Open Cloud Consortium
      • OpenCirrus & M45
    30. THANK YOU
      Surendra Reddy
      @sureddy twitter
      sureddy@yahoo-inc.com email
      yCloudBlog.com blog
    31. TUESDAY, 11/3
      4:50pm – 5:35 pm
      Hadoop @ Yahoo! – Internet
      Scale Data Processing
      Eric Baldeschwieler
      VP, Hadoop Software Development
      WEDNESDAY, 11/4
      9:10 am - 9:55 am
      Yahoo! Scalable Storage and
      Delivery Services
      Chuck Neerdaels
      VP, Storage and Edge Services
      VISIT BOOTH #103
      TO TALK WITH YAHOO! ENGINEERS AND LEARN MORE ABOUT YAHOO!’S VISION FOR CLOUD COMPUTING.

    + Yahoo Developer NetworkYahoo Developer Network, 2 weeks ago

    custom

    774 views, 12 favs, 1 embeds more stats

    From Surendra Reddy's presentation "Walking Through more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 774
      • 773 on SlideShare
      • 1 from embeds
    • Comments 0
    • Favorites 12
    • Downloads 0
    Most viewed embeds
    • 1 views on http://static.slidesharecdn.com

    more

    All embeds
    • 1 views on http://static.slidesharecdn.com

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories