• Save
Walking Through Cloud Serving at Yahoo!
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Walking Through Cloud Serving at Yahoo!

on

  • 6,766 views

From Surendra Reddy's presentation "Walking Through Cloud Serving at Yahoo!" at the 2009 Cloud Computing Expo in Santa Clara, CA, USA. Here's the talk description on the Expo's site: ...

From Surendra Reddy's presentation "Walking Through Cloud Serving at Yahoo!" at the 2009 Cloud Computing Expo in Santa Clara, CA, USA. Here's the talk description on the Expo's site: http://cloudcomputingexpo.com/event/session/508

Statistics

Views

Total Views
6,766
Views on SlideShare
6,739
Embed Views
27

Actions

Likes
18
Downloads
0
Comments
0

3 Embeds 27

http://www.slideshare.net 25
http://static.slidesharecdn.com 1
https://my.jolicloud.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • _____We believe, Cloud computing is about Driving Innovation.At Yahoo!, we are developing and deploying Cloud Services to help us drive innovationBy providing a set of horizontal Cloud services that are broadly applicable to building many Yahoo! products We are able to reallocate many developers to building products and features from building product-centric infrastructure. We are providing a stable base of Cloud Services that enable product developers to iterate quickly and innovate without risking product quality We are providing the ability to process and analyze enormous amounts of data easily and quickly for purposes of improving consumer experience with our products.
  • Hundreds of properties/productsEnormous Scale 500M+ unique visitors monthly 300M+ Mail users 100’s of petabytes of storage 100’s of billions of objects stored Billions of daily requestsGlobal footprint

Walking Through Cloud Serving at Yahoo! Presentation Transcript

  • 1. Yahoo Cloud Serving
    Surendra Reddy
    Cloud Computing Conference & ExpoNovember 3, 2009
     
  • 2. Imagining the Future of Cloud!
  • 3. Cloud is a Promise, But …
  • 4. Reference:
    Tussle in Cyberspace: Defining Tomorrow’s Internet
    http://conferences.sigcomm.org/sigcomm/2002/papers/tussle.pdf
  • 5. Customer Tussles:
    • Outages: 76% outages caused by change management
    • 6. Unmet Needs: Rapidly changing needs
  • Management Tussles:
    • Growing Costs and Complexity
    • 7. Long Deployment Cycles: Inflexible, peak-provisioned infrastructure
  • Operations Tussles:
    • Entangled in many day-to-day failures & troubleshooting
    • 8. Crippled Infrastructure
    • 9. Building on weak foundation
  • Developers Tussles:
    • Bloated Operating System, App Stacks
    • 10. Staggering Complexities of Dependencies
    • 11. No time for QA – Bugs Creeping in all over
  • Security Tussles:
    • Whose problem is it?
    • 12. Auditability and Assurance
    • 13. Governance
    • 14. Compliance
  • 15. As a Result:
    • More Friction, Waste, and Frustration
    • 16. Less Innovation
  • So, What is Cloud?
    • C – Customer Driven
    • 17. L – Lean
    • 18. O – Optimized
    • 19. U – Utility
    • 20. D – Demand-Driven
    • Cloudis NOT about Cost cutting
    • 21. It is about Value Creation enabling Innovation
    • Cloud is an opportunity to clean up the Infrastructure mess!
  • 22. Yahoo! is Perfect for Cloud Computing
    600M
    UNIQUE USERS / MONTH
    300M+
    YAHOO! MAIL USERS / MONTH
    HUNDREDS
    OF PROPERTIES / PRODUCTS
    BILLIONS
    OF OBJECTS STORED
    HUNDREDS
    OF PETABYTES OF STORAGE
    PETABYTES
    OF TRAFFIC DAILY
  • 23.
    • Declarative language for specifying the full, end-to-end structure of a service
    • 24. Includes multiple environments: Dev, QA, Staging, and production
    • 25. Extreme Automation
  • Yahoo! Cloud Serving - Principles
    • Metastructure: Abstracts concerns of the underlying infrastructure and the network communication
    • 26. Declarative application structure
    • 27. Continuous Integration, Continuous Deployment, & Continuous Compliance*
    • 28. End-to-End Security
    * Rich Miller @RHM2K first coined “Continuous Compliance”
  • 29. Yahoo! Cloud Serving - Principles
    • Virtualized hardware
    • 30. Standardized software stacks and packaging
    • 31. Integrated service management
    • 32. Containers vs. Machines
  • Yahoo! Cloud Serving - Principles
    • Elastic serving of changing workloads
    • 33. Controlled/Intelligent traffic direction
    • 34. Controlled execution environment
    • 35. Managed Communication
    • 36. Service Associations, Bindings, and Access Controls
  • Yahoo! Cloud: Tiers without Tears
    Entrypoint
    Naming
    Presentation
    Tier
    Application Logic
    Tier
    Bindings
    Connections
    Database Tier
  • 37. Yahoo! Cloud: Tiers without Tears
    Entrypoint
    Naming
    Load
    Balancer
    Presentation
    Tier
    Application Logic
    Tier
    node
    node
    node
    Bindings
    Connections
    Database Tier
  • 38. Yahoo! Cloud: Extreme Automation
    Abstracting Services
    Component AnswersCore {
    Entry root;
    Tier pres {
    app = “answers_web_tier”;
    Exit applogic;
    nsize = small;
    nnodes = 3;
    };
    Tier applogic {
    app = “answers_al_tier”;
    Exit data;
    nsize = medium;
    nnodes = 3;
    };
    Tier data;
    Connections {
    root -> pres;
    pres.applogic -> applogic;
    applogic.data -> data;
    }
    }
    Presentation
    Tier
    Application Logic
    Tier
    Database Tier
  • 39. Yahoo! Cloud: Extreme Automation
    QA Environment
    Environment qa(ne1) {
    TrafficDirector root {
    name = “qa.answers.yahoo.com”;
    access = testing;
    };
    AnswersCore core;
    Connections {
    root -> core;
    }
    }
    Presentation
    Tier
    Presentation
    Tier
    Application Logic
    Tier
    Application Logic
    Tier
    Testing Testing
    Environment production(ac4,sk1) {
    TeeTrafficDirector root {
    name = “answers.yahoo.com”;
    access = public;
    split_percent = 10;
    split_destination = qa.root;
    };
    AnswersCore core;
    Connections {
    root -> core;
    }
    }
    Database Tier
    Database Tier
  • 40. Yahoo! Cloud: Extreme Automation
    Apps written against rich stacks
    Includes core infra and platforms
    • Infrastructure: Security, monitoring, metering, abuse prevention
    • 41. Platforms: data, user info, ad systems, configuration bundles
    • 42. All “auto configured”
  • Serving Cloud Serving for Y!’s Cloud
  • 43.
    • Infrastructure to Metastructure
    • 44. Machines to Containers
    • 45. Friction to Innovation
    • 46. Tears to Tiers
    • 47. Complexity to Simplexity
    • 48. Costs to Revenue Streams
  • Cloud is a Promise, Service is the fulfillment!
  • 49. OPEN CLOUD ACCESS PROTOCOL
  • 50. OPEN CLOUD ACCESS PROTOCOL
    • Definition, Deployment, and Life Cycle Management of Cloud Resources
    • 51. Allocation, Provisioning, and Metering of Cloud Resources
    • 52. Metadata/Registry for Cloud Resources
    • 53. Virtual Infrastructure Management
  • Yahoo! Submitted OpenCAP to IETF
    • Why IETF?
    • 54. Specification behind Yahoo Implementation
    • 55. Plan to open source implementation
  • Open Cloud Access Protocol
    Resource Model
    • All Infrastructure and Functional Components are modeled as Resources
    • 56. Nodes, Networks, Storage Devices, Databases, Load Balancers, Identities, Key Value Store, Object Stores, Caching Services
    Resource Properties
    • Modeled as JSON objects
    • 57. Standard Catalog of attributes, extensible
    Resource Operations
    • Operation, Control, and Notification
  • Open Cloud Access Protocol
    Management Operation Services
    • CREATE - Create an instance of a managed resource
    • 58. DELETE - Delete an instance of a managed resource
    • 59. GET - Request managed resource properties (for one resource or a set of resource)
    • 60. CANCEL - Cancel an outstanding request
    • 61. SET - Set managed resource properties
    • 62. ACTION - Request an action to be performed on a managed resource
  • Open Cloud Access Protocol
    Management Notification Services
    • EVENT- Send events occurring on managed resources (VM events, VM stats etc, SNMP events etc)
    • 63. ALERT- Send alerts for resource failures and exceptions
    Life Cycle Management Services
    • START- Starts the resource
    • 64. TERMINATE – Stop the resource
    • 65. SUSPEND – Suspend the resource
    • 66. RESUME – Resume the resource
    • 67. ARCHIVE – Archive the resource
  • Yahoo! Open Source & Research
    Open Source
    • Cloud Serving Platform, Yahoo Implementation of OpenCAP
    • 68. Traffic Server
    Research Activities
    • Open Cloud Consortium
    • 69. OpenCirrus & M45
  • THANK YOU
    Surendra Reddy
    @sureddy twitter
    sureddy@yahoo-inc.com email
    yCloudBlog.com blog
  • 70. TUESDAY, 11/3
    4:50pm – 5:35 pm
    Hadoop @ Yahoo! – Internet
    Scale Data Processing
    Eric Baldeschwieler
    VP, Hadoop Software Development
    WEDNESDAY, 11/4
    9:10 am - 9:55 am
    Yahoo! Scalable Storage and
    Delivery Services
    Chuck Neerdaels
    VP, Storage and Edge Services
    VISIT BOOTH #103
    TO TALK WITH YAHOO! ENGINEERS AND LEARN MORE ABOUT YAHOO!’S VISION FOR CLOUD COMPUTING.