Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Lighthouse: Intercloud Metadata Service Rich Miller Surendra Reddy Infrastructure 2.0 Working Group January 20, 2010
  2. 2. Agenda •  Intercloud & Lighthouse Objectives •  Use cases (as drivers & definition) •  Lighthouse Requirements & Concepts •  Available technologies & standards •  Architectural Guiding Principles •  Call(s) to action
  3. 3. Intercloud & Objectives
  4. 4. Intercloud Requires the dissemination & exchange of operational metadata - among clouds, - between cloud services and consumers of their services.
  5. 5. Lighthouse
  6. 6. Lighthouse Where to start? •  Agreement on identification, location and ID-Loc resolution •  A registry for the discovery and description of intercloud constituents •  A mechanism for the delivery of cloud service descriptive & operational data •  A governance structure for admission & ejection, assurance, permissions & entitlements
  7. 7. Lighthouse The concept: •  Each member takes responsibility for its own metadata access services •  Membership in a communal registry of metadata access services, with identification – location resolution •  Agreement on mechanisms for - pub/sub/search/query - asynchronous message delivery
  8. 8. Lighthouse Scope Scope is limited to providing the Service Access Point and related metadata to service Consumers
  9. 9. Use Cases
  10. 10. Intercloud: Use Case #1 •  Customer A, EDA company, seeks a list of IaaS services which claim to provide: •  cloud data management •  Linux OS image management •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS service metadata to make selection •  Access the Service Access Point (SAP) of a vendor to validate claims •  Subscribes to Service Access Point for receipt of service announcements, rate changes, etc
  11. 11. Intercloud: Use Case #2 •  Customer B, an insurance company, seeks a single IaaS provider to continuously satisfy service requirements (constraints) •  E.g. latencies, geography, SLAs etc. •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS metadata to make selection •  Access the SAPs of vendor to create Cloud Service Account Instance •  Subscribes to SAP for receipt of relevant requirement-specific metadata •  Takes specific actions based on timely notifications (near realtime alerts) via Service Provider APIs and management functions
  12. 12. Intercloud: Use Case #3 •  Customer C, a globally distributed online service looking for an IaaS Providers in Europe and in USA with specific SLAs. •  Using the Intercloud registry, locates services meeting needs in two locations. •  Identifies alternative providers for the business continuity (DR, backup, …) functions. •  Customer C’s application management system subscribes to failure events & performance sensors from the IaaS providers. •  Based monitored event/sensor feeds, C’s service monitoring application dynamically scales up/down the resources (computing, networking, and storage) to their applications
  13. 13. Intercloud: Use Case #4 •  Customer D, a financial services company, runs applications that are either (or both) •  latency sensitive •  throughput sensitive •  After selecting IaaS provider: •  Sets up the virtual network between on-premise data center and the IaaS provider cloud. •  Customer D runs their own application mobility controller within their data center. •  Application Mobility Controller subscribes to IaaS and data center metadata related to: •  traffic flows, performance metrics •  log feeds from the IaaS cloud service.
  14. 14. Intercloud: Use Case #5 •  PaaS E, a security broker service, provides an anti-phishing service for e-mail: “whitelist”, analytics and forensics •  Operates on behalf of domain holders •  A list management and forensics for multiple receiver services (e.g. web mail services) •  After establishing service w/ receiver: •  Each receiver establishes a metadata access point (MAP) regarding failed email •  PaaS E publishes notifications of phishing attempts to subs, on behalf of domain holder •  All new events and changes in state/status distributed as pub/sub metadata
  15. 15. Lighthouse: Requirements & Concepts
  16. 16. Lighthouse Requirements •  Defines a dynamically extensible set of identifiers and metadata •  Automatically aggregates and associates real-time info from many different sources •  Provides real-time pub/sub/search mechanism for data regarding cloud instances, their state and their activities •  Scales for cloud to cloud coordination
  17. 17. Lighthouse Concept Autonomous Metadata Access Point •  All interested and authenticated cloud services, acting in ‘good faith’, provide their own Metadata Access Point. •  A Metadata Access Point publishes to the intercloud community any information about itself.
  18. 18. Lighthouse Concept A Registry of Registries •  Identity and location of individually and autonomously managed Metadata Access Services •  Authoritatively establishes the status of any individual cloud service and its standing within the Intercloud community
  19. 19. Lighthouse Concept Process / Event Coordination •  All 'interested' consumers of a cloud’s MAP Service may subscribe to metadata updates that result in a 'property' change •  Many systems can coordinate through a Metadata Access Protocol with no in- depth knowledge of each other's APIs
  20. 20. Lighthouse Concept Share operational metadata •  Near Real-time •  Cloud Information Service + Cloud Operations Coordination
  21. 21. Intercloud Registry: Features •  Discovery of a registry’s specific interfaces / capabilities •  Auditable logging mechanism •  For element / value changes •  For publishing event
  22. 22. Intercloud Registry: Features Forms of Search & Query •  search and report of items based on (…) •  comparison of object to ‘checklist’ of elements and parameters •  ‘standing’ search/query established as subscription •  query and retrieval of items based on published / recognized (?) data scheme
  23. 23. Intercloud Registry: Operational •  Distributed MAP Servers: Each Cloud Service is responsible for establishing and administering •  its own Registry Server, or •  publication of metadata by a trusted party •  Authoritative compilation of Registries (and, therefore, of Cloud Services) •  Unambiguous identification •  Authentication method associated with ID
  24. 24. Available Standards
  25. 25. Current Standards/Protocols Federated UDDI Registry • Pros: •  Federated UDDI consisting of multiple repositories that are synchronized periodically. •  Federated UDDI is an efficient solution for service discovery in distributed service networks. • Cons: •  too expensive to replicate frequently updated information •  it is hard to directly utilize this approach to support discovery of dynamic information •  Governance nightmare…
  26. 26. Current Standards/Protocols Service Location Protocol (SLP) • Pros: •  agent based service discovery framework •  designed for service discovery in for local area network •  extensions to SLP proposed aiming to the WAN environment • Cons: •  Not suitable for wide area network environment •  unsuitable for Cloud environment due to the scale and distribution complexities involved.
  27. 27. Current Standards/Protocols IF-MAP • Pros: •  Client-Server based, real-time pub/sub/search •  Designed to disseminate network security info on objects & events (dynamic state and activity data) •  Easily extensible to components other than network and security components •  XML-based SOAP protocol •  Supports standardized, dynamic data interchange •  Provides an uniform mechanism to securely discover, consume, and manage a single management domain’s metadata.
  28. 28. Current Standards/Protocols IF-MAP (continued) •  Cons: •  SOAP based only, heavy messaging structure •  Scale for Cloud •  Need lot of extensions to existing metadata model •  IF-MAP access point becomes a central authority •  TBD •  Federation to support intercloud scale? •  Wider range of protocols / RESTful interface? •  A MAP-to-MAP (P2P) approach to bi-directional pub/sub? •  Asynch messaging queues? •  “Economical” message encoding system ? hierarchical, binary, self-describing
  29. 29. Current Standards/Protocols Other Standards/Protocols to Consider •  WebDAV/DASL •  DAV Provides Versioned Metadata Access of Resources •  DASL: Provides Searching and Location
  30. 30. Current Standards/Protocols And, what about asynchronous messaging? •  AMQP •  Session Initiation Protocol (SIP) •  XMPP •  HTTP •  JMS Not to mention message encoding… •  ASN.1 •  FUDGE •  …
  31. 31. Lighthouse: Architectural Model
  32. 32. Lighthouse: Metadata Model
  33. 33. Lighthouse: Conceptual Architecture 1 Cloud Service Provider CSP CSP CSP MAP Metadata Access Point IC- MAP " InterCloud Registry IC Registry Metadata Access Point
  34. 34. Lighthouse: Conceptual Architecture 2 Cloud Service Provider CSP CSP CSP IC MAP InterCloud Registry " IC IC- ROOT Metadata Access Point IC Registry Metadata “Root Server”
  35. 35. Lighthouse: Call(s) to Action Rich Miller Surendra Reddy Infrastructure 2.0 Working Group