Your SlideShare is downloading. ×
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply



Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Lighthouse: Intercloud Metadata Service Rich Miller Surendra Reddy Infrastructure 2.0 Working Group January 20, 2010
  • 2. Agenda •  Intercloud & Lighthouse Objectives •  Use cases (as drivers & definition) •  Lighthouse Requirements & Concepts •  Available technologies & standards •  Architectural Guiding Principles •  Call(s) to action
  • 3. Intercloud & Objectives
  • 4. Intercloud Requires the dissemination & exchange of operational metadata - among clouds, - between cloud services and consumers of their services.
  • 5. Lighthouse
  • 6. Lighthouse Where to start? •  Agreement on identification, location and ID-Loc resolution •  A registry for the discovery and description of intercloud constituents •  A mechanism for the delivery of cloud service descriptive & operational data •  A governance structure for admission & ejection, assurance, permissions & entitlements
  • 7. Lighthouse The concept: •  Each member takes responsibility for its own metadata access services •  Membership in a communal registry of metadata access services, with identification – location resolution •  Agreement on mechanisms for - pub/sub/search/query - asynchronous message delivery
  • 8. Lighthouse Scope Scope is limited to providing the Service Access Point and related metadata to service Consumers
  • 9. Use Cases
  • 10. Intercloud: Use Case #1 •  Customer A, EDA company, seeks a list of IaaS services which claim to provide: •  cloud data management •  Linux OS image management •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS service metadata to make selection •  Access the Service Access Point (SAP) of a vendor to validate claims •  Subscribes to Service Access Point for receipt of service announcements, rate changes, etc
  • 11. Intercloud: Use Case #2 •  Customer B, an insurance company, seeks a single IaaS provider to continuously satisfy service requirements (constraints) •  E.g. latencies, geography, SLAs etc. •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS metadata to make selection •  Access the SAPs of vendor to create Cloud Service Account Instance •  Subscribes to SAP for receipt of relevant requirement-specific metadata •  Takes specific actions based on timely notifications (near realtime alerts) via Service Provider APIs and management functions
  • 12. Intercloud: Use Case #3 •  Customer C, a globally distributed online service looking for an IaaS Providers in Europe and in USA with specific SLAs. •  Using the Intercloud registry, locates services meeting needs in two locations. •  Identifies alternative providers for the business continuity (DR, backup, …) functions. •  Customer C’s application management system subscribes to failure events & performance sensors from the IaaS providers. •  Based monitored event/sensor feeds, C’s service monitoring application dynamically scales up/down the resources (computing, networking, and storage) to their applications
  • 13. Intercloud: Use Case #4 •  Customer D, a financial services company, runs applications that are either (or both) •  latency sensitive •  throughput sensitive •  After selecting IaaS provider: •  Sets up the virtual network between on-premise data center and the IaaS provider cloud. •  Customer D runs their own application mobility controller within their data center. •  Application Mobility Controller subscribes to IaaS and data center metadata related to: •  traffic flows, performance metrics •  log feeds from the IaaS cloud service.
  • 14. Intercloud: Use Case #5 •  PaaS E, a security broker service, provides an anti-phishing service for e-mail: “whitelist”, analytics and forensics •  Operates on behalf of domain holders •  A list management and forensics for multiple receiver services (e.g. web mail services) •  After establishing service w/ receiver: •  Each receiver establishes a metadata access point (MAP) regarding failed email •  PaaS E publishes notifications of phishing attempts to subs, on behalf of domain holder •  All new events and changes in state/status distributed as pub/sub metadata
  • 15. Lighthouse: Requirements & Concepts
  • 16. Lighthouse Requirements •  Defines a dynamically extensible set of identifiers and metadata •  Automatically aggregates and associates real-time info from many different sources •  Provides real-time pub/sub/search mechanism for data regarding cloud instances, their state and their activities •  Scales for cloud to cloud coordination
  • 17. Lighthouse Concept Autonomous Metadata Access Point •  All interested and authenticated cloud services, acting in ‘good faith’, provide their own Metadata Access Point. •  A Metadata Access Point publishes to the intercloud community any information about itself.
  • 18. Lighthouse Concept A Registry of Registries •  Identity and location of individually and autonomously managed Metadata Access Services •  Authoritatively establishes the status of any individual cloud service and its standing within the Intercloud community
  • 19. Lighthouse Concept Process / Event Coordination •  All 'interested' consumers of a cloud’s MAP Service may subscribe to metadata updates that result in a 'property' change •  Many systems can coordinate through a Metadata Access Protocol with no in- depth knowledge of each other's APIs
  • 20. Lighthouse Concept Share operational metadata •  Near Real-time •  Cloud Information Service + Cloud Operations Coordination
  • 21. Intercloud Registry: Features •  Discovery of a registry’s specific interfaces / capabilities •  Auditable logging mechanism •  For element / value changes •  For publishing event
  • 22. Intercloud Registry: Features Forms of Search & Query •  search and report of items based on (…) •  comparison of object to ‘checklist’ of elements and parameters •  ‘standing’ search/query established as subscription •  query and retrieval of items based on published / recognized (?) data scheme
  • 23. Intercloud Registry: Operational •  Distributed MAP Servers: Each Cloud Service is responsible for establishing and administering •  its own Registry Server, or •  publication of metadata by a trusted party •  Authoritative compilation of Registries (and, therefore, of Cloud Services) •  Unambiguous identification •  Authentication method associated with ID
  • 24. Available Standards
  • 25. Current Standards/Protocols Federated UDDI Registry • Pros: •  Federated UDDI consisting of multiple repositories that are synchronized periodically. •  Federated UDDI is an efficient solution for service discovery in distributed service networks. • Cons: •  too expensive to replicate frequently updated information •  it is hard to directly utilize this approach to support discovery of dynamic information •  Governance nightmare…
  • 26. Current Standards/Protocols Service Location Protocol (SLP) • Pros: •  agent based service discovery framework •  designed for service discovery in for local area network •  extensions to SLP proposed aiming to the WAN environment • Cons: •  Not suitable for wide area network environment •  unsuitable for Cloud environment due to the scale and distribution complexities involved.
  • 27. Current Standards/Protocols IF-MAP • Pros: •  Client-Server based, real-time pub/sub/search •  Designed to disseminate network security info on objects & events (dynamic state and activity data) •  Easily extensible to components other than network and security components •  XML-based SOAP protocol •  Supports standardized, dynamic data interchange •  Provides an uniform mechanism to securely discover, consume, and manage a single management domain’s metadata.
  • 28. Current Standards/Protocols IF-MAP (continued) •  Cons: •  SOAP based only, heavy messaging structure •  Scale for Cloud •  Need lot of extensions to existing metadata model •  IF-MAP access point becomes a central authority •  TBD •  Federation to support intercloud scale? •  Wider range of protocols / RESTful interface? •  A MAP-to-MAP (P2P) approach to bi-directional pub/sub? •  Asynch messaging queues? •  “Economical” message encoding system ? hierarchical, binary, self-describing
  • 29. Current Standards/Protocols Other Standards/Protocols to Consider •  WebDAV/DASL •  DAV Provides Versioned Metadata Access of Resources •  DASL: Provides Searching and Location
  • 30. Current Standards/Protocols And, what about asynchronous messaging? •  AMQP •  Session Initiation Protocol (SIP) •  XMPP •  HTTP •  JMS Not to mention message encoding… •  ASN.1 •  FUDGE •  …
  • 31. Lighthouse: Architectural Model
  • 32. Lighthouse: Metadata Model
  • 33. Lighthouse: Conceptual Architecture 1 Cloud Service Provider CSP CSP CSP MAP Metadata Access Point IC- MAP " InterCloud Registry IC Registry Metadata Access Point
  • 34. Lighthouse: Conceptual Architecture 2 Cloud Service Provider CSP CSP CSP IC MAP InterCloud Registry " IC IC- ROOT Metadata Access Point IC Registry Metadata “Root Server”
  • 35. Lighthouse: Call(s) to Action Rich Miller Surendra Reddy Infrastructure 2.0 Working Group