Lighthouse:
Intercloud Metadata Service



                                   Rich Miller
                            Sure...
Agenda
•  Intercloud & Lighthouse Objectives

•  Use cases (as drivers & definition)

•  Lighthouse Requirements & Concept...
Intercloud & Objectives
Intercloud

Requires the dissemination &
exchange of operational metadata
- among clouds,
- between cloud services and
con...
Lighthouse
Lighthouse
  Where to start?
  •  Agreement on identification, location
     and ID-Loc resolution
  •  A registry for the...
Lighthouse
  The concept:
  •  Each member takes responsibility for
     its own metadata access services
  •  Membership ...
Lighthouse Scope

Scope is limited to providing the
Service Access Point and related
 metadata to service Consumers
Use Cases
Intercloud: Use Case #1
•  Customer A, EDA company, seeks a list of
   IaaS services which claim to provide:
       •  clo...
Intercloud: Use Case #2
•  Customer B, an insurance company, seeks a
   single IaaS provider to continuously satisfy
   se...
Intercloud: Use Case #3
•  Customer C, a globally distributed online
   service looking for an IaaS Providers in Europe
  ...
Intercloud: Use Case #4
•  Customer D, a financial services company,
   runs applications that are either (or both)
     •...
Intercloud: Use Case #5
•  PaaS E, a security broker service, provides an
   anti-phishing service for e-mail:
   “whiteli...
Lighthouse:
Requirements & Concepts
Lighthouse Requirements
•  Defines a dynamically extensible set of
   identifiers and metadata
•  Automatically aggregates...
Lighthouse Concept

Autonomous Metadata Access Point

  •  All interested and authenticated cloud
     services, acting in...
Lighthouse Concept

A Registry of Registries

  •  Identity and location of individually and
     autonomously managed
   ...
Lighthouse Concept

Process / Event Coordination

  •  All 'interested' consumers of a cloud’s
     MAP Service may subscr...
Lighthouse Concept

Share operational metadata

  •  Near Real-time

  •  Cloud Information Service
     +
     Cloud Oper...
Intercloud Registry: Features

•  Discovery of a registry’s specific
   interfaces / capabilities

•  Auditable logging me...
Intercloud Registry: Features

Forms of Search & Query
  •  search and report of items based on
     (…)
  •  comparison o...
Intercloud Registry: Operational
•  Distributed MAP Servers:
  Each Cloud Service is responsible for
  establishing and ad...
Available Standards
Current Standards/Protocols
Federated UDDI Registry
• Pros:
   •      Federated UDDI consisting of multiple repositories
 ...
Current Standards/Protocols
Service Location Protocol (SLP)
• Pros:
  •    agent based service discovery framework
  •    ...
Current Standards/Protocols
IF-MAP
• Pros:
  •    Client-Server based, real-time pub/sub/search
  •    Designed to dissemi...
Current Standards/Protocols
IF-MAP (continued)
•  Cons:
  •    SOAP based only, heavy messaging structure
  •    Scale for...
Current Standards/Protocols
Other Standards/Protocols to Consider

  •  WebDAV/DASL
    •  DAV Provides Versioned Metadata...
Current Standards/Protocols
  And, what about asynchronous
  messaging?
  •    AMQP
  •    Session Initiation Protocol (SI...
Lighthouse: Architectural
         Model
Lighthouse: Metadata Model
Lighthouse: Conceptual Architecture 1

    Cloud Service Provider


                                             CSP
     ...
Lighthouse: Conceptual Architecture 2

     Cloud Service Provider


                                                   CS...
Lighthouse: Call(s) to Action




                                   Rich Miller
                            Surendra Redd...
Lighthouse 20100120
Upcoming SlideShare
Loading in …5
×

Lighthouse 20100120

956 views

Published on

Rich Miller & Surendra Reddy
Lighthouse is a concept for the creation of an intercloud registry service, based on (1) access points established and maintained by cloud instances to disseminate operational metadata; and, (2) the use of publish/subscribe (pub/sub) asynchronous messaging as the dominant means of disseminating operational metadata among the constituents of the intercloud.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
956
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Rich, add some talking points why did you choose Lighthouse? Helping us to get to the shore 
  • 12/27/09 RHM: modify use case to include not just finding the directory / registry, but establishes requirements / criteria
  • 12/27/09 RHM: I’m not sure I understand the reference here to “Customer B.” Are you saying that Customer B (from previous page) is using Customer C’s services? Or, is this a typo and you’re referring to Customer C? I assume it’s the latter, and that you were referring to Customer C.
  • Customer’s application controller helps them to negotiate the resources from various cloud service providers using the Intercloud registry as a matchmaking service utilizing Searching, Location services offered by the Intercloud registry. NOTE: the difference between earlier and this use case, the timely meta-data delivery and consistency of mata-data are two critical needs for this use case.
  • Customer’s application controller helps them to negotiate the resources from various cloud service providers using the Intercloud registry as a matchmaking service utilizing Searching, Location services offered by the Intercloud registry. NOTE: the difference between earlier and this use case, the timely meta-data delivery and consistency of mata-data are two critical needs for this use case.
  • Marketplace for Cloud Services offers matchmaking of service providers based on competitive pricing, SLA, and Location preferences. 12/27/09 RHM: I’m not sure I would include the Marketplace as part of Lighthouse. The idea of marketplaces, brokers of various kinds, intercloud ‘core services’ … any of the service offerings that require a ‘trusted third party’ … require the existence of the Lighthouse underpinnings: Autonomously managed metadata access services a registry for the discovery of metadata access services, identification – location resolution a common set of mechanisms and protocols for messaging and pub/sub/search/query
  • Lighthouse 20100120

    1. 1. Lighthouse: Intercloud Metadata Service Rich Miller Surendra Reddy Infrastructure 2.0 Working Group January 20, 2010
    2. 2. Agenda •  Intercloud & Lighthouse Objectives •  Use cases (as drivers & definition) •  Lighthouse Requirements & Concepts •  Available technologies & standards •  Architectural Guiding Principles •  Call(s) to action
    3. 3. Intercloud & Objectives
    4. 4. Intercloud Requires the dissemination & exchange of operational metadata - among clouds, - between cloud services and consumers of their services.
    5. 5. Lighthouse
    6. 6. Lighthouse Where to start? •  Agreement on identification, location and ID-Loc resolution •  A registry for the discovery and description of intercloud constituents •  A mechanism for the delivery of cloud service descriptive & operational data •  A governance structure for admission & ejection, assurance, permissions & entitlements
    7. 7. Lighthouse The concept: •  Each member takes responsibility for its own metadata access services •  Membership in a communal registry of metadata access services, with identification – location resolution •  Agreement on mechanisms for - pub/sub/search/query - asynchronous message delivery
    8. 8. Lighthouse Scope Scope is limited to providing the Service Access Point and related metadata to service Consumers
    9. 9. Use Cases
    10. 10. Intercloud: Use Case #1 •  Customer A, EDA company, seeks a list of IaaS services which claim to provide: •  cloud data management •  Linux OS image management •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS service metadata to make selection •  Access the Service Access Point (SAP) of a vendor to validate claims •  Subscribes to Service Access Point for receipt of service announcements, rate changes, etc
    11. 11. Intercloud: Use Case #2 •  Customer B, an insurance company, seeks a single IaaS provider to continuously satisfy service requirements (constraints) •  E.g. latencies, geography, SLAs etc. •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS metadata to make selection •  Access the SAPs of vendor to create Cloud Service Account Instance •  Subscribes to SAP for receipt of relevant requirement-specific metadata •  Takes specific actions based on timely notifications (near realtime alerts) via Service Provider APIs and management functions
    12. 12. Intercloud: Use Case #3 •  Customer C, a globally distributed online service looking for an IaaS Providers in Europe and in USA with specific SLAs. •  Using the Intercloud registry, locates services meeting needs in two locations. •  Identifies alternative providers for the business continuity (DR, backup, …) functions. •  Customer C’s application management system subscribes to failure events & performance sensors from the IaaS providers. •  Based monitored event/sensor feeds, C’s service monitoring application dynamically scales up/down the resources (computing, networking, and storage) to their applications
    13. 13. Intercloud: Use Case #4 •  Customer D, a financial services company, runs applications that are either (or both) •  latency sensitive •  throughput sensitive •  After selecting IaaS provider: •  Sets up the virtual network between on-premise data center and the IaaS provider cloud. •  Customer D runs their own application mobility controller within their data center. •  Application Mobility Controller subscribes to IaaS and data center metadata related to: •  traffic flows, performance metrics •  log feeds from the IaaS cloud service.
    14. 14. Intercloud: Use Case #5 •  PaaS E, a security broker service, provides an anti-phishing service for e-mail: “whitelist”, analytics and forensics •  Operates on behalf of domain holders •  A list management and forensics for multiple receiver services (e.g. web mail services) •  After establishing service w/ receiver: •  Each receiver establishes a metadata access point (MAP) regarding failed email •  PaaS E publishes notifications of phishing attempts to subs, on behalf of domain holder •  All new events and changes in state/status distributed as pub/sub metadata
    15. 15. Lighthouse: Requirements & Concepts
    16. 16. Lighthouse Requirements •  Defines a dynamically extensible set of identifiers and metadata •  Automatically aggregates and associates real-time info from many different sources •  Provides real-time pub/sub/search mechanism for data regarding cloud instances, their state and their activities •  Scales for cloud to cloud coordination
    17. 17. Lighthouse Concept Autonomous Metadata Access Point •  All interested and authenticated cloud services, acting in ‘good faith’, provide their own Metadata Access Point. •  A Metadata Access Point publishes to the intercloud community any information about itself.
    18. 18. Lighthouse Concept A Registry of Registries •  Identity and location of individually and autonomously managed Metadata Access Services •  Authoritatively establishes the status of any individual cloud service and its standing within the Intercloud community
    19. 19. Lighthouse Concept Process / Event Coordination •  All 'interested' consumers of a cloud’s MAP Service may subscribe to metadata updates that result in a 'property' change •  Many systems can coordinate through a Metadata Access Protocol with no in- depth knowledge of each other's APIs
    20. 20. Lighthouse Concept Share operational metadata •  Near Real-time •  Cloud Information Service + Cloud Operations Coordination
    21. 21. Intercloud Registry: Features •  Discovery of a registry’s specific interfaces / capabilities •  Auditable logging mechanism •  For element / value changes •  For publishing event
    22. 22. Intercloud Registry: Features Forms of Search & Query •  search and report of items based on (…) •  comparison of object to ‘checklist’ of elements and parameters •  ‘standing’ search/query established as subscription •  query and retrieval of items based on published / recognized (?) data scheme
    23. 23. Intercloud Registry: Operational •  Distributed MAP Servers: Each Cloud Service is responsible for establishing and administering •  its own Registry Server, or •  publication of metadata by a trusted party •  Authoritative compilation of Registries (and, therefore, of Cloud Services) •  Unambiguous identification •  Authentication method associated with ID
    24. 24. Available Standards
    25. 25. Current Standards/Protocols Federated UDDI Registry • Pros: •  Federated UDDI consisting of multiple repositories that are synchronized periodically. •  Federated UDDI is an efficient solution for service discovery in distributed service networks. • Cons: •  too expensive to replicate frequently updated information •  it is hard to directly utilize this approach to support discovery of dynamic information •  Governance nightmare…
    26. 26. Current Standards/Protocols Service Location Protocol (SLP) • Pros: •  agent based service discovery framework •  designed for service discovery in for local area network •  extensions to SLP proposed aiming to the WAN environment • Cons: •  Not suitable for wide area network environment •  unsuitable for Cloud environment due to the scale and distribution complexities involved.
    27. 27. Current Standards/Protocols IF-MAP • Pros: •  Client-Server based, real-time pub/sub/search •  Designed to disseminate network security info on objects & events (dynamic state and activity data) •  Easily extensible to components other than network and security components •  XML-based SOAP protocol •  Supports standardized, dynamic data interchange •  Provides an uniform mechanism to securely discover, consume, and manage a single management domain’s metadata.
    28. 28. Current Standards/Protocols IF-MAP (continued) •  Cons: •  SOAP based only, heavy messaging structure •  Scale for Cloud •  Need lot of extensions to existing metadata model •  IF-MAP access point becomes a central authority •  TBD •  Federation to support intercloud scale? •  Wider range of protocols / RESTful interface? •  A MAP-to-MAP (P2P) approach to bi-directional pub/sub? •  Asynch messaging queues? •  “Economical” message encoding system ? hierarchical, binary, self-describing
    29. 29. Current Standards/Protocols Other Standards/Protocols to Consider •  WebDAV/DASL •  DAV Provides Versioned Metadata Access of Resources •  DASL: Provides Searching and Location
    30. 30. Current Standards/Protocols And, what about asynchronous messaging? •  AMQP •  Session Initiation Protocol (SIP) •  XMPP •  HTTP •  JMS Not to mention message encoding… •  ASN.1 •  FUDGE •  …
    31. 31. Lighthouse: Architectural Model
    32. 32. Lighthouse: Metadata Model
    33. 33. Lighthouse: Conceptual Architecture 1 Cloud Service Provider CSP CSP CSP MAP Metadata Access Point IC- MAP " InterCloud Registry IC Registry Metadata Access Point
    34. 34. Lighthouse: Conceptual Architecture 2 Cloud Service Provider CSP CSP CSP IC MAP InterCloud Registry " IC IC- ROOT Metadata Access Point IC Registry Metadata “Root Server”
    35. 35. Lighthouse: Call(s) to Action Rich Miller Surendra Reddy Infrastructure 2.0 Working Group

    ×