Lighthouse 20100120

Lighthouse: Intercloud Metadata Service Rich Miller Surendra Reddy Infrastructure 2.0 Working Group January 20, 2010

Agenda •  Intercloud & Lighthouse Objectives •  Use cases (as drivers & definition) •  Lighthouse Requirements & Concepts •  Available technologies & standards •  Architectural Guiding Principles •  Call(s) to action

Intercloud & Objectives

Intercloud Requires the dissemination & exchange of operational metadata - among clouds, - between cloud services and consumers of their services.

Lighthouse

Lighthouse Where to start? •  Agreement on identification, location and ID-Loc resolution •  A registry for the discovery and description of intercloud constituents •  A mechanism for the delivery of cloud service descriptive & operational data •  A governance structure for admission & ejection, assurance, permissions & entitlements

Lighthouse The concept: •  Each member takes responsibility for its own metadata access services •  Membership in a communal registry of metadata access services, with identification – location resolution •  Agreement on mechanisms for - pub/sub/search/query - asynchronous message delivery

Lighthouse Scope Scope is limited to providing the Service Access Point and related metadata to service Consumers

Use Cases

Intercloud: Use Case #1 •  Customer A, EDA company, seeks a list of IaaS services which claim to provide: •  cloud data management •  Linux OS image management •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS service metadata to make selection •  Access the Service Access Point (SAP) of a vendor to validate claims •  Subscribes to Service Access Point for receipt of service announcements, rate changes, etc

Intercloud: Use Case #2 •  Customer B, an insurance company, seeks a single IaaS provider to continuously satisfy service requirements (constraints) •  E.g. latencies, geography, SLAs etc. •  Queries the Intercloud registry, returns IDs of services that meet criteria •  Searches IaaS metadata to make selection •  Access the SAPs of vendor to create Cloud Service Account Instance •  Subscribes to SAP for receipt of relevant requirement-specific metadata •  Takes specific actions based on timely notifications (near realtime alerts) via Service Provider APIs and management functions

Intercloud: Use Case #3 •  Customer C, a globally distributed online service looking for an IaaS Providers in Europe and in USA with specific SLAs. •  Using the Intercloud registry, locates services meeting needs in two locations. •  Identifies alternative providers for the business continuity (DR, backup, …) functions. •  Customer C’s application management system subscribes to failure events & performance sensors from the IaaS providers. •  Based monitored event/sensor feeds, C’s service monitoring application dynamically scales up/down the resources (computing, networking, and storage) to their applications

Intercloud: Use Case #4 •  Customer D, a financial services company, runs applications that are either (or both) •  latency sensitive •  throughput sensitive •  After selecting IaaS provider: •  Sets up the virtual network between on-premise data center and the IaaS provider cloud. •  Customer D runs their own application mobility controller within their data center. •  Application Mobility Controller subscribes to IaaS and data center metadata related to: •  traffic flows, performance metrics •  log feeds from the IaaS cloud service.

Intercloud: Use Case #5 •  PaaS E, a security broker service, provides an anti-phishing service for e-mail: “whitelist”, analytics and forensics •  Operates on behalf of domain holders •  A list management and forensics for multiple receiver services (e.g. web mail services) •  After establishing service w/ receiver: •  Each receiver establishes a metadata access point (MAP) regarding failed email •  PaaS E publishes notifications of phishing attempts to subs, on behalf of domain holder •  All new events and changes in state/status distributed as pub/sub metadata

Lighthouse: Requirements & Concepts

Lighthouse Requirements •  Defines a dynamically extensible set of identifiers and metadata •  Automatically aggregates and associates real-time info from many different sources •  Provides real-time pub/sub/search mechanism for data regarding cloud instances, their state and their activities •  Scales for cloud to cloud coordination

Lighthouse Concept Autonomous Metadata Access Point •  All interested and authenticated cloud services, acting in ‘good faith’, provide their own Metadata Access Point. •  A Metadata Access Point publishes to the intercloud community any information about itself.

Lighthouse Concept A Registry of Registries •  Identity and location of individually and autonomously managed Metadata Access Services •  Authoritatively establishes the status of any individual cloud service and its standing within the Intercloud community

Lighthouse Concept Process / Event Coordination •  All 'interested' consumers of a cloud’s MAP Service may subscribe to metadata updates that result in a 'property' change •  Many systems can coordinate through a Metadata Access Protocol with no in- depth knowledge of each other's APIs

Lighthouse Concept Share operational metadata •  Near Real-time •  Cloud Information Service + Cloud Operations Coordination

Intercloud Registry: Features •  Discovery of a registry’s specific interfaces / capabilities •  Auditable logging mechanism •  For element / value changes •  For publishing event

Intercloud Registry: Features Forms of Search & Query •  search and report of items based on (…) •  comparison of object to ‘checklist’ of elements and parameters •  ‘standing’ search/query established as subscription •  query and retrieval of items based on published / recognized (?) data scheme

Intercloud Registry: Operational •  Distributed MAP Servers: Each Cloud Service is responsible for establishing and administering •  its own Registry Server, or •  publication of metadata by a trusted party •  Authoritative compilation of Registries (and, therefore, of Cloud Services) •  Unambiguous identification •  Authentication method associated with ID

Available Standards

Current Standards/Protocols Federated UDDI Registry • Pros: •  Federated UDDI consisting of multiple repositories that are synchronized periodically. •  Federated UDDI is an efficient solution for service discovery in distributed service networks. • Cons: •  too expensive to replicate frequently updated information •  it is hard to directly utilize this approach to support discovery of dynamic information •  Governance nightmare…

Current Standards/Protocols Service Location Protocol (SLP) • Pros: •  agent based service discovery framework •  designed for service discovery in for local area network •  extensions to SLP proposed aiming to the WAN environment • Cons: •  Not suitable for wide area network environment •  unsuitable for Cloud environment due to the scale and distribution complexities involved.

Current Standards/Protocols IF-MAP • Pros: •  Client-Server based, real-time pub/sub/search •  Designed to disseminate network security info on objects & events (dynamic state and activity data) •  Easily extensible to components other than network and security components •  XML-based SOAP protocol •  Supports standardized, dynamic data interchange •  Provides an uniform mechanism to securely discover, consume, and manage a single management domain’s metadata.

Current Standards/Protocols IF-MAP (continued) •  Cons: •  SOAP based only, heavy messaging structure •  Scale for Cloud •  Need lot of extensions to existing metadata model •  IF-MAP access point becomes a central authority •  TBD •  Federation to support intercloud scale? •  Wider range of protocols / RESTful interface? •  A MAP-to-MAP (P2P) approach to bi-directional pub/sub? •  Asynch messaging queues? •  “Economical” message encoding system ? hierarchical, binary, self-describing

Current Standards/Protocols Other Standards/Protocols to Consider •  WebDAV/DASL •  DAV Provides Versioned Metadata Access of Resources •  DASL: Provides Searching and Location

Current Standards/Protocols And, what about asynchronous messaging? •  AMQP •  Session Initiation Protocol (SIP) •  XMPP •  HTTP •  JMS Not to mention message encoding… •  ASN.1 •  FUDGE •  …

Lighthouse: Architectural Model

Lighthouse: Metadata Model

Lighthouse: Conceptual Architecture 1 Cloud Service Provider CSP CSP CSP MAP Metadata Access Point IC- MAP " InterCloud Registry IC Registry Metadata Access Point

Lighthouse: Conceptual Architecture 2 Cloud Service Provider CSP CSP CSP IC MAP InterCloud Registry " IC IC- ROOT Metadata Access Point IC Registry Metadata “Root Server”

Lighthouse: Call(s) to Action Rich Miller Surendra Reddy Infrastructure 2.0 Working Group

Lighthouse 20100120

by Infrastructure 2.0 on Jan 20, 2010

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Lighthouse 20100120 Presentation Transcript