0
perfSONAR Architecture <ul><li>February 11 th  2010, APAN 29 – perfSONAR Workshop </li></ul><ul><li>Jeff Boote, Senior Net...
<ul><li>Motivation and Goals </li></ul><ul><li>Service Oriented Architecture (SOA) </li></ul><ul><li>Architecture Overview...
<ul><li>Thus far we have seen that the  perfSONAR  framework can be used to solve end-to-end performance problems for mult...
<ul><li>Interoperable network measurement middleware (SOA): </li></ul><ul><ul><li>Modular </li></ul></ul><ul><ul><li>Web s...
<ul><li>Measurement Point (MP) Service </li></ul><ul><ul><li>Enables the initiation of performance tests </li></ul></ul><u...
<ul><li>Lookup Service </li></ul><ul><ul><li>Allows the client to discover the existing services and other LS services. </...
Where is link utilization for - IPs d,e,f? Client Network A Network B LS A LS B MA A MA B a b c d e f Where is link utiliz...
 –  05/06/10 , © 2009 Internet2 Architecture Overview
 –  05/06/10 , © 2009 Internet2 perfSONAR Architecture Overview
<ul><li>Measurement Point (MP) form the  lowest layer  of the monitoring infrastructure </li></ul><ul><ul><li>Directly int...
<ul><li>Measurement Archive (MA) stores the results of network and performance measurements </li></ul><ul><ul><li>WS inter...
<ul><li>The Transformation Service (TrS) performs operations on data sets (e.g. aggregation, correlation).  </li></ul><ul>...
<ul><li>The Resource Protector (RP) monitors the relative performance and availability of the monitoring infrastructure  <...
<ul><li>The Lookup Service (LS) is a general name for the service and and data discovery infrastructure </li></ul><ul><ul>...
<ul><li>The Home Lookup Service (hLS) interacts directly with the other portions of the perfSONAR framework </li></ul><ul>...
<ul><li>The Global Lookup Service (gLS) serves as the  oracle  of the perfSONAR framework </li></ul><ul><ul><li>Global  cl...
 –  05/06/10 , © 2009 Internet2 Architecture Overview – Lookup Service
<ul><li>Communities = Web 2.0 Content Tagging </li></ul><ul><ul><li>Think Flicker (tag your pictures with a category) </li...
<ul><li>Example: Some VO is setting up monitoring. </li></ul><ul><ul><li>All sites want to test with each other </li></ul>...
<ul><li>Screenshot from the toolkit (when setting up the host): </li></ul><ul><li>Top: Communities the host has chosen to ...
<ul><li>Adding hosts via the scheduled testing screen </li></ul><ul><ul><li>Manually add </li></ul></ul><ul><ul><li>See wh...
<ul><li>We are interested in who is involved in the LHC community: </li></ul> –  05/06/10 , © 2009 Internet2 Architecture ...
<ul><li>List of hosts from the LHC community: </li></ul> –  05/06/10 , © 2009 Internet2 Architecture Overview - Communities
<ul><li>The Topology Service (TS) gathers and stores network topology information similar to the Lookup Service (LS) </li>...
<ul><li>The Authentication and Authorization (AS) service serves as a front end for identity management. </li></ul><ul><ul...
<ul><li>All communication in  perfSONAR  is done using well defined XML protocols </li></ul><ul><ul><li>Client communicati...
<ul><li>perfSONAR schema and protocol </li></ul><ul><ul><li>Understanding the structure of the schema is informative when ...
<ul><li>Key Goals: Extensibility, Normalization, Readability </li></ul><ul><li>Break representation of performance measure...
<ul><li>Can simplify the database representation for many types of measurement data </li></ul><ul><ul><li>While optimizati...
<ul><li>Subject (Noun) </li></ul><ul><ul><li>The measured/tested entity </li></ul></ul><ul><ul><li>E.g. An pair of hosts (...
<ul><li>Some sort of value - Datum </li></ul><ul><ul><li>Existence of an event might point to the case where there no addi...
Message Message  –  05/06/10 , © 2009 Internet2 Metadata Data A Message - Transient Information
Store  –  05/06/10 , © 2009 Internet2 Metadata Data An Object Store – Stationary information
Metadata <id>someId</id> Data <metadataIdRef> someId </metadataIdRef>  –  05/06/10 , © 2009 Internet2 A Data is Linked to ...
Metadata <id>someId</id> Metadata <id>someOtherId</id> <metadataIdRef> someId </metadataIdRef>  –  05/06/10 , © 2009 Inter...
<ul><li>Most elements contain an ID and IDref element </li></ul><ul><ul><li>ID = unique identification for this element </...
<ul><li>Metadata can be linked in series in two ways </li></ul><ul><ul><li>Merge  chaining allows for elements to be reuse...
<ul><li>Functions applied to the data have URI-based names </li></ul><ul><ul><li>Common parameters </li></ul></ul><ul><li>...
<ul><li>A namespace: </li></ul><ul><ul><li>http://ggf.org/ns/nmwg/base/2.0/ </li></ul></ul><ul><ul><li>MAY NOT  be a URL <...
<ul><li>We encode the measurement/event type in the namespace </li></ul><ul><ul><li>And as a standalone element </li></ul>...
<ul><li>One key to extensibility is the use of hierarchy with delegation </li></ul><ul><ul><li>Similar to OIDs in the IETF...
<ul><li>Organization-rooted tools namespace addresses this </li></ul><ul><li>Some top-level tools </li></ul><ul><ul><ul><l...
 –  05/06/10 , © 2009 Internet2 Message Example
<ul><li>Topology schema grew from network measurement description </li></ul><ul><ul><li>Reusable “Subject” elements for co...
 –  05/06/10 , © 2009 Internet2 Topology
 –  05/06/10 , © 2009 Internet2 Topology – Recursive Links
<ul><li>Structured by layers and the same elements recurring there </li></ul><ul><li>Varied by namespaces </li></ul><ul><u...
<ul><li>To completely capture the relationships, we need to do a few more things </li></ul><ul><li>Recursive definition of...
<ul><li>Elements at the same layer have relationships </li></ul><ul><ul><li>A link references two ports/interfaces </li></...
<ul><li>A scheme for identifying network elements </li></ul><ul><li>Each network element gets a unique identifier </li></u...
<ul><li>Use Cases: </li></ul><ul><ul><li>A topology service can be used to find the identifier for a network element </li>...
<ul><li>Identifiers use URN notation </li></ul><ul><ul><li>Prefixed with “urn:ogf:network:” </li></ul></ul><ul><ul><li>Con...
<ul><li>Examples </li></ul><ul><ul><li>urn:ogf:network:domain=Internet2.edu </li></ul></ul><ul><ul><li>urn:ogf:network:dom...
<ul><li>Utilizes the basic message container </li></ul><ul><ul><li>With type attribute – will vary by service </li></ul></...
<ul><li>How can I extend perfSONAR? </li></ul><ul><li>Definition of metric schema </li></ul><ul><ul><li>If you are publish...
<ul><li>The following examples will illustrate the mechanics of services and protocols: </li></ul><ul><ul><li>Deploying a ...
<ul><li>A perfSONAR service is deployed alongside the measurement infrastructure </li></ul><ul><li>Interactions with the l...
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
 –  05/06/10 , © 2009 Internet2 Deploying a Service
<ul><li>EchoRequest </li></ul><ul><ul><li>Sent to a service to test connectivity </li></ul></ul><ul><ul><li>Can be made ar...
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
<ul><li>MetadataKeyRequest </li></ul><ul><ul><li>For a given (partial) metadata, ask the service to verify that it does or...
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
<ul><li>SetupDataRequest </li></ul><ul><ul><li>Given a key or (partial) metadata, return measurement information. </li></u...
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
 –  05/06/10 , © 2009 Internet2 Client/Service Interaction
<ul><li>Services register with an hLS </li></ul><ul><li>hLSs summarize what they know and pass to the gLSs </li></ul><ul><...
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
 –  05/06/10 , © 2009 Internet2 Lookup Service Interaction
perfSONAR Architecture <ul><li>February 11 th  2010, APAN 29 – perfSONAR Workshop </li></ul><ul><li>Jeff Boote, Senior Net...
Upcoming SlideShare
Loading in...5
×

20100211-APAN29-pSArchitecture-1.ppt

156

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
156
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The services register themselves to the LS and mention their capabilities. They can also leave or be removed from a LS if a service gets down (keepalives).
  • The services register themselves to the LS and mention their capabilities. They can also leave or be removed from a LS if a service gets down (keepalives).
  • Transcript of "20100211-APAN29-pSArchitecture-1.ppt"

    1. 1. perfSONAR Architecture <ul><li>February 11 th 2010, APAN 29 – perfSONAR Workshop </li></ul><ul><li>Jeff Boote, Senior Network Software Engineer </li></ul>
    2. 2. <ul><li>Motivation and Goals </li></ul><ul><li>Service Oriented Architecture (SOA) </li></ul><ul><li>Architecture Overview </li></ul><ul><ul><li>Service Components </li></ul></ul><ul><ul><li>Client Examples </li></ul></ul><ul><li>Communication Protocols </li></ul><ul><li>Framework Interaction </li></ul><ul><ul><li>Deploying a Service </li></ul></ul><ul><ul><li>Simple Client/Service Interaction </li></ul></ul><ul><ul><li>Lookup Service Interaction </li></ul></ul> – 05/06/10 , © 2009 Internet2 Outline
    3. 3. <ul><li>Thus far we have seen that the perfSONAR framework can be used to solve end-to-end performance problems for multi-domain paths. </li></ul><ul><li>The framework is made up of several unique components and design considerations, all of which operate in a cooperative yet independent manner </li></ul><ul><ul><li>Each functionality is separated into a specific function </li></ul></ul><ul><ul><li>Clients and servers interact through scripted, XML Based protocols </li></ul></ul><ul><ul><li>Measurement data is encoded in expressive XML formats </li></ul></ul><ul><li>By the end of this talk we will see: </li></ul><ul><ul><li>A complete description of services for the entire framework </li></ul></ul><ul><ul><li>Overview of protocols </li></ul></ul><ul><ul><li>Use cases for clients and services alike </li></ul></ul> – 05/06/10 , © 2009 Internet2 Motivation and Goals
    4. 4. <ul><li>Interoperable network measurement middleware (SOA): </li></ul><ul><ul><li>Modular </li></ul></ul><ul><ul><li>Web services-based </li></ul></ul><ul><ul><li>Decentralized </li></ul></ul><ul><ul><li>Locally controlled </li></ul></ul><ul><li>Integrates: </li></ul><ul><ul><li>Network measurement tools and data archives </li></ul></ul><ul><ul><li>Data manipulation </li></ul></ul><ul><ul><li>Information Services </li></ul></ul><ul><ul><ul><li>Discovery </li></ul></ul></ul><ul><ul><ul><li>Topology </li></ul></ul></ul><ul><ul><ul><li>Authentication and authorization </li></ul></ul></ul><ul><li>Based on: </li></ul><ul><ul><li>Open Grid Forum Network Measurement ( NM ) Working Group schema </li></ul></ul><ul><ul><li>Currently attempting to formalize specification of perfSONAR protocols in the Network Measurement Control ( NMC ) working group. </li></ul></ul><ul><ul><li>Network topology description being defined in the Network Markup Language ( NML ) Working Group </li></ul></ul> – 05/06/10 , © 2009 Internet2 Service Oriented Architecture (SOA)
    5. 5. <ul><li>Measurement Point (MP) Service </li></ul><ul><ul><li>Enables the initiation of performance tests </li></ul></ul><ul><li>Measurement Archive (MA) Service </li></ul><ul><ul><li>Stores and publishes performance monitoring results </li></ul></ul><ul><li>Transformation Service </li></ul><ul><ul><li>Transform the data (aggregation, concatenation, correlation, translation, etc) </li></ul></ul><ul><li>Resource protector </li></ul><ul><ul><li>Arbitrate the consumption of limited resources </li></ul></ul><ul><ul><li>Other services delegate a limited portion of the authorization decision here </li></ul></ul><ul><li>These services are specifically concerned with the job of network performance measurement and analysis </li></ul> – 05/06/10 , © 2009 Internet2 Service Oriented Architecture (SOA)
    6. 6. <ul><li>Lookup Service </li></ul><ul><ul><li>Allows the client to discover the existing services and other LS services. </li></ul></ul><ul><ul><li>Dynamic: services registration themselves to the LS and mention their capabilities, they can also leave or be removed if a service goes down. </li></ul></ul><ul><li>Topology Service </li></ul><ul><ul><li>Make the network topology information available to the framework. </li></ul></ul><ul><ul><li>Find the closest MP, provide topology information for visualisation tools </li></ul></ul><ul><li>Authentication Service </li></ul><ul><ul><li>Based on Existing efforts: Internet2 MAT, GN2-JRA5 </li></ul></ul><ul><ul><li>Authentication & Authorization functionality for the framework </li></ul></ul><ul><ul><li>Users can have several roles, the authorization is done based on the user role. </li></ul></ul><ul><ul><li>Trust relationship between networks </li></ul></ul><ul><li>These services are the infrastructure concerned with discovering federating the available network services </li></ul> – 05/06/10 , © 2009 Internet2 Service Oriented Architecture (SOA)
    7. 7. Where is link utilization for - IPs d,e,f? Client Network A Network B LS A LS B MA A MA B a b c d e f Where is link utilization for – IPs a,b,c? a,b,c : Network A, MA A Get link utilization d,e,f Here you go Get Link utilization a,b,c Here you go Useful graph gLS Where can I get more about network Doman B/IP d,e,f and Domain A/IP a,b,c? LS A, LS B d,e,f : Network B, MA B – 05/06/10 , © 2009 Internet2 Service Oriented Architecture (SOA)
    8. 8. – 05/06/10 , © 2009 Internet2 Architecture Overview
    9. 9. – 05/06/10 , © 2009 Internet2 perfSONAR Architecture Overview
    10. 10. <ul><li>Measurement Point (MP) form the lowest layer of the monitoring infrastructure </li></ul><ul><ul><li>Directly interacts with the measurement tool </li></ul></ul><ul><ul><li>Can offer WS control over on-demand measurement </li></ul></ul><ul><ul><li>Can offer interface to a regular scheduled measurements </li></ul></ul><ul><li>Rolls of the Measurement Point: </li></ul><ul><ul><li>Utilize well known tools to perform measurements </li></ul></ul><ul><ul><li>Offer, at a minimum, cache storage of recently performed measurements </li></ul></ul><ul><ul><li>Interact with Measurement Archives (MAs) to archive stored measurements </li></ul></ul><ul><li>Examples: </li></ul><ul><ul><li>perfSONAR-BUOY (OWAMP and BWCTL Testing) </li></ul></ul><ul><ul><li>PingER (Ping Testing) </li></ul></ul><ul><ul><li>Command Line MP </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - MP
    11. 11. <ul><li>Measurement Archive (MA) stores the results of network and performance measurements </li></ul><ul><ul><li>WS interface for storage and query </li></ul></ul><ul><ul><li>Interacts with backend databases (e.g. SQL, RRD) </li></ul></ul><ul><li>Rolls of the Measurement Archive: </li></ul><ul><ul><li>Expose historical and current measurements of diverse types </li></ul></ul><ul><ul><li>Draw data queries away from the Measurement Points (MPs) </li></ul></ul><ul><li>Examples: </li></ul><ul><ul><li>perfSONAR-BUOY (OWAMP and BWCTL Data) </li></ul></ul><ul><ul><li>PingER (Ping Data) </li></ul></ul><ul><ul><li>SNMP MA </li></ul></ul><ul><ul><li>Status MA </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - MA
    12. 12. <ul><li>The Transformation Service (TrS) performs operations on data sets (e.g. aggregation, correlation). </li></ul><ul><ul><li>WS interface </li></ul></ul><ul><ul><li>Potential to store well known operations, and replay later </li></ul></ul><ul><li>Rolls of the Transformation Service : </li></ul><ul><ul><li>Draw complex statistical queries from Measurement Archives </li></ul></ul><ul><ul><li>Provide a conduit for popular operations (e.g. running statistics over several changing dataset). </li></ul></ul><ul><li>Examples (Planned): </li></ul><ul><ul><li>Path diagnostics tools </li></ul></ul><ul><ul><ul><li>Combining multiple metrics (network path, utilization, latency, bandwidth) </li></ul></ul></ul><ul><ul><li>Data presentation </li></ul></ul><ul><ul><ul><li>Statistical results for raw measurements. </li></ul></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - TrS
    13. 13. <ul><li>The Resource Protector (RP) monitors the relative performance and availability of the monitoring infrastructure </li></ul><ul><ul><li>Knowledge of the services in a given deployment </li></ul></ul><ul><ul><li>Defined policy regarding access and resources </li></ul></ul><ul><li>Rolls of the Transformation Service : </li></ul><ul><ul><li>Protects the time and resources of services from being overrun </li></ul></ul><ul><ul><ul><li>Too many queries from a single source </li></ul></ul></ul><ul><ul><ul><li>Too much data for a given query </li></ul></ul></ul><ul><ul><li>Cooperate with the Authentication and Authorization (AA) entities </li></ul></ul><ul><li>Examples (Panned): </li></ul><ul><ul><li>Data Protection </li></ul></ul><ul><ul><ul><li>Limits the size, duration, or frequency of a query </li></ul></ul></ul><ul><ul><li>Service Protection </li></ul></ul><ul><ul><ul><li>Limits access to functionality of the service </li></ul></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - RP
    14. 14. <ul><li>The Lookup Service (LS) is a general name for the service and and data discovery infrastructure </li></ul><ul><ul><li>Facilitates service and data discovery through the concept of registration </li></ul></ul><ul><ul><li>“ Summarizes” and distributes the job of location across layers of lookup </li></ul></ul><ul><ul><ul><li>Home Lookup Services – Local cache of data for several services </li></ul></ul></ul><ul><ul><ul><li>Global Lookup Services – Works similar to DNS for locating information through general queries </li></ul></ul></ul><ul><li>Rolls of the Lookup Service : </li></ul><ul><ul><li>Draws specific queries about the data and services away from the Measurement Points and Archives </li></ul></ul><ul><ul><li>Distribute information globally based on local conditions </li></ul></ul><ul><ul><li>Assure the ‘freshness’ of information in a dynamic infrastructure </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - LS
    15. 15. <ul><li>The Home Lookup Service (hLS) interacts directly with the other portions of the perfSONAR framework </li></ul><ul><ul><li>Recommended deployment is per domain </li></ul></ul><ul><li>Accepts Registration information from around the framework </li></ul><ul><ul><li>E.g. An MA will register its name, location, and available metadata </li></ul></ul><ul><ul><li>Metadata = static portion of a measurment (‘ subject ’, not results) </li></ul></ul><ul><li>Responds to Queries about services and data </li></ul><ul><ul><li>Services looking for each other (e.g. MP looking for an MA) </li></ul></ul><ul><ul><li>Client applications looking for data </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - hLS
    16. 16. <ul><li>The Global Lookup Service (gLS) serves as the oracle of the perfSONAR framework </li></ul><ul><ul><li>Global cloud of services cooperating together to distribute information </li></ul></ul><ul><ul><li>Manage the hLSs at the lower layer </li></ul></ul><ul><li>Accepts Registration information from hLSs only (!) </li></ul><ul><ul><li>E.g. An hLS will register its name, location, and a summary of the services and data it contains </li></ul></ul><ul><ul><li>Summary = condensed list of domains, ip addresses, data types </li></ul></ul><ul><li>Responds to Queries about services and data </li></ul><ul><ul><li>Similar to hLS queries, but more focused on where instead of what </li></ul></ul><ul><ul><li>Answer is typically an hLS to contact, not a direct result </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - gLS
    17. 17. – 05/06/10 , © 2009 Internet2 Architecture Overview – Lookup Service
    18. 18. <ul><li>Communities = Web 2.0 Content Tagging </li></ul><ul><ul><li>Think Flicker (tag your pictures with a category) </li></ul></ul><ul><ul><li>Think iTunes (tag your music with a genre) </li></ul></ul><ul><li>How does this help measurement lookup and discovery? </li></ul><ul><ul><li>One more axis to search on </li></ul></ul><ul><ul><li>More human readable and understandable than IP address or hostnames </li></ul></ul><ul><li>Use as many (or as few) as required: </li></ul><ul><ul><li>Networks (e.g. Campus, Regional, Network) </li></ul></ul><ul><ul><li>VO or Project (e.g. USATLAS, eVLBI, etc.) </li></ul></ul><ul><ul><li>Organization (DOE) </li></ul></ul><ul><ul><li>Other? </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - Communities
    19. 19. <ul><li>Example: Some VO is setting up monitoring. </li></ul><ul><ul><li>All sites want to test with each other </li></ul></ul><ul><ul><li>Not everyone is coming online at once, and VO membership may be volatile. </li></ul></ul><ul><ul><li>Strategy 1: </li></ul></ul><ul><ul><ul><li>Central VO coordinator maintains a list of participants (and must update it often) </li></ul></ul></ul><ul><ul><ul><li>All monitoring is manual: add/remove test hosts when the list changes </li></ul></ul></ul><ul><ul><li>Strategy 2: </li></ul></ul><ul><ul><ul><li>VO recommends a tag for all new hosts </li></ul></ul></ul><ul><ul><ul><li>All VO members search for test hosts (periodically) that share this tag – N.B. the GUIs on the disk can organize this automatically </li></ul></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - Communities
    20. 20. <ul><li>Screenshot from the toolkit (when setting up the host): </li></ul><ul><li>Top: Communities the host has chosen to associate with </li></ul><ul><li>Bottom: ‘Popular’ communities </li></ul><ul><ul><li>The word cloud is based on what we found in the GLS – the larger the word = the more people that are using this classification </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - Communities
    21. 21. <ul><li>Adding hosts via the scheduled testing screen </li></ul><ul><ul><li>Manually add </li></ul></ul><ul><ul><li>See what hosts are available for a particular community </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - Communities
    22. 22. <ul><li>We are interested in who is involved in the LHC community: </li></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - Communities
    23. 23. <ul><li>List of hosts from the LHC community: </li></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - Communities
    24. 24. <ul><li>The Topology Service (TS) gathers and stores network topology information similar to the Lookup Service (LS) </li></ul><ul><ul><li>Interfaces with external network tools (Dynamic Circuits, NOC databases) </li></ul></ul><ul><ul><li>Provides a query interface </li></ul></ul><ul><li>Rolls of the Transformation Service : </li></ul><ul><ul><li>Gather network topology from various sources </li></ul></ul><ul><ul><li>Correlate information found in other TS sources to provide a complete view of network availability </li></ul></ul><ul><ul><li>Interface with measurement tools to associate measurements with specific portions of the infrastructure </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - TS
    25. 25. <ul><li>The Authentication and Authorization (AS) service serves as a front end for identity management. </li></ul><ul><ul><li>Identity management relies on assigning roles to a given user via attributes , e.g. permission to do something </li></ul></ul><ul><ul><li>The AS will communicate via WS with a client and pass along credentials in order to validate an action or task </li></ul></ul><ul><ul><li>The AS will protect access to services and data </li></ul></ul><ul><li>Rolls of the Authentication Service : </li></ul><ul><ul><li>Validate services and clients given credentials </li></ul></ul><ul><ul><li>Act on behalf of the users to acquire the necessary permissions </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture Overview - AS
    26. 26. <ul><li>All communication in perfSONAR is done using well defined XML protocols </li></ul><ul><ul><li>Client communication with services </li></ul></ul><ul><ul><ul><li>Informative requests </li></ul></ul></ul><ul><ul><ul><li>Measurement Data requests </li></ul></ul></ul><ul><ul><ul><li>Discovery (e.g. “tell me where to find X”) queries </li></ul></ul></ul><ul><ul><li>Service to service communication </li></ul></ul><ul><ul><ul><li>Where are you type of queries </li></ul></ul></ul><ul><ul><ul><li>Registration of data into the LS infrastructure </li></ul></ul></ul><ul><ul><ul><li>Self-organizing domains </li></ul></ul></ul><ul><li>The following section outlines the basics of the communication protocol used in perfSONAR </li></ul><ul><ul><li>Too much information for a simple slide show – detailed references available </li></ul></ul> – 05/06/10 , © 2009 Internet2 Communication Protocols
    27. 27. <ul><li>perfSONAR schema and protocol </li></ul><ul><ul><li>Understanding the structure of the schema is informative when discussing the architecture and extensibility mechanisms </li></ul></ul><ul><li>Information Services </li></ul><ul><ul><li>The Lookup Service and Topology Service utilize the base model and provide system “glue” that allows for service discovery and contextualization of both data and services </li></ul></ul><ul><li>Extensibility and ongoing work </li></ul> – 05/06/10 , © 2009 Internet2 Architecture and Protocol Interaction
    28. 28. <ul><li>Key Goals: Extensibility, Normalization, Readability </li></ul><ul><li>Break representation of performance measurements down into basic elements </li></ul><ul><li>Data and Metadata </li></ul><ul><li>Measurement Data </li></ul><ul><ul><li>A set of of measurement events that have some value or values at a particular time </li></ul></ul><ul><li>Measurement Metadata </li></ul><ul><ul><li>The details about the set of measurement data </li></ul></ul> – 05/06/10 , © 2009 Internet2 Architecture and Protocol Interaction
    29. 29. <ul><li>Can simplify the database representation for many types of measurement data </li></ul><ul><ul><li>While optimizations are possible, many measurement types can be viewed as one value measured over time </li></ul></ul><ul><li>Assists Combination/Concatenation of metrics </li></ul><ul><ul><li>Creating derived metrics </li></ul></ul><ul><li>Normalization helps with inferring relationships between types of metrics </li></ul> – 05/06/10 , © 2009 Internet2 Schema Normalization
    30. 30. <ul><li>Subject (Noun) </li></ul><ul><ul><li>The measured/tested entity </li></ul></ul><ul><ul><li>E.g. An pair of hosts (end-point-pair), or a Layer 3 interface </li></ul></ul><ul><li>EventType (Verb) </li></ul><ul><ul><li>What type of measurement, value, or event occurred </li></ul></ul><ul><ul><li>Characteristic, tool output, or generic event </li></ul></ul><ul><ul><li>E.g. latency , bandwidth , utilization , or simply iperf </li></ul></ul><ul><li>Parameters (Adjectives and Adverbs) </li></ul><ul><ul><li>How, or under what conditions, did this event occur? </li></ul></ul><ul><ul><li>E.g. buffer sizes used, TCP vs ICMP packets </li></ul></ul> – 05/06/10 , © 2009 Internet2 Schema Basic Elements - Metadata
    31. 31. <ul><li>Some sort of value - Datum </li></ul><ul><ul><li>Existence of an event might point to the case where there no additional value </li></ul></ul><ul><ul><ul><li>As in “Link up/down” or threshold events </li></ul></ul></ul><ul><li>Time </li></ul><ul><ul><li>Is extensible since various representations are appropriate in different cases </li></ul></ul><ul><ul><ul><li>E.g. UNIX timestamp vs NTP time </li></ul></ul></ul> – 05/06/10 , © 2009 Internet2 Schema Basic Elements - Data
    32. 32. Message Message – 05/06/10 , © 2009 Internet2 Metadata Data A Message - Transient Information
    33. 33. Store – 05/06/10 , © 2009 Internet2 Metadata Data An Object Store – Stationary information
    34. 34. Metadata <id>someId</id> Data <metadataIdRef> someId </metadataIdRef> – 05/06/10 , © 2009 Internet2 A Data is Linked to Metadata, via IDs
    35. 35. Metadata <id>someId</id> Metadata <id>someOtherId</id> <metadataIdRef> someId </metadataIdRef> – 05/06/10 , © 2009 Internet2 A Metadata can be Linked to Another
    36. 36. <ul><li>Most elements contain an ID and IDref element </li></ul><ul><ul><li>ID = unique identification for this element </li></ul></ul><ul><ul><li>IDref = pointer to another unique ID </li></ul></ul><ul><li>IDs have a defined scope </li></ul><ul><ul><li>Varies by element type </li></ul></ul><ul><ul><li>Ex: Message IDs span a request/response pair </li></ul></ul><ul><ul><li>Ex: Metadata/Data ids can be long lived in storage </li></ul></ul><ul><ul><li>Check service documentation for specific scope rules </li></ul></ul> – 05/06/10 , © 2009 Internet2 IDs
    37. 37. <ul><li>Metadata can be linked in series in two ways </li></ul><ul><ul><li>Merge chaining allows for elements to be reused and a complete metadata can be built </li></ul></ul><ul><ul><li>Operation chaining requests or describes operations on data sets </li></ul></ul>A B A B A B B(A) – 05/06/10 , © 2009 Internet2 Linking Metadata
    38. 38. <ul><li>Functions applied to the data have URI-based names </li></ul><ul><ul><li>Common parameters </li></ul></ul><ul><li>Ideally, a series of these metadata elements can completely describe the provenance of any resultant dataset </li></ul><ul><ul><li>As well as requesting selection and reduction operations at query time </li></ul></ul> – 05/06/10 , © 2009 Internet2 Operation Metadata
    39. 39. <ul><li>A namespace: </li></ul><ul><ul><li>http://ggf.org/ns/nmwg/base/2.0/ </li></ul></ul><ul><ul><li>MAY NOT be a URL </li></ul></ul><ul><li>All measurements have some sort of Data and Time </li></ul><ul><li>All measurements can be described by the Metadata identifying who/what/how </li></ul><ul><li>The specific structures of the Data and Metadata elements depend on the measurement </li></ul><ul><li>Approach: Use Data and Metadata elements and vary the namespaces of the specific elements </li></ul> – 05/06/10 , © 2009 Internet2 Schema Namespaces
    40. 40. <ul><li>We encode the measurement/event type in the namespace </li></ul><ul><ul><li>And as a standalone element </li></ul></ul><ul><li>Some components of the system can pass Data and Metadata elements through without understanding their specific structure </li></ul><ul><li>Allows and implementation to decide whether it supports a particular type of data or not </li></ul><ul><li>Allows validation based on extended (namespace-specific) schemata </li></ul> – 05/06/10 , © 2009 Internet2 Schema Namespaces
    41. 41. <ul><li>One key to extensibility is the use of hierarchy with delegation </li></ul><ul><ul><li>Similar to OIDs in the IETF management world </li></ul></ul><ul><li>The NM-WG has a hierarchy of network characteristics </li></ul><ul><ul><li>Good starting point </li></ul></ul><ul><li>However, not all tools are cleanly mapped onto the Characteristic space </li></ul><ul><ul><li>Often a matter of some debate </li></ul></ul><ul><ul><li>Can be self defined with an ORG </li></ul></ul> – 05/06/10 , © 2009 Internet2 Schema Namespaces & Extensibility
    42. 42. <ul><li>Organization-rooted tools namespace addresses this </li></ul><ul><li>Some top-level tools </li></ul><ul><ul><ul><li>ping, traceroute </li></ul></ul></ul><ul><li>Easy to add new tools in organization-specific namespaces </li></ul><ul><li>Performance Event Repository </li></ul><ul><ul><li>Add a schema and get a URI </li></ul></ul><ul><ul><li>Add Java classes </li></ul></ul> – 05/06/10 , © 2009 Internet2 Schema Namespaces & Extensibility
    43. 43. – 05/06/10 , © 2009 Internet2 Message Example
    44. 44. <ul><li>Topology schema grew from network measurement description </li></ul><ul><ul><li>Reusable “Subject” elements for common cases </li></ul></ul><ul><ul><ul><li>Also reduces redundancy </li></ul></ul></ul><ul><ul><li>Relationships between measurement Subjects </li></ul></ul><ul><li>Same basic structure at all layers </li></ul><ul><ul><li>Networks are graphs </li></ul></ul><ul><li>Define: </li></ul><ul><ul><li>Node </li></ul></ul><ul><ul><li>Port (Interface) </li></ul></ul><ul><ul><li>Link </li></ul></ul><ul><ul><li>Domain </li></ul></ul><ul><ul><li>Network </li></ul></ul><ul><ul><li>Path </li></ul></ul><ul><ul><li>Service </li></ul></ul> – 05/06/10 , © 2009 Internet2 Topology Schema
    45. 45. – 05/06/10 , © 2009 Internet2 Topology
    46. 46. – 05/06/10 , © 2009 Internet2 Topology – Recursive Links
    47. 47. <ul><li>Structured by layers and the same elements recurring there </li></ul><ul><li>Varied by namespaces </li></ul><ul><ul><li>Reuse visualization logic, etc. </li></ul></ul><ul><ul><li>Validate layer- or technology-specific attributes </li></ul></ul><ul><li>4 Layers: Base (both abstract and L1), L2, L3, L4 </li></ul><ul><li>Also technology-specific layers like Ethernet, SONET/SDH </li></ul><ul><li>Defining topology is tricky, and lots of related software will use: </li></ul><ul><ul><li>Network Descriptions (static and dynamic) </li></ul></ul><ul><ul><li>Measurement tools </li></ul></ul><ul><ul><li>Research areas (path finding) </li></ul></ul> – 05/06/10 , © 2009 Internet2 Topology Schema
    48. 48. <ul><li>To completely capture the relationships, we need to do a few more things </li></ul><ul><li>Recursive definition of links </li></ul><ul><ul><li>Logical links consist of physical links </li></ul></ul><ul><li>Ordered lists of links - Path </li></ul><ul><ul><li>Like above, but we need to introduce an Index attribute </li></ul></ul><ul><li>Networks </li></ul><ul><ul><li>Physically consist of links but that is not always the most convenient logical view </li></ul></ul><ul><ul><li>Special element to which Interfaces or Links belong </li></ul></ul> – 05/06/10 , © 2009 Internet2 Relationships between Subject
    49. 49. <ul><li>Elements at the same layer have relationships </li></ul><ul><ul><li>A link references two ports/interfaces </li></ul></ul><ul><ul><ul><li>At Layer2 or Layer3 </li></ul></ul></ul><ul><li>Elements of the same sort have relationships between themselves at different layers </li></ul><ul><ul><li>A Layer 1 Interface (physical NIC) can have one or more Layer 2 Interfaces, which can each have one or more Layer 3 Interfaces </li></ul></ul><ul><li>Node is special </li></ul><ul><ul><li>Since a Node doesn’t really have any higher-layer characteristic independent of its Interfaces </li></ul></ul> – 05/06/10 , © 2009 Internet2 Relationships between Subject
    50. 50. <ul><li>A scheme for identifying network elements </li></ul><ul><li>Each network element gets a unique identifier </li></ul><ul><li>This identifier will be included with any measurement associated with that element. </li></ul> – 05/06/10 , © 2009 Internet2 Schema – Network Element Identifiers
    51. 51. <ul><li>Use Cases: </li></ul><ul><ul><li>A topology service can be used to find the identifier for a network element </li></ul></ul><ul><ul><li>An LS could then be queried to find all measurements associated with that element </li></ul></ul><ul><ul><li>Dynamic service path-finding can be integrated with ongoing measurements </li></ul></ul> – 05/06/10 , © 2009 Internet2 Network Element Identifiers
    52. 52. <ul><li>Identifiers use URN notation </li></ul><ul><ul><li>Prefixed with “urn:ogf:network:” </li></ul></ul><ul><ul><li>Consists of name/value pairs separated by colons </li></ul></ul><ul><ul><li>Possible field names: domain, node, port, link, path, network </li></ul></ul><ul><ul><li>Note: GLIF only recognizes domain= as required , everything else is opaque </li></ul></ul><ul><ul><li>Set of rules defined for each field to keep identifiers compact and finite </li></ul></ul> – 05/06/10 , © 2009 Internet2 Network Element Identifiers
    53. 53. <ul><li>Examples </li></ul><ul><ul><li>urn:ogf:network:domain=Internet2.edu </li></ul></ul><ul><ul><li>urn:ogf:network:domain=internet2.edu:node=packrat </li></ul></ul><ul><ul><li>urn:ogf:network:domain=internet2.edu:node=rtr.seat:port=so-2%2F1%2F0.16 </li></ul></ul><ul><ul><li>urn:ogf:network:domain=internet2.edu:node=rtr.seat:port=198.32.8.200 </li></ul></ul><ul><ul><li>urn:ogf:network:domain=Internet2.edu:node=packrat:port=eth0:link=1 </li></ul></ul><ul><ul><li>urn:ogf:network:domain=internet2.edu:link=WASH to ATLA OC192 </li></ul></ul><ul><ul><li>urn:ogf:network:path=anna-11537-176 </li></ul></ul> – 05/06/10 , © 2009 Internet2 Network Element Identifiers
    54. 54. <ul><li>Utilizes the basic message container </li></ul><ul><ul><li>With type attribute – will vary by service </li></ul></ul><ul><li>Contains various data and metadata elements </li></ul><ul><ul><li>Linked via ID or independent </li></ul></ul><ul><li>Uses a message-level parameter element to communicate message-level options </li></ul><ul><ul><li>Output format, service annotation </li></ul></ul><ul><li>Messages return a result datum with a type system identical to that of a measurement datum </li></ul> – 05/06/10 , © 2009 Internet2 perfSONAR Protocol
    55. 55. <ul><li>How can I extend perfSONAR? </li></ul><ul><li>Definition of metric schema </li></ul><ul><ul><li>If you are publishing a new type of data, schema definition is the first step </li></ul></ul><ul><li>Reuse or re-implement protocol processing </li></ul><ul><ul><li>Examples in Java and Perl </li></ul></ul><ul><li>Register with the Lookup Service </li></ul><ul><ul><li>Defining a new service type </li></ul></ul><ul><li>Analysis modules are extended by assigning a URI, defining parameters </li></ul> – 05/06/10 , © 2009 Internet2 Extending perfSONAR
    56. 56. <ul><li>The following examples will illustrate the mechanics of services and protocols: </li></ul><ul><ul><li>Deploying a Service </li></ul></ul><ul><ul><li>Simple Client/Service Interaction </li></ul></ul><ul><ul><ul><li>Echo </li></ul></ul></ul><ul><ul><ul><li>Metadata Request </li></ul></ul></ul><ul><ul><ul><li>Data Request </li></ul></ul></ul><ul><ul><li>Lookup Service Interaction </li></ul></ul> – 05/06/10 , © 2009 Internet2 Framework Interaction
    57. 57. <ul><li>A perfSONAR service is deployed alongside the measurement infrastructure </li></ul><ul><li>Interactions with the lookup service and clients are described </li></ul> – 05/06/10 , © 2009 Internet2 Deploying a Service
    58. 58. – 05/06/10 , © 2009 Internet2 Deploying a Service
    59. 59. – 05/06/10 , © 2009 Internet2 Deploying a Service
    60. 60. – 05/06/10 , © 2009 Internet2 Deploying a Service
    61. 61. – 05/06/10 , © 2009 Internet2 Deploying a Service
    62. 62. – 05/06/10 , © 2009 Internet2 Deploying a Service
    63. 63. – 05/06/10 , © 2009 Internet2 Deploying a Service
    64. 64. – 05/06/10 , © 2009 Internet2 Deploying a Service
    65. 65. – 05/06/10 , © 2009 Internet2 Deploying a Service
    66. 66. – 05/06/10 , © 2009 Internet2 Deploying a Service
    67. 67. – 05/06/10 , © 2009 Internet2 Deploying a Service
    68. 68. – 05/06/10 , © 2009 Internet2 Deploying a Service
    69. 69. – 05/06/10 , © 2009 Internet2 Deploying a Service
    70. 70. – 05/06/10 , © 2009 Internet2 Deploying a Service
    71. 71. <ul><li>EchoRequest </li></ul><ul><ul><li>Sent to a service to test connectivity </li></ul></ul><ul><ul><li>Can be made arbitrarily complex by the service designer </li></ul></ul><ul><ul><ul><li>Test backend storage </li></ul></ul></ul><ul><ul><ul><li>Test internal self-checks </li></ul></ul></ul><ul><ul><li>Minimum is an ‘are you alive’ ping </li></ul></ul> – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    72. 72. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    73. 73. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    74. 74. <ul><li>MetadataKeyRequest </li></ul><ul><ul><li>For a given (partial) metadata, ask the service to verify that it does or does not exist </li></ul></ul><ul><ul><li>Return a ‘key’, e.g. replayable token, to access the data </li></ul></ul> – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    75. 75. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    76. 76. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    77. 77. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    78. 78. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    79. 79. <ul><li>SetupDataRequest </li></ul><ul><ul><li>Given a key or (partial) metadata, return measurement information. </li></ul></ul><ul><ul><li>Can be ‘filtered’ by time to prevent getting more results than necessary. </li></ul></ul> – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    80. 80. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    81. 81. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    82. 82. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    83. 83. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    84. 84. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    85. 85. – 05/06/10 , © 2009 Internet2 Client/Service Interaction
    86. 86. <ul><li>Services register with an hLS </li></ul><ul><li>hLSs summarize what they know and pass to the gLSs </li></ul><ul><li>gLSs exchange the information as needed </li></ul><ul><li>Clients will need a multi-step process to find information </li></ul><ul><ul><li>Query the gLS </li></ul></ul><ul><ul><li>Query the appropriate hLS </li></ul></ul><ul><ul><li>Query the appropriate services </li></ul></ul> – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    87. 87. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    88. 88. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    89. 89. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    90. 90. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    91. 91. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    92. 92. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    93. 93. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    94. 94. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    95. 95. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    96. 96. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    97. 97. – 05/06/10 , © 2009 Internet2 Lookup Service Interaction
    98. 98. perfSONAR Architecture <ul><li>February 11 th 2010, APAN 29 – perfSONAR Workshop </li></ul><ul><li>Jeff Boote, Senior Network Software Engineer </li></ul><ul><li>For more information, visit psps.perfsonar.net </li></ul> – 05/06/10 , © 2009 Internet2
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×