My SRB
Upcoming SlideShare
Loading in...5
×
 

My SRB

on

  • 872 views

 

Statistics

Views

Total Views
872
Views on SlideShare
872
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 18

My SRB My SRB Presentation Transcript

  • MySRB a Virtual Data Grid Solution (An interface to the Storage Resource Broker) Arcot Rajasekar SDSC
  • Introduction
    • Motivation
    • SRB Background
    • mySRB
    SRB
  • Data Handling Problems
    • Large Datasets; Large Number of Datasets; Scaling
    • Distributed, Heterogeneous Storage
    • Virtualization & Transparency
    • Collaboration, Access Control, Authentication, Security
    • Replication, Coherency, Synchronization
    • Fault Tolerance and Load Distribution
    • Scheduling, Caching & Data Placements
    • Data Migration over Time & Space
    • Data/Collection Curation
    • Uniform Name Space
    • Handling Legacy Data and Data/Resource Evolution
    • User-friendly Interfaces
  • Metadata Problems
    • Types of Metadata – Relational to XML to unstructured
    • Standardized to User-defined Metadata
    • Large Number of Attributes;
    • Large Size; Scaling
    • Federation - integration over space
    • Evolution - integration over time
    • Evolution - integration over contexts
    • Discovery and Search
    • Presentation – user friendly
    • Extraction and Maintenance
  • Virtual Data Grid Requirements
    • Seamless access to data and information stored at local and remote sites
    • Virtualization of data, collection and meta information
    • Handle Dataset Scaling – size & number
    • Integrate Data Collections & Associated Metadata
    • Handle Multiplicity of Platforms, Resource & Data Types
    • Handle Seamless Authentication
    • Handle Access Control
    • Provide Auditing Facilities
    • Handle Legacy Data & Methods
  • Solution SRB SDSC Storage Resource Broker & Meta-data Catalog SRB Archives HPSS, ADSM, UniTree, DMF Databases DB2, Oracle, Sybase File Systems Unix, NT, Mac OSX Application C, C++, Linux I/O Unix Shell Dublin Core Resource, Mthd, User User Defined Application Meta-data Remote Proxies DataCutter Metadata Extraction Java, NT Browsers Web Prolog Predicate MCAT HRM
  • Features SRB
    • Data Virtualization
      • Uniform Data Name Space & Virtual Collection-concept
    • Resource Transparency
      • Local or Remote, Resource Type & Access Method
    • Location/Format Transparency
      • Path Names, Schemas, Containers
    • User Transparency
      • Uniform User Name Space
      • Tickets & Groups
    • Method Transparency
    • Other Features
      • Rich Access Control
      • Cross-Domain Authentication
      • Segmented & Replicated Data Mgmt
      • Data Discovery
    Virtual Data Grid (SRB) MCAT Duke UCLA NCMIR CalTech SDSC
  • Solution SRB
    • Federated Server Architecture
    • Uniform Access Interface – Thread-safe Client
      • Programmatic API (C, C++, Java, Perl-through-C)
      • GUI (Java for Unix, Windows Browser [ NT, Me, 98, 2000 ])
      • Web Support (CGI-Scripts, Portals )
      • Command Line Interface (Unix, DOS)
    • Metadata Catalog (Oracle, DB2, Sybase, SQLServer)
      • Handles transparencies, authentication, access control,
      • replication, container support, …
      • User-defined Metadata
    • Multi-Platform Support
      • Unix, Linux, Windows, MacOSX (from Cray to Desktop)
      • HPSS, ADSM, UniTree, …, UnixFS, NTFS,…, Oracle, DB2,…, FTP,HTTP
    • Integration of Data Handling & Digital Library Functionalities
  • SRB Space DL DR - Data Repository DL - Dig Library MC - Meta Catalog SRB SRB SRB SRB SRB SRB SRB SRB SRB SRB DR DR DR DR DR DR DL DL DL MC Client Client Client Client Client Client
  • SRB Projects
    • Digital Libraries
      • UCB, Umich, UCSB, Stanford,CDL
      • NSF NSDL - UCAR / DLESE
    • NASA Information Power Grid
    • DOE ASCI Data Visualization Corridor
    • Astronomy
      • National Virtual Observatory
      • 2MASS Project (2 Micron All Sky Survey)
    • Particle Physics
      • Particle Physics Data Grid (DOE)
      • GriPhyN
      • SLAC Synchrotron Data Repository
    • Medicine
      • Visible Embryo (NLM)
    • Earth Systems Sciences
      • ESIPS
      • LTER
    • Persistent Archives
      • NARA
      • LOC
    • Neuro Science & Molecular Science
      • TeleScience, Brain Images, BIRN
      • JCSG (SSRL/SLAC), AfCS, …
  • MySRB
    • Web-bases Access to the SRB
    • Secure HTTP
    • Uses Cookies for Session Control
    • Self Registration of Users Supported
      • Currently limited to SDSC users
    • Self Registration of Resources (soon)
    • Access to Both Data and Metadata
  • MySRB Features
    • Data & File Management
    • Collection Creation and Management
    • Collection of Varied Objects
      • Files, SQL Objects, Databases, URLs, directories, archives, …
    • Metadata Handling
    • Browsing & Querying Interface
    • Access Control
    • Version Control (soon)
    • Support proxy (remote) operations
  • Data Management
    • Browse in Hierarchical Collections
    • Registration of
    • (remote) Legacy Files & Directories
    • Registration of SQL Objects
    • Registration of URLs
    • Data Movement Operations
      • Ingest & Re-Ingest, Delete, Unlink
      • Replicate, Copy, Move, S-Link
    • Access Control Operations
      • Read, Write, Own, Curate, Annotate, …
      • Ticket-based Access
    • Version Control Operations (soon)
      • Read Lock, Write Lock, Unlock
      • Check In Check Out
  • Types of Meta data
    • System-level Metadata
      • Size, resource, owner, date, access control, …
    • User-defined Meta data
      • for data & collections
      • <name,value,unit> triples
      • No limits in number of metadata
      • Support for Collection-level schemas
        • Comments, default values, drop-down lists
      • Support for Standardized Schemas
        • (eg. Dublin Core)
    • Annotations
      • Supports textual annotations
      • Annotator, date, context also registered
  • Meta Data Management
    • Insert, Update and Delete of Metadata
    • Access Control for Metadata (soon in mySRB)
    • Querying across system-level, user-defined metadata and annotations
      • Query under collections & across collections
    • Browsing on user-defined metadata
    • Metadata supported for legacy files & directories
    • Extract Metadata (using proxy operations)
  • Conclusion
    • Demonstration: goto mySRB
    • For Additional Information:
      • http://www.npaci.edu/dice/srb