Your SlideShare is downloading. ×
My SRB
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

My SRB

747

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
747
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • 18
  • Transcript

    • 1. MySRB a Virtual Data Grid Solution (An interface to the Storage Resource Broker) Arcot Rajasekar SDSC
    • 2. Introduction
      • Motivation
      • SRB Background
      • mySRB
      SRB
    • 3. Data Handling Problems
      • Large Datasets; Large Number of Datasets; Scaling
      • Distributed, Heterogeneous Storage
      • Virtualization & Transparency
      • Collaboration, Access Control, Authentication, Security
      • Replication, Coherency, Synchronization
      • Fault Tolerance and Load Distribution
      • Scheduling, Caching & Data Placements
      • Data Migration over Time & Space
      • Data/Collection Curation
      • Uniform Name Space
      • Handling Legacy Data and Data/Resource Evolution
      • User-friendly Interfaces
    • 4. Metadata Problems
      • Types of Metadata – Relational to XML to unstructured
      • Standardized to User-defined Metadata
      • Large Number of Attributes;
      • Large Size; Scaling
      • Federation - integration over space
      • Evolution - integration over time
      • Evolution - integration over contexts
      • Discovery and Search
      • Presentation – user friendly
      • Extraction and Maintenance
    • 5. Virtual Data Grid Requirements
      • Seamless access to data and information stored at local and remote sites
      • Virtualization of data, collection and meta information
      • Handle Dataset Scaling – size & number
      • Integrate Data Collections & Associated Metadata
      • Handle Multiplicity of Platforms, Resource & Data Types
      • Handle Seamless Authentication
      • Handle Access Control
      • Provide Auditing Facilities
      • Handle Legacy Data & Methods
    • 6. Solution SRB SDSC Storage Resource Broker & Meta-data Catalog SRB Archives HPSS, ADSM, UniTree, DMF Databases DB2, Oracle, Sybase File Systems Unix, NT, Mac OSX Application C, C++, Linux I/O Unix Shell Dublin Core Resource, Mthd, User User Defined Application Meta-data Remote Proxies DataCutter Metadata Extraction Java, NT Browsers Web Prolog Predicate MCAT HRM
    • 7. Features SRB
      • Data Virtualization
        • Uniform Data Name Space & Virtual Collection-concept
      • Resource Transparency
        • Local or Remote, Resource Type & Access Method
      • Location/Format Transparency
        • Path Names, Schemas, Containers
      • User Transparency
        • Uniform User Name Space
        • Tickets & Groups
      • Method Transparency
      • Other Features
        • Rich Access Control
        • Cross-Domain Authentication
        • Segmented & Replicated Data Mgmt
        • Data Discovery
      Virtual Data Grid (SRB) MCAT Duke UCLA NCMIR CalTech SDSC
    • 8. Solution SRB
      • Federated Server Architecture
      • Uniform Access Interface – Thread-safe Client
        • Programmatic API (C, C++, Java, Perl-through-C)
        • GUI (Java for Unix, Windows Browser [ NT, Me, 98, 2000 ])
        • Web Support (CGI-Scripts, Portals )
        • Command Line Interface (Unix, DOS)
      • Metadata Catalog (Oracle, DB2, Sybase, SQLServer)
        • Handles transparencies, authentication, access control,
        • replication, container support, …
        • User-defined Metadata
      • Multi-Platform Support
        • Unix, Linux, Windows, MacOSX (from Cray to Desktop)
        • HPSS, ADSM, UniTree, …, UnixFS, NTFS,…, Oracle, DB2,…, FTP,HTTP
      • Integration of Data Handling & Digital Library Functionalities
    • 9. SRB Space DL DR - Data Repository DL - Dig Library MC - Meta Catalog SRB SRB SRB SRB SRB SRB SRB SRB SRB SRB DR DR DR DR DR DR DL DL DL MC Client Client Client Client Client Client
    • 10. SRB Projects
      • Digital Libraries
        • UCB, Umich, UCSB, Stanford,CDL
        • NSF NSDL - UCAR / DLESE
      • NASA Information Power Grid
      • DOE ASCI Data Visualization Corridor
      • Astronomy
        • National Virtual Observatory
        • 2MASS Project (2 Micron All Sky Survey)
      • Particle Physics
        • Particle Physics Data Grid (DOE)
        • GriPhyN
        • SLAC Synchrotron Data Repository
      • Medicine
        • Visible Embryo (NLM)
      • Earth Systems Sciences
        • ESIPS
        • LTER
      • Persistent Archives
        • NARA
        • LOC
      • Neuro Science & Molecular Science
        • TeleScience, Brain Images, BIRN
        • JCSG (SSRL/SLAC), AfCS, …
    • 11. MySRB
      • Web-bases Access to the SRB
      • Secure HTTP
      • Uses Cookies for Session Control
      • Self Registration of Users Supported
        • Currently limited to SDSC users
      • Self Registration of Resources (soon)
      • Access to Both Data and Metadata
    • 12. MySRB Features
      • Data & File Management
      • Collection Creation and Management
      • Collection of Varied Objects
        • Files, SQL Objects, Databases, URLs, directories, archives, …
      • Metadata Handling
      • Browsing & Querying Interface
      • Access Control
      • Version Control (soon)
      • Support proxy (remote) operations
    • 13. Data Management
      • Browse in Hierarchical Collections
      • Registration of
      • (remote) Legacy Files & Directories
      • Registration of SQL Objects
      • Registration of URLs
      • Data Movement Operations
        • Ingest & Re-Ingest, Delete, Unlink
        • Replicate, Copy, Move, S-Link
      • Access Control Operations
        • Read, Write, Own, Curate, Annotate, …
        • Ticket-based Access
      • Version Control Operations (soon)
        • Read Lock, Write Lock, Unlock
        • Check In Check Out
    • 14. Types of Meta data
      • System-level Metadata
        • Size, resource, owner, date, access control, …
      • User-defined Meta data
        • for data & collections
        • <name,value,unit> triples
        • No limits in number of metadata
        • Support for Collection-level schemas
          • Comments, default values, drop-down lists
        • Support for Standardized Schemas
          • (eg. Dublin Core)
      • Annotations
        • Supports textual annotations
        • Annotator, date, context also registered
    • 15. Meta Data Management
      • Insert, Update and Delete of Metadata
      • Access Control for Metadata (soon in mySRB)
      • Querying across system-level, user-defined metadata and annotations
        • Query under collections & across collections
      • Browsing on user-defined metadata
      • Metadata supported for legacy files & directories
      • Extract Metadata (using proxy operations)
    • 16. Conclusion
      • Demonstration: goto mySRB
      • For Additional Information:
        • http://www.npaci.edu/dice/srb

    ×