• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Status Of Hpss

Status Of Hpss






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Status Of Hpss Status Of Hpss Presentation Transcript

    • Status of HPSS New Features, Requirements, and Installations Otis Graf IBM Global Services - Federal Houston, Texas October 1999
    • Topics
      • HPSS New Features since Release 3.2
      • Next incremental updates
      • Feature to be added in 4.2
      • Status of HPSS Installations
      • Focus on Support Services
      • Next Requirements Development Cycle
    • Summary of HPSS Releases and Patches
      • Includes through 4.1.1 (July 1999)
      • File families
        • Segregate files onto different groups of tapes.
        • Storage Server uses family id passed in segment create calls for selecting the tape for the segment create
      Features Added Since Rel. 3.2 Level 1 F1 F2 FileA FileB (F1) FileC (F2) Level 2 Unassigned FileC FileB FileA
    • Features Added Since Rel. 3.2 (Cont.)
      • Filsets : A hierarchical collection of files and directories managed as a single unit.
      work play games play fs_goodwinj fs_teaff usr bin home goodwinj ... teaff Root Fileset junction junction
    • Features Added Since Rel. 3.2 (Cont.)
      • HPSS Filesets
        • Managed by name server
        • Can be either HPSS-only or HPSS/DFS
      • DFS Filesets
        • DFS-only
        • HPSS/DFS archived
          • HPSS & DFS name spaces not kept in synch
          • Only way to get data is through DFS
          • Performance similar to DFS-only if data is resident
        • HPSS/DFS mirrored
          • HPSS & DFS name spaces kept in synch
          • Data can be seen and altered from either DFS or HPSS
          • HPSS high speed data movement can be used
      • PFTP Enhancements
        • Allows the use of multiple network and/or multiple systems for child processes for multi-network stripe transfers
      • MPI-IO Interface
        • Implements a subset of the MPI-2 standard
        • Coordinates access to HPSS files from multiple processes
      Features Added Since Rel. 3.2 (Cont.)
      • Scaleable Accounting
        • Accounting summary records support the following collection options during the accounting period:
          • By account index & COS,
            • Number of accesses
            • Number of files transferred
            • bytes used
          • By account index, COS, & storage class
            • number of files transferred
            • number of accesses
      Features Added Since Rel. 3.2 (Cont.)
      • Performance Enhancements
        • Support disk partitions > 2 GB
        • Bigger virtual volumes (up to 16,384 segments per VV)
        • Faster disk file creates and purges
      • Support for Shelf Tapes
        • Provide utility to identify and move volumes to shelf
        • Mark cartridge metadata with shelf descriptor
        • Generate operator mount requests via SSM pop-up
      • Client API Enhancements
        • Manually migrate/purge on a file basis
        • Lock/unlock to prevent a file from being purged
      Features Added Since Rel. 3.2 (Cont.)
      • Non-DCE client API (Solaris, IRIX & AIX)
      • Non-DCE Mover
      • Mover ported to other platforms
        • Solaris (DCE)
        • SGI IRIX (non-DCE)
      • Additional Tape Drive Support
        • Ampex DST 314
        • IBM 3590E
      • Additional Library Support
        • ADIC Automated Media Library (AML)
      Features Added Since Rel. 3.2 (Cont.)
    • Number of HPSS Systems *
      • *Notes:
        • “ System” is a name space instance intended for production
        • 1999 data is “year to date”.
    • Listing of HPSS Sites as of October 1999
    • Listing of HPSS Sites (Cont.) as of October, 1999
    • Listing of HPSS Sites (Cont.) as of October, 1999
    • Current Activities
      • Next scheduled patch ( in Nov 99)
        • Sun Phase 2
          • STK PVR and IBM 3494 PVR
          • PFTP Client and Daemon
          • MPI-IO
        • SFS Backup utilities
        • Some manageability improvements
      • Follow-on patch (first quarter 00)
        • IBM 3590 tape drives on Solaris
      • StorageTek RAIT - testing and prototyping activity, no HPSS software change
    • Current Activities (Cont.)
      • Development of Release 4.2
        • Planned date is 4th Quarter 00
      • Enhancement of On-Line Resources
        • Tools repository
        • On-line help (FAQ & problem/solution DB)
        • On-line documentation (HTML & PDF)
      • New Training Classes
        • Advanced System Admin Class
        • HPSS API programming class
    • Current Activities (Cont.)
      • Enhancement of Customer Support
        • Better coordination with IBM Transarc
        • Deployment/support procedures
    • Features in Rel. 4.2
      • Multiple Storage Subsystems *
        • Comprised of Name Server, Bitfile Server, Migration Purge Server, & 1-n Storage Servers
        • May be used to enhance concurrency
        • May be used to partition servers to handle parts of the name space Data assigned to Storage Subsystem based on name
          • Fileset creation established association with Name Server
          • Junction attached fileset to name space
        • *Note: HPSS Contract terms are pending for multiple name spaces.
    • Features in Rel. 4.2 (Cont.)
      • Federated Name Space
        • Defined as cooperative, distributed HPSS systems linked at the internal server level
        • Supports use of all HPSS interfaces across multiple HPSS sites
        • Maintains autonomy of individual HPSS installations
        • Name spaces linked by junctions which point to filesets
    • Features in Rel. 4.2 (Cont.)
      • Command Line Utilities
        • Provide command line interface to SSM to enable management of HPSS from automated scripts
        • Command-line program (hpssadm) which can be run interactively or in batch mode
        • Allows user to manage servers, devices, storage classes, PVL jobs, and, to a limited extent, volumes
    • Features in Rel. 4.2 (Cont.)
      • Non-DCE Client API Security
        • Note: The majority of the non-DCE Client API has been developed for released in Release 4.1.1
        • Support client authorization / authentication by:
          • None
          • DCE (for platforms with DCE)
          • Kerberos
        • Provide Non-DCE Client Gateway (NDCG) with the ability to authenticate clients’ identities on a per connection basis
    • Features in Rel. 4.2 (Cont.)
      • Mass Configuration
        • Provide mechanisms to create multiple server or drive / device records in a single request
        • Provide mechanism to set persistent default values used in creating new configuration records
        • A configurable default Log Policy will be provided, which may reduce the number of specific policies which need to be created
    • Features in Rel. 4.2 (Cont.)
      • Gatekeeper
        • Provides optional client interface to allow sites to schedule and monitor use of HPSS file resources
        • Supports file create, open, stage, and close requests
        • May be associated with one or many Storage Subsystems
        • Sites will be able to implement site specific user level scheduling of storage requests. This may include:
          • Limit the number of open files per user/host
          • Prevent create requests for a user/host
          • Collect statistics
    • Features in Rel. 4.2 (Cont.)
      • Account Validation
        • Maintain integrity of user account information. Users will only be able to use valid accounts
        • Support cross-linked namespaces (remote sites). Individual sites select their own style of accounting
        • While still supported, no longer be necessary to have user default account information in DCE registry
        • Sites that do not need accounting or validation of user accounts may continue to run in the current fashion
    • Features in Rel. 4.2 (Cont.)
      • Sun Solaris Phase 3
        • Port remaining servers
        • Target OS is Solaris 8
      • HPSS back-end the IBM General Parallel File System (GPFS)
    • Post Release 4.2 Discussions
      • No definitive release plans yet
      • Features that were delayed from Rel. 4.2
        • Tape import/export
        • Multiple distributed movers - allow multiple, possibly distributed movers to access a single device
      • Begin new requirements development cycle: 2 Qtr 00
      • Features under discussion
        • Improved manageability features
        • Steps towards server consolidation and decreasing SFS dependencies
        • Improved GPFS/HPSS performance through parallel data paths
        • Backup/archive feature (file servers, workstations)
        • Critical HEP requirements
    • Next Requirements Development Cycle
      • Process
        • Process begins at start of a new release cycle.
        • Requirements contributed by customer and IBM.
        • Requirements team consists of a representative from each customer site.
        • Release contents are prioritized by requirements team.
        • Developers responsible decomposing requirements into subsystem level.
    • Requirements Development Cycle (Cont.)
      • Process (Cont.)
        • Requirements are costed, and then reprioritized as an iterative process. Customer balloting influences release contents, but is not binding.
        • Project metrics used to determine the number of requirements accepted in the next baseline.
        • Requirements not making baseline are included in the requirements document, but annotated as future release.
        • Formal inspection of requirements is performed.
    • Requirements Development Cycle (Cont.)
      • Process (Cont.)
        • Any changes to the approved baseline require Technical and Executive Committee approval.
        • Subsystem Requirements documents are published once a baseline is approved.
      • Opportunity for the HEP Community
        • HEP community develop common set of requirements
        • Provide unified voice on requirements committee
    • Additional Resources
      • HPSS User Forum Presentations
        • www5.clearlake.ibm.com  “What’s New”
          • Site Reports
          • Details for current capabilities
          • Details of Release 4.2
          • Details of DFSS and HPSS
      • Multi-Platform Capability of HPSS Components
        • www5.clearlake.ibm.com  “Product Information”  ”Multi-Platform Capability of HPSS”
          • Core Servers and Movers
          • User Interface Clients