NATIONAL STRATEGY FOR THE PRESERVATION OF DIGITAL RESOURCES : CASE STUDIES & A PROPOSAL Smita Chandra Email : [email_address] Dr. Vivek Patkar Independent Researcher, Mumbai. Email : [email_address]

Outline of Presentation Importance and need for digital preservation Stakeholders of preservation and their role Strategies and approaches to digital preservation Web archiving strategies Metadata preservation, implementation and OAIS IPR and copyrights International initiatives Indian scenario A proposal relevant to Geomagnetism and in general Conclusion

Importance and need for digital preservation

Stakeholders of preservation and their role

Strategies and approaches to digital preservation

Web archiving strategies

Metadata preservation, implementation and OAIS

IPR and copyrights

International initiatives

Indian scenario

A proposal relevant to Geomagnetism and in general

Conclusion

Importance of Digital Preservation 1975 – Two Viking space probes sent to Mars by USA. Data generated by unrepeatable mission cost $1 billion. Recorded data on magnetic tapes was corrupted / unidentifiable after 2 decades despite being kept in climate controlled environment. Scientists could not access data, unable to decode the formats used.

1975 – Two Viking space probes sent to Mars by USA.

Data generated by unrepeatable mission cost $1 billion.

Recorded data on magnetic tapes was corrupted / unidentifiable after 2 decades despite being kept in climate controlled environment.

Scientists could not access data, unable to decode the formats used.

Importance of Digital Preservation Original format developers not alive. Finally old printouts tracked and retyped. NASA therefore is biggest supporters of Digital Projects. This illustrates wide gap in information generation and its management.

Original format developers not alive.

Finally old printouts tracked and retyped.

NASA therefore is biggest supporters of Digital Projects.

This illustrates wide gap in information generation and its management.

Need for Digital Preservation Huge scientific data 259 MB/person/yr. is being generated (having economic value). 90% is stored digitally. This provides huge opportunities for long term digital preservation. Initiatives for long term preservation needed with common goal, centrally funded activities with partnership and collaboration.

Huge scientific data 259 MB/person/yr. is being generated (having economic value).

90% is stored digitally.

This provides huge opportunities for long term digital preservation.

Initiatives for long term preservation needed with common goal, centrally funded activities with partnership and collaboration.

Stakeholders of Preservation Libraries Archives Museums Scientific Institution

Libraries

Archives

Museums

Scientific Institution

Role of Libraries / Archives in Preservation activities Maintaining digital material for long term Conservation : Treatment for stability Restoration (Bring back document to its original state)

Maintaining digital material for long term

Conservation : Treatment for stability

Restoration (Bring back document to its original state)

Strategies for Preservation Technical Preservation (hardware / software of object) Technology Emulation (Mimic original environment using latest software) Data Migration (Transfer digital materials from one format to current one).

Technical Preservation (hardware / software of object)

Technology Emulation (Mimic original environment using latest software)

Data Migration (Transfer digital materials from one format to current one).

Two Main Digital Preservation Approaches Preservation through digitization (for greater security / saving space / producing multiple copies). Born digital – (Convert information form from Analog to Digital). Digitization costly but necessary for human progress. Appropriate action by individuals / organization needed.

Preservation through digitization (for greater security / saving space / producing multiple copies).

Born digital – (Convert information form from Analog to Digital).

Digitization costly but necessary for human progress.

Appropriate action by individuals / organization needed.

Technical Approaches to Archiving Open Archival Information System (OAIS) Published by consultative committee for space data system (CCSDS) 2002 ISO 14721 : 2003 standard An archive consists of organization of people and systems with responsibility to preserve information and make it available to users.

Open Archival Information System (OAIS)

Published by consultative committee for space data system (CCSDS) 2002

ISO 14721 : 2003 standard

An archive consists of organization of people and systems with responsibility to preserve information and make it available to users.

Technical Approaches to Archiving Three key players of OAIS model in archiving are : OAIS MODEL Creator Archive Operator Designated user community

Three key players of OAIS model in archiving are :

Creator : Some of players are Authors Copyright owners Professional societies Publishers Distributors With so many players rights of ownership and responsibility to provide content to archive needs to be addressed.

Authors

Copyright owners

Professional societies

Publishers

Distributors

Archive Operator Collection policy needs attention (Based on topic / subject / user community addressed) Boundary of archiving activity needs to be defined. E.g. National library of New Zealand collects electronic material using PUSH / PULL Business Process & online publication with purview copyright and community licensing agreements.

Collection policy needs attention (Based on topic / subject / user community addressed)

Boundary of archiving activity needs to be defined.

Users Archiving should consider needs of specified user community. Changing needs of user community Change of users with time Issue of access to archive, domain of material (Private vs. Public) needs to be addressed specifically. Responsive and automated access control system required to facilitate use and save archive to problems arising out of user and third partly interaction.

Archiving should consider needs of specified user community.

Changing needs of user community

Change of users with time

Issue of access to archive, domain of material (Private vs. Public) needs to be addressed specifically.

Responsive and automated access control system required to facilitate use and save archive to problems arising out of user and third partly interaction.

Tools for quality of Digital Objects Preservation Web Services Architecture for New Media and Interactive Collections (PANIC), Australia Preserving Access to Digital Information (PADI), the National Library of Australia PRONOM Project, National Archives, UK Preservation and Long-Term Access via Networked Services (PLANETS), European Union

Preservation Web Services Architecture for New Media and Interactive Collections (PANIC), Australia

Preserving Access to Digital Information (PADI), the National Library of Australia

PRONOM Project, National Archives, UK

Preservation and Long-Term Access via Networked Services (PLANETS), European Union

Web Archiving Started in 1996 to tap wealth of information on web before they vanish. Special software products i.e. web crawlers / spiders employed. They have limitations like non-coverage of some hidden content, unable to interpret certain codes for links and long time to retrieve archived web pages.

Started in 1996 to tap wealth of information on web before they vanish.

Special software products i.e. web crawlers / spiders employed.

They have limitations like non-coverage of some hidden content, unable to interpret certain codes for links and long time to retrieve archived web pages.

Strategies for Web Archiving Making a local copy of the files of a web site and navigating through this copy Running a web server to make accessible the contents to the user Reorganizing documents in a non-web manner Examples: - Minerva, U.S. Library of Congress - Way Back Machine , Internet Archive, USA

Making a local copy of the files of a web site and navigating through this copy

Running a web server to make accessible the contents to the user

Reorganizing documents in a non-web manner

Metadata and Digital Preservation Creation and maintenance of metadata required for long term preservation of digital information. Within Archive Metadata (maintained / migrated format to format & standard to standard independent of objects it describes). Refers each digital object and provides Descriptive Structural Administrative Rights Management Other information

Creation and maintenance of metadata required for long term preservation of digital information.

Within Archive

Metadata (maintained / migrated format to format & standard to standard independent of objects it describes).

Refers each digital object and provides

Descriptive Structural Administrative Rights Management Other information

Metadata and Digital Preservation Metadata preservation is popular area of research in last decade. Research Library Group (RLG) at NLA has developed model to identify entities that supports its PANDORA (Preserving & Accessing Networked Documentary Resources of Australia) to support archive concept.

Metadata preservation is popular area of research in last decade.

Research Library Group (RLG) at NLA has developed model to identify entities that supports its PANDORA (Preserving & Accessing Networked Documentary Resources of Australia) to support archive concept.

OAIS Model and Metadata Source: Day, M.,

OAIS and Metadata Implementation NEDLIB Project National Library of Netherlands DSEP (Deposit System for Electronic Publication) CURL (Consortium of Univ. Res. Libraries of Cambridge, Leeds & Oxford). PREMIS (Preservation Metadata : Implementation Strategies Working Group) sponsored by OCLC / RLG.

NEDLIB Project National Library of Netherlands

DSEP (Deposit System for Electronic Publication)

CURL (Consortium of Univ. Res. Libraries of Cambridge, Leeds & Oxford).

PREMIS (Preservation Metadata : Implementation Strategies Working Group) sponsored by OCLC / RLG.

IIPC (International Internet Preservation Consortium) developed archival metadata scheme) PID (Persistent identifier) assigns unique name to a digital object that points to bibliographic record. URN (Uniform Resource Name) refers to identifiers. OAIS and Metadata Implementation

IIPC (International Internet Preservation Consortium) developed archival metadata scheme)

PID (Persistent identifier) assigns unique name to a digital object that points to bibliographic record.

URN (Uniform Resource Name) refers to identifiers.

IPR & Copyright Different countries are addressing issues are IPR and copyright with respect to digital preservation. National Libraries responsible for archiving national domain. Examples : Library of Congress : best practices for digital preservation Koninklyke Bibliotheek (KB), the National Library of The Netherlands : individual, collective and the ‘opt-out’ approach to archiving scholarly journals UK, Legal Deposit Libraries Act 2000 LOCKSS

Different countries are addressing issues are IPR and copyright with respect to digital preservation.

National Libraries responsible for archiving national domain.

Examples :

Library of Congress : best practices for digital preservation

Koninklyke Bibliotheek (KB), the National Library of The Netherlands : individual, collective and the ‘opt-out’ approach to archiving scholarly journals

UK, Legal Deposit Libraries Act 2000

LOCKSS

Data Archiving “ ...is primarily a program of practices and procedures that support the collection, long-term preservation and low-cost access to & dissemination of scientific and technical data…” Some Initiatives: International Council of Scientific Unions (ICSU)/CODATA Group on Data and Information World Meteorological Organization (WMO) World Intellectual Property Organization (WIPO) Creative Commons - The Research Index - Public Commons Project

“ ...is primarily a program of practices and procedures that support the collection, long-term preservation and low-cost access to & dissemination of scientific and technical data…”

Some Initiatives:

International Council of Scientific Unions (ICSU)/CODATA Group on Data and Information

World Meteorological Organization (WMO)

World Intellectual Property Organization (WIPO)

Creative Commons

- The Research Index

- Public Commons Project

International Initiatives USA The National Digital Information Infrastructure and Preservation Program (NDIIPP) by the Library of Congress with research focus on: (i) Technical infrastructure and building a preservation layer (ii) Building core capacity and organizational issues (iii) Long-term research program

International Initiatives UK The Continuing Access and Digital Preservation Strategy of the Joint Information Systems Committee (JISC) with Main objectives: Establish best practice and guidelines Generate support and collaborative funding Develop a long-term strategy Areas covered: Collecting and preserving the World Wide Web Legal issues related to internet archiving Archiving e-journals UK LOCKSS Data curation for e-Science Preservation of e-Prints Survey and assessment of information resources Study of records lifecycle

UK

The Continuing Access and Digital Preservation Strategy of the Joint Information Systems Committee (JISC) with

Main objectives:

Establish best practice and guidelines

Generate support and collaborative funding

Develop a long-term strategy

Areas covered:

Collecting and preserving the World Wide Web

Legal issues related to internet archiving

Archiving e-journals

UK LOCKSS

Data curation for e-Science

Preservation of e-Prints

Survey and assessment of information resources

Study of records lifecycle

International Initiatives Germany The Network of Expertise in Long-term Storage of Online Resources (nestor) Kopal Digital Preservation Europe @ http://www.digitalpreservationeurope.eu

Germany

The Network of Expertise in Long-term

Storage of Online Resources (nestor)

Kopal

Digital Preservation Europe

@ http://www.digitalpreservationeurope.eu

The Indian Scenario Initiative for information organization and management by Govt. and NGOs Library Initiatives INFLIBNET (UGC) : e-Journal consortia, retro-conversion, web-site hosting, etc. Indian National Digital Library in Engineering Sci & Tech (INDEST) by MHRD. Proposal : (i) Consortia agreements incorporate compulsory option of ‘archiving’ scholarly journals (ii) Consortia efforts for web archiving initiatives

Initiative for information organization and management by Govt. and NGOs

Library Initiatives

INFLIBNET (UGC) : e-Journal consortia, retro-conversion, web-site hosting, etc.

Indian National Digital Library in Engineering Sci & Tech (INDEST) by MHRD.

Proposal :

(i) Consortia agreements incorporate compulsory option

of ‘archiving’ scholarly journals

(ii) Consortia efforts for web archiving initiatives

The Indian Scenario Digital Library Initiatives Digital Library of India (DLI) to digitize books in Indian languages. The Traditional Knowledge Digital Library (TKDL) by CSIR. NISCAR by CSIR, MST and Department of AYUSH Proposal : OAIS model for digital preservation

Digital Library Initiatives

Digital Library of India (DLI) to digitize books in Indian languages.

The Traditional Knowledge Digital Library (TKDL) by CSIR.

NISCAR by CSIR, MST and

Department of AYUSH

Proposal :

OAIS model for digital preservation

The Indian Scenario Organizations involved in CMD Institutional Repositories Eprints of the Indian Institute of Science <http://etc.ncsi.ernet.in> National Chemical Laboratory <http://dspace.ncl.res.in/dspace/index.jsp> Proposal : Facilitate the option of digital preservation in the software

Organizations involved in CMD

Institutional Repositories

Eprints of the Indian Institute of Science <http://etc.ncsi.ernet.in>

National Chemical Laboratory

<http://dspace.ncl.res.in/dspace/index.jsp>

Proposal :

Facilitate the option of digital preservation in the software

The Indian Scenario Social Science Data -- The Census of India -- M.S.Swaminathan Research Foundation Proposal : Centralized Data Managament and Preservation Strategy Museums and Art Galleries -- Ministry of Culture, GOI -- The National Archives

Institutions involved in Data Production and Storage Scientific and Technical Data Oceanic data Ministry of Ocean Development (MOD) Meteorological and seismic data Indian Meteorological Department (IMD) Satellite imageries, land use and wasteland maps on different scales National Remote Sensing Agency (NRSA) Geological maps on various scales, geological and seismic data Geological Survey of India (GSI) Topographical maps, geodetic trigonometric and levelling data, gravity & geomagnetic data, GPS data, tidal data, repetitive geodetic & geophysical data Survey of India (SOI) Soil maps and land use data National Bureau of Soil Survey and Soil Maps Command area maps Central Water Commission Resource Institute

A Proposal Relevant to Geomagnetism Huge number of photographic magnetograms are kept at Alibagh (1901-1920) and at different World Data Centers (WDCs) under ICSU. 50 years after International Geophysical Year (IGY, 1957-1958), Electronic Geophysical Year (eGY, 2007-2008) is planned to digitize old magnetograms. Foster International Co-operation in digitizing old magnetograms because aged films are brittle and cracking.

Huge number of photographic magnetograms are kept at Alibagh (1901-1920) and at different World Data Centers (WDCs) under ICSU.

50 years after International Geophysical Year (IGY, 1957-1958), Electronic Geophysical Year (eGY, 2007-2008) is planned to digitize old magnetograms.

Foster International Co-operation in digitizing old magnetograms because aged films are brittle and cracking.

A Proposal Relevant to Geomagnetism Digital Preservation for longer life and make them amenable to computational analysis. The mission is universal (equal) access by everyone having an internet connection and a computer thus cut down the travel cost and time of researchers. Encouraging library professionals to address issues related to preservation and intellectual property right (IPR) and allied issues.

Digital Preservation for longer life and make them amenable to computational analysis.

The mission is universal (equal) access by everyone having an internet connection and a computer thus cut down the travel cost and time of researchers.

Encouraging library professionals to address issues related to preservation and intellectual property right (IPR) and allied issues.

A Proposal (General) Maintaining digital material by building institutional repositories and digital libraries. Providing adequate financial and technical resources for establishing certification system. Archiving and maintaining access to S & T data with the help from ICSU / CODATA. National Informatics Center (NIC) entrusted with framing guidelines and policy. or establishing a new agency For handling digital preservation, for collaboration, sharing and avoiding duplication. Trusted Digital Repository for accurate and reliable information.

Maintaining digital material by building institutional repositories and digital libraries.

Providing adequate financial and technical resources for establishing certification system.

Archiving and maintaining access to S & T data with the help from ICSU / CODATA.

National Informatics Center (NIC) entrusted with framing guidelines and policy.

or establishing a new agency

For handling digital preservation, for collaboration, sharing and avoiding duplication.

Trusted Digital Repository for accurate and reliable information.

Legally sustainable digital preservation policy. Joining the Digital Preservation Consortium. Attention to collection management of digital material in libraries. Amendment of the Delivery of Books Act and Press and Registration Act to cover the digital material. Training of manpower for the management and preservation of electronic records. Research in the area of digital preservation. A Proposal (General)

Legally sustainable digital preservation policy.

Joining the Digital Preservation Consortium.

Attention to collection management of digital material in libraries.

Amendment of the Delivery of Books Act and Press and Registration Act to cover the digital material.

Training of manpower for the management and preservation of electronic records.

Research in the area of digital preservation.

Conclusion Implementation of suitable preservation strategy for digital data and documents is required for our country. Best practices for digital preservation in India can be established by combining OIAS model and trusted digital depository. Collaborative efforts and sharing information will help preservation rather than duplication. Experienced LIS professionals can play a major role in preserving the library material and handling the IPR like allied issues.

Implementation of suitable preservation strategy for digital data and documents is required for our country.

Best practices for digital preservation in India can be established by combining OIAS model and trusted digital depository.

Collaborative efforts and sharing information will help preservation rather than duplication.

Experienced LIS professionals can play a major role in preserving the library material and handling the IPR like allied issues.

Thank You