Electronic Records Preservation: ERA Research and Development Lagring – Arkivering Två världar – en vardag? 13 October 2008 Mark Conrad ERA Research National Archives and Records Administration

NARA’s Mission To ensure access to records of three branches of the U.S. Government. Records that: Protect citizen’s rights Hold Government officials accountable Facilitate historical understanding of our national experience

To ensure access to records of three branches of the U.S. Government. Records that:

Protect citizen’s rights

Hold Government officials accountable

Facilitate historical understanding of our national experience

Electronic Records NARA’s Challenges Scope The entire U.S. Federal Government Obsolescence Constantly Changing Technology Access Ability to view records over time Volume Large amounts of records arriving to NARA Variety Different/Complex Types of Records Complexity and Records Formats

Scope The entire U.S. Federal Government

Obsolescence Constantly Changing Technology

Access Ability to view records over time

Volume Large amounts of records arriving to NARA

Variety Different/Complex Types of Records

Complexity and Records Formats

ERA Program History 1970 First electronic records transferred to NARA, begin preservation

1970 First electronic records transferred to NARA, begin preservation

Transfers of Digital Files to NARA (1970 – 1995)

Transfers of Digital Files to NARA: Reagan/Bush Presidential Records

Transfers of Digital Files to NARA: Clinton Presidential Records

ERA Program History (cont.) 1970 First electronic records transferred to NARA, begin preservation 1998 NARA is heading for mission failure

1970 First electronic records transferred to NARA, begin preservation

1998 NARA is heading for mission failure

NARA’s Strategic Response: The ERA Program Research and exploratory development on technologies that offer promise for addressing electronic records challenges. Acquiring and building a system that meets our requirements and our mission for NARA, the Presidential Libraries, and Federal Records Centers Organizational and cultural Change Management

Research and exploratory development on technologies that offer promise for addressing electronic records challenges.

Acquiring and building a system that meets our requirements and our mission for NARA, the Presidential Libraries, and Federal Records Centers

Organizational and cultural Change Management

ERA Program History 1998 ERA Research Begins Understand the Issues Feasibility Relevant Technologies

1998 ERA Research Begins

Understand the Issues

Feasibility

Relevant Technologies

ERA Program History 2000 ERA becomes an Official Program 2003 ERA Requirements Issued http://www.archives.gov/era/pdf/requirements-amend0001.pdf Based on ERA Research Some Standards Referenced OAIS DoD 5015.2

2000 ERA becomes an Official Program

2003 ERA Requirements Issued

http://www.archives.gov/era/pdf/requirements-amend0001.pdf

Based on ERA Research

Some Standards Referenced

OAIS

DoD 5015.2

ERA Program History 2008 ERA IOC 2008 - ERA Research Continues No one knows how to preserve and provide sustained access to authentic electronic records for most types of electronic records No one knows what information technology will be in the future

2008 ERA IOC

2008 - ERA Research Continues

No one knows how to preserve and provide sustained access to authentic electronic records for most types of electronic records

No one knows what information technology will be in the future

ERA Research Program “ Next-generation methods, technologies, and tools are needed to...manage massive stores of distributed, heterogeneous information (e.g., science and engineering research data, Federal records , health information).

Human Computer Interaction and Information Management (HCI&IM) NITRD Agencies: NSF, DARPA, OSD and DoD Service research organizations, NIH, NASA, NIST, AHRQ, NOAA, EPA, NARA President’s 2009 Request Strategic Priorities Underlying This Request Today’s increasingly data-centric world requires the effective and strategic use of information assets. To advance the role of HCI&IM in providing strategic support for national priorities, R&D in this area focuses on: Information integration: To support complex human ideas, analysis, and timely decision-making, large amounts of disparate forms of raw information must be managed, assimilated, and accessible in formats responsive to the user needs. Next-generation methods, technologies, and tools are needed to fully integrate and efficiently manage massive stores of distributed, heterogeneous information (e.g., science and engineering research data, Federal records , health information) Key research issues include: – Information standards: Data interoperability and integration of distributed data; usability; provenance and integrity (metadata); generalizable ontologies; accessibility – Decision support: Timeliness of and access to data; user-oriented techniques and tools for summarization , synthesis, analysis, and visualization of information for decision-making; measurement and management of human responses to data – Information management (IM): Intelligent rule-based data management , efficient integration, maintenance, and access to complex, large-scale collections of heterogeneous data; innovative systems architecture; scalable technologies ; integration of policies ( differential sensitivity , security, user authentication) with data; integrated distributed data repositories ; testbeds ; sustainability and validation of complex models

Some examples of ERA Research Partnerships Data-Intensive Cyber Environments (DICE) Group NIST Army Research Laboratory National Science Foundation

Fundamental Requirements for The ERA System Evolvability Scalability Extensibility

Evolvability

Scalability

Extensibility

ERA Research Data Intensive Cyber Environments Group Transcontinental Persistent Archives Prototype (TPAP) intelligent Rule Oriented Data System (iRODS) Make Policies/Rules Explicit http://www.irods.org

Data Intensive Cyber Environments Group

Transcontinental Persistent Archives Prototype (TPAP)

intelligent Rule Oriented Data System (iRODS)

Make Policies/Rules Explicit

http://www.irods.org

ERA Research A Sample of Continuing Collaborations ARL – High Confidence Computing NCSA – Storage and Retrieval of 3D+Time Data Representations NSF – Cyberinfrastructure Pittsburgh Supercomputing Center - SLASH

A Sample of Continuing Collaborations

ARL – High Confidence Computing

NCSA – Storage and Retrieval of 3D+Time Data Representations

NSF – Cyberinfrastructure

Pittsburgh Supercomputing Center - SLASH

Building for the Future Anticipate changes characteristics of electronic records, Preservation and Access technologies Researcher expectations and behaviors Recognize those things that will not or should not change Archival science provides stable principles, concepts, requirements and understanding. NARA’s mission and the functions Make reasonable assumptions about the future Use of computers will continue to become more common Information use and Technology will continue to grow Decline in Information Technology costs The Internet will continue to grow

Anticipate changes

characteristics of electronic records,

Preservation and Access technologies

Researcher expectations and behaviors

Recognize those things that will not or should not change

Archival science provides stable principles, concepts, requirements and understanding.

NARA’s mission and the functions

Make reasonable assumptions about the future

Use of computers will continue to become more common

Information use and Technology will continue to grow

Decline in Information Technology costs

The Internet will continue to grow

ERA Research [email_address] The ERA Web site is: http://www.archives.gov/era Your Contact in the ERA Program Office