1.
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
ISO standards and Audit & Certification
Barbara Sierman, KB National Library of the Netherlands
Dasish Meeting 17-10-2014, The Hague

2.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification: introduction
–History of the standards for audit and certification
–The ISO standards 16363 and 16919
–The APARSEN test audits
–Final remarks and further reading

3.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification: what is it and what not
•Audit:
planned and documented investigation by an independent qualified group of the compliance of an organization against a certain standard
Not a simple Yes or No, but recommendation for improvements
•Certification:
Confirmation that organization meets the requirements of the standard to which it is audited
Temporarily: regularly revised

4.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The history
: Infrastructure and Security Risk Management
2002
•OAIS ISO 14721 published (updated 2012)
•Par. 1.5: standard(s) for accreditation of archives.
2005
•Checklist for Certification of Trusted Digital Repositories (RLG/NARA)
•Testaudits performed by RLG
2007
•DRAMBORA (2007), NESTOR (2006)
•Trusted Repositories Audit and Certification final report.
•(Input for Repositories Audit and Certification Working Group (RAC-WG)
2012-
•ISO 16363 Audit and Certification of Trustworthy Digital Repositories (RAC-WG)
•Draft ISO 16919 Requirements for bodies providing Audit and Certification for candidate trustworthy repositories (RAC- WG)
•Primary Trustworthy Digital Repository Authorisation Body (PTAB)

5.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification : European Framework
3 Levels of Certification
•Basic Certification (based on DSA)
•Extended Certification (self-assessment based on DSA plus self-audit based on ISO 16363 or DIN 31644)
•Formal Certification (self-assessment based on DSA plus full external audit of ISO 16363 or DIN 31644)
This Framework is supported by and coordinated with the help of the European Commission

6.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363
•ISO 16363- 2012 Audit and Certification of Trustworthy Digital Repositories
: Infrastructure and Security Risk Management
Organisational Infrastructure
Digital Objects Management
Infrastructure and Security Risk Mgmt.
Metrics
•Statement of requirement
•Supporting text
•Examples: repository demonstrates it is meeting this requirement
•Discussion

7.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363
: Infrastructure and Security Risk Management
The ISO standard follows the chapters in TRAC and distinguished 3 areas of auditing:
Organisational Infrastructure
Digital Objects Management
Infrastructure and Security Risk Management
as this one is also dealt with in other IT-related standards, only the specific requirements in relation to preservation are mentioned.
The standard consitis of metrics
In comparison with the TRAC document the explanation of the “ metrics” is extended
Statement of requirement
Supporting text
Examples: repository demonstrates it is meeting this requirement
Discussion part to explain various points of view in relation to the statement or “metric”
•TRAC 2005
•TRAC 2007
•ISO 16363

8.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363: example
: Infrastructure and Security Risk Management
The ISO standard follows the chapters in TRAC and distinguished 3 areas of auditing:
Organisational Infrastructure
Digital Objects Management
Infrastructure and Security Risk Management
as this one is also dealt with in other IT-related standards, only the specific requirements in relation to preservation are mentioned.
The standard consitis of metrics
In comparison with the TRAC document the explanation of the “ metrics” is extended
Statement of requirement
Supporting text
Examples: repository demonstrates it is meeting this requirement
Discussion part to explain various points of view in relation to the statement or “metric”
Metric: “3.3.1 The repository shall have defined its Designated Community and associated knowledge base(s) and shall have these definitions appropriately accessible”
Evidence: “A written definition of the Designated Community. “
Discussion:

9.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363: example
: Infrastructure and Security Risk Management
The ISO standard follows the chapters in TRAC and distinguished 3 areas of auditing:
Organisational Infrastructure
Digital Objects Management
Infrastructure and Security Risk Management
as this one is also dealt with in other IT-related standards, only the specific requirements in relation to preservation are mentioned.
The standard consitis of metrics
In comparison with the TRAC document the explanation of the “ metrics” is extended
Statement of requirement
Supporting text
Examples: repository demonstrates it is meeting this requirement
Discussion part to explain various points of view in relation to the statement or “metric”
Metric: 3.3.2 The repository shall have Preservation Policies in place to ensure its Preservation Strategic Plan will be met.
Evidence: Preservation Policies; Repository Mission Statement.
Discussion:

10.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363
•ISO 16363- 2012 Audit and Certification of Trustworthy Digital Repositories
•Guidance for auditors
•Other standards also applicable (security)
•Dependent on auditors experience
Consistency!

11.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16919
•ISO : standards of good auditing practices , accreditation of auditors
•Basis: ISO/IEC 17021
–Standard requirements for A&C general management systems
–Adapted for Trustworthy Digital Repositories (TDR)
Reference to OAIS
Reference to ISO 16363 as the set of criteria
Dealing with sensitive collection / confidentiality
List of competencies (normative in annex)
•PTAB group created new standard
•ISO 16919-2014 Requirements for bodies providing Audit and Certification for candidate trustworthy digital repositories

12.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16919
Process of accreditation
ISO
CASCO: Committee on Conformity Assesment: advice
IAF: International Accreditation Forum
Assessors, Training/Accreditation Group
National standards bodies
Monitoring & Approving

13.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16919

14.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
What to expect from an auditor?
In general:
•Impartiality,
•Competence,
•Responsibility,
•Openness,
•Confidentiality,
•Responsiveness to complaints

15.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
What to expect from an auditor?

16.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The APARSEN test audits: what?
“Trust” is one of the pillars in APARSEN
2011: Testing of practical use of (draft) standards
•Metrics understandable and usable
•How much effort and time is needed for a repository
•Consistency in evaluation of the evidence
•Is the standard ISO 16363 applicable on different kind of repositories?

17.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The APARSEN test audits
Europe
Data Archiving and Networked Services (DANS),
UK Data Archive (UKDA),
Centre Informatique National de l’Enseignement Supérieur:
Département Archivage et Diffusion (CINES-DAD, France),
German National Library (DIN 31644 standard)
United States
Socio-economic Data and Applications Center (SEDAC),
National Space Science Data Center (NSSDC)
Kentucky Department for Libraries and Archives (KDLA).
International Group of “test-auditors”
Members of the RAC-WG

18.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test Audit preparations
How much time will it take?
•Greater effort than expected to prepare the audit. Preparation varied between 1.5 to 3 months
•Time spent on:
–Internal discussions about the standard
–Writing documentation that was not there yet
–Collecting existing documentation
–Improving existing documentation
•“Difficult to evaluate level of compliance”

19.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test Audit procedure
•Expectations document: test-audit!
•Two Stages:
–1. Repositories completed a Self-Audit template (Checklist based on 16363)
Checklist plus documentation returned to audit team to prepare audit
–2. Site visit (2 days)
Verbal feedback with first impressions
Detailed audit report: areas for improvement

20.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test audits: benefit quotes
Benefits as stated in the APARSEN report:
•DNB: “to have their own processes and documentation reviewed, scrutinized, and ideally approved by some external professionals. “
•DANS: “it sheds a clear light on what the strengths and the weaknesses are in the archiving activities of our institute. It gave us confidence that we are well on our way to fulfil the requirements.
•CINES-DAD : [it] certainly helped them to evaluate the progress made since the previous audits and the relevance of the actions taken over the past couple of years

21.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audits: benefits for organisations
•3rd Party view of qualified people
•Better understanding of requirements
•Identification of areas for improvement
•Incentive to take action

22.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test audits: benefits for organisations
•In line with report of 4C project:
–“To improve work processes
–To meet contractual obligation
–Publicly understandable statement of quality and reliability”
•In line with experience self-assesment SB Denmark:
–Improvement common vision organisation
–Competency development
–Organisational awareness digital preservation
–Good overview available documentation

23.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification : costs
•Factor costs often discussed
•4C project showed:
–The only figures we have are of the APARSEN test audits
–Distinguish
Procurement of standards (preparation)
Staff costs
Certification costs
•Audit and certification will cost time and money

24.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification : risks
•Digital preservation is pioneering area
•Need for qualified auditors
•Growth path in audit and certification

25.
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Further information
•APARSEN : Report on peer review of Digital Repositories http://bit.ly/1jxRorz
•4C project on audit & certification: http://bit.ly/1yGDpvc
•iPRES 2014 G. Elstroem & J. Junge:
Self-assessment of the Digital Repository at the State and University Library, Denmark - a Case Study
•Blogposts David Rosenthal about recent TRAC audit http://bit.ly/1vyLzEI
•PTAB group: http://www.iso16363.org/
– News and updates about these standards
– Self-Assessment Template