This document discusses challenges and best practices for archiving qualitative data from policy research and evaluations. It describes the Irish Qualitative Data Archive, which archives several major datasets. The RACcER project explored issues around archiving community-based evaluation data. Interviews with researchers found a lack of archiving policies, challenges anonymizing small communities, concerns about consent and data access, and risks to reputation. However, benefits include exploring new research questions and increasing knowledge. The document outlines a process for archiving data from the Tallaght West CDI involving consultation, consent, anonymization, and restricting access depending on sensitivity level.

1. Archiving qualitative data for
policy research: meeting
challenges and establishing
best practices
Aileen O’Carroll
Manager, Irish Qualitative Data
Archive

2. Irish Qualitative Data Archive
• Funded by the Irish Government under the Programme for
Research in Third Level Institutions (Cycle 4)
• Several major datasets being archived, including –
• Growing up in Ireland study
• Life Histories and Social Change in Ireland in 20th Century Ireland
• Re-use & Archiving of Complex Community-based
Evaluation Research (RACcER) project explores issues
specific to archiving community-based evaluation data,
also aims to raise awareness throughout research
community

3. Tallaght West Childhood
Development Initiative (CDI)
1 of 3 Prevention and Early Intervention
Programmes jointly funded by government &
philanthropy
Aims to improve the education and well-being
outcomes for children and families in historically
disadvantaged areas of Tallaght West
5 constituent projects: early years, literacy, pro
social behaviour, healthy schools, community
safety
Major emphasis on evaluation

4. RACcER
RACcER co-funded by IRCHSS and CDI: demonstrator
project for qualitative archiving
Objectives:
Scope concerns, issues and requirements in qualitative
data archiving through interviews
Explore issues and challenges for researchers, participants,
funders, potential data users
Test feasibility of archiving evaluation data
Develop protocols and procedures for archiving evaluation
data
Research Process
Consultation > Pilot Strategy > Consultation > Archiving process

5. Findings – qualitative data gathered in
evaluations
Range of qualitative material collected
Variety of participant groups
Most material recorded, anonymised and
transcribed

6. Findings – archiving policy
Evaluators’ research centres didn’t generally have
archiving policies for qualitative material
Range of experience of what happens to
qualitative data after project
No evidence of researchers sharing datasets
informally
Default ethics position is to destroy after 5 years

7. Findings – archiving experience
No evidence of formal archiving
Limited experience of accessing qualitative
archives,
One example for use in teaching
Some had archived quantitative material
Some had accessed quantitative archives
Range of views on resource implications of
prepare qualitative material for archiving

8. Findings – issues with archiving
Difficulties of anonymising in small community
“Just to follow on really from that, archiving
to do what really? Archiving to make
something as a resource that can be used
again. In the context of some of my own
work, to anonymise it sufficiently is to
make it meaningless. ”

9. Findings – issues with archiving
“There’s interconnections, there’s politics, and there’s
alliances, and there’s you know, sort of, tensions,
between people and characters and actors and you
know there’s a finite cast of characters. I think in that
context, it is challenging about how anonymise that
work and then allow it to be available and maybe you
can’t, I don’t know, maybe that’s not possible”.
“I think there are enormous issues, if you talk about
archiving do you anonymise transcripts to make... so
then you do all this anonymisation activity, you invest
time and money in it, and what you come out with is
nothing. If you remove all the contextual information
and so on. Then you’ve got a set of data which would
essentially be akin to what was in a report. ”

10. Findings – issues(Consent)
“…. could you actually be happy that you would have
informed consent from somebody? That they actually fully
understand what it means to have this available and what
the implications are. Because, to a degree, you don’t
actually know what the implications are.”
“If we were to put on our consent forms or our information
sheets that data generated from interviews will be
available, anonymised, as part of an archive, we’re
screwed. That’s basically it. Sorry for using such
unparliamentary language, we would not get responses,
not in a country like ours. ”

11. Findings – issues(3)
Wide perception of fundamental difference
between evaluation research and broader
topics
“we’re generally speaking about evaluation and evaluation is
different. It’s not asking people whether the sky is blue or
green or how you perceive that, you’re talking about
somebody else’s actions or activities and putting some,
essentially, value on those in what you say. So therefore,
there’s another level that you have to add into the mix.”

12. Findings – issues(4)
Lack of clarity on who can access data and how
it could be used generates fear and reduces
inclination to archive
Concern over data being used out of context,
‘twisted’, sensationalising populations, journalists
gaining access

13. Findings – issues(5)
Researchers concerned about being opened up to
scrutiny
PhD/masters students using archived material
could reduce valuable learning opportunities
gained from fieldwork
Resource implication of archiving

14. Findings – advantages
Potential to explore alternative research
questions
Contributing to cumulative knowledge of
implementation of children’s services
Facilitate comparative analysis over time or
place

15. Findings – advantages (2)
A valuable teaching resource
Increase the standing of qualitative
research if it is open to external
examination, improved validity and
reliability
Contribute to corporate memory of
commissioning organisation

16. Findings - risks
Reputational risk for organisations involved at
all levels
Negative publicity risk for researched
community
User agreement could be breached
Anonymity could be compromised –
But...
Need to consider likelihood of risk occurring and
how serious it would be if it happened and
develop approperiate archiving strategies

17. Findings – mitigating risks
Strict controls on user access
Archive to facilitate collaboration between
researchers
Hierarchy of access, tailored depositor and user
agreements

18. Process of archiving CDI
datasets
Consultation Building Data Data
Data
With Anonymisation Costing
Collection
CDI Board Informed Consent

19. Accessing Sensitivity level
of the data
Risk of Identification Risk Of Harm Sensitivity Level
Little Low Low
Some Low Medium
Any High High

20. Archiving Strategy Appropriate
to the Level of Sensitivity
Participant Anon- Access Remove Embargo 30 Year
Sensitivity Consent ymised Sensitive (Anonymised Embargo
Level Text Sensitive (Sensitive
Text Text Not
Removed) Removed
Low Yes Yes Restricted N/A No No
Medium Yes Yes Restricted No No No
High Yes Yes Restricted Yes No No

21. Thank You
RACcER Team
Irish Qualitative Data Archive
Jane Gray
Julius Komolafe
Hazel O’Byrne
Aileen O’Carroll
Tallaght West CDI
Tara Murphy