This presentation is based on an analysis of 248 impact case studies citing support from the UK's National Institute for Health Research (NIHR) submitted to REF2014. It provides a high-level view of the features of this research, and categorises a number of levels at which they provide evidence of wider societal impacts.

1. From singing clubs, to newborn screening, to stroke care:
an analysis of the impact of research supported by the
UK National Institute for Health Research (NIHR)
Adam Kamenetzky, Saba Hinrichs-Krapels, Steven Wooding, Jonathan Grant
INRA, Paris 3 November 2015

2. Background (1)
UK REF2014 was the largest research impact assessment
exercise of its kind:
- £1.6 billion annual public funding
- 154 Universities
- >52,000 research staff
- 6,975 impact case studies
Higher Education Funding Council for England impact definition:
‘an effect on, change or benefit to the economy, society, culture, public policy
or services, health, the environment or quality of life, beyond academia’

3. Background (2)
What does UK research impact look like?
(King’s College London
and Digital Science 2015)

4. Aims & methods

5. Aims & methods (1)
Largest supporter of
clinical research in the UK:
“…to improve the health
and wealth of the nation.”
Aims:
1) How was NIHR support referred to in REF case studies?
2) Describe the features of this research
3) Curate a database that NIHR could use to develop narrative and use
for advocacy purposes

6. Aims & methods (2)
We compiled a list of named sources of NIHR research support
using data from NIHR’s website:
(http://www.nihr.ac.uk)

7. Aims & methods (3)
National Institute for Health Research (NIHR) OR
National Institute of Health Research
Research Programmes
Efficacy and Mechanism Evaluation (EME)
Health Service and Delivery (HS&DR) OR
Service Delivery and Organisation (SDO)
Health Technology Assessment (HTA)
Health Technology Devices (HTD)
Methodology Research Programme
Invention for Innovation (i4i)
Programme Grants for applied Research (PGfAR)
Programme Development Grants (PDG)
Public Health Research (PHR)
Research for Patient Benefit (RfPB)
Systematic Reviews Programme (SRP)
Centre for Reviews and Dissemination (CRD)
Cochrane Review Group (CRG)
Technology Assessment Reviews (TAR)
Research Schools
School for Primary Care Research (SPCR)
School for Public Health Research (SPHR)
School for Social Care Research (SSCR)
Other Programme Work
Blood and Transplant Research Unit (BTRU)
Centre for Surgical Reconstruction and Microbiology
(CSRM)
Clinical Trials Units (CTU)
Health Protection Research Units (HPRU)
Horizon Scanning Centre (HSC)
James Lind Alliance Priority Setting Partnership (PSP)
Research Design Service (RDS)
Cochrane Centre (UKCC)
Clinical Research Units
Office for Clinical Research Infrastructure (NOCRI)
Translational Research Partnership (TRP)
Biomedical Research Centre (BRC)
Biomedical Research Unit (BRU)
Clinical Research Facilities (CRF)
Experimental Cancer Medicine Centre (ECMC)
Translational Research Collaboration (TRC)
Patient Safety Translational Research Centres (PSTRC)
Healthcare Technology Co-operatives (HTC)
Diagnostic Evidence Co-operatives (DEC)
Collaborations for Leadership in Applied Health Research
and Care (CLAHRC)
MRC/NIHR Phenome Centre
Coordinating centres
Evaluation, Trials and Studies Coordinating Centre
(NETSCC)
Trainees Coordinating Centre (TCC)
Central Commissioning Facility (CCF)
Clinical Research Network Coordinating Centre (CRN CC)
Supporting NIHR Infrastructure
BioResource Centre
BioSample Centre
Join Dementia Research (JDR)
Health Informatics Collaborative (HIC)
NIHR investment into data and informatics
Clinical Practice Research Datalink (CPRD)
Clinical Records Interactive Search (CRIS)
Dementia Clinical Records Interactive Search (D-CRIS)
Other schemes where NIHR contributes funding
Flexibility and Sustainability Funding (FSF)
Clinical and Translational Radiotherapy Research Working
Group (CTRad)
Supportive and Palliative Care Collaboratives (SuPaC)
National Prevention Research Initiative (NPRI)
National Awareness & Early Diagnosis Initiative (NAEDI)
Dermatology Clinical Trials Network (DCTN)
Health Challenge Innovation Fund

8. Aims & methods (4)
We combined the names of these NIHR support streams into
Boolean search strings, and used these to query the publicly-
available REF impact case study database:1
(http://impact.ref.ac.uk/CaseStudies/search1.aspx)

9. Aims & methods (5)
We ran searches for mentions of NIHR support streams within
the references to the research (section 3) or details of the impact
(section 4) sections of the case studies.
We also ran searches for DH/NIHR grant reference numbers for
awards made since 2004, to broaden the pool of potential case
studies included for analysis.
This resulted in a pool of 371 case studies identified as potentially
of relevance to NIHR.
Following rapid read-through, 248 included for further analysis.

10. Aims & methods (6)
We read each of the remaining 248 case studies in some detail
and coded them according to a framework adapted for the
purposes of this analysis (Annex 2: coding framework).
Coding was carried out using a data entry form hosted on
Google Forms, and written to a Google Sheet.
Final quantitative analysis and combining of the annotated impact
case study data with the original REF-submitted data was
carried out in Excel.
Hinrichs, S et al. (2015) The non-academic impact of international development research in UK
Higher Education: analysis using the REF 2014 impact case studies
http://www.kcl.ac.uk/sspp/policy-institute/publications/non-academic-impact-of-international-development-research.pdf

11. The nature of NIHR support

12. The nature of NIHR support (1)
0
10
20
30
40
50
60
70
80
Counts of impact case studies by NIHR support stream

13. The nature of NIHR support (2)
Most case studies (n=191; 77%) noted NIHR as having supported
research within the last 10 years, with only a quarter (n=60; 24%)
exclusively noting NIHR support more recently than 2010.
A number of case studies (n=21; 8%) – mostly HTA-funded –
referred to support prior to the NIHR’s establishment in 2006.
In almost all cases, NIHR’s primary involvement was as a
contributor to the impacts described in the case studies (n=204;
82%) or as a supporter of ongoing research (n=31; 13%).5

14. The scale of NIHR support

15. The scale of NIHR support (1)
0
10
20
30
40
50
60
70
80
90
Counts of impact case studies by primary HRCS health category

16. The scale of NIHR support (2)
Word cloud produced from case study disease tags, generated at:
www.jasondavies.com/wordcloud/

17. The scale of NIHR support (3)
NIHR support was associated with a high degree of cooperative
funding between different public and private agencies.
Almost half (n=114; 46%) of NIHR-supported case studies noted
support from UK charities, and over a third (n=93; 38%) support
from one or more of the UK Research Councils. Fewer case
studies noted support from the European Union (n=20; 8%) and/
or other international (n=24; 10%) funders.
Overall, three-quarters of NIHR-supported case studies (n=185;
75%) made reference to funding from another partner.

18. The impact of NIHR support

19. To understand how NIHR-supported research led to impacts, we
annotated case studies by the evidence provided of research
contributing to one or more steps along a pathway to impact:
Almost all case studies noted impacts on regional or UK
practice. High proportions noted evidence of adoption into
international practice.
The impact of NIHR support (1)

20. The impact of NIHR support (2)
We further sought to capture and quantify, where possible, those
case studies that provided evidence of the number of patients
directly benefitting from the research (n=79; 32%):

21. The impact of NIHR support (3)
Similarly, we sought to capture in our analysis those case studies
that provided evidence of financial returns (such as cost savings
to the NHS or licensing revenue) in absolute terms (n=27; 11%):9

22. The impact of NIHR support (4)
A third of case studies (n=84; 34%) made reference to specific
beneficiary groups (i.e. over and above medical groupings of
patients), which we also sought to capture:

23. Example narratives of NIHR impact

24. Example narratives of NIHR impact (1)
Intervention: trials of the benefits of singing groups to improve health outcomes in
the elderly.
Impact: Research at the Centre for Arts and Health at Canterbury Christ Church
University has supported singing groups to improve health and wellbeing in elderly
people. Since establishing a charity in 2005 to transfer research findings into
practice, these groups now reach over 1k people per month. An NIHR-funded
randomised controlled trial of singing for 300 people aged 60 or over showed
evidence of improvements in mental wellbeing.
The team have since established local singing for mental health networks, and
singing groups for Parkinson's and dementia. The Royal Society for Public Health
incorporated research findings into national practitioner training in the arts,
health and wellbeing. Lead researchers established an international arts and
health journal, cited for its contributions to health care policy and public opinion.
NIHR support streams: Research for Patient Benefit
Scale: Financial return: n/a Beneficiaries: elderly people

25. Example narratives of NIHR impact (2)
Intervention: feasibility of a newborn bloodspot screening programme, and a
cluster trial to evaluate delivery of care, for patients with sickle cell disease.
Impact: A follow up to earlier HTA-funded research at King’s College London (KCL)
led to the establishment of a national newborn bloodspot screening programme
for sickle cell disease and thalassaemia at KCL. Since 2008, this has led to the
implementation of new UK-wide standards, care pathways, education and training,
data collection and monitoring and evaluation of patients with blood disorders.
The programme has resulted in a majority of antenatal screening tests (61%) for
sickle cell anaemia and thalassaemia now being performed before the 1st
trimester. The research has been recognised both nationally, and internationally
(highlighted as a model for screening in other countries). It has contributed to
efforts to improve pain management and the long-term care of patients with
sickle cell disease, and fed into guidance for commissioners.
NIHR support streams: Health Technology Assessment
Scale: Financial return: n/a Beneficiaries: n/a

26. Example narratives of NIHR impact (3)
Intervention: Establishing a stroke register of over 5k patients in South London to
inform long-term care and policy, and trials of more effective care models (e.g.
early supported discharge).
Impact: Registry data using risk of stroke (in particular ethnic variations in risk)
and long-term outcomes informed the National Stroke Strategy, and enabled
modelling of cost-effective options for stroke care. This fed into reconfiguration of
stroke services in London, resulting in an estimated 12% reduction in deaths at 90
days, a reduction in median length of hospital stay, and cost savings of £5.2m
(£811 per patient).
Recommendations on early supported discharge informed national guidelines,
and have been implemented in 66% of English hospitals, with indications of
reduced mortality in hospitals with better organised care.
NIHR support streams: Research for Patient Benefit, Programme Grants for
Applied Research
Scale: Financial return: £££ Beneficiaries: ethnic minorities

27. Conclusion/next steps

28. Conclusion/next steps
While by no means a comprehensive evaluation of all of the
NIHR’s research funding throughout the REF analysis period
(2008-2013), we provide a birds-eye view of how NIHR’s
investment in research has sustained a diverse swathe of health,
social and economic benefits.
Our analysis likely underestimates impacts attributable to NIHR.
Though mindful of the purpose for which REF impact case
studies were written, we recommend them as a rich and diverse
source for further enquiry.

29. Policy
Research
in
Science and
Medicine
@PRISMscience