The document outlines guidelines for writing a 4-star impact case study, emphasizing the need for underpinning research of at least 2-star quality published after 2000 and impacts demonstrated between August 2013 and July 2020. It stresses the importance of evidence-based testimonials and clear connections between research and impacts, highlighting differences between high and low-scoring case studies. Moreover, it provides insights into qualitative and quantitative aspects that contribute to obtaining a high score in evaluations, particularly in the context of REF2021.

1. HOW TO WRITE A 4* IMPACT CASE STUDY

2. 1. Case study fields remain, but with metadata and 5 pages
2. Underpinning research must be 2* quality and published >2000
3. Impacts must occur between Aug 2013 and July 2020
4. Minimum 2 case studies up to 20 FTE, then 1 per 15 FTE
5. Impacts from a body of research (e.g. collaborative project) must
demonstrate substantive contribution from institution
6. Public engagement is a pathway, not an impact
7. Underpinning research 2* as a body with quality justification
8. All evidence submitted with case studies and independently
verifiable (testimonials should be based on evidence, not opinion)
9. Continuation case studies have 1) no significant new underpinning
research, AND 2) similar impacts and beneficiaries to those in 2014
10. A researcher’s outputs & impact can be submitted to different UoAs
11. Panel A will treat qualitative evidence and continuation case studies
without prejudice (despite consultation showing disciplinary biases)
Impact in REF2021: a summary

3. Has the bar been raised? Will it be harder to
get 4* in REF2021?
•Definitely
•Probably
•Probably not
•A 4* case study in 2014 should be equivalent
to a 4* case study in 2021
The REF2021 bar
Vote
now

4. •Investment in evidence:
•Longer-term (sometimes longitudinal) evidence
•Cause and effect relationships proven across
causal chains
•Evidence-based testimonials (but also
testimonial carnage)
The REF2021 bar

5. Before you start:
• Ethics approval
• Informed consent
Evidence-based testimonials
Format:
• Part catch up and offer of
further help
• Part evaluation
1. Significance
2. Reach
3. Attribution
4. What could we have
done better?
Transcribe interview and
draft testimonial

6. 1. Case study fields remain, but with metadata and 5 pages
2. Underpinning research must be 2* quality and published >2000
3. Impacts must occur between Aug 2013 and July 2020
4. Minimum 2 case studies up to 20 FTE, then 1 per 15 FTE
5. Impacts from a body of research (e.g. collaborative project) must
demonstrate substantive contribution from institution
6. Public engagement is a pathway, not an impact
7. Underpinning research 2* as a body with quality justification
8. All evidence submitted with case studies and independently
verifiable (testimonials should be based on evidence, not opinion)
9. Continuation case studies have 1) no significant new underpinning
research, AND 2) similar impacts and beneficiaries to those in 2014
10. A researcher’s outputs & impact can be submitted to different UoAs
11. Panel A will treat qualitative evidence and continuation case studies
without prejudice (despite consultation showing disciplinary biases)
Impact in REF2021
Open
mic

7. What made a 4*case study in REF2014?
Based on PhD research by Bella Reichard
@BellaReichard based on quantitative analysis of 217
and qualitative analysis of 180 of the highest and lowest
scoring cases, spread across Panels A, B, C and D

9. Quantitative linguistic analysis
1. Highly-rated case studies provided specific,
high-magnitude and well-evidenced
articulations of significance and reach
• 84% of high-scoring cases articulated significant and far-reaching
benefits, compared to 32% of low-scoring cases, which typically focused
on pathway
Phrases more common in high-
scoring:
• Significance and reach (specific and
high): in England and, in the US, the
UK’s, millions of, long-term, the
government’s, the department of,
the House of Commons, for the first
time, prime minister, select
committee
Phrases more common in low-scoring:
• Significance and reach (non-specific or low): a number
of, a range, nationally and internationally, in local, of
local, the north, city council, policy and practice, an
impact on, impact on the, the impact
• Pathways to impact: has been disseminated, disseminated
through, dissemination of, been disseminated, and
workshops, the event, the book
• Beneficiaries (not benefits): and community, practitioners
and, group of, members of the
Qualitative analysis

10. • 97% of high-scoring cases clearly linked the
underpinning research to claimed impacts,
compared to 50% of low-scoring case studies
• 42% high-scoring policy cases described policy and
implementation, compared to 17% in low
Phrases more common in high-scoring
• Attribution between research and
impact: cited in the, (was) used to
inform, to improve the, led to the,
resulting in, showing that, was
subsequently, produced by, reported in,
evidence for, cited in, led by
2. Highly-rated case studies established
links between research (cause) and
impact (effect) convincingly
Phrases more common in low-scoring
• Research outputs/process: the paper, peer-
reviewed, journal of, et al, research project, this
research has, by Dr, of Dr, research team
• Attribution between research and pathways:
work has, has informed, through the
0
10
20
30
40
50
60
70
80
Generally high
quality
corroborating
evidence
Some
questionable
quality evidence
Vague and/or not
clearly linked to
impacts
Number
of
case
studies
High-scoring Low-scoring
Quantitative linguistic analysis
Qualitative analysis
• Coh-metrix analysis shows higher-scoring cases had more explicit causal connections between
ideas and more logical connective words (and, or, but) than low-scoring cases

11. 3. Highly-rated case studies were
easy to understand and well
written
• Low-scoring cases more likely to have
academic phrasing: in relation to, in
terms of, the way(s) in which
• Flesch Reading Ease score, out of 100,
was 30.0 on average for 4* and 27.5 on
average for 1*/2* (all “college-graduate”
difficulty). Panels C & D high-scoring
case studies significantly easier to read
than low-scorers
• High scoring cases had more sub-
headings (especially pronounced when
comparing high to low cases in Panel D)
• High-scoring cases used more direct, plain language, had fewer
expressions of uncertainty or hedging statements, and were less likely to
contain unsubstantiated or vague use of adjectives to describe impacts
Qualitative analysis
Quantitative linguistic analysis

12. •Add quick/simple questions to chat to start
•Open mic: pose challenging questions for
discussion
• Context: what is your research and impact?
• Challenge: what are you stuck on?
• Share screen so we can see details (if relevant)
• Initial response from Mark
• Additional responses from Mark and group via chat
• Wider discussion with group (answer in chat or open
mic)
• Concluding response from case study author
Case study surgery
Open
mic

13. www.fasttrackimpact.com
@fasttrackimpact