The document discusses challenges faced by research funders. It describes how funders must decide which people, ideas, environments, and teams to fund. They also must determine if their initial funding decisions were correct and if further funding is warranted. Impact is defined as effects on the economy, society, or quality of life beyond academia. Funders face challenges in accurately measuring impact when they may only be able to count measurable outcomes rather than all important outcomes. Steps are being taken to develop better infrastructure for gathering evidence of impact and considering a wider range of research contributions and products beyond traditional authorship metrics.

1. Challenges in Research Assessment:
A funder’s view

2. Challenges in Research Assessment:
A funder’s view (ish)

3. Wellcome Trust
 Spend approximately £700 million
on research per year on
biomedicine & societal context
 Wide range of funding schemes
supporting people, places, teams,
innovation

4. Funders’ challenge part 1: who & what to fund?
 Person - track record; potential
 Idea – fit with strategic focus? excellent? new thinking?
 Environment/place
 Team/collaborators

5. who & what to fund: toolkit

6. Challenge of authorship

7. Funders’ challenge part 2: did we make right decision?
 Understanding ‘scientific’ progress & impact
 Should we fund further?
 Did it help to achieve organisation’s goals & strategy?

8. did we make right decision: toolkit
 Markers of research progress & impact (qualitative & quantitative)
 Measure of use & re-use
 Use of proxies

9. Challenge of impact
Impact is defined as ‘any effect on, change or benefit to the
economy, society, culture, public policy or services, health,
the environment or quality of life, beyond academia’
HEFCE 2014

10. Impact is defined as ‘any effect on, change or benefit to the
economy, society, culture, public policy or services, health,
the environment or quality of life, beyond academia’
HEFCE 2014
Challenge of impact
do we measure what counts or
count what we can measure?

11. o Building infrastructure to provide ‘evidence’ & insight
 Automated ‘evidence’ gathering
 System connectors
 Standards & definitions
o Ability to consider wider range of research contributions &
products: ‘beyond authorship’
o Research on research (‘science of science’)
Steps towards more greater insight …