Exploratory presentation on tools to characterize biomedical research portfolios, given at the STI 2014 conference in Leiden

1. Portfolios and landscapes of avian influenza research:
new tools to inform policymaking
1
Matthew L. Wallace · INGENIO (CSIC-UPV)
Ismael Rafols · INGENIO (CSIC-UPV) · SPRU (University of Sussex)
19th International Conference on Science and Technology Indicators
Leiden, September 3-5, 2014

2. OUTLINE
2
What tools and analyses can help allocate resources
to a complex societal challenge such as avian
influenza?
1. Context (research agenda): research portfolios as a
science policy tool
2. Case study: avian influenza as a “societal challenge”
3. What can and can’t we learn about avian flu
landscape and funding from bibliometrics?

3. The drivers for portfolio-level analysis
• Problem: perceived mismatch between research and
desired outcomes
• Respond to current policy demands in grand societal
challenges
3
• Tackling large-scale, multi-stakeholder issues
• Transparency, accountability, cost-effectiveness, etc.
• Seeking out alternative research avenues spawning new
solutions.

4. Examples of current portfolio practices
4

5. Moving beyond rhetoric and financial analogies
Other science policy literature can point to some promising
pathways…
• Consider a broad range of outcomes and pathways for
5
achieving them
– From risk to uncertainty
• Think about how to link policy pull and science push
– E.g., public-value mapping
• Take a systems approach to sets of research project
– Look for “positive interactions” between projects
What might this look like?
(and can bibliometrics help?)

6. Overview of avian flu as a case study
6
The Independent,
June 11, 2014

7. From public health governance to research: is there a
mismatch?
7
Global governance /
narratives of control
- Veterinary
- Pandemic
preparedness
- Public health
Science policy
priorities
Portfolio design or
evaluation?
Research enterprise
Collaboration?
Coordination?

8. From public health governance to research: is there a
mismatch?
8
Global governance /
narratives of control
- Veterinary
- Pandemic
preparedness
- Public health
Science policy
priorities
Portfolio design or
evaluation?
Research enterprise
Collaboration?
Coordination?
Editorial topics of top
journals: vaccine and
treatment, “dual-use”,
control, characterization of
disease, pandemic risk
In-depth, in-person
interviews with 15
policymakers, scientists and
stakeholders across Europe

9. From public health governance to research: is there a
mismatch?
9
Global governance /
narratives of control
- Veterinary
- Pandemic
preparedness
- Public health
Science policy
priorities
Portfolio design or
evaluation?
Mapping the
research
landscape
and
research
portfolios
Research enterprise
Collaboration?
Coordination?
Editorials topics of top
journals: vaccine and
treatment, “dual-use”
control, characterization of
disease, pandemic risk
In-depth, in-person
interviews with 15
policymakers, scientists and
stakeholders across Europe

10. Bibliometric data and methods
10
Combine Medical
Subject Headings
(Medline)
Web of Knowledge
database
Databases of Avian
influenza and
Influenza A
publications
Mapping and
clustering
techniques (co-citation,
co-term)
Other funding
data on avian
flu and
Influenza A
Overlays and cluster
analysis

11. Avian flu publications and funding
Avian influenza funding by main funding
agencies ($M)
Source: Über Research Data
11
Influenza A funding by main funding
agencies ($M)

12. The research landscape at different scales (co-term)
Influenza A, 2004-2013
Avian influenza, 2004-2013
12

13. Characterizing the overall landscape
13
HUMAN
ANIMAL

14. Characterizing the overall landscape
14
BASIC
CLINICAL

15. Characterizing the overall landscape
15
FIELD-BASED
LAB-BASED

16. Characterizing clusters and research options
• Journals (and sub-disciplines) cut across clusters
• Keywords are often ambiguous
• Some scale/granularity dependency (of clusters and overall
16
landscape)
But we need clarity to understand / inform the allocation
of resources!!

17. Landscape as a continuum
“Vaccine/vaccination” as a keyword in abstracts
17

18. Landscape as a continuum
Veterinary science as a WoS subject category
18

19. Landscape as a continuum
Virology as a WoS subject category
19

20. Landscape as a continuum
Virology as a MeSH qualifier
20

21. Overlays from funding data in WoS
GlaxoSmithKline
(Influenza A, 2010-12)
Welcome Trust
(Influenza A, 2010-12)
Canadian Institutes for
Health Research
(Influenza A, 2010-12)
21

22. Overlays from funding application data (ÜberResearch)
Canadian Institutes
for Health Research
(Influenza A, 2003-13)
Welcome Trust
(Influenza A,
2003-13)
22

23. What does this tell us about avian flu?
• Cannot be described according to disciplinary lines
• We can tentatively identify the dominant lines of
23
research: epidemiology, molecular biology,
immunology, vaccine development
• We can see where connections lie (overall and within
portfolios): e.g., very few connections between
epidemiology and virology (lab-based)
• Public funders generally tend to diversify their
portfolios, but overall objectives (“problems”) are not
reflected
• We can identify gaps according to influenza research
overall

24. CONCLUSION
• Portfolios lens has the potential to help tackle complex
24
societal problems: a way to move beyond “excellence”
• Mapping and clustering of bibliometric data can be useful,
but cannot produce “one-dimensional”, prescriptive
description
• Co-word overlays present data/technical difficulties, but are
good means to visualize research and especially to elicit
opinions

25. 25
THANK YOU
INGENIO [CSIC-UPV] Ciudad
Politécnica de la
Innovación | Edif 8E 4º
Camino de Vera s/n
46022 Valencia
tel +34 963 877 048
fax +34 963 877 991