Presentation at the annual GLOBVAC conference, Oslo 2015

1. Multi-criteria value-maximization methods for the prioritization of R&D
investments in global health product development
International Collaboration for Capitalizing on Life-Saving and Cost-
Effective Commodities (i4C)

2. Limited resources invested in R&D for
global health product development
• Neglected disease R&D funding 2007-2013 (G-FINDER 2014)

3. Women and children particularly at risk

4. R&D investments key for sustaining product
development pipelines
*BVGH (2007) Closing the Global Health Innovation Gap: A Role for the Biotechnology Industry in Drug Discovery for Neglected Diseases

5. A systematic prioritization of R&D investments is
required
Which
diseases?
• NTDs?
• The big three
(HIV/AIDS, TB,
malaria)?
• Emerging
infectious
diseases?
• Maternal and
child health
conditions?
• other?
What
products?
• Vaccines?
• Drugs?
• Diagnostics?
• Vector control
products?
• MCH essential
commodities?
• Other medical
devices?
What R&D
areas?
• Basic
research?
• Discovery?
• Preclinical?
• Clinical?
• Post-
licensure?
Which actors?
• Industry?
• Academic and
government
research
organizations?
• Product
Development
Partnerships?

6. Risk of misallocation of scarce resources in absence of
systematic prioritization
© Can Stock Photo – csp23071257

7. Dearth of industry R&D investments due to time, risk,
cost and lack of incentives
$236
$359
$391
$447
$418
$401 $401
$0
$50
$100
$150
$200
$250
$300
$350
$400
$450
$500
FY 2007 FY 2008 FY 2009 FY 2010 FY 2011 FY 2012 FY 2013
Milions
Industry investments in neglected disease R&D (G-FINDER 2014)
Discovery &
preclinical
Clinical I/II Clinical III Licensure
Basic research /
translational researchBasic & translational
research
Average Time: 10 – 15 years
Average Risk: 95% chance of failure at discovery
Average Cost: $500mm - > $1.3bn

8. A Complex funding flow structure
Funders
Fund
Managers
Product
Developers

9. Variable concerns and motivations to R&D priority
setting
x
CSR /
reputational
concerns?
Social
returns?
Health
returns?
Financial
constraints
R&D
awareness
Technical
constraints
Optimal R&D priority setting and investment decision making

10. Project outline
Goal- Assess whether a multi-criteria value-maximization tool can
be applied for the prioritization of R&D investments in global
health product development
Conceptual
Framework
Global
Survey-based mapping of motivations,
objectives and criteria for investing in
ND R&D
MCDA model development and
testing for prioritizing global R&D
investments in NDs
Cluster
MCDA model development and
testing for prioritizing funders’ R&D
investments in PDPs
Organization
-specific
MCDA model development and
testing for prioritizing R&D
investments in a product developer’s
R&D candidate portfolio
Methodological
Framework

11. The partnership
Organization Name
Harvard Global Health Institute Suerie Moon
Harvard School of Public Health Joshua Salomon
Instituto de Evaluación Tecnológica en Salud Javier Guzman
K Marsh Consulting Ltd. Kevin Marsh
Technical University of Crete Constantin Zopounidis; Michael Doumpos
Universitair Medisch Centrum St Radboud Rob Baltussen
University of Bergen Ole Frithjof Norheim
University of Bergen Trygve Ottersen
University of California, San Francisco Dean Jamison
University of California, San Francisco Gavin Yamey
WHO Global Health R&D Observatory Riek Viergever
World Health Organization - PHI Zafar Mirza
World Health Organization - TDR Rob Terry

12. Stage 1: Towards a conceptual
framework
Purpose
conceptual framework
for systematic
prioritization of R&D
investments in global
health product
development
Targeted review
 Nature of the prioritization
decision
 Type of options considered
 Who is making the choices
 What level of aggregation
choices are made at
 What concerns, objectives
and criteria influencing
choices
 What the model is for
processing objectives and
criteria
 What additional processes
are being deployed
Appraisal of
pros / cons
 identifying options
 identifying criteria
 valuing options

13. Issues emerging from priority setting literature in health
research
Examples
• 3D CAM
• ENHR
• CHNRI
• COHRED
• WHO Ad Hoc
Committee method
• Numerous informal
methodologies
Emerging issues
• consensus building process around research
issues requiring urgent attention
• making use of stakeholder judgments based
on multiple values and objective criteria
• suitability of approaches varies according to
• level of application
• comprehensiveness of topics addressed
• mix of technical and interpretive
techniques
• nature and degree of stakeholder
involvement

14. Issues emerging from priority setting literature in health
care more broadly
Examples
• PBMA
• health economic evaluation
• QALY league tables and BoD
approaches
• value of information
approaches
• historical allocation
processes
• partnership approaches
• clinical variations and
payback approaches
• conjoint analysis and DCE
methods
• MCDA methods
• ethical frameworks
Emerging issues
• a process of rationing limited resources to
meet target group needs
• stakeholder preferences matter and how
target groups are approached is critical,
including ethical considerations
• central are also issues of:
• what evidence is provided to inform the
prioritization process
• what analytic methods are applied to assess
performance of options against a range of
values and quantitative criteria

15. Issues emerging from priority setting literature in public
policy (linked to health)
Examples
•category-based (cost-benefit)
models
•user-based models
•institutional models
•political models
Emerging issues
• systemic process of benefit-cost trade-offs between
preferred areas of science towards certain national goals,
• with interplays occur between user-based, institutional,
and political forces
• dynamic process whereby priorities change with goals
• recognition of complex political interactions involving
multiple stakeholders
• who makes the choices, at what level of decision making
and in what relation to national goals and needs also central
• consideration of funding outcomes transform priorities
from ways of stressing issues for attention to methods for
managing resource allocation

16. Issues emerging from priority setting literature in R&D
management
Examples
• capital budgeting and
pharmacoeconomic
models
• efficiency and
productivity analysis
models
• optimization and
simulation models
• MCDA models
Emerging issues
• organization-specific decision problem of R&D project
selection or portfolio management
• under specified constraints and criteria
• accompanied by good managerial judgment
• earlier models of limited practical application due to over-
emphasizing mathematical sophistication while being
contextually naïve
• recent models recognize importance of stakeholder
interactions in resource allocation decisions
• with managerial perception and satisfaction acquiring a
central role

17. Common issues emerging across disciplines
 Priority setting can be seen as a resource allocation
problem and as an investment decision aiding process
when funding outcomes are considered
 Priority setting dependent on data for observing reality, and
on preference relations for interpreting data in a
meaningful way
 Valuation of options conducted against pre-defined or
interactively constructed metrics to which a specific type of
trade-off is always associated
 Suitability of empirical methods varies, depending on
context of application & balance between methodological
sophistication and intuitive judgment of stakeholders
involved

18. Conceptual Framework: Next steps
x 5,492 records
identified
391 duplicates
removed
5,101 records
screened by
title/abstract
5,010 records
excluded
91 records pre-
selected for full
text review
 Google
scholar
views
1,000 records viewed
13 additional records selected
for full text review
 Excluded if:
 Not a method for pharma R&D priority setting;
 Theoretical approach without application on pharma R&D priority setting;
 Method from chemical or biology perspective without focus on strategy or resource
allocation problems
 Database
searches

19. Stage 1: Towards a Methodological Framework
• Research already conducted
– Viergever R, Gouglas D (Draft). The role
of multi-criterion decision analysis
(MCDA) in health research priority
setting. (Eds) Rob Baltussen, Kevin Marsh
‘Multi-criteria Decision Analysis for
Health Interventions’ Springer Editions.
2014.
• Relevant findings:
– many health research priority setting
approaches apply MCDA elements
– better use of well-established MCDA
approaches can increase
transparency, explicitness, and
analytic robustness of methods
deployed
• Next step: Determine how to apply
MCDA in R&D priority setting and
investment decision making for global
health product development
Method Papers (No)
MCDA / Value measurement methods 46
Scoring / weighted sum model (CHNRI methodology) 24
Scoring model (not weighted) 14
Scoring / weighted sum model 7
Nominal Group Technique (not weighted) 1
Qualitative 31
Consultative group process 14
Consensus-based 9
Nominal Group Technique (consensus-based) 4
stepwise approach (literature review, key informant interviews, consultative group process) 2
Survey 2
Mixed methods 21
Listing / Sorting model 11
ENHR 7
stepwise approach (literature review, key informant interviews, consultative group process) 2
Concept mapping 1
MCDA / Qualitative 20
Listing / Sorting model 14
CAM 3
Consensus-based 3
39%
26%
18%
17%
MCDA /
Qualitative
MCDA / Value
measurement
methods
Mixed
methods

20. Thank you
Dimitrios Gouglas
Adviser / Stipendiat
Norwegian Institute of Public Health
Postbox 4404 Nydalen, 0403 Oslo
Norway
Email: Dimitrios.Gouglas@fhi.no
Phone: +47 2107 8013
www.fhi.no