Qualitative
Prioritization
ISPC 12 Meeting, Rome, 15 Sep 2015
Introduction
“CGIAR’s Independent Science and Partnership Council (ISPC)
will strengthen the quality, relevance, and impact of new
investments through the provision of expert scientific guidance
through an appropriate qualitative prioritization for the next
generation of CRPs at both pre-proposal and final proposal
stage. This will ensure that only the strongest, most directly
relevant CRP proposals are approved for funding.”
Quote from the SRF (May 2015)
http://ispc.cgiar.org/ISPC 12, Rome
Improved food and nutrition
security for health
Increased
household
capacity to cope
with shocks
Reduced poverty
Increased
incomes and
employment
Increased
productivity
Enhanced
smallholder
market access
Increased
resilience of the
poor to climate
change and
other shocks
Improved natural resource
systems and ecosystem services
Improved diets
for poor and
vulnerable
people
Improved
food
safety
More
sustainably
managed agro-
ecosystems
Enhanced
benefits from
ecosystem
goods and
services
Improved
human and
animal health
through better
agricultural
practices
Natural capital
enhanced and
protected,
especially from
climate change
Cross-cutting issues: Gender and youth , Climate change , Policies and institutions , Capacity development
Increased access
to productive
assets, including
natural resources
Reduced pre- and
post-harvest
losses, including
those caused by
climate change
Increased
availability of
diverse nutrient-
rich foods
Reduced
production risk
Reduced
biological and
chemical hazards
in the food
system
Increased
livelihood
opportunities
Reduced market
barriers
Increased
resilience of agro-
ecosystems and
communities
especially those
including
smallholders
More productive
and equitable
management of
natural resources
Land, water and
forest
degradation
(including
deforestation)
minimized and
reversed
Improved water
quality
Increased access
to diverse
nutrient-rich
foods
More efficient
use of inputs
Increased value
capture by
producers
Closed yield gaps
through improved
agronomic and
animal husbandry
practices
Increased
conservation and
use of genetic
resources
Diversified
enterprise
opportunities
Enhanced genetic
gain
Improved access
to financial and
other services
Optimized
consumption of
diverse nutrient-
rich foods
Appropriate
regulatory
environment for
food safety
Reduced livestock
and fish disease
risks associated
with
intensification
and climate
change
Increased safe
use of inputs
Enhanced
conservation of
habitats and
resources
Increased genetic
diversity of
agricultural and
associated
landscapes
Agricultural
systems
diversified and
intensified in
ways that protect
soils and water
Enhancement of
plant and animal
biodiversity for
multiple goods
and services
Enhanced
adaptive capacity
to climate risks
Reduced net
greenhouse gas
emissions from
agriculture,
forests and other
forms of land use
Climate change Gender and youth Policies and institutions Capacity development
Mitigation and adaptation
achieved
Reduced net GHG emission from
agriculture, forestry and other
forms of land use
Increased above- and below-
ground biomass for carbon
sequestration
Improved forecasting of impacts
of climate change and targeted
technology development
Enhanced capacity to deal with
climatic risks and extremes
Equity and inclusion
achieved
Enabling environment
improved
National partners and
beneficiaries enabled
Gender equitable control of
productive assets and resources
Technologies which reduce
women’s labour and energy
expenditure developed and
disseminated
Improved capacity of women and
young people to participate in
decision-making
Enabled environment for climate
resilience
Increased capacity of
beneficiaries to adopt research
outputs
Increased capacity of partner
organisations as evidenced by
rates of investment in
agricultural research
Conducive agricultural policy
environment
Conducive environment for
managing shocks and
vulnerability as evidenced in
rapid response mechanisms
Enhanced institutional capacity of
partners research organisations
Enhanced individual capacity in
partner research organisations
through training and exchange
Increased capacity for innovation
in partner research organisations
Enhanced capacity for innovation
in partner development
organisations and in poor and
vulnerable communities
The Prioritization Exercise
http://ispc.cgiar.org/ISPC 12, Rome
Methodology
The analysis was undertaken in two separate spheres:
http://ispc.cgiar.org/ISPC 12, Rome
Donors
Asked to indicate
their organisational
priorities for sub-
IDOs
Experts
Drawing on experts’
knowledge of
agricultural R & D
pathways to rate sub-
IDOs against specific
criteria that feed into
priorities
Caveats
Sub-IDOs are open to interpretation.
Sub-IDOs are interlinked; some may be a necessary condition
for others or impacting one may mean impacting several.
Development strategies are contextual; variation by country or
even within countries.
http://ispc.cgiar.org/ISPC 12, Rome
I. The Donor Survey
Donors allocated 45 points across the 45 unique sub-IDOs
(forcing choices !).
Allocations reflected the relative importance of each sub-IDO
for the organisation.
Of 70 stakeholders and donors approached, 19 respondents.
15 of these 19 contributed 59% of CGIAR funding in 2014.
http://ispc.cgiar.org/ISPC 12, Rome
Results: Varying Discrimination between Sub-IDOS
0
5
10
15
20
25
30
35
40
#ofsub-IDOsassignedzero
Donor
0
5
10
15
20
25
30
35
4.3 4.1 3.3 3.2 4.2 4.4 8.1 9.2 4.5 2.1 1.1 5.2 10.110.2 3.4 5.1 7.2 1.2 8.3 9.1 2.2 3.1 7.1 8.2 6.1 5.3 6.2 9.3 7.3
PointsAssigned
Sub-IDO
Reduced pre- and
post-harvest losses
Enhanced genetic gain
Increased value capture
by producers
Increased livelihood opportunities
Optimised
consumption of
diverse
nutrient rich
foods
Enhancement of
animal and
plant
biodiversity
Increased use of
safe inputs
Regulatory environment for food
safety
Donor Priorities: Total Points Assigned
II. Expert Opinion
7 experts (international agricultural research and development)
were asked to undertake a prioritization exercise.
Multi-disciplinary: economists, agricultural R&D expert, bio-
physical scientist, international development expert, donors, ex
CGIAR
http://ispc.cgiar.org/ISPC 12, Rome
Methodology
http://ispc.cgiar.org/ISPC 12, Rome
• Assign scores of 1-5 to sub-IDOs according to some of these criteria:
The relevance of
the sub-IDO for
achieving the SLO.
The centrality of
agricultural
research to this
sub-IDO.
The comparative
advantage of the
CGIAR for
achieving the sub-
IDO.
The International
Public Goods
orientation of CG to
achieve the sub-
IDO.
Expectation of
delivery in the
short and long
term.
Methodology
Initially wanted to establish some sort of consensus/majority.
• Normalised scores
• Divided into quintiles
• Grouped into high, moderate and low
• Combined relevance and centrality to form ISPC rating
(high, moderate-high, moderate or low)
http://ispc.cgiar.org/ISPC 12, Rome
Results: Priority Levels for all Sub-IDOs under the SLOs
Results: Priority Levels for Sub-IDOs under the Cross
Cutting Themes
Comparing prioritisation results
to sub-IDO targets in the CRP
pre-proposals
http://ispc.cgiar.org/ISPC 12, Rome
Going Forward
“CGIAR’s Independent Science and Partnership Council (ISPC)
will strengthen the quality, relevance, and impact of new
investments through the provision of expert scientific guidance
through an appropriate qualitative prioritization for the next
generation of CRPs at both pre-proposal and final proposal
stage. This will ensure that only the strongest, most directly
relevant CRP proposals are approved for funding.”
http://ispc.cgiar.org/ISPC 12, Rome
Going Forward
To what extent is this information useful for evaluating CRP
strength and relevance in this pre-proposal stage?
How can this methodology be refined for a more robust analysis
of full proposals?
http://ispc.cgiar.org/ISPC 12, Rome

Qualitative prioritization - Tim Kelley

  • 1.
  • 2.
    Introduction “CGIAR’s Independent Scienceand Partnership Council (ISPC) will strengthen the quality, relevance, and impact of new investments through the provision of expert scientific guidance through an appropriate qualitative prioritization for the next generation of CRPs at both pre-proposal and final proposal stage. This will ensure that only the strongest, most directly relevant CRP proposals are approved for funding.” Quote from the SRF (May 2015) http://ispc.cgiar.org/ISPC 12, Rome
  • 3.
    Improved food andnutrition security for health Increased household capacity to cope with shocks Reduced poverty Increased incomes and employment Increased productivity Enhanced smallholder market access Increased resilience of the poor to climate change and other shocks Improved natural resource systems and ecosystem services Improved diets for poor and vulnerable people Improved food safety More sustainably managed agro- ecosystems Enhanced benefits from ecosystem goods and services Improved human and animal health through better agricultural practices Natural capital enhanced and protected, especially from climate change Cross-cutting issues: Gender and youth , Climate change , Policies and institutions , Capacity development Increased access to productive assets, including natural resources Reduced pre- and post-harvest losses, including those caused by climate change Increased availability of diverse nutrient- rich foods Reduced production risk Reduced biological and chemical hazards in the food system Increased livelihood opportunities Reduced market barriers Increased resilience of agro- ecosystems and communities especially those including smallholders More productive and equitable management of natural resources Land, water and forest degradation (including deforestation) minimized and reversed Improved water quality Increased access to diverse nutrient-rich foods More efficient use of inputs Increased value capture by producers Closed yield gaps through improved agronomic and animal husbandry practices Increased conservation and use of genetic resources Diversified enterprise opportunities Enhanced genetic gain Improved access to financial and other services Optimized consumption of diverse nutrient- rich foods Appropriate regulatory environment for food safety Reduced livestock and fish disease risks associated with intensification and climate change Increased safe use of inputs Enhanced conservation of habitats and resources Increased genetic diversity of agricultural and associated landscapes Agricultural systems diversified and intensified in ways that protect soils and water Enhancement of plant and animal biodiversity for multiple goods and services Enhanced adaptive capacity to climate risks Reduced net greenhouse gas emissions from agriculture, forests and other forms of land use
  • 4.
    Climate change Genderand youth Policies and institutions Capacity development Mitigation and adaptation achieved Reduced net GHG emission from agriculture, forestry and other forms of land use Increased above- and below- ground biomass for carbon sequestration Improved forecasting of impacts of climate change and targeted technology development Enhanced capacity to deal with climatic risks and extremes Equity and inclusion achieved Enabling environment improved National partners and beneficiaries enabled Gender equitable control of productive assets and resources Technologies which reduce women’s labour and energy expenditure developed and disseminated Improved capacity of women and young people to participate in decision-making Enabled environment for climate resilience Increased capacity of beneficiaries to adopt research outputs Increased capacity of partner organisations as evidenced by rates of investment in agricultural research Conducive agricultural policy environment Conducive environment for managing shocks and vulnerability as evidenced in rapid response mechanisms Enhanced institutional capacity of partners research organisations Enhanced individual capacity in partner research organisations through training and exchange Increased capacity for innovation in partner research organisations Enhanced capacity for innovation in partner development organisations and in poor and vulnerable communities
  • 5.
  • 6.
    Methodology The analysis wasundertaken in two separate spheres: http://ispc.cgiar.org/ISPC 12, Rome Donors Asked to indicate their organisational priorities for sub- IDOs Experts Drawing on experts’ knowledge of agricultural R & D pathways to rate sub- IDOs against specific criteria that feed into priorities
  • 7.
    Caveats Sub-IDOs are opento interpretation. Sub-IDOs are interlinked; some may be a necessary condition for others or impacting one may mean impacting several. Development strategies are contextual; variation by country or even within countries. http://ispc.cgiar.org/ISPC 12, Rome
  • 8.
    I. The DonorSurvey Donors allocated 45 points across the 45 unique sub-IDOs (forcing choices !). Allocations reflected the relative importance of each sub-IDO for the organisation. Of 70 stakeholders and donors approached, 19 respondents. 15 of these 19 contributed 59% of CGIAR funding in 2014. http://ispc.cgiar.org/ISPC 12, Rome
  • 9.
    Results: Varying Discriminationbetween Sub-IDOS 0 5 10 15 20 25 30 35 40 #ofsub-IDOsassignedzero Donor
  • 10.
    0 5 10 15 20 25 30 35 4.3 4.1 3.33.2 4.2 4.4 8.1 9.2 4.5 2.1 1.1 5.2 10.110.2 3.4 5.1 7.2 1.2 8.3 9.1 2.2 3.1 7.1 8.2 6.1 5.3 6.2 9.3 7.3 PointsAssigned Sub-IDO Reduced pre- and post-harvest losses Enhanced genetic gain Increased value capture by producers Increased livelihood opportunities Optimised consumption of diverse nutrient rich foods Enhancement of animal and plant biodiversity Increased use of safe inputs Regulatory environment for food safety Donor Priorities: Total Points Assigned
  • 11.
    II. Expert Opinion 7experts (international agricultural research and development) were asked to undertake a prioritization exercise. Multi-disciplinary: economists, agricultural R&D expert, bio- physical scientist, international development expert, donors, ex CGIAR http://ispc.cgiar.org/ISPC 12, Rome
  • 12.
    Methodology http://ispc.cgiar.org/ISPC 12, Rome •Assign scores of 1-5 to sub-IDOs according to some of these criteria: The relevance of the sub-IDO for achieving the SLO. The centrality of agricultural research to this sub-IDO. The comparative advantage of the CGIAR for achieving the sub- IDO. The International Public Goods orientation of CG to achieve the sub- IDO. Expectation of delivery in the short and long term.
  • 13.
    Methodology Initially wanted toestablish some sort of consensus/majority. • Normalised scores • Divided into quintiles • Grouped into high, moderate and low • Combined relevance and centrality to form ISPC rating (high, moderate-high, moderate or low) http://ispc.cgiar.org/ISPC 12, Rome
  • 14.
    Results: Priority Levelsfor all Sub-IDOs under the SLOs
  • 15.
    Results: Priority Levelsfor Sub-IDOs under the Cross Cutting Themes
  • 16.
    Comparing prioritisation results tosub-IDO targets in the CRP pre-proposals http://ispc.cgiar.org/ISPC 12, Rome
  • 19.
    Going Forward “CGIAR’s IndependentScience and Partnership Council (ISPC) will strengthen the quality, relevance, and impact of new investments through the provision of expert scientific guidance through an appropriate qualitative prioritization for the next generation of CRPs at both pre-proposal and final proposal stage. This will ensure that only the strongest, most directly relevant CRP proposals are approved for funding.” http://ispc.cgiar.org/ISPC 12, Rome
  • 20.
    Going Forward To whatextent is this information useful for evaluating CRP strength and relevance in this pre-proposal stage? How can this methodology be refined for a more robust analysis of full proposals? http://ispc.cgiar.org/ISPC 12, Rome