Measuring and Improving the Effectiveness of R&D Systems in Sub-Saharan AfricaLeonard OrukoIAAE Symposium on Improving Ret...
R&D Results Measurement Challenges• What is an effective R&D system?   – Outputs impact on poverty, food security and inco...
R&D Results Measurement Challenges..• Research evaluation has supported the case for R&D   – The tools and information dev...
Status of Results Measurement in SSA Ag. R&DInstitutions• Ex-ante impact evaluation   *CAADP Framework   – Evidence of som...
Impact of Ag. R&D: Common practice• Primary focus has been that of establishing the impact on development  outcomes       ...
Outcomes of Ag. R&D• Estimates of adoption primarily case specific   – Targeted studies estimate adoption levels and deter...
Inputs and outputs         • The DIVA Initiative             •   Research expenditure             •   Full Time Equivalent...
Improving the Results measurement of R&Dsystems   “A major knowledge gap in understanding and strengthening    R&D systems...
Empirics from Ghana and Nigeria : PerceptionRatings)• Output, outcome and impact indicators are standard   -Technologies g...
Way Forwards for results Measurement• There is no substitute for valid and credible data    – Real time data for operation...
Way Forwards for Results Measurement• Strategic Focus in SSA   – ASTI initiative to support the NARS in the institutionali...
Way Forwards for Results Measurement• Rigorous impact evaluation approaches recommended   – Selective use of RCT and other...
THANK YOU
Upcoming SlideShare
Loading in …5
×

Measuring and improving effectiveness of african ag research systems asti - iaae

374 views
327 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
374
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Measuring and improving effectiveness of african ag research systems asti - iaae

  1. 1. Measuring and Improving the Effectiveness of R&D Systems in Sub-Saharan AfricaLeonard OrukoIAAE Symposium on Improving Returns toAgricultural Research in Sub-Saharan AfricaFoz do Iguaçu | 20 August 2012
  2. 2. R&D Results Measurement Challenges• What is an effective R&D system? – Outputs impact on poverty, food security and income growth – Generates relevant products and services in a timely fashion – Has adequate human capacity and financial resources• Can we demonstrate Results of Ag. R&D Investments? – Human capital, infrastructure and operational funding constraints – Long time lags from the point of investment to the manifestation of returns – “ A fishing expedition or a shooting range” – Well functioning support services and institutions must be in place for research outputs to have an impact on development outcomes
  3. 3. R&D Results Measurement Challenges..• Research evaluation has supported the case for R&D – The tools and information developed for evidence-based policy were linked to the development imperatives of the day• Research evaluation responded to questions being asked – Economic returns , welfare analysis, priority setting, funding of research issues – Concerns with poverty (well beyond producer and consumer surplus), NRM and sustainability (beyond production systems) , and later climate change at increasing scale• Impact assessment has had to balance the needs for accountability to funders versus learning and change by actors – Economic return, experiments and quasi experiments (quantitative) – Utilization-focused evaluation, qualitative
  4. 4. Status of Results Measurement in SSA Ag. R&DInstitutions• Ex-ante impact evaluation *CAADP Framework – Evidence of some NARS adopting objective criteria, with support from the CG Centres – ASARECA, CORAF and CCARDESA focus on estimating spillover potential• Managing research implementation process – Accountability focus pushing NARS towards RBM-PABRA – FARA and SRO gravitating around RBM derived CPMF – CG-CRP• Ex post impact evaluation – Adoption – Precise measurement of impact with RCT emerging as “ the Gold Standard”
  5. 5. Impact of Ag. R&D: Common practice• Primary focus has been that of establishing the impact on development outcomes Source: Block, 2010
  6. 6. Outcomes of Ag. R&D• Estimates of adoption primarily case specific – Targeted studies estimate adoption levels and determinants of adoption; guide research planning and priority setting• Important lessons for R&D results measurement systems – LSMS-ISA – DIVA Initiative Adoption of improved varieties Crop % cropped area Maize (in west and central Africa) 67 Cassava 39 Beans 32 Sorghum 14 Source: Alene, et al, 2011
  7. 7. Inputs and outputs • The DIVA Initiative • Research expenditure • Full Time Equivalent Scientists(FTEs) • Research Intensity • Mean Incidence of varietal Output Changes in researcher Intensity ratio over time Commodity 2010 1998 Rice 10.9 6 Maize (west and central Africa) 9.2 10 Cassava 1.2 3 Sorghum 1.7 5 *Beans 33.7 21 Source: Alene, et al,2011
  8. 8. Improving the Results measurement of R&Dsystems “A major knowledge gap in understanding and strengthening R&D systems stems from the lack of empirical application of framework, metrics, and benchmarks to measure organizational performance and institutional impact in the context of agricultural research” Ragasa, 2011• Organization design theory in the context of innovation system – Coordination mechanisms that provide incentives for innovation – Demand responsiveness and connectivity to other actors in the innovation system
  9. 9. Empirics from Ghana and Nigeria : PerceptionRatings)• Output, outcome and impact indicators are standard -Technologies generated -Publications – Adoption of technologies (most researchers unaware of the adoption rates) – Limited complementarity and consistency across the indicators• Connectivity – Linkage with other researchers exist, limited in the case of extension, farmers and other innovation actors• Organization culture and job satisfaction – Satisfaction with outputs – Staff morale – Perception on effectiveness of the organization
  10. 10. Way Forwards for results Measurement• There is no substitute for valid and credible data – Real time data for operational management not available in the majority of cases – Measurement error arising from reported area and output data (LSMS-ISA) – Data is a valuable resourcetreasure often kept in “armory” – Challenges with data sharing protocols hence despite the noble intentions espoused in; CAADP, CRP, SRO• Getting adequate data for results measurement is costly – Owing to scarcity of resources , collection of performance data and information is rarely given priority-donors are pushing for a reversal! – Greater chances of getting resources when framed as a research endeavor – Operational management data is often treated as confidential
  11. 11. Way Forwards for Results Measurement• Strategic Focus in SSA – ASTI initiative to support the NARS in the institutionalization of data collection and expand to include output indicators – Work with SRO and RECS – FARA and SRO’s to focus on quantifying the externalities/spillovers and, support the NARS in developing measurement approaches for effective coordination and management – NARS to plug into the broader innovation system and NIMES in order to demonstrate contribution to broader development agenda
  12. 12. Way Forwards for Results Measurement• Rigorous impact evaluation approaches recommended – Selective use of RCT and other quasi experimental methods given the associated costs – Develop rapid and robust approaches for measuring the impact of R&D on development outcomes• Operational management support – Great research opportunity in the area of agricultural innovation systems employing management science tools• Effective results measurement systems respond to information needs in a timely fashion-an art – Proactive strategic analyses – Consistent data collection effort – Focus on generating evidence and catalyzing use *Duplication of efforts arising from fragmented approach to data collection
  13. 13. THANK YOU

×