This document discusses the challenges of conducting impact evaluations of agricultural extension programs in Sub-Saharan Africa. It notes that while most impact evaluations have found positive effects, some studies have produced contradictory results. There are several challenges to impact evaluation, including difficulties establishing a counterfactual, attribution of outcomes to the program given multiple influences, and methodological limitations like lack of baseline data and small sample sizes. The document argues for moving beyond a focus on proving impact to instead emphasize learning from evaluations and using flexible, low-cost methods to improve programs over time.

1. Impact Evaluation as Learning and Accountability Tool for Agricultural Extension Programs: Challenges and Prospects HAILEMICHAEL TAYE BEYENE INTERNATONAL LIVESTOCK RESEARCH INSTITUE (ILRI)

2. Outline Introduction and Objectives Materials and Methods Results and Discussions Challenges of IE of Agricultural Extension Programs Future Prospects of IE of Agricultural Extension Programs Conclusions and Recommendations

3. Introduction and Objectives… Agricultural extension has long been recognized as an important factor in promoting agricultural development (Birkhaeuser et al., 1991; and Anderson and Feder, 2007). Consequently, the Sub-Saharan African (SSA) countries have been implementing various agricultural extension programs to improve agricultural production and productivity. Impact evaluation is an attempt to establish a casual relationship between an intervention and change in outcomes. If impact evaluations are not conducted appropriately, could lead to wrong conclusions which may lead to wrong decisions.

4. Introduction and Objectives… The difficulty of tracing the casual relationship between extension input and its impact is appreciated by various authors (Anandajayasekeram et al., 2008; Davis. K, 2008; Anderson, 2007). Several evaluations have been conducted on agricultural extension programs in SSA by various parties. This paper, based on analysis of some impact evaluation studies conducted in SSA attempts to identify challenges and future prospects of impact evaluation of agricultural extension programs in the region.

5. Materials and Methods Review of some impact evaluation reports on agricultural extension programs in SSA. A total of 21 impact evaluation results from 10 countries were collected from various websites for the analysis. Agricultural productivity trends reported by various authors are used. Mainly based on qualitative analysis. The quantitative data are analyzed using simple descriptive statistics mainly percentages.

6. Results and Discussions Agricultural Production and Productivity Most literatures agree that the performance of the agricultural sector in SSA has been disappointing. Per capita agricultural production was deteriorating in Africa during the past 45 years (Steven H and Peter B. R. H, 2010) Due to this the continent has turned from a food exporter to a net food importer. Yields per ha is stagnant, particularly for cereals, in contrast to substantial yield increases in other regions of the world in the years from 1960 to 2005 ( Staatz. JM & Dembele NN, 2007). Yield per ha is stagnant shows that the yield production growth is due to mostly area expansion (limited technological change )

7. Results and Discussions... Agricultural Extension Impact Evaluation Results 15(71%) impact evaluation studies reported positive significant impact. 6 (29%) reported non significant impact. Some of the impact evaluation studies also contradict each other. For instance: Bindlish & Evenson (1997) vs Gautam & Anderson (1999) Ayele et al (2005) and IFPRI (2008) vs EEA/EEPRI (2006)

8. Results and Discussions... Contradictions Agricultural productivity trends..low technological change. The IE results themselves.  Such exaggerations and contradictions lead to skepticism on the results (Davis, K. 2008).

9. Challenges of IE of Agricultural Extension Programs Impact evaluation is about attribution . it is difficult to attribute a change in outcomes to a specific intervention since there are so many different factors involved. Two challenges: Controlling the extraneous effects and design effects for estimating the net impact. The other inherent challenge in impact evaluation is estimating the counterfactual .

10. Challenges of IE of Agricultural Extension Programs… Comparable Control Group: difficult to establish control groups both from operational and ethical considerations. Selection bias: People involvement in agricultural extension programs is based on interest and resource availability. Additionally, trying new technology requires the capacity to take risks. Endogeneity in extension-farmer interaction: Extension contact which plays central role for success of extension programs is highly affected by farmers’ characteristics and actions.

11. Challenges of IE of Agricultural Extension Programs… Other sources of information and technology: There are sources of information and technology other than the extension program. Extension to be effective needs other support services. Agriculture especially in the Sub-Saharan African countries is highly dependent on nature and it is complex, diverse and risk prone.

12. Challenges of IE of Agricultural Extension Programs… Methodological Factors Indicators used to measure outcomes: This is related to the definition of impact evaluation. ‘ Impact’ as the final level of the causal chain. The difference in the indicator of interest (Y) with the intervention (Y1) and without the intervention (Y0). That is, impact = Y1 – Y0. It is about attribution by identifying the counterfactual value of Y (Y0) in a rigorous manner. The two definitions could overlap if Y is an outcome indicator.

13. Challenges of IE of Agricultural Extension Programs… Impact Evaluation Designs : impact evaluations in SSA have shortfall in employing appropriate evaluation designs. Sampling: The sample should not only be large enough but also random so that it can represent the population and provide adequate statistical power. Statistical analysis: Regression techniques are preferred for analysis in impact evaluations so that the effects of confounding factors could be controlled. Research Design and Implementation: In SSA due to lack of skilled manpower, budget, time and logistics evaluations are done in tight schedule and in a hurry.

14. Challenges of IE of Agricultural Extension Programs… Lack of baseline and reliable data Shortage of Skilled Manpower to Conduct Rigorous Impact Evaluation Poor M&E system Budget constraints for Impact Evaluation Studies Future Prospects of IE in SSA Paradigm shift in agricultural research and extension calls for review the linear M&E approaches. Creating casual relationships between interventions and outcomes will not be easy due to the complex, dynamic and institutional pluralism. More focus on learning. Shift from proving to improving impact.

15. Conclusions and Recommendations Conclusions Agricultural productivity is disappointing. Majority of IE reported positivity results. In some cases contradictory results. There are methodological, data and capacity related factors that affect the results of IEs. Recommendations Building evaluation capacity, establishing appropriate M&E systems. Employing easy, less costly ,flexible, participatory, and learning focused tools. From proving to improving impact.