See more here http://www.pim.cgiar.org/2014/08/22/advisory-and-knowledge-services-evidence-for-what-works/

1. Advisory and Knowledge Services: Evidence for What Works
PIM Advisory Services Research Team 2014
PIM/IFPRI SEMINAR
17 September 2014, 12:00 – 2:00 pm
Washington, DC

2. How Did We Get Here?
IFPRI “Best Fit” Framework
GFRAS Evaluation Guide
PIM Research Area

3. Past Extension Work with PIM
•Building R4D learning platforms in Latin America, Africa Asia (CIAT)
•Evaluation of innovative extension approaches (ICRAF)
•Analyzing effects of decentralization and governance environment on policy processes and outcomes (IFPRI)
•Does agricultural training and female representation in extension foster investments among female farmers? Lessons from a policy experiment in Mozambique (IFPRI)
•Approaches for evaluating and increasing access to rural services by women and the poor (ILRI)

4. PIM’s “Advisory Services” Research
•October 2013 workshop
•2014 work plan under PIM
•IFPRI
•CIRAD
•GFRAS
•ICRAF

5. Overview of 2014 Work
Rationale
•Extension…advisory services…knowledge sharing…
•Critical institution for agricultural development
Goal
•Evidence & outreach to strengthen advisory services
Photo: Flickr, part of the image collection of the International Rice Research Institute (IRRI)

6. 2014 Activities
1.Framework to assess performance of pluralistic extension systems
2.Evaluation of impact of extension on ag productivity
3.Systematic collection of “global good practices”
4.Assessments in Central Asia, Brazil, North Africa
5.Knowledge sharing platform
6.Proposal for 2015-2016 work

7. Impact of Agricultural Extension
•AIM: Provide new set of evidence on impact of public investment
•Past meta-reviews on economic impact and ROR (Birkhaueser et al. 1991; Alston et al. 2000; Evenson 2001): generally positive, but results widely varied, averages misleading, too much noise, suffer econometric deficiencies
•Recent studies addressed some econometric deficiencies (Dercon et al. 2013; Owens et al. 2003; Gautam 2000; Benin et al. 2011; Davis et al. 2012)
•We will use existing datasets with components on extension to provide new empirical evidence on impact of agricultural extension
•We will further disaggregate analysis and unpack extension input to explain reasons for low or high marginal impact and suggest ways to move forward

8. Malawi Case
•What is differentiated impact across regions, zones and groups of access to extension services?
•Do different sources of information or service providers matter in explaining differences in productivity?
•Does access to other service providers or other sources of information a substitute or complementary to public extension services?
•Do different types of advice or information provided matter in explaining productivity?
•Does the gender of the receiver of information matter in explaining productivity?
IHS3 2010/11 covering 31 districts; 560 rural EAs; 10,038 rural HH; and follow-up panel in 2013/14 covering 150 EAs; 2,400 rural HH

9. Assessments and Evaluations
•How do we know RAS functions as supposed to?
•How to develop best-fit practices from wide range of contexts and policy regimes?
•Set of criteria for RAS program assessment and evaluations - Effectiveness, Efficiency, Relevance, Impact, Sustainability
•GFRAS evaluation indicators as starting point
•Currently engaged in applying methodology for Brazil, Central Asia, & North Africa

10. What does this mean for our team?

11. Which framework to assess the performance and impact of pluralistic extension systems ?
•Objective : design and test a framework to support national policy makers and top managers of service providers:
•Which extension programs suit which needs under what circumstances?
•How much public funding can be justified as a good investment?
•How performance and impact can be monitored?

12. Evaluations of Extension Performance: What do we have?
•Many evaluations of performance to assess the relevance of investments in extension (Neuchatel Initiative, World Bank, GFRAS, etc.)
•Mainly based on comparison between objectives of the project and achievements
•Using OECD criteria (relevance, efficiency, effectiveness, sustainability)
•Mix of surveys (SWOT, focus group, statistical surveys, etc.)
•More comprehensive studies about the evolution of RAS and the reforms undertaken:
•Labarthe et al. on EU advisory systems, Babu in India, etc.

13. Evaluation of Extension Impact: What do we have?
•A few research to assess change of farmers’ skills (Cameron in Australia, de Romemont in Benin, etc.)
•A large number of evaluations to assess extension programs (T&V, FFS, and other) (see GFRAS documents)
• Impacts on adoption of technologies, yields, and incomes,
•Some research to compare impacts of
•different methods (video, farmer to farmer, etc.)
•type of service provider (NGO, private provider)
•A few research to assess spill-over

14. Conclusions about Evaluations
•Positive impacts but variability and over-estimation with regards to the real evolution of agriculture
•Methodological questions (the quality of the data, the design of samples, the attribution)
•Such methods consider the service providers as a black box (not very useful to provide recommendations)
•There is no study about the impacts of an extension system (with a set of providers providing different type of advice) at territorial level/national level

15. Proposal to Assess Complex Pluralistic Extension Services
•Holistic approach based on IFPRI framework (Birner el al. 2009)
•Need to operationalize taking into account the national level and the local level
•Improving the impact pathway approach (Douthwaite 2003)
•Designing a mix of qualitative and quantitative methods

16. Context- ual factors
Policy,
farming systems, access to market)
Perfor- mance efficiency effectiveness inclusion sustainability
Impact
Economic
Social
Environmental
Governance structure (funding)
Advisory methods
Service provider capacities
Farm house- hold
Representation
Decision
Change of practices

17. Methodological Proposal
•Mapping of RAS at national level (governance, funding, capacities, methods)
•Case studies to represent the diversity of service providers and diversity of innovation processes
•Impact pathway linking key outputs (information, training, platform, etc.), outcomes (change at farm level) and impacts (economic, social, environmental, etc.)
•Two levels to assess impacts : direct impacts, indirect impacts
•Impact at national level = sum of impacts at case studies level

18. Tools to be Designed
•Timeline diagram describing actions undertaken by service provider and external events influencing advisory services
•Impact pathway diagram showing place where interactions advisors/farmers really take place, and changes at different levels
•Performance diagram is aiming to quantify an visualize the performances (effectiveness, efficiency, equity (targeting), sustainability)
•Impacts diagram to quantify and visualize different impacts

19. Key Methodological Questions to be Addressed
•How to select case studies to have relevant results?
•How to select performance indicators that make sense for actors and in line with national public policies on extension?
•How to calibrate a method to quantify the impacts (using existing surveys, carrying out specific surveys) (see Wellard 2014)?
•Next step : test the method in one or two countries

20. Global Best-fit Practices
The problem:
•Much available about how a particular practice works in a particular context: “case studies”
•But little synthetic material available - how a particular practice works in different contexts
•E.g. farmer to farmer extension: dozens of case studies available about a particular experience but only one study found that cuts across countries and facilitating organizations: Selener et al 1997 – covering only Latin America

21. •High demand from practitioners for simple materials that present how particular practices perform in different contexts and key lessons for implementation
•Our proposed solution:
•Commission preparation of syntheses of best-fit practices and produce reports and training materials useful for policy, rural advisory services managers and practitioners (different products likely needed for different target groups)

22. •Syntheses will be sought from 5 key dimensions of advisory services, using a structured framework for case selection:
•Governance structures (e.g., level and sources of financing)
•Policy (e.g., experiences with national advisory services policies)
•Capacity and management (e.g., staff numbers, expertise)
•Advisory methods (e.g., the use of ICTs)
•Cross cutting (e.g. gender, nutrition)

23. 5 Dimensions of extension
Theme
Case
Country Typology
Inter-country
Intra-country
Socio- economic
Political
Organiza- tional
Ecological
1.Governance
Decentrali- zed extension systems
Ethiopia
Agrarian
Federal
(decentrali- zed)
Government
providers
Low natural resource base, subsistence, medium density
2. Policies
3. Capacity & management
4. Methods
5. Cross-cutting
23

24. Implementation plan
•Fund raising. GIZ already on board
•Establish advisory committee composed mainly of potential users of the information
•The committee, in consultation with regional ag. advisory system networks, selects potential topics
•Issue call for proposals to conduct syntheses
•Award grants; develop and complete studies
•Develop informational materials and implement communication and capacity strengthening strategy to promote their use

25. Where Do We Go from Here?
•Team meetings, 18 Sep, 6-7 Nov
•Develop common framework and proposal
•Feedback from experts*
•Engage partners, find funding
*Mark Lundy (CIAT), Cheryl Doss (Yale), Arame Tall (CCAFS), Aden Aw-Hassan (ICARDA), Laurens Klerkx (Wageningen), Cathy Colverson (University of Florida), Ismail Moumouni (University of Parakou, Benin), Victor Manyong (IITA), Pierre Labarthe (INRA), Andrea Knierim (Hohenheim University), Evelyne Kiptot and Anne Degrande (ICRAF) and Katarina Kosec, David Spielman, Ephraim Nkonya, Tewodaj Mogues (IFPRI).

26. Response by the Panel
•Cheryl Doss, Yale
•David Spielman, IFPRI
•Valerie Mueller, IFPRI
•Pierre Labarthe, INRA

27. Questions? Discussion & Conclusions