PIALA presentation IFAD 26 oct 2015 v8
•Download as PPTX, PDF•
2 likes•305 views
The Participatory Impact Assessment and Learning Approach (PIALA) has been developed over the past three years as an potentially scalable approach that can help IFAD and its partners collaboratively assess, explain and debate its contributions to rural poverty impact. Above all, it aims to improve our understanding of who it is that benefits from IFAD-supported projects, as well as the nature, dimensions of the impacts achieved, and the factors that have combined to drive that impact. However, it also seeks to provide reliable and relevant quantitative evidence on impact. Ultimately, it is envisaged as an approach on the frontier of impact evaluation theory, complementing conventional regularity- and counterfactual-based approaches using statistical, quasi-experimental and RCT methods. During the three years, the PIALA approach has been designed, tested and refined; and it has been piloted in two IFAD-supported projects: the Doing Business with the Rural Poor project (DBRP) in Viet Nam, and the Root and Tuber Improvement and Marketing Programme (RTIMP) in Ghana. This has been funded by IFAD’s DFID-financed Innovation Mainstreaming Initiative and the Bill and Melinda Gates Foundation. This presentation by the project leader Adinda van Hemelrijck offers an overview of the approach and the main findings from the Ghana case study.
More Related Content
Viewers also liked
Similar to PIALA presentation IFAD 26 oct 2015 v8
Similar to PIALA presentation IFAD 26 oct 2015 v8 (20)
Recently uploaded
Recently uploaded (20)
PIALA presentation IFAD 26 oct 2015 v8
- 1. Results from the impact evaluation of RTIMP in Ghana Participatory Impact Assessment & Learning Approach (PIALA) Adinda Van Hemelrijck IFAD 26 October 2015 Rome 1
- 2. IMIInitiative $ 90 K (+100K) $ 230 K (+60 K) DBRP RTIMP 2 PIALA Research strategy DBRP Evaluation Vietnam PIALA reflections Vietnam PIALA debriefs RTIMP Evaluation Ghana PIALA reflections PIALA learning event PIALA @ Conferences (AEA‘13, EES’14, UKES’15, EAS’15, IDEAS’15) PIALA Practice Paper PIALA guide for commissioners
- 3. Project Objectives Assessing to what extent impacts occurred (or not) Debating how impacts can be enhanced Explaining why impacts occurred (or not) 1. Produce rigorous qualitative and quantitative evidence for global reporting and advocacy 2. Facilitate inclusive analysis and reflection for collaborative learning 3. Generate a scalable model for strengthening IFAD’s self-evaluation system PIALA Purposes 3
- 4. Design Challenges • Causal inference in the absence of a clean or credible comparison group • Program effects spill over • Influence of other R&T livelihoods programs • Heterogeneous pattern of program treatment • Generate solid evidence-based debate about “what has worked for whom, under which conditions and why?” • Look at systemic impact broader than the program performance to enhance collaborative learning around emerging issues that undermine program theory 4
- 5. PIALA features & standards Focus & frame the evaluation Collect & link data Manage quality Synthesise evidence Analyse & debate contributions Rigour Inclusiveness Feasibility Systemic ToC approach Concurrent participatory mixed-methods Cluster sampling around market-bounded systems 2-stage participatory sensemaking Configurational analysis 5
- 6. DBRP THEORY OF CHANGE Enhanced access for small producers to local inputs providers, agro-processing SMEs and traders Increased rural job and business opportunities for the rural poor Enhanced provincial competitiveness Sustainable pro-poor value chain development ECONOMIC EMPOWERMENT: Rural poor people can work, save money, take a loan, invest in their land and businesses, access markets and influence socio-economic development plans ß SUSTAINABLE & EQUITABLE POVERTY REDUCTION: Improved livelihoods and increased wealth and wellbeing THREATS: · Low prices for value chain products · High prices for value chain inputs · Unstable market behaviour · Salinity intrusion due to CC · Land degradation · Increasing pests · Uncontrolled pestices Effective implementation of governmental and international infrastructure, training and market access programs Effective local implementation of SEDP and poverty reduction plans DBRP: FU Mobilising commune investments (CIF) in infrastructure (90%) and market training (10%) Capacity-building of commune staff in participatory and market-oriented pro-poor SEDP Well targeted and Informed poverty-reduction planning Development and capacity- building of CIGs with 70% poor members (CIG Dev Fund)WU DBRP: Management capacity-building of district and commune staff (M&E, procurement, accounting, etc.) DBRP: Capacity building of provincial business facility agencies (One-Stop-Shops for business registration, business forums/exhibitins, trade promotion, legal assistance, etc.) PPPs for piloting new credit models and setting up local SMEs in support of local short value chain development Management capacity-building of provincial line agencies Annual HH surveys on labour, poverty status and causes of poverty, informing commune-level poverty reduction planning Agribank Capacity-building of district-level business facility and extension services (One-Stop-Shops for business registration, agricultural training, study tours, etc.) Strengthening the district-level Vocational Training Centres providing training for poor HHs in poverty- reducing livelihood activities (garment, chickens, making brooms, weaving, etc.) clothes and mats DARD DPI DOLISA Access to credit for poor HHs involved in production and trade in short value chains IFAD contribution: Agribank and CIG Dev Fund since 2012 Access to training and services for enhancing poor HHs’ productivity and businesses IFAD contribution: vocational training by DOLISA; market training and study tours by the Commune (10% CIF); market facilitation by the IPA and DOIT Sustainable construction and maintenance of infrastructure involving local people IFAD contribution: roads, bridges & markets (90% CIF) DOIT Access to credit for poor HHs Access to training & extension for poor HHs Access to markets Jobs & livelihood opportunities Poor people’s voice District capacity to provide training & services Commune capacity in SEDP & poverty reduction planning and management Provincial capacity in facilitation of local businesses, PPPs and entrepreneurship Wealth & wellbeing Institutional relationships Framing the evaluation in Vietnam
- 7. M2b: Training & starter pack for commercial seed growers to multiply certified R&T seeds C3a: R&T processors grow and develop into GPCs that are profitable enterprises O3: Enhanced R&T processed volumes of high quality at scale O2: Enhanced R&T productivity and production at scale M2c: Farmer Field Forums (FFF) engage farmers, extension agents and researchers in developing, demonstrating and promoting appropriate R&T production technologies C2a: Resource-poor R&T farmers & seed producers gain access to and adopt improved R&T seed varieties, technologies & inputs to improve crop husbandry, soil fertility and pest management practices C2b: Resource-poor R&T farmers organise and register as FBOs that can access credit and bargain better market prices C1b: Resource-poor R&T processors, farmers & seed producers commercialize and establish effective supply chain linkages C1a: R&T supply chain farmers & processors are capable of developing and implementing viable business and marketing plans C3c: R&T supply chain farmers and processors gain access to business financing and market-linking services M3b: Subsidized upgrading of advanced R&T processors into Good Practice Centres (GPCs) that demonstrate and promote good quality processing & management practices C3b: R&T supply chain processors gain access to and adopt standardized processing technology and good quality management practices O1: R&T supply chain actors effectively solve their supply & demand issues and timely obtain technical support, resulting in sustainable and inclusive CCs linked to old and new markets I2: Improved R&T- based livelihoods for the rural poor in CC catchment areas M2a: R&D for developing bio-agents M1c: Information, Education & Communication (IEC) about CC support services, inputs and technologies M1a: Training of resource-poor farmers and processors involved in the R&T supply chains in business development and marketing M3c: Co-financing of R&T supply chain farmers and processors by matching 40% RTIMP funds with 50% loans from PFIs and 10% self-financing through the Micro-Enterprise Fund (MEF) M3a: Training of artisans to produce and maintain standardized processing equipment for R&T supply chain processors and GPCs I1: Rural poor people in CC catchment areas have increased access to food & income to sustain an active and healthy life M1: District Stakeholder Forums (DSFs) for addressing supply & demand issues and technical support needs of R&T supply chain actors members M1b: Supply Chain Facilitation (SCF) and market linking through the Initiative Fund (IF) EC1 EC3c EC1 EO2 EO1 EC3b MEF GPC DSF FFF Framing the evaluation in Ghana
- 8. Changes & causes of access to food & income R&T livelihood changes and causes Reach and effects of program mechanisms (DSF, FFF, GPC, MEF) Household Survey (n 840) on changes in food, assets, income, R&T revenues Generic Change Analysis (n 439) Ranking & causal flow mapping of changes in wealth & wellbeing Livelihood Change Analysis (n 400) Livelihood matrix & causal flow mapping SenseMaker Lithe (n 393) Constituent Feedback (n 341) with DSF, FFF & GPC/MEF beneficiaries Key Informant Interviews (n 100) with officials & service providers Participatory Sensemaking Workshops with research participants in 23 districts (n 640) and national (n 106) Review of secondary data (Ghana Living Standard Survey 2010, RTIMP cost-benefit studies 2014, SPN & MTR 2010, 12, 13 & 14) Collecting and linking data 8 Parti FGDs
- 9. 3 Zones - 8 regions 4 commodity chains (HQCF, PCF, Gari & FYE) 25 random districts 30 random supply chains 840 random HHs 1180 FGD participants (45% women) 152 Parti FGDs (109 gender-specific) 840 HH Surveys (24% female-headed) Parti Sensemaking WSs (in 23 districts with 640 ppts; national with 106 ppts) 100 KIIs (75 districts + 25 national)9
- 10. Configurational causal analysis 1. Per district: – binary coding (0-1) of program mechanisms – Scoring (0-6) of causal links and evidence 2. Aggregated: – Unzipping evidence along the ToC: cluster and compare districts with different configurations of scores & explanations for each causal claim – zipping up findings along the ToC: draw conclusions about program contributions to impact Analysing & debating contributions Strength of causal link (necessity) Consistency of causal link (sufficiency) HIGH LOW LOW HIGH 1 2 3 4 5 6 0 10
- 11. Scoring of causal links and evidence DSF: District Stakeholder Forum; FFF: Farmer Field Forum; GPC: Good Practice Centre; MEF: Micro-Enterprise Fund 11
- 12. 12 Participatory sensemaking Involving stakeholders in collective analysis & debate of evidence of impact and areas for future investment (640 pps in districts 23 WSs, 106 in nat WS) • Cross-validate the evidence and the ToC • Obtain extra layer of data • Enable voice (30 % beneficiaries) • Debate “less success” needing “more effort” • Rate program contributions
- 14. Roots & Tubers Improvement and Marketing Program (2007-2014) PCR 2015: RTIMP operated in 106 districts (out of 216), spent USD 23.6 m (incl. USD 18.83 m IFAD) and reached 15,000 - 20,000 farmers & processors with training, finance and market-linking Program goal: Enhance household access to food & income through improving R&T livelihoods and strengthen market-based systems generating profitability along R&T commodity chains Intervention components: 1. Market-linking of supply chains, incl. providing market-oriented information and training, supporting innovation, and building networks for supply, investment and market linking through District Stakeholder Forums (DSF) 2. Enhance smallholders production, incl. multiplication and distribution of improved R&T planting materials and technologies through Farmer Field Forums (FFFs) 3. Grow smallholder root and tuber processing businesses, incl. upgrading of MSEs to Good Practices Centres (GPC) to serve as demonstration and market hubs, and creating access to finance through Micro-Enterprise Fund (MEF) 14
- 15. M2b: Training & starter pack for commercial seed growers to multiply certified R&T seeds C3a: R&T processors grow and develop into GPCs that are profitable enterprises O3: Enhanced R&T processed volumes of high quality at scale O2: Enhanced R&T productivity and production at scale M2c: Farmer Field Forums (FFF) engage farmers, extension agents and researchers in developing, demonstrating and promoting appropriate R&T production technologies C2a: Resource-poor R&T farmers & seed producers gain access to and adopt improved R&T seed varieties, technologies & inputs to improve crop husbandry, soil fertility and pest management practices C2b: Resource-poor R&T farmers organise and register as FBOs that can access credit and bargain better market prices C1b: Resource-poor R&T processors, farmers & seed producers commercialize and establish effective supply chain linkages C1a: R&T supply chain farmers & processors are capable of developing and implementing viable business and marketing plans C3c: R&T supply chain farmers and processors gain access to business financing and market-linking services M3b: Subsidized upgrading of advanced R&T processors into Good Practice Centres (GPCs) that demonstrate and promote good quality processing & management practices C3b: R&T supply chain processors gain access to and adopt standardized processing technology and good quality management practices O1: R&T supply chain actors effectively solve their supply & demand issues and timely obtain technical support, resulting in sustainable and inclusive CCs linked to old and new markets I2: Improved R&T- based livelihoods for the rural poor in CC catchment areas M2a: R&D for developing bio-agents M1c: Information, Education & Communication (IEC) about CC support services, inputs and technologies M1a: Training of resource-poor farmers and processors involved in the R&T supply chains in business development and marketing M3c: Co-financing of R&T supply chain farmers and processors by matching 40% RTIMP funds with 50% loans from PFIs and 10% self-financing through the Micro-Enterprise Fund (MEF) M3a: Training of artisans to produce and maintain standardized processing equipment for R&T supply chain processors and GPCs I1: Rural poor people in CC catchment areas have increased access to food & income to sustain an active and healthy life M1: District Stakeholder Forums (DSFs) for addressing supply & demand issues and technical support needs of R&T supply chain actors members M1b: Supply Chain Facilitation (SCF) and market linking through the Initiative Fund (IF) EC1 EC3c EC1 EO2 EO1 EC3b Livelihoods and poverty status will improve, if smallholders commercialise and become part of strong and inclusive commodity chains • FFFs will help small R&T farmers commercialise by organising into FBOs and demonstrating improved planting materials and technologies • R&T smallholders will commercialise and supply the chains, if they have knowledge and capacity to increase production, access markets and develop businesses • DSFs can help develop inclusive and sustainable R&T commodity chains by linking supply chains to markets • GPCs will help small R&T processors commercialise by demonstrating improved technologies & equipment, hence helping them access finance through the MEF • Well-trained R&T smallholders will be able to obtain a loan through the MEF to invest in their businesses and adopt improved equipment and technologies Assumptions 15
- 16. Evaluation questions 1. To what extent did these assumptions hold true (or not) and under which conditions? 2. What were the major barriers for farmers and processors (particularly women & young adults) to commercialize and access old and new markets? 16
- 17. 17 • 83 % HHs reported NO food shortage in 2014 (51 % in 2013) • 2009 WFP: 95 % population had access to sufficient food • 2008 RTIMP baseline: av. 85 % HHs living from R&T could feed themselves • 15 % HH moved up to income categories > USD 2/day due to R&T livelihood improvements • Increase in total HH income, and HH income from R&T, appear to have reached a ceiling at USD 4-5 /day, indicating improved but limited profitability Increased access to food & income
- 18. 18 (sig.ooo) “The main effect of the livelihood change in my experience is...” Experiences about negative changes (130) access to education access to income Experiences about positive changes (243) access to education access to foodaccess to income 25% (60) 24% (58) 19% (47) 22% 6% (14) 18% (23) 16% (21) 38% (49) 7% (9) access to food • Statistical analysis of 837 HH surveys showed a more direct correlation of HH income from R&T with total HH income than with HH access to food • SenseMaker analysis of 373 micro-narratives from FGDs showed 94 % participants experienced R&T livelihood changes affecting income and education, rather than food • R&T livelihood changes affected access to food through creating access to income rather than directly
- 19. 19 0 5 10 15 20 25 30 0-500 501-1000 1001-2000 2001-5000 >5001 2009 27 20.1 14.1 7.6 26.9 2013 21.4 21.3 20.2 12.1 22 2014 17.8 23.3 22 12.7 20.4 Change in distribution of the range of HH income from R&T in 2009-2014 Change in distribution of the range of total HH income in 2009-2014 0 5 10 15 20 25 30 0-500 501-1000 1001-2000 2001-5000 > 5001 2009 17.8 24.6 23.9 12.3 9.6 2013 12.8 16.6 27 23.5 10.7 2014 11.7 15.2 24.9 25.8 13.6
- 20. R&T livelihood improvements • R&T livelihood improvements were found relatively strong but inconsistent in 52 % and weak in 48 % supply chains • SenseMaker analysis (n 378) showed that 33 % FGD participants attributed R&T livelihood changes to RTIMP rather then to other or own efforts • Configurational analysis found livelihood changes attributable to RTIMP in 32 % districts (strong in 12 % ; weak in 20 %), of which 88 % showing weak market linking • Very weak/no livelihood improvements were found in districts where RTIMP mechanisms were dysfunctional or not in place • R&T livelihood improvements occurred country-wide due to new R&T varieties and technologies, causing an influx into R&T farming spilling into processing • Inadequate business and market linkages + economic downturn turned the tide and caused prices to drop, negatively affecting R&T livelihoods from 2013 • Statistical analysis (n 837) showed overall limited R&T profits and investments, while Configurational analysis found market failure as the main cause 20
- 21. Market-linking of R&T supply chains • Market linking of supply chains through DSFs was found weak in 84 % districts – In 57 % districts, DSFs somewhat contributed to strengthening supply chains, but failed linking them to markets – In 43 % districts, DSFs’ contribution to developing supply chains was virtually nil, with no market linking efforts • In 16 % districts, market linking was found stronger due to stronger DSF & GPC performance, but insufficient to withstand external threats and prevent market saturation 21
- 22. R&T supply chain development & smallholder commercialization • R&T has changed from food to cash crop for the resource-poor, but market saturation has offset initial gains from enhanced production and rendered R&T livelihoods fragile • Commercialisation of smallholders has remained limited and proven unsustainable in 88 % districts due to limited market opportunities, caused by – Weak market linking combined with overproduction – Poor infrastructure and inappropriate competition regulations, rendering smallholders more vulnerable to unfair competition (particularly in districts with low/no RTIMP presence) • Stronger but inconsistent commercialisation of smallholders was found in 12 % districts due to stronger GPC performance 22
- 23. Enhanced smallholder R&T production • Enhanced smallholder R&T production and productivity was found strong in 76 % districts due to adoption of new planting materials & technologies (of which 73 % attributed to RTIMP and 27 % to RTIMP + WAAP/others) • Weak results in 24 % supply chains due to limited adoption + other factors (incl. yam beetle, weather patterns, land tenure issues, limited extension, input prices) • 36 % districts had FBOs, but none stemmed from FFFs or were influenced by RTIMP, and in only 2 cases were farmers able to access business finance (Nanumba North and Tano North) – SPN Nov 2014: FBOs successfully developed out of FFFs in 4 locations, but none had yet been able to obtain credit or bargain better market prices • At present there is no evidence of whether FBOs could be sufficient for helping farmers bargain better prices, fight unfair competition, obtain business finance, access markets and commercialize – All evidence points to the need for more market opportunities in the first place, requiring better roads, policies and regulations more supportive of smallholder businessess 23
- 24. Access to new R&T planting materials & technologies • Access to and adoption of new planting materials & technologies was found strong and consistent in 72 % districts, due to the success of FFFs and the visible efficiency and benefits of adoption – PCR 2015: 187,275 farmers received improved R&T planting materials, and 15,154 farmers were trained through 451 FFF • Women participated and benefitted less since FFFs mostly targeted and reached male farmers (between 40-60 owning 1-2 ha land) – Women generally are more involved in cassava production than men and traditionally do most of the work – Since R&T changed from food to cash crop, men took a greater interest and FFFs have encouraged and supported this • 74 % FFF participants reported they were able to apply what they learned, which helped them expand their businesses • Young farmers (< 25) and women were less positive about FFF benefits and less confident to express their needs and ask for help 24
- 25. Enhanced R&T processing • Enhanced processing as a result of more people growing and processing high quality cassava was found strong and mostly inconsistent in 50 % cassava supply chains, and weak in the other 50 %. – In 17 % supply chains, this was found strong and consistent due to stronger GPC performance in terms of market creation, reach, strength and inclusiveness of supply chains, and adoption of improved technologies and equipment – In 33 % supply chains, this was found strong but inconsistent due to the limited reach of GPC operations and spill-over of excess R&T production to processing using both new and traditional equipment – In 50 % supply chains, this was found weak due to dysfunctional GPCs (more than half) or a very limited reach and adoption of improved technologies and equipment (nearly nil in more than half of the cases) • New technologies and equipment have proven cost-efficient and potentially profitable. Yet adoption was limited in 83 % supply chains due to: – limited reach and effectiveness of GPC’s as learning and good practice centres, and – limited investment capital of GPCs and individual processors (mostly women) 25
- 26. • Over half of GPC beneficiaries reported they were able to apply what they learned at the GPC • Nearly one third found GPCs helped expand their businesses • Women were more positive than men, although less confident to express their needs and ask for help at GPCs (as were young people < 25 felt less confident) • Only 9 % GPC leaders were positive about GPC influence on beneficiaries’ businesses and their ability to apply what they learn 26 Fig 10.1.3.
- 27. Access to business finance • HH survey and FGDs showed limited access to financial support to invest in existing or new livelihood activities – Only 15 % HHs obtained finance in 2009-2014 to invest in R&T (of which 55 % from relatives and friends, 22 % from Susu or local moneylenders, 14 % from RCBs, 4 % from gov programs and 5 % from other sources) – MEF funding was found “obtained” by processors in 2 districts (Techiman & Mangong) and GPCs in 2 other districts (Assin South & Abura Assebu) • MEF not available and accessible to most smallholders & GPCs – MEF was found formally unavailable in 50 % districts (RTIMP 2014: only in few) – Procedure for obtaining and paying off MEF funding was too onerous, making smallholders pre-invest and sustain operations with insufficient capital or immediate returns on investment (RTIMP 2014) – Reluctance of PFIs to approve applications due to investment risks in present conjuncture (RTIMP 2014) 27
- 28. Immerging issues Answers to the evaluation questions 28
- 29. Assumptions true (or not)? 29 • Livelihoods & poverty status will improve, if smallholders commercialise and become part of strong and inclusive commodity chains PARTLY TRUE • Stronger GPC and DSF performance in 12 % distr • Concerted efforts by partners to develop solid supply links, build capacities & relations in 12 % distr • Limited R&T profits & investments due to market failure • R&T smallholders will commercialise, if they have knowledge and capacity • DSFs can help develop inclusive and sustainable R&T commodity chains UNTRUE • Weak DSF performance in 84 % distr • Limited reach and capacity of DSFs and GPCs to expand, innovate and develop markets • Limited market opportunities & unfair competition • FFFs will help small farmers commercialise by organising into FBOs and demonstrating improved planting materials & technologies PARTLY TRUE • Large adoption of plant materials & techs in 84 % distr • Commercialization limited by lack of market opportunities and unfair competition in 88 % distr • Farmer organisation insufficient to overcome these • GPCs will help small R&T processors commercialise by demonstrating improved technologies & equipment • Well-trained R&T smallholders will obtain finance through the MEF to invest in R&T PARTLY UNTRUE • Limited adoption of new techs & equipm in 83 % distr due to (a) limited GPC reach & effectiveness, and (b) smallholders’ limited investment capital • MEF largely unavailable and inaccessible in >83 % distr
- 30. ] Validating the ToC Sufficient HH access to sufficient food & income Improved R&T livelihoods Enhanced market- linking of smallholders Strong R&T supply chain linking and smallholder commercialisation through DSFs + support services Enhanced smallholder production Access to improved R&T planting materials & technologies through FFFs Enhanced smallholder processing Access to improved R&T processing equipment & technologies and business finance through GPCs and MEF 30 ✗ P+T ✔P+T T ✔ ✗P
- 31. • Lack of market opportunities and unfair competition due to a failing rural infrastructure and inadequate policy and regulations for smallholder business development 31 Barriers to smallholder commercialization and market linking? • Lack of investment capital (particularly for women and young adults <25) due the lack of access to finance caused by (a) investment risks and (b) limited capacity • Limited capacity of DSFs and GPCs to develop strong supply chains, link these to existing and new markets, and expand their catchments/reach • Lack of capacity of small farmers and processors to organise into sustainable agri-businesses that are able to create market value
- 32. Recommendations • Rethink the DSF mechanism as a forum for inclusive VC linking • Clarify and monitor the role of GPCs as supply chain leaders • Develop women- and youth-specific FFFs that are more market/business-oriented • Develop and pilot an appropriate mechanisms for VC funding • Advocate for investments, policies & regulations supportive of smallholder commercialization 32
- 34. PIALA benefits, costs & requirements • Rigorous evidence of impact of programs with no/limited counterfactuals and baselines • ToC of multiple interventions engaging multiple stakeholders in evidence-based reflections on collective impact • Empowerment-related benefits from engaging beneficiaries and other constituents in data collection and collective sensemaking • Inclusive analysis and evidence- based debate of emerging issues related to impact and sustainability 34 • Cost USD 100 - 300 K Ghana (full scope – full scale) • 18 K for design, training & ToC • 146 K for method & fieldwork • 70 K for analysis & reporting (incl. loc & nat sensemaking) • Committed research team with sufficient capacity and experience in mixed methods impact evaluation • Evaluation manager (consultant) to support and supervise procurement, design, fieldwork and analysis
- 35. RTIMP Evaluation commissioners: GoG/RTIMP & IFAD Country Office RTIMP evaluation team: Participatory Development Associates (PDA) PIALA management team: Edward Heinemann (IFAD) Adinda Van Hemelrijk (IFAD) Richard Caldwell (BMGF) PIALA design support group: IFAD PMD, SKM & IOE; BMGF PIALA core design team: Adinda Van Hemelrijck (team leader) Irene Guijt, Andre Proctor, Jeremy Holland PIALA external reference group: Robert Chambers (IDS, UK) Marie Gaarder (World Bank IEG) Kent Glenzer (MIIS, USA)
Editor's Notes
- Intro by Ed Heinemann and Paul Winter (10’) PIALA and evaluation design in Ghana (15 incl. Q&A’) RTIMP evaluation results & recommendations (20’) Final reflections (5’) Reflections by Glowen Kyei-Mensah (5’) Reflections by Ulac Demriraq (5’) Questions and discussion (50’) Final notes by Paul Winter and Ed Heinemann (10’) ED’s intro: Background to IMI: 80m people out of poverty target, requiring rigorous quantitative IMI -> learning initiative International development desperate to ‘prove impact’, while IFAD9 committed to 30 impact evaluations for assessing contributions to lifting 80m people out of poverty. Need for appropriate approaches serving both learning and accountability and capturing complexity of IFAD-funded projects. = approaches that can: •Produce rigorous evidence to justify investments at scale •Generate debate and learning with partners •Provide credible explanations to influence policy •Create space for participation SDGs: demands of sustainability and inclusiveness added to those of effectiveness, making these approaches even more necessary
- GSS May 2013: 31 % of rural HHs in Ghana are female-headed.
- Different perspectives of causality (Punton and Welle 2015 ; Befani 2014 ; Stern 2012) Counterfactual –with/without comparison (Mill’s method of difference) Regulatory –frequency of association (Mill’s method of agreement) Configurational –necessity & sufficiency of causal constellations (logical inference) Generative –strength and consistency of hypotheses + evidence (process tracing)
- The code doesn’t say anything about the quality, reach and effects of the mechanisms and thus their influence on the causal links. The formal availability of a certain mechanism doesn’t imply it was actually accessible. From impact level to program mechanisms - look for major similarities & differences between districts across three agro-ecological and administrative zones findings from analysis of impact-related changes and causes used to examine the changes and causes in each of the contributions claims separately Districts/zones clustered according to their scores on the enhancement of market linking for the first claim, enhancement of production for the second claim, and enhancement of processing for the third claim Differences and similarities were then examined within and between the different clusters for each of the three claims.
- Programme Beneficiaries: 187,275 farmers received improved R&T planting materials 15,154 farmers were trained through 451 FFF 26 existing R&T processing businesses (12 groups and 14 individuals) were upgraded as GPCs 201 exposure visits to GPCs were organized involving 3,777 persons (2,777 females and 1000 males) 3,959 smallholders (58.4 % women) were trained in business development and marketing 1,235 processors received MEF funding
- First chart: 15 % movement of HHs from 501-2000 to 2001-5000 income categories only 1 % to > 5000 Second chart: 10 % movement of HHs from 0-500 to 501-2000 R&T value categories only 4 % beyond to 2000-5000 -0.1 % to > 5000
- USD 2-4 /day = 2001-5000 GHS /year First chart: 15 % movement of HHs from 501-2000 to 2001-5000 income categories only 1 % to > 5000 Second chart: 10 % movement of HHs from 0-500 to 501-2000 R&T value categories only 4 % beyond to 2000-5000 -0.1 % to > 5000
- Sensemaker: +ve changes (n 246) were attributed to RTIMP (37 %), own efforts (23 %), and a combination of both (16 %) -ve changes (n 132) were attributed to RTIMP (28 %), own efforts (23 %), and combination of RTIMP & others (20 %) Very weak/no livelihood improvements were found in districts where RTIMP mechanisms were dysfunctional or not in place: North Dayi, Agona East, Pru, Tano North, Wassa Amenfi West) Ghana experienced an economic crisis since 2013, with the Ghana cedi dropping up to 40% against the US dollar in 2014. Cf. The Guardian (8 August 2014), Reuters (13 May 2015), The Economist (20 June 2015). HH analysis: 50 % HHs (of which 67 % female-headed) invested in R&T production and 11 % HHs (70 % female-headed) invested in R&T processing Increase in HH income from R&T reached a ceiling at USD 4 /day, indicating limited profitability
- Wa East somewhat forms an exception: there has been an improvement in people’s livelihoods and wellbeing in the area due to the supply chain development of cassava flour for local consumption, which generated increases in savings and investments in fixed and moveable assets and other livelihoods activities. Since there was no GPC, farmers and processors mostly engaged in drying cassava for processing it into flour to make Kokonte, which is part of the local staple food. While the production of cassava improved due to the introduction of the new varieties and technologies, the DSF helped stimulate market activity in the area. In the FGDs, supply chain actors confirmed they benefited from the DSF activities, the use of the ESOKO platform, and the learning visit to Damongo. But unfair competition created distrust and disadvantaged resource-poor farmers who were exploited by middlemen and the so-called “market queens” of Wa. They sell their produce on credit to processors and traders, yet processors and traders are not willing to pre-finance farm inputs, and transporters are not prepared to carry farmers’ produce on credit. Trust and collaboration seem an important precondition for successful supply chain linking and commercialisation. Appropriate regulation and mechanisms need to be in place that must ensure fair competition. The lack of access to improved processing equipment (e.g. for gari) also has helped prevent the market saturation that occurred in almost all other districts. Production, processing and market linking were clearly more attuned to the limited capacity and infrastructure available in this district. THIS SHOWS THE NEED FOR AN INTEGRATED VC DEVELOMENT APPROACH, in which activities and budgets are planned for each of the supply chains based on the analysis of market opportunities, infrastructural conditions, and local capacity to create market value, attract investors and prevent unfair competition.
- Example of an FBO that works well under such circumstances in Tano North (FBO at Adwenepa GPC formed an FFF, obtained finance from the district to buy tractors, increased quality and qunatity, and bargained good prices at Tepa market ) A study on the efficiency and effectiveness of FFFs under RTIMP conducted in 2014, revealed that farmers’ membership of formal or informal farmer-based organizations (FBOs) across the country was quite high, in particular among FFF participants. The study further stated that most farmers either maintained their FFF group or converted their group into registered FBOs. PIALA evaluation found insufficient evidence to confirm this (although in multiple occasions FFF participants expressed the desire to maintain their groups
- Farmers started to process their excess cassava into gari but mostly in the old fashioned way, yet were able to produce more and better quality gari due to the new cassava variety. Access to new processing technologies and equipment was created by introducing a cassava processing equipment package, training local artisans, and by upgrading small processing enterprises to GPCs that could serve as demonstration, learning and practice centres and as market-hubs for processors and farmers. 26 existing R&T processing businesses (12 groups and 14 individuals) were upgraded as GPCs 201 exposure visits to GPCs were organized involving 3,777 persons (2,777 females and 1000 males) The success of GPCs largely depended on its capacity to innovate and create new market value/demand, its reach of farmers and processors in the catchment area, and the trust it had built between the various supply chain actors and their buyers and service providers needed to form strong supply chains Where livelihoods were the strongest, this was undeniably the result of a strong RTIMP presence with strong performing GPCs and better functioning DSFs, which proved that under more conducive conditions it is possible for resource-poor farmers and processors to adopt or use the improved technologies and equipment and develop more profitable smallholder businesses, and for GPCs to become innovative and profitable social enterprises with a wide reach of the surrounding communities. It is clear that for this to be feasible and sustainable, much more work need to be done on the market-linking and market creation, infrastructure development, the development of appropriate regulations with regards to fair competition and trade, and the development of a mechanism such as the DSF or VC committees that empower all actors involved in the VC to address issues of unfair competition, breaches of agreement, transportation and supply & demand linking.
- The limited availability of the MEF was also confirmed by the 2014 MEF study in which the fund was found operational in only a few districts. The MEF arrangement required the beneficiary to contribute 10% of the total cost of the improved equipment, while the PFI provided 50 % in the form of credit and RTIMP/IFAD added 40 % in the form of a matching grant. The MEF procedure was the following: The DADU undertook a needs assessment on the prospective beneficiaries as a basis for possible financing. Upon submission of an application (mostly ranging between 728 GHS to 60,000 GHS), the PFI then inquired if the potential beneficiary met the requirements. In the case of the Ecobank for instance, processors were required to submit firm orders with pro-forma invoices and contracts from key customers before loan approval. If the potential beneficiary met all criteria, then the loan was approved and the application was sent to the RTIMP national office that then granted authorization to transfer the matching grant component to the requested PFI. Finally, a supplier was paid to manufacture and deliver the requested equipment to the MEF beneficiary after s/he fulfilled his/her 10 % contribution to the investment. Smallholders’ limited capital to sustain and expand businesses insufficiently factored in in the design of the MEF mechanism (2014 MEF study) Smallholders generally have limited capital to expand their businesses. According to the 2014 MEF study, this was insufficiently factored in in the design of the MEF mechanism. To meet the requirement of a 10 % pre-investment, many MEF beneficiaries used nearly all the capital they had, hence ended up with improved equipment but little capital for operations. (MoFA, 2014c) The hierarchy of the PFIs often delayed the process of approval, which on its turn also delayed the supply of the equipment to the MEF beneficiaries, leaving them to wait without any return on their pre-investment and with little or no capital left to continue their businesses, while loan deduction started with immediate effect, causing a serious risk for many to default. (MoFA, 2014c) In Upper West Akim for instance, the GPC had applied for the MEF and put in 10,000 GHS without receiving any funds or getting back its money for more than 2 years, which caused a financial deficit that affected GPC operations and payment of farmers.
- RTIMP has undoubtedly made an important contribution to creating opportunities for resource-poor smallholders to improve their lives and livelihoods by turning R&T from a merely subsistence into a cash crop Without sufficient markets, returns from investments in enhanced R&T smallholder production and processing are unlikely to sustain and remain positive Risks & sustainability threats of further investments in commercializing R&T? FFFs initially effective but insufficient Necessity of strong supply chains and market links
- 26 existing R&T processing businesses (12 groups and 14 individuals) were upgraded as GPCs The potential power of such a business relationship was shown in a few cases where GPCs functioned as open social-private profit centres where resource-poor farmers and processors learned to use improved technologies and equipment and create added value of their produce. Where GPCs were profitable and attractive businesses in particular for women and young processors, this was largely due to its capacity to innovate and demonstrate innovation and thus create new market value/demand, its reach of farmers and processors in the surrounding communities, and the trust it built between the various supply chain actors and their buyers and service providers to establish strong and inclusive supply chains. Rethink the DSF mechanism as a forum for inclusive VC linking Capacity for market analysis & value creation Regular DSF meetings involving all VC actors Expansion of reach/membership (incl. women and young adults) Monitor conditions for fair and inclusive competition Clarify and monitor the role of GPCs as supply chain leaders Define “effective supply chain leadership” needed for inclusive VCs Monitor quality and effectiveness of business relationships with VC actors Capacity for product diversification/innovation Expansion of reach/membership (incl. young adults) Develop women- and youth-specific FFFs that are more market/business-oriented Monitor market opportunities and conditions Develop and pilot an appropriate mechanisms for VC funding Credit packages accessible for smallholders Risk transfer and distribution mechanisms Advocate for investments, policies & regulations supportive of smallholder commercialization Investment in minimal rural infrastructure Policy and regulations promoting inclusive and fair competition Incentives for industrial use of cassava in end products (e.g. beer, starch, bio-fuels)