Successfully reported this slideshow.

RIMS+ surveys: A tool for project design and evaluation


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

RIMS+ surveys: A tool for project design and evaluation

  1. 1. RIMS+ surveys:A tool for project design and evaluation Nicholas Minot (IFPRI/Uganda) Atsuko Toda (IFAD/Vietnam) Nguyen Ngoc Ahn (DEPOCEN) Presented to IFAD/Rome From Hanoi, 29 March 2012
  2. 2. Outline of presentation Background on RIMS Changes in RIMS+ Cost and implementation issues Benefits: Results from three RIMS+ surveys in Vietnam Summary and conclusions
  3. 3. Background on RIMSResults and Information Management System (RIMS) Provides information at three levels  1st -level results refer to project activities and outputs  2nd -level results relate to project outcomes and reflect changes in beneficiaries behavior, improved performance and sustainability of groups, institutions and infrastructure  3rd -level results are associated with project impact on child malnutrition and household living standards. We focus on the household survey used to collect third-level results
  4. 4. Background on RIMSRIMS survey guidelines Should be implemented for large, national IFAD projects Should be done before, during, and at end of project Sample size: 900 beneficiary households Returning to same households not recommended  Concern about concentration of IFAD program efforts  Administrative complications of finding old households
  5. 5. Background on RIMSRIMS questionnaire Objective is to measure assets and child nutrition 6 pages (plus cover) divided into three sections  Section 1 – Household demographics  Section 2 – Housing, assets, and food security  Section 3 – Anthropometry
  6. 6. Background on RIMSStandardization of RIMS questionnaire Ensures comparability across countries Makes analysis relatively quick Assures quality But little flexibility in questionnaire design & analysis Does not collect intermediary indicators
  7. 7. Changes in RIMS+Overview of changesChanges Rationale1. Expanded questionnaire Collect additional information to diagnose farmer constraints, improve design of interventions, and measure impact on intermediate indicators2. Use of control group Better measurement of impact of project by controlling for broader changes in rural conditions3. Additional training and Improve quality of datasupervision4. GPS to geo-reference Facilitate return to same households (panel) andhouseholds better supervision of enumerators5. Flexible questionnaire & Address information needs of the IFAD project andanalysis IFAD planning in general
  8. 8. Changes in RIMS+ 1. Expanded questionnaire (cover + 14 pages)RIMS+ RIMS New info in RIMS+A. Member 1. Household + ethnicity, school attendance, & reasons forcharacteristics demographics not attendingA. Housing 2. Survey questions + roof material, ownership status, location of toiletA. Assets 2. Survey questions + agricultural equipmentA. Land (no info) Farm size, ownership, irrigation, distanceE. Crop production (no info) Production, sales, & prices for 25 crops; cost of 6 inputsF. Livestock & (no info) Herd size, sales, & costs for 12 types of animals,fisheries use of vet services, type of feeding
  9. 9. Changes in RIMS+ 1. Expanded questionnaire (continued)RIMS+ RIMS New info in RIMS+G. Extension & (no info) Access to extension, who uses, cooperatives,market access details of sales, distance to marketsH. Non-farm (no info) Income and business expenses for 11 non-farmactivities income sources, gender rolesI. Food security 2. Survey questions + coping strategies and quality of dietJ. Credit & (no info) Access to credit, info on loans receivedborrowingK. Socio-Economic (no info) Knowledge of and participation in SEDP processDevelopment PlanL. Risk & (no info) Perceived risk of six natural disastersvulnerabilityM. Anthropometry 3. Anthropometry No new information
  10. 10. Changes in RIMS+ 2. Use of control group  Control group is 300 households that are similar to beneficiaries but not in project area  Useful to control for changes in rural areas due to other factors Beneficiary Control Impact according Actual impact using households households to current info from control before-after group comparisonExample 1 Income rises Income rises 4% Suggests that project Actually, only a 4% 8% due to economic caused 8% increase increase due to project growth in incomeExample 2 Income does Income falls 4% Suggests that project Actually, 4% increase in not change due to drought had no effect income due to project
  11. 11. Changes in RIMS+2. Use of control group (continued)Outcomeindicator Beneficiary households Actual effect of project Before-after difference Control group is hypothetical path of beneficiary households without the project, based on growth in control group Before project After project Time
  12. 12. Changes in RIMS+3. Additional training and supervision Because questionnaire is longer and somewhat more complicated, need for additional training & supervision of enumerators IFPRI & DEPOCEN prepared detailed enumerator manual DEPOCEN provided 5 days of training plus testing of questionnaire DEPOCEN also provided additional supervision during data collection, particularly important in first week of data collection
  13. 13. Changes in RIMS+4. Use of GPS units GPS units are sometimes used in RIMS surveys Main purpose is to make it easier to find household to interview in later round of survey Additional benefit of verifying that enumerators have visited households in village
  14. 14. Changes in RIMS+5. Flexible questionnaire & analysis of results Original RIMS is analyzed in a “black box”  Advantage is analysis is fast, reliable, and comparable  But little opportunity to customize results for project RIMS+ questionnaire can be customized for projectType of IFAD project Possible customization of questionnaireFarmer training & Access to extension, sources of info, perception ofextension usefulness, adoption of advice, yieldLinking farmers to Travel time to markets, types of buyers, degree ofmarket competition, prices received, share soldPromotion of non-farm Number & composition of NFEs, profitability, trainingenterprises needs, perceived constraints, factors affecting successImproved access to Sources of credit, interest rates paid, use of credit, reasonscredit for use of informal credit, factors affecting repayment rate
  15. 15. Changes in RIMS+5. Flexible questionnaire & analysis of results RIMS+ analysis can be customized to address questions relevant for project design & implementation  Is access to extension services different for female-headed farmers?  Can pepper be successfully grown by small-scale farmers with limited resources?  Is targeting landless households more (or less) pro-poor than targeting farmers with less than 0.5 hectares?  Is satisfaction with project services higher in one district than in another?
  16. 16. Cost and implementation issuesExpanded questionnaire More information and more complicated questionnaire  Requires additional training and supervision  Longer interview time (double at least) Requires a new data entry program  Separate data entry in CSPro for 1200 questionnaires  At least 2 days in preparing CSpro entry data form  Another 2 days for training in data entry in CSPro in addition to RIMS training. Increased complexity in analysis and reporting
  17. 17. Cost and implementation issuesUse of control group Increased workload with financial implication (additional 300 non- project household) Implementing survey in non-project area is more difficult due to logistics, cooperation Data entry in both RIMS and CSPro  RIMS software to enter RIMS core questions for 900 beneficiary households  Data entry in CSPro for full questionnaire for1200 household sample  Additional training/supervision Project managers do not see immediate benefit
  18. 18. Cost and implementation issuesUse of GPS Increased training time (1/2 day) and additional time at household (10 minutes) Not easy to use due to language barrier Additional burden due to the fact that interviewers already have to carry weight and scale
  19. 19. Cost and implementation issuesCost estimates Component First-time costs Per survey costs Expanded questionnaire in Already carried out under Interview time is data collection IFAD-IFPRI Partnership approximately doubled Use of control group No fixed cost Increases field costs by 50-100% Additional training & Enumerator manual Approximately US$ supervision prepared under 10-15k per survey Partnership Use of GPS units Cost to purchase = Modest - GPS units can be US$ 100 x 20 units = shared across projects or US$ 2000 rented Analysis of data Large initial cost of For standard analysis, preparing analysis negligible. For programs, already customized analysis, undertaken by Partnership requires Stata skills
  20. 20. Results of Vietnam RIMS+Questions Which crops are pro-poor? How does crop commercialization vary across farmers? Do female-headed farmers have equal access to modern inputs? How important is income from non-farm activities? How to farmers perceive the risks of natural disasters? Is food security threatened by crop commercialization? How involved are farmers in the preparation of the Socio- Economic Development plans? Will raising farmer income improve child nutrition?
  21. 21. Results of Vietnam RIMS+ Which crops are pro-poor?• Rice is grown by majority of the poor, but fewer high-income households• Maize, groundnut, red onion, bananas, tea, and vegetables are grown by both poor and non-poor• Avocado, mango, durian, pepper, sugarcane, coffee, and cashew are grown disproportionately by high- income farms• This is not to say they can’t be grown by poor farmers, but any untargeted support to these crops will not be pro- poor
  22. 22. Results of Vietnam RIMS+Is input use less among female-headed households?• Not much evidence that input use per hectare is lower• But smaller farm sizes lead to smaller crop production and lower income
  23. 23. Results of Vietnam RIMS+What is the importance of non-farm income?• Even the 20% of farms with the smallest area (less than 0.10 hectares) earns the bulk of their income from crop production• 45% of smallest farms rent, sharecrop, borrow, or use illegally other land
  24. 24. Results of Vietnam RIMS+How do farmers perceive the risk of different natural disasters?• Perception of disaster risk varies by province• Also, perception of likely losses is greater for poor households
  25. 25. Results of Vietnam RIMS+Is food security threatened by commercialization?• Commercialization is defined as the share of the value of crop production that is sold• Relationship holds even after controlling for per capita income and farm size in regression analysis
  26. 26. Results of Vietnam RIMS+Will raising farmer income improve child nutrition? 5 Z-scores 0 -5 10 12 14 16 18 Log of per capita income Length/height-for-age Z-score Weight-for-length/height Z-score lowess haz06 lnpcinc lowess whz06 lnpcinc• Yes, but effect is weak• Many other variables influence child nutrition: sanitation, health care, education, child rearing practices, etc.
  27. 27. Summary & conclusionsAdvantages of RIMS+ Additional costsExpanded questionnaire gives much Fixed costs of preparing questionnaire,more information for diagnosis of manual, and data analysis programsproblems and measuring project- have already been incurred by IFAD-specific indicators IFPRI PartnershipControl group allows better Per-survey cost is increased 3x bymeasurement of impact of project, expanded questionnaire and controltaking into account trends in rural group. GPS and additional qualityareas control also imply costs.Additional training & supervision Cost of analyzing basic results usingprovides higher-quality data existing programs, but customization of questionnaire or analysis implies additional costs.Use of GPS units makes it easier torevisit same respondents in laterrounds of surveyAbility to customize questionnaire &analysis to meet project needs
  28. 28. Summary & conclusionsWhen is RIMS+ most suitable?  RIMS+ surveys probably not suitable for all IFAD projects because of additional costs  Conditions under which it is most suitable:  IFAD project design is flexible, can be revised in light of new information from survey  IFAD project focuses on a new topic or new region, so there is a need for information  There are gaps in knowledge about farm household livelihoods and behavior relevant to project  IFAD project is relatively large, implying an adequate M&E budget
  29. 29. Summary & conclusionsAdditional issues  Size of control group  At the moment, 900 treatment to meet standard RIMS requirement and 300 control  But typically control group is similar size  It would reduce costs to develop a Core Module and additional modules that are selected depending on project (e.g. agricultural marketing, credit, extension)  RIMS+ would require additional capacity building for IFAD project staff  Project has prepared an enumerator manual and data entry programs and could also prepare an implementation guidelines if needed