Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Time matters: ‘short’ term vs. ‘medium’ term impacts

893 views

Published on

Soumya Balasubramanya
Making Impact Evaluation Matter: Better Evidence for Effective Policies and Programs
September 3-5, 2014
Manila, Philippines

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Time matters: ‘short’ term vs. ‘medium’ term impacts

  1. 1. Photo: Photo :DToavmid v Banra Cziaekre/InWbeMrgI he/IWMI www.iwmi.org Water for a food-secure world Photo: Andrea Silverman/IWMI Name Date Event Location Time matters: ‘short’ term vs. ‘medium’ term impacts 1 Soumya Balasubramanya Making Impact Evaluation Matter: Better Evidence for Effective Policies and Programs September 3-5, 2014 Manila, Philippines Balasubramanya, S., Pfaff, A., Bennear, L., Tarozzi, A., Ahmed, K.M., van Geen, A. (2014). Evolution of households’ responses to the groundwater arsenic crisis in Bangladesh: information on environmental health risks can have increasing behavioral impact over time. Environment and Development Economics, 19(05), pp. 631-647.
  2. 2. www.iwmi.org Water for a food-secure world Summary • Impact of an arsenic information and testing campaign on adoption of risk-avertive behaviors over time. •Followed households over two time periods and tracked behaviors and perceptions over 2003-05 and 2005-08. •Impact of information on risk-avertive behaviors depends on ability to recall information –Households that recalled: behaviors were persistent and double the # of households took up behaviors over time –Households that didn't’ recall: observed perverse behaviors 2
  3. 3. www.iwmi.org Water for a food-secure world Background • GW is primary drinking source; accessed through private tubewells (shallow, < 30 m; 90% of wells are shallow ) • GW contaminated with naturally occurring arsenic; unpredictable in shallow aquifer • WB- funded countrywide groundwater testing program b/w 1999-2003 • Tested households’ tubewells for arsenic; painted spouts red/green • Encouraged households to share green wells • Village level awareness campaigns • Media campaigns 3
  4. 4. www.iwmi.org Water for a food-secure world What happens within two years of testing? • Schoenfeld, 2005 – Census of all tubewells tested by WB in 2003; in 62 villages of Araihazar Upazila – Interviewed the wives of the well owners – Recorded color of well (observed by enumerator) – Elicited safety perception (asked respondent if well was safe or unsafe) – Elicited switching behavior b/w 2003-05 • Switching behaviors from unsafe wells ~20% 4
  5. 5. www.iwmi.org Water for a food-secure world What happens over time? • Household values both convenience (well nearby) and safe drinking water (green well) – Switch if cost (switching) < benefit (switching) •Can the costs and benefits change over time? –‘Social learning’ by observing peers (Miguel and Kremer, 2004; Munshi and Myaux, 2006) –Household ‘loses’ information –Concern about health risk reduces (Karlan et al., 2010) –Safe well owners reduce access (Fehr and Fischbacher, 2003; Hanchett et al., 2002) 5
  6. 6. www.iwmi.org Water for a food-secure world Two Questions • Is well-switching a persistent behavior change? • How does adoption of well-switching evolve over time? 6
  7. 7. www.iwmi.org Water for a food-secure world Methods and Data • Follow-up study on households whose switching behaviors had been previously studied in Schoenfeld, 2005 – Randomly selected subset of households in 59 villages – Interviewed same person as in previous survey – 1557 households/wells (1038 unsafe; 519 safe) – Use sampling weights 7
  8. 8. www.iwmi.org Water for a food-secure world Data 8
  9. 9. www.iwmi.org Water for a food-secure world Consistency in recall 9 • 78% recalled switching behavior of 2003-2005 consistently • Always consider households with consistent recall of 2003-2005 switching behavior •By 2008, no color left on any well spout •78% recalled well safety consistently (well color = safety perception in 2005 = safety perception in 2008)
  10. 10. www.iwmi.org Water for a food-secure world Did impact of testing increase or erode over time? 10 • Consider households with consistent recall of well tests •If switched in 2003-2005, check if switches were persistent –16% of households at unsafe wells switched during 2003-2005 •14% stayed put •2% switched to a second well they thought safe •If did not switch during 2003-2005, check if switched during 2005-2008 –17% of households at unsafe-tested wells had switched –Switching over five years doubles that over two years
  11. 11. www.iwmi.org Water for a food-secure world 11
  12. 12. www.iwmi.org Water for a food-secure world What about those that ‘forget’ information? 12 • Consider households with inconsistent recall of well tests (remembered correctly in 2005, but not in 2008) •In 2005 –Paint visible on all wells (1039 red/ 519 green) –Almost all households recalled well safety correctly (97% unsafe/ 95% safe) •In 2008 –Paint washed off –77% unsafe tested households recalled safety accurately –81% safe tested households recalled safety accurately
  13. 13. www.iwmi.org Water for a food-secure world Would it help to reinforce provision of arsenic information? 13 •Safe-tested hhs that changed safety perceptions after 2005: 26% switched in 2005-2008 •Unsafe-tested hhs that changed safety perceptions after 2005: 11% switched in 2005-2008
  14. 14. www.iwmi.org Water for a food-secure world 14
  15. 15. www.iwmi.org Water for a food-secure world Discussion • Behavioral: • Recall of arsenic information changes over time • Recall is critical for behavior •Evaluation: •Short-term evaluations present a snap shot •Impacts are heterogeneous •Policy perspective: •Cheap, responsible for most reduction in arsenic exposure (World Bank, 2005; Johnston et al., 2010; Ahmed et al., 2006) •Supporting continued well testing: new well constantly installed; 1/3 wells untested 15
  16. 16. www.iwmi.org Water for a food-secure world Thank you! s.balasubramanya@cgiar.org 16
  17. 17. www.iwmi.org Water for a food-secure world 17
  18. 18. www.iwmi.org Water for a food-secure world 18

×