Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
FIGHTING POVERTY WITH WHAT WORKS (EVIDENCE): The case for impact evaluations 
Thomas Chataghalala Munthali, PhD 
ECAMA Ina...
Reflection 
But test everything; hold fast what is good. 
1 Thessalonians 5:21 (ESV)
Introducing IPA 
•Innovations for Poverty Action (IPA) is an international non-profit research organization that has a str...
IPA’s VISION 
More evidence, less poverty 
IPA’s MISSION 
To discover and promote effective solutions to global poverty pr...
IPA works on three core areas: 
•Research 
•Capacity building on randomized evaluations 
•Policy outreach and scaling-up
HOW ARE WE CURRENTLY ACHIEVING OUR GOALS? 
CREATE EVIDENCE 
•Randomized evaluations, with a focus on understanding why pro...
WHAT WE HAVE ACHIEVED 
CREATE EVIDENCE 
•Over 150 Principal Investigators 
•14 Country Offices 
•300 number of randomized ...
WHERE WE WORK? 
14 country Offices – planning to expanding to 20 in next five years.
Innovations for Poverty Action (IPA) Malawi
Basic History 
IPA started work in Malawi in 2009 but got registered in 2010. IPA-Malawi has grown over the years and now ...
Priority sectors 
IPA-Malawi now has a five year Strategic Plan (2014-2018) that focuses on four priority sectors: 
i.Educ...
Main Challenge with evidence-based planning 
•Randomized Control Trials (RCTs) not fully known/appreciated/utilised by sta...
Introduction to impact evaluation
Reflection 
Friend tells Marx: “Life is difficult!” 
Marx asks Friend: “Compared to what?”
What is Evaluation? 
Evaluation 
Program Evaluation 
Impact Evaluation
Monitoring and Evaluation 
Evaluation 
Program Evaluation 
Impact Evaluation 
Monitoring
Components of Program Evaluation 
•What is the problem? 
Needs Assessment 
•How, in theory, does the program fix the probl...
Evaluation should usually be conducted: 
A. Externally and independent from the implementers of the program being evaluate...
Impact evaluation methods 
1. Non- or Quasi-Experimental Methods 
a. Pre-Post 
b.Simple Difference 
c.Differences-in-Diffe...
Ice Breaker 
For every complex question there is a simple answer -- and it's wrong. 
-- H.L. Mencken
Impact evaluation methods Contd. 
2. Randomized Experiments, also known as: 
–Random Assignment Studies 
–Randomized Field...
Clip on randomized control experiments/trials
What is a randomized experiment? 
Start with simple case: 
•Take a sample of program applicants 
•Randomly assign them to ...
Key advantage of randomized experiments 
Because members of the groups (treatment and control) do not differ systematicall...
Intervention 
Time 
Primary outcome 
Counterfactual 
Impact 
What is Impact?
How to measure impact? 
•What would have happened in the absence of the program? 
•Take the difference between 
what happe...
Some responses Contd. 
Access to obstetric care and distribution of reproductive health commodities such as medication an...
Some of the research (impact evaluations) done in Malawi 
Active Projects: 
•Finance Sector and private sector development...
Overview: IPA Malawi 
Completed Projects: 
•Agriculture 
–Making Networks Work (MNW) : The Project aims at evaluating how ...
Completed projects Contd. 
•Enterprise Development: 
–Malawi Savings Project (GLASE): to measure impact of increasing acce...
Ice Breaker 
According to a new survey, women say they feel more comfortable undressing in front of men than they do undre...
For More … 
Visit: www.poverty-action.org/malawi
You need technical support? 
•Partner with Innovations for Poverty Action (IPA) for impact evaluations on your project(s)!...
Sister organizations 
•The Abdul Latif Jameel Poverty Action Lab (J-PAL), based at the Massachusetts Institute of Technolo...
Reflection 
Therefore, I will always remind you about these things, [since] you are well- grounded in the truth that you n...
ZIKOMO KWAMBIRI! 
IPA-Malawi
• 
• THANK YOU! 
Each question you ask will be evaluated, so do you really want to ask questions? 
CONTACTS 
Thomas Chataghalala Munthali, PhD COUNTRY DIRECTOR 
Innovations for Poverty Action (IPA) Malawi Country Office 
...
Fighting poverty with what works - The IPA mission across the globe by Sarah de Tournemire
Upcoming SlideShare
Loading in …5
×

Fighting poverty with what works - The IPA mission across the globe by Sarah de Tournemire

404 views

Published on

Fighting poverty with what works

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Fighting poverty with what works - The IPA mission across the globe by Sarah de Tournemire

  1. 1. FIGHTING POVERTY WITH WHAT WORKS (EVIDENCE): The case for impact evaluations Thomas Chataghalala Munthali, PhD ECAMA Inaugural Research Symposium: Golden Peacock, 10th October 2014
  2. 2. Reflection But test everything; hold fast what is good. 1 Thessalonians 5:21 (ESV)
  3. 3. Introducing IPA •Innovations for Poverty Action (IPA) is an international non-profit research organization that has a strong local presence through its 14 country offices serving 51 countries. •IPA works towards the goal of reducing poverty by ensuring that policy is based on scientifically proven evidence.
  4. 4. IPA’s VISION More evidence, less poverty IPA’s MISSION To discover and promote effective solutions to global poverty problems.
  5. 5. IPA works on three core areas: •Research •Capacity building on randomized evaluations •Policy outreach and scaling-up
  6. 6. HOW ARE WE CURRENTLY ACHIEVING OUR GOALS? CREATE EVIDENCE •Randomized evaluations, with a focus on understanding why programs work or not •Partnership with academics and implementing organizations •Creating research infrastructure around the world TURN EVIDENCE INTO ACTION •Dissemination through policy-briefs and conferences •Working closely with implementation partners throughout the evaluation •Direct role in implementing successful programs at scale
  7. 7. WHAT WE HAVE ACHIEVED CREATE EVIDENCE •Over 150 Principal Investigators •14 Country Offices •300 number of randomized evaluations completed or underway •More than 500 staff trained in implementing randomized evaluations across 50 countries TURN EVIDENCE INTO ACTION •Two programs scaled across countries, leading to 40 Mn children dewormed and thousands chlorine dispensers •Several organizations changed their program based on research results. •Thousands of practitioners attended conferences . To see studies done by IPA in Malawi and beyond visit: www.poverty-action.org
  8. 8. WHERE WE WORK? 14 country Offices – planning to expanding to 20 in next five years.
  9. 9. Innovations for Poverty Action (IPA) Malawi
  10. 10. Basic History IPA started work in Malawi in 2009 but got registered in 2010. IPA-Malawi has grown over the years and now has long-term staffing of 17 with an annual project budget of over US$1.6million. Over 11 research projects have been completed with considerable influence on policy especially at the decentralized levels. Most of the research work has been in private sector and enterprise development as well as agriculture. Notably, most of the research has this far been largely driven by PI research interests. Currently, IPA-Malawi is developing a research agenda that reflects PI interests as well as Government’s development priorities as defined in the Malawi Growth and Development Strategy (MGDS II: 2011-2016).
  11. 11. Priority sectors IPA-Malawi now has a five year Strategic Plan (2014-2018) that focuses on four priority sectors: i.Education (Especially basic education and early child development), ii.Health (especially reproductive health), iii.Enterprise development (especially SMEs), and iv.Agriculture (especially climate smart and smallholder commercial farming).
  12. 12. Main Challenge with evidence-based planning •Randomized Control Trials (RCTs) not fully known/appreciated/utilised by stakeholders as an important impact evaluation tool for measuring attribution of programmes to observed impacts.
  13. 13. Introduction to impact evaluation
  14. 14. Reflection Friend tells Marx: “Life is difficult!” Marx asks Friend: “Compared to what?”
  15. 15. What is Evaluation? Evaluation Program Evaluation Impact Evaluation
  16. 16. Monitoring and Evaluation Evaluation Program Evaluation Impact Evaluation Monitoring
  17. 17. Components of Program Evaluation •What is the problem? Needs Assessment •How, in theory, does the program fix the problem? Program Theory Assessment •Does the program work as planned? Process Evaluation •Were its goals achieved? The magnitude? Impact Evaluation •Given magnitude and cost, how does it compare to alternatives? Cost Effectiveness
  18. 18. Evaluation should usually be conducted: A. Externally and independent from the implementers of the program being evaluated B. Externally and closely integrated with program implementers C. Internally D. Don’t know
  19. 19. Impact evaluation methods 1. Non- or Quasi-Experimental Methods a. Pre-Post b.Simple Difference c.Differences-in-Differences d.Multivariate Regression e.Statistical Matching f.Interrupted Time Series g.Instrumental Variables h.Regression Discontinuity
  20. 20. Ice Breaker For every complex question there is a simple answer -- and it's wrong. -- H.L. Mencken
  21. 21. Impact evaluation methods Contd. 2. Randomized Experiments, also known as: –Random Assignment Studies –Randomized Field Trials –Social Experiments –Randomized Controlled Trials (RCTs) –Randomized Controlled Experiments
  22. 22. Clip on randomized control experiments/trials
  23. 23. What is a randomized experiment? Start with simple case: •Take a sample of program applicants •Randomly assign them to either: Treatment Group – is offered treatment Control Group - not allowed to receive treatment (during the evaluation period)
  24. 24. Key advantage of randomized experiments Because members of the groups (treatment and control) do not differ systematically at the outset of the experiment, any difference that subsequently arises between them can be attributed to the program rather than to other factors. 25
  25. 25. Intervention Time Primary outcome Counterfactual Impact What is Impact?
  26. 26. How to measure impact? •What would have happened in the absence of the program? •Take the difference between what happened (with the program) …and - what would have happened (without the program) = IMPACT of the program
  27. 27. Some responses Contd. Access to obstetric care and distribution of reproductive health commodities such as medication and contraceptives may be seriously hampered after disasters caused by climate change such as floods, putting women at risk for unintended pregnancies, sexually transmitted infections, including HIV. Women and girls are generally expected to care for the sick, particularly in times of disaster and environmental stress. This limits the time they have available for income generation.
  28. 28. Some of the research (impact evaluations) done in Malawi Active Projects: •Finance Sector and private sector development: –Direct Deposits and Financial Literacy : measuring the changes in household financial behavior that are caused by promoting savings in the form of financial literacy trainings and the introduction of regular or labeled savings accounts •PIs: Dean Yang, Jessica Goldberg, Lasse Brune –Fingerprinting to reduce risky borrowing: aims at expanding the knowledge base about the impact of fingerprinting on repayment behavior and thus costs and risks of lending that microfinance organizations generally encounter in rural markets. •PIs: Dean Yang, Jessica Goldberg, Xavier Gine –Business Registration and Impact Evaluation Project (BRIE): Evaluating whether or not business registration improves enterprise performance. • PIs: Francisco Campos and Marcus Goldstein •Agriculture –‘Making family farmers more productive’ – Fomento project: The goal of this project is to implement and evaluate a modified version of a capital and information transfer program for extremely poor farmers . •PIs: Susan Godlonton, Kate Ambler, Allan de Brauw
  29. 29. Overview: IPA Malawi Completed Projects: •Agriculture –Making Networks Work (MNW) : The Project aims at evaluating how best to leverage communication within networks to increase the spread of two productive but under- utilized agricultural technologies in Malawi (pit planting for maize and improved crop residue management). •PIs: Lori Beaman, Jeremy Magruder, Paul Fatch •Rural Development: –Access to Transport in Rural Malawi: Assessing the socio-economic impact of access to public transport on the lives of those living in remote rural areas. •PIs: Jessica Goldberg, Dean Yang, Rebecca Thornton •Social protection: –Public Works Programme: aims at examining the impact of the World Bank supported public works program (PWP) as an income promoting mechanism via its impact on agricultural productivity through increased access to yield-improving inputs and any impacts on labor markets. •PIs: Jessica Goldberg, Kathleen Beegle, Emanuela Galasso
  30. 30. Completed projects Contd. •Enterprise Development: –Malawi Savings Project (GLASE): to measure impact of increasing access to saving services on business creation, survival and growth, as well as on farming productivity and returns •PIs: Dupas, Karlan, Robinson –Borrower Responses to Fingerprinting for Loan Enforcement in Malawi: Assessing ability of fingerprinting in improving borrowers’ repayment rates •PIs: Dean Yang, Jessica Goldberg, Xavier Gine –Reducing Barriers to Saving in Malawi: the impact of enhanced savings products on the use of different agricultural inputs, farm output, and overall well-being in rural farming households? •PIs: Dean Yang, Jessica Goldberg, Xavier Gine •Governance –Domestic Election Observers: to assess the impact of domestic election observers on detecting and preventing electoral fraud. •PIs: George Ofosu and Daniel Posner
  31. 31. Ice Breaker According to a new survey, women say they feel more comfortable undressing in front of men than they do undressing in front of other women. They say that women are too judgmental, where, of course, men are just grateful. -- Jay Leno
  32. 32. For More … Visit: www.poverty-action.org/malawi
  33. 33. You need technical support? •Partner with Innovations for Poverty Action (IPA) for impact evaluations on your project(s)! •Remember we are not a team of consultants – won’t charge you professional fees!
  34. 34. Sister organizations •The Abdul Latif Jameel Poverty Action Lab (J-PAL), based at the Massachusetts Institute of Technology (MIT) and the University of Cape Town (UCT), consists of a network of leading academics who use randomized impact evaluations to test the effectiveness of alternative ways to improve welfare. –Runs free Executive Education course on randomized evaluations www.povertyactionlab.org •Evidence Action responsible for taking proven impact initiatives to scale – just successfully chlorine dispensers in Zomba. www.evidenceaction.org
  35. 35. Reflection Therefore, I will always remind you about these things, [since] you are well- grounded in the truth that you now have. 2 Peter 1:12 (God’s Word Translation)
  36. 36. ZIKOMO KWAMBIRI! IPA-Malawi
  37. 37. • • THANK YOU! Each question you ask will be evaluated, so do you really want to ask questions? 
  38. 38. CONTACTS Thomas Chataghalala Munthali, PhD COUNTRY DIRECTOR Innovations for Poverty Action (IPA) Malawi Country Office P.O Box 31093 Lilongwe 3 Email: tmunthali@poverty-action.org Skype: thomas.chataghalala.munthali Malawi Office: + 265 1 762 424 | Mobile: +265 999 803140 | USA Office (HQ): 101 Whitney Ave. New Haven, CT 06510 USA | +1 203-772-2216 www.poverty-action.org/malawi

×