Impact Evaluations                         for                  Social Inclusion                  Hélène Giacobino        ...
The Need for Evaluation• We have little hard evidence on key questions   – What is the most cost-effective program to redu...
Why Are Programs Not Evaluated?• Partisans want to show their program had an  impact   – Further funding, re-election, pro...
J-PAL: Jameel Poverty Action Lab• A network of researchers  at universities around the  world• Focused on randomized  eval...
Build             capacity                         J-PAL                            ImproveEvaluate  social               ...
J-PAL today: 5 Offices & 64 Affiliates
J-PAL Evaluations around the World• 318 randomized evaluations in 52 countries• About 162 are completed
J-PAL Europe• Started in 2008 with 9 affiliates• Today: 18 affiliates (Belgium, Denmark, Germany,  France, United Kingdom,...
How to evaluate the impact of an idea?• Implement it on the ground in the form of a real  program, and see if it works• Co...
Randomized Evaluation• Randomized evaluations (or RCTs) allow us to  identify impact rigorously   – People are randomly as...
Basic set up of a randomized evaluation                 Not in               evaluation   Target  Population              ...
Randomized assignment• This method works because of the law of large  numbers• Both groups (treatment and control) have th...
The question of ethic• Resources are very limited any way• This random assignment is usually seen as very fair• Inefficien...
La Mallette des Parents
La Mallette des Parents (the parentstoolkit)Questions:   • Is it really possible to improve parents’     involvement ?   •...
The program• Children and their parents in first year after primary  school in low-income schools• A toolkit:   – a bookle...
Messages to the parents• All parents can help their children• Work outside of school is extremely important for  success• ...
Protocol: Four groups    Test Classes              Control classes    Volonteer                 Volonteer                 ...
Parents involvement• More interactions with school:  – Meetings with teachers  – Participation in parents organizations• H...
Childrens behavior• Large improvement, even for classmates whose  parents were not volunteers• 34% less likely to be sanct...
Policy implication• Simple and inexpensive program• Rigorous evaluation: can convince schools or  governments that such ac...
Policy lessons• Directed parent discussion groups are an effective  policy tool for increasing parental involvement, even ...
Other Programs…• Counseling the unemployed• Counseling and job placement for young graduate job  seekers• Counseling welfa...
Scale-ups: million of lives improved
More information           www.povertyactionlab.org           www.povertyaction.org          www.worldbank.org/dime      w...
• Many thanks !
Upcoming SlideShare
Loading in …5
×

Hélène Giacobino előadása

280 views

Published on

Hélène Giacobino (Abdul Latif Jameel Poverty Action Lab) előadása a Haza és Haladás Alapítvány szociálpolitikai konferenciáján - 2012. március 29.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
280
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hélène Giacobino előadása

  1. 1. Impact Evaluations for Social Inclusion Hélène Giacobino Director J-PAL EuropeJbjb;kjbkjbkb
  2. 2. The Need for Evaluation• We have little hard evidence on key questions – What is the most cost-effective program to reduce unemployment? What is the real impact of microcredit?• Evidence is important: – for maximizing the impact of limited resources – to give donors and policy makers evidence to select better programs• Evidence provides an objective platform for debate• Evaluations sometimes demonstrate that conventional wisdom needs to be rethought
  3. 3. Why Are Programs Not Evaluated?• Partisans want to show their program had an impact – Further funding, re-election, promotion, depend on it• Evaluation is not that easy – Participants are often very different from non- participants • Often programs are implemented in specific areas, at a specific moment for a specific reason • Volunteers are often more motivated or better informed or… – But we cannot observe the same person both: • exposed and • not exposed to the program
  4. 4. J-PAL: Jameel Poverty Action Lab• A network of researchers at universities around the world• Focused on randomized evaluations• Founded in 2003 by Esther Duflo and Abhijit Banerjee, MIT Professors of Economics
  5. 5. Build capacity J-PAL ImproveEvaluate social the livesprograms of the poor Disseminate the results
  6. 6. J-PAL today: 5 Offices & 64 Affiliates
  7. 7. J-PAL Evaluations around the World• 318 randomized evaluations in 52 countries• About 162 are completed
  8. 8. J-PAL Europe• Started in 2008 with 9 affiliates• Today: 18 affiliates (Belgium, Denmark, Germany, France, United Kingdom, Sweden)• About 50 projects: – Many in developed countries – Mostly ongoing
  9. 9. How to evaluate the impact of an idea?• Implement it on the ground in the form of a real program, and see if it works• Common approaches: – Before and after (But many things happen over time?) – Participants vs. Non-participants (But are they different? More motivated? Live in a different region?)• Need an adequate comparison group – individuals who, except for the fact that they were not beneficiaries of the program, are similar to those who received the program
  10. 10. Randomized Evaluation• Randomized evaluations (or RCTs) allow us to identify impact rigorously – People are randomly assigned to a treatment group, who receives intervention, or to a control group – Program beneficiaries are not more motivated, richer, educated, than non-beneficiaries, so changes in outcomes are purely because of program – Gives nice clean results everyone can understand—no fancy econometrics: rigor and transparency
  11. 11. Basic set up of a randomized evaluation Not in evaluation Target Population Treatment Group Sample Random Population Assignment Control Group Based on Orr (1999) 11
  12. 12. Randomized assignment• This method works because of the law of large numbers• Both groups (treatment and control) have the same characteristics, except for the program• Differences in the outcomes can confidently be attributed to the program• We can use this knowledge to set policy – Identify programs that actually solve social problems – Make government policies and programs more effective
  13. 13. The question of ethic• Resources are very limited any way• This random assignment is usually seen as very fair• Inefficient programs are not ethical!• Every project has to go through an ethic committee• J-PAL never works on projects if the cost of the evaluation means less beneficiaries. After the evaluation is over, if necessary, the control group will also get the program
  14. 14. La Mallette des Parents
  15. 15. La Mallette des Parents (the parentstoolkit)Questions: • Is it really possible to improve parents’ involvement ? • Has increased parental involvement any effect on children? • Does the effect on program participants spread out on other families?
  16. 16. The program• Children and their parents in first year after primary school in low-income schools• A toolkit: – a booklet with thematic guidelines – 3 meetings with the parents – A DVD in 10 different languages• Cost: 1000€ per school (7€/child)
  17. 17. Messages to the parents• All parents can help their children• Work outside of school is extremely important for success• Parents should be involved in their childrens homework• Children need to feel that their parents understand how school functions and care that they adhere to the demands of teachers and administration
  18. 18. Protocol: Four groups Test Classes Control classes Volonteer Volonteer Compare Non Non volonteers Compare volonteers
  19. 19. Parents involvement• More interactions with school: – Meetings with teachers – Participation in parents organizations• Helped more their children at home• No more difference in involvement between blue- collar and white-collar families
  20. 20. Childrens behavior• Large improvement, even for classmates whose parents were not volunteers• 34% less likely to be sanctioned for disciplinary reasons• Similar results in reduction of absenteeism• Cognitive achievement: – Limited but significant impact in test scores in French – No impact in Math
  21. 21. Policy implication• Simple and inexpensive program• Rigorous evaluation: can convince schools or governments that such action is worth taking• Ongoing generalization in France• New program launched for children in the last class of compulsory school
  22. 22. Policy lessons• Directed parent discussion groups are an effective policy tool for increasing parental involvement, even in underprivileged area• Increasing parental involvement and awareness of school structure improved student behavior and positively impacted learning• Even though only a small fraction of parents choose to participate, the benefits of their involvement were felt by all children in the classroom.
  23. 23. Other Programs…• Counseling the unemployed• Counseling and job placement for young graduate job seekers• Counseling welfare recipients• Discrimination in hiring and anonymous CVs• Facilitating youths access to apprenticeships and encouraging them to complete them• Small business training and loans for aspiring entrepreneurs in disadvantaged neighborhoods• Supporting 18-25 year-olds through long-term mentoring plus financial assistance• …
  24. 24. Scale-ups: million of lives improved
  25. 25. More information www.povertyactionlab.org www.povertyaction.org www.worldbank.org/dime www.worldbank.org/impactevaluation www.3ieimpact.org www.usaid.gov/div
  26. 26. • Many thanks !

×