The mexican family life survey within the surveys

1. The Mexican Family Life Survey within the Surveys´ contexts
regarding the evaluation of social programs in Mexico
Luis Rubalcava

2. OBJECTIVE
 The role of population based panel surveys as a tool to evaluate social programs and
policies over time
 Its link to administrative records

3. roadmap
1. Panel Surveys
2. Advantages of a Panel for evaluation
3. On going panel survey in Mexico, MxFLS
4. Example of Evaluation of Health Insurance Program
5. Example of Evaluation of OPORTUNIDADES (CCT)

4. 1. Panel survey
 Panel study is one in which the unit of analysis is interviewed in different periods of
time
 Rotative panel studies in which the sample is followed only for a limited period of
time. The sample is refreshed every certain period
 Longitudinal studies that only go back to original locations (interview whoever
is there)
 Longitudinal studies that do follow up of people who leave the original
household
 Longitudinal studies over generations (PSID)

5. 2. Advantages
 Important tool for the evaluation of natural experiments
 Economies of scale-- it is useful for many evaluations and policies, as opposed to
evaluations done purposefully for one program
 Long term exposure analysis—if you have a long enough panel and low attrition
 Difficult to do in randomized trials

6. 2. Advantages
Excellent tool to study transitions
 Health transition
 Labor transitions
 Poverty transitions

7. 2. Advantages
 Important for the study of migration
 Domestic migration
 International migration
 Return migration

8. 3. MEXICAN FAMILY LIFE SURVEY
CAMBS/UIA/CIDE
México

9. GOAL
 Collect information on the welfare of Mexican households
 Representative of the population at the national, urban-rural and regional level
 Multi-thematic
 Longitudinal
 Community component

10. Characteristics
 Conducts in-person interviews (to all household members)
 Collects objective health information (anthropometric measures and
biomarkers) with specialized health personnel
 Comprises information on ONE single instrument (households and individuals)
 Follows individuals overtime, regardless of their place of residence
 Re-contacts in Mexico
 Re-contacts in the USA

11. Characteristics
 First round (baseline survey) in 2002
 Second round in 2005-2006
 Third round in 2010-2011
 Original sample: 35,000 individuals

12. Topics
Household Level
 Economic characteristics (income, expenditure, wealth, savings, credits)
 Transfers inside and outside the household
 Victimization
 Agricultural land use and management (rural households)

13. Topics
INDIVIDUAL Level
 Human capital (schooling, cognitive ability and health status)
 Employment and time use
 Migration (national and international)
 Reproductive health
 Marriage history
 Anthropometric measures and biomarkers
 Reproductive health, contraceptive methods and pregnancy history
 Health perception, mental health

14. HEALTH
Objective health information (anthropometric measures and biomarkers) of all
household members:
 Weight
 Height
 Waist circumference
 Hip circumference
 Blood pressure >15 years old
 Hemoglobin >6 months old

15. HEALTH
Objective health information (anthropometric measures and biomarkers) of all
household members:
 2005 + cholesterol LDL, cholesterol HDL, total glucose and dried blood samples
 In 2010+Hba1c, leg length

16. Topics
COMMUNITY Level
Collects information on the infrastructure of the communities where the sample
households live:
 Schools
 Health centers
 Small health providers
 Local authorities
 Commercial establishments

17. Advantages
Evaluation of Public Policies
The MxFLS allows:
 Analyzing socioeconomic and demographic indicators of the same individuals over time
 PANEL For example, who is in poverty over an extended period? Who can leave poverty? Who falls
into poverty?
 MULTIPURPOSE Studying the relationship between different variables
 For example, what is the relationship between poverty and migration to the US? What is the
relationship between economic productivity and health?
 Conducting causal analyses.

18. Advantages
Evaluation of Public Policies
The MxFLS allows:
 For example, how does having a US migrant affect mental health?
 Dynamic Analysis: Mobility of the same individuals over time. Do they remain in the formal sector?
Do they exit the formal sector to go to the informal?
 Local Infrastructure and its impact on household welfare
 Example: What is the relationship between the local infrastructure and economic well-being? Does the
lack of public schools affect educational levels? By how much?

19. Advantages
Evaluation of Public Policies
The MxFLS allows:
 Program Evaluation of large programs represented in the sample.
 For example, Evaluation of Prospera (formerly Oportunidades). Evaluation of Seguro
Popular, health insurance program.

20. 4. Example: Evaluation of seguro popular

21. Background
 By 2000, 50 million of the Mexican population did not have health insurance
 => Catastrophic health expenditures were covered directly by the uninsured affecting the poorest
families.
 Seguro Popular (SP) represents the effort of the Mexican government to:
 Reduce catastrophic health expenditures.
 Reduce health inequalities
 Improve health status of the uninsured.
 Government goal to achieve by 2012 universal coverage.

22. Background
Fig1. Number of individuals affiliated to the three main public health institutions in Mexico
Millions
of
people

23. Study Objective
 Since SP is close to achieve the goal of universal coverage, we evaluate the impact of
SP on:
 Health services utilization
 Health Status
 Out-of-pocket health expenditure
 Using longitudinal data from the Mexican Family Life Survey (MxFLS) implemented in
2002 and 2009-2011.

24. DATA
Advantages of using MxFLS
 Baseline was conducted in 2002, before the period in which SP was expanded,
 Provides socio-economic and demographic characteristics of the individuals before the
implementation of the program, which can be used as controls to analyze the impact of the programs.
 Provides information on several health outcomes before and after the implementation of the
program.

25. Hypotheses
Hypothesis 1
for coverage >
Hypothesis 2
<
for coverage >
Hypothesis 3
for coverage <
Non-beneficiaries eligible
Health Status
Out-pocket health SP Beneficiaries
expenditure
SP Beneficiaries Non-beneficiaries eligible
SP Beneficiaries Non-beneficiaries eligible
Health services
utilization

26. METHODS
Difference-in-Difference Propensity Score Matching
 Estimate propensity scores
 P(i) = Pr (Di =1|i) (1)
 Xi Observable characteristics before treatment
Di =1 if at least one household member is a beneficiary of the SP
 Match propensity scores
 Estimate difference-in-Differences
 ATTDD = E(1-0|T1 =1) - E(1-0|T1 =0) (2)

27. results
Health services utilization
 Increase of 6% in the probability of visiting
a doctor in the case of SP beneficiary
men living in rural areas.
0% 1% 2% 3% 4% 5% 6% 7%
Beneficiary men living in rural areas
6%

28. results
Health services utilization
 Increase of 47% in the number of visits to
the doctor of SP beneficiary women
living in urban areas.
 No impact for children from 0 to 5 years
old.
Beneficiary women living in urban areas
50%
40%
30%
20%
10%
0%
47%

29. results
Graph2. Impact of SP on Health Status of Rural Women 20-60 years old
7%
Relative Self-Reported Health (Very
Good, Good)
-17%
ADLS for age>50: Can not Stand Up
from the floor
10%
5%
0%
-5%
-10%
-15%
-20%
Rural Women 7% -17%

30. results
Graph3. Impact of SP on Health Status of Urban Men 20-60 years old
-6%
-5%
-4%
-3%
Cold Cough Abdominal pain Heart Disease
0%
-1%
-2%
-3%
-4%
-5%
-6%
-7%
Urban Men -6% -5% -4% -3%

31. results
Graph6. Impact of SP on Out-of-Pocket Household Health Expenditure
-56.0%
National- Total Reported Household
-100.0%
Health Expenditure Urban- Contraceptive Methods
0.0%
-10.0%
-20.0%
-30.0%
-40.0%
-50.0%
-60.0%
-70.0%
-80.0%
-90.0%
-100.0%
Households -56.0% -100.0%

32. 5. Example: Evaluation of OPORTUNIDADES (CCT)

33. “Do Conditional Cash Transfer Programs Improve Work and Earnings
among its Youth Beneficiaries? Evidence after a Decade of a Mexican
Cash Transfer Program.”
Susan W. Parker, CIDE
Luis N. Rubalcava, Spectron/Cambs
Graciela M. Teruel, Universidad Iberoamericana

34. introduction
 CCT Program
 Longer-term effects can be directly measured only after a significant number of
years of program operation and follow-up.
 Little is known about the longer-term impacts of CCTs on work and earnings of
youth beneficiaries.

35. objective
We estimate long term impacts of Oportunidades on youth education and labor market
outcomes in rural areas using 3 rounds (2002, 2005, 2009) of the longitudinal Mexican
Family Life Survey.
 Does additional education induced by CCT’s improve employment and
salaries as youth beneficiaries enter adulthood?
 Intergenerational effects on reducing poverty?

36. objective
 Sample: Rural children age 10 to14 when Oportunidades began in 1997-
1998, followed until the last round of the MxFLS (2009) when they were in
their early to mid twenties.
 Impacts observed in 2002, 2005 and 2009.

37. hypotheses
 Schooling:
Transfers reduce the shadow wage of children’s time in activities other than school.
 Work:
Initially school subsidies reduce time spent in work. (substitutes)
But with more schooling, youth receive higher wages offers. Eventually should increase
working as well as wages/job conditions.

38. DATA
 Use MxFLS--Low attrition, migrants followed and interviewed.
 Administrative data on Oportunidades receipt at community level merged to
MxFLS.

39. Definitions of groups
 Treatment group: “Early” beneficiaries, youth in communities incorporated in
1997 or 1998 (T1998)
 Control group: “Late” beneficiaries, youth in communities incorporated in
2004 or afterwards (T2004)

40. results
 Initial results confirm impacts of Oportunidades on accumulating additional education.
0.18
Grades of completed schooling
0.47
0.37

41. results
 Some apparent increases in labor market participation of Oportunidades youth.
 No impact in 2002
0
Currently working
0.13
0.11

42. results
 No impacts on income per hour or job benefits.