Health policy program effects on institutional delivery in peru


Published on

Published in: Health & Medicine, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Health policy program effects on institutional delivery in peru

  1. 1. Health Policy 77 (2006) 221–232 Evaluating program effects on institutional delivery in Peru Michael J. McQuestiona,∗, Anibal Velasquezb a Johns Hopkins Bloomberg School of Public Health, Population and Family Health Sciences, 615 N Wolfe Street, E-4142, Baltimore, MD 21205, USA b Consultant, PHRPlus, Antequera 777 Piso 8, San Isidro, Lima, Peru Abstract We evaluate the joint effects of two targeted Peruvian health programs on a mother’s choice of whether to deliver in a public emergency obstetric care (EmOC) facility. The national maternal and child health insurance, or SMI Program, provided delivery care coverage to Peru’s poorest households beginning in 1998. During 1996–2002, Proyecto 2000 sought to improve the quality of EmOC and increase utilization of public EmOC facilities in the districts reporting the highest maternal and neonatal mortality levels. Our data come from the Proyecto 2000 endline evaluation, which sampled 5335 mothers living in the catchment areas of 29 treatment and 29 matched control EmOC facilities. Using propensity scoring and two quality of care indices, we find significantly higher quality of care in Proyecto 2000 treatment facilities. Using variance components logistic models, we find a mother enrolled in the SMI Program was more likely to have delivered her last child in a public EmOC, controlling for household constraints. Residence in a Proyecto 2000 treatment area did not significantly affect the choice. A cross-level interaction term was insignificant, indicating the two program effects were independent. © 2005 Elsevier Ireland Ltd. All rights reserved. Keywords: Quality of care; Evaluation; Developing countries; Safe motherhood 1. Introduction This study examines two very different efforts to increase institutional delivery in Peru. During 1992–1997, Peru implemented large-scale health sec- tor decentralization reforms. The reforms were criti- cized for widening health disparities, particularly in hospital utilization [1]. Peru’s DHS III (1996) and DHS IV (2000) surveys documented a relative decline in physician-assisted deliveries among rural and poorly ∗ Corresponding author. Tel.: +1 410 502 6037; fax: +1 410 955 2303. E-mail address: (M.J. McQuestion). educated women over the period. To correct this, the Peruvian Ministry of Health initiated a series of tar- geted maternal and child health interventions, two of which we evaluate. The first intervention was Proyecto 2000, a USAID-funded effort begun in 1996 in the 12 of Peru’s 25 departmentos reporting the highest mater- nal mortality levels. Proyecto 2000 aimed to increase the proportion and quality of institutional deliveries, thereby reducing maternal mortality and improving birth outcomes. The project began with mass media, health education and social mobilization efforts pro- moting delivery in the nearest public emergency obstet- ric care (EmOC) facility. Its emphasis, however, was on improving the quality of services on offer. The sec- 0168-8510/$ – see front matter © 2005 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.healthpol.2005.07.007
  2. 2. 222 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 ond intervention was the Maternal and Child Health Insurance (SMI) Program, launched in 1998. The SMI Program covered most maternal and child health costs, including institutional delivery in public EmOC facil- ities [2]. It was a means-tested program in that only households in the poorest wealth quintile were eligi- ble to participate. By 2000, this program was reaching about 50% of eligible households in two pilot regions [3], and the following year it was extended nationwide. The two programs thus incorporated different target- ing strategies. Proyecto 2000 targeted high-risk dis- tritos, specific EmOC facilities and their surrounding communities while the SMI Program directly targeted the country’s poorest households. Did either program increase EmOC utilization? In this study we use quasi- experimental data to probe this question. We model a woman’s choice of where she delivered her last baby, conditional on exposure to these two programs. 2. Background 2.1. Recent perinatal health trends Demographic data show perinatal health in Peru improved over this period. The country’s neonatal death rate fell from 27 to 18 deaths per 1000 live births during the 1990s [4]. Peru’s estimated mater- nal mortality ratio also fell, from 265/100,000 live births in 1990–1996 to 185 in 1994–2000 [5], yet it remained third highest among 14 Latin American countries reporting in 1999 [6]. Maternal and neona- tal mortality are largely influenced by two factors: a woman’s decision whether or not to utilize institutional delivery care and the quality of that care. High-quality EmOC can prevent an estimated one-third of mater- nal deaths [7], and 40–62% of neonatal deaths [8]. Regardingmaternalbehaviors,the1996DHSIIIsurvey showed that 55% of women who had given birth in the previous5yearsdidsoathome.Another38%usedpub- lic health care facilities and 5% used private delivery facilities [9]. Over the succeeding 5 years, the propor- tion of home deliveries fell to 47%, the public sector’s share rose to 48% and the proportion using private facilities stayed at about 5% [10]. There are no com- parable EmOC quality of care estimates, however, a recent qualitative study ranked Peru second of 13 Latin American countries evaluated in terms of maternal and neonatal program effort [11,12]. It is thus plausible that the observed perinatal health improvements were due to increased institutional deliveries that in turn resulted from program improvements. There were other important factors affecting mater- nal and perinatal health in Peru over this period. The country’s per capita GNP grew by a mean 2.4% per annum during the 1990s [13], an improvement over the chaotic 1980s. Total fertility rates declined from 4.8 in 1986 to 2.1 in 2000, lengthening birth intervals and reducing the proportion of high-parity births [4]. These changing background forces may have been more deci- sive health behavioral determinants that the program effects we attempt to elucidate. 2.2. Maternal health risk factors In Peru, as elsewhere, it is the poorest, most remote and most socially excluded women who least use maternal health services [14], and are at highest risk of maternal, perinatal and post-perinatal mortality [15,16]. A 2000 survey in Peru’s Ayacucho Depart- ment, for example, found that only about one-fourth of women with complications were delivered in adequate EmOC facilities [17]. In Peru’s DHS IV survey some 83% of women identified at least one barrier to access- ing local maternal health services. Expense was the leading problem, followed by lack of female caregivers [10]. Other cultural factors act as barriers to EmOC uti- lization, particularly among the 47% of Peruvians who do not speak Spanish as their first language. Reports of discrimination and mistreatment by health work- ers are commonplace [18,19]. The DHS data suggest that more high-risk women chose to utilize the public EmOC facilities over this period. 2.3. National SMI Program TheFujimoriAdministrationinstitutedtheSMIPro- gram in 1998. It was Peru’s first attempt to subsidize preventive and maternal care for low-income pregnant women, mothers and children ages 0–4 years. Many saw it as an attempt to restore basic health rights that had been infringed by decentralization. In 2001, the program was supplanted by a national Integral Health Insurance Plan, which offered a wider gamut of tar- geted benefits to low-income Peruvians of all ages. Until 1998, any woman could have accessed any public
  3. 3. M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 223 EmOC facility where she had to pay fees for service on a sliding scale. The targeted insurance programs elim- inated these fees for the eligible poor. However, by 2001 many eligible households were still not enrolled in the program. Peru’s public health system still lacked the infrastructure and level of performance needed to extend MCH services to all those eligible. Production levels remained exceedingly low. The median num- ber of consultations that year was less than three per day in half of the Ministry of Health’s peripheral PHC facilities. To date there has been no comprehensive evaluation of these targeted insurance efforts [20]. 2.4. Proyecto 2000 2.4.1. Phase I We describe Proyecto 2000 in greater detail because it generated the data we analyze. Proyecto 2000 was implemented by a team of Ministry of Health and exter- nal expert consultants. The team sought to make the Ministry’s EmOC services culturally acceptable and to ensure that the facilities delivered high-quality care. A hallmark of Proyecto 2000 and other Safe Motherhood projects is an emphasis on making services “woman- friendly”. An EmOC facility is woman-friendly if: (a) it is easily accessible and convenient to use; (b) high- quality services are offered; (c) local cultural beliefs and social norms are incorporated into treatment proto- cols and (d) confidentiality is guaranteed, information is shared and clients’ choices are respected [21]. The Proyecto 2000 team worked at facility and commu- nity levels to accomplish these aims. At baseline, team members and Regional Ministry of Health educators gathered and analyzed qualitative data on mothers’ perceptions and preferences regarding pregnancy and childbirth. They used these data to mount a multime- dia Safe Motherhood campaign in the treatment areas. In addition, expert trainers trained 3692 community- based traditional birth attendants (promotoras), and EmOC staff formally engaged newly constituted com- munity health committees (Comites Locales de Admin- istracion en Salud) in their catchment areas. Facility inputs included physical plant improvements, retrain- ing of 409 facility-based providers, incorporation of local birthing practices into clinical protocols and the introduction of a continuous quality of care (“autoeval- uacion”) model in some 89 public hospitals and health centers. In brief, the autoevaluacion model incorpo- rated the Donabedian continuous quality of care [22], and the McCarthy and Maine maternal mortality deter- minants frameworks [23]. The autoevaluacion instru- ment included a battery of detailed indicators regarding essential obstetric and neonatal care, physical facilities, patient interaction and management. The expectation was that greater autonomy and participation in the self-appraisal process would stimulate improved staff performance, and the resulting improved quality of care would generate more institutional deliveries as client satisfaction improved. All facilities were expected to attain quality of care improvements sufficient to merit formal certification by expert evaluators. These 89 facilities comprised the original treatment arm. 2.4.2. Midterm evaluation (2000) As of 1998, 72 treatment facilities were still active in the program, all of which had attained formal qual- ity of care certification as high-quality perinatal care centers [24]. By October 2000, the number of active treatment facilities had fallen to 60. At that time a midterm evaluation was carried out. An external evalu- ation team examined a random sample of 37 treatment facilities. They also identified a group of 37 similar EmOC facilities not exposed to the project to serve as a comparison group. The control facilities were drawn from six Ministry of Health districts (DISAs) with ser- vice population characteristics (literacy, contraceptive prevalence, use of institutional delivery services, mal- nutrition and poverty levels) similar to the Proyecto 2000 areas. The control facilities had received only rou- tine Ministry of Health supervision over the period. The mid-term evaluation was entirely facility-based. Expert observersusedstandardizedchecklistsandinstitutional record reviews to assess the quality of EOC on offer. They found evidence of improved quality of care and a relative increase in the numbers of institutional deliver- ies in the treatment group facilities as compared to the control facilities (Table 1). Additionally, the observers interviewed samples of prenatal clients. They found users of treatment facilities were more knowledgeable about pregnancy, more satisfied with their experiences and more likely intended to deliver their babies in that treatment area facility [25]. 2.4.3. Phase II (2001–2002) During Phase II, Proyecto 2000 inputs were con- centrated on the 31 treatment facilities judged to have
  4. 4. 224 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 Table 1 Selected EOC facility indicators, Proyecto 2000 Variable Control facilities Treatment facilities Mean S.D. Mean S.D. 1997 Institutional births 1463 1937 1486 845 Prop births <2.5 kg 0.07 0.08 0.10 0.09 Prop births c-section 0.24 0.13 0.19 0.08 2000 Institutional births 1052 1434 1542 681 Prop births <2.5 kg 0.07 0.04 0.09 0.07 Prop births c-section 0.24 0.10 0.23 0.11 Ob-gyns 7.92 7.61 7.50 5.37 Births/ob-gyn 140 59 284 152 2002 Autoevaluacion scorea 26.80 8.30 51.20 14.40 EmOC capacity score 69.30 8.90 72.19 7.18 a First principal component of nine-factor index. performed best in Phase I. Project supervisors regu- larly visited these facilities to ensure the autoevalua- ciones were performed in each facility each quarter. Project data show the autoevaluaciones were in fact implemented. Of the 29 treatment establishments that participated to endline (2002), all carried out at least two autoevaluaciones, 25/29 carried out three, 13/29 carried out four and 3/29 carried out five. The auto- evaluacion scores reported by the facilities increased with each round (Fig. 1). These data indicate the institu- tional Proyecto 2000 interventions were implemented and suggest the interventions could have been strong enough to improve the quality of EmOC services on Fig. 1. Autoevaluacion scores by evaluation round, 2000–2001, Proyecto 2000. offer. Our task is thus to disentangle two distinct treat- ment effects, one operating through the health sys- tem, the other directly on household health production. We expect the two effects will be synergetic: insured women in high-quality EmOC catchment areas ought to be the most likely to use that facility. With these points in mind, we model the probability a Peruvian mother chose to deliver her youngest child at the nearest public EmOC facility, conditional on the qualityofcareatthatfacility,herhouseholdconstraints, SMI Program participation, and whether her commu- nity and facility participated in Proyecto 2000. 3. Data and methods 3.1. Facility data The Proyecto 2000 evaluators collected a second round of endline evaluation data in mid-2002 and it is these data we analyze in the present paper. The Phase II treatment group included all 19 Phase I hospitals and a subset of 12 Phase I health centers. The eval- uators selected a new control group, consisting of 15 of the Phase I control establishments and 14 additional establishments. As in Phase I, the 14 new control facili- ties were purposively selected from six newly matched DISAs that were unexposed to the project. Expert teams again evaluated essential obstetric care in the EmOC facilities using the same extensive standard checklist used in the midterm evaluation. They also evaluated the quality of services using the autoevaluacion instrument itself. Thirdly, they collected selected service indica- tors routinely reported by each facility to the Ministry of Health. We used these data to derive two EmOC quality of care measures, which we described below. 3.2. Household data To assess changes in local utilization patterns and measure SMI Program participation, the Proyecto 2000 evaluators carried out a household survey in all treat- ment and control facility service areas. The survey instrument incorporated selected items from Peru’s DHS III and DHS IV survey questionnaires [26], particularly household characteristics, birth histories and pregnancy-related behaviors. Sampling procedures were similar to those used in the DHS. Peru’s 1993 cen- sus of households provided the sampling frame. Within
  5. 5. M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 225 each Proyecto 2000 catchment area, census tracts were listed and selected at random. Within each tract house- holds were selected systematically following cardinal directions from the approximate center of each cluster. Ten women who had given birth in the previous 5 years were surveyed in each cluster. The measures we derive from these household-level data described below. We merged the facility and household data to make a hier- archical dataset consisting of 5335 women nested in 420 clusters in 58 facility catchment areas. 3.3. Facility quality of care measures Our two quality of care and basic EmOC capac- ity measures, along with other facility indicators, are shown in Table 1. The first measure (autevaluacion score) is the first principal component (eigenvector) from a factor score analysis of nine items from the autoevaluacion checklist. The nine items were: blood is routinely filtered, an incinerator is present, there are generic versus proprietary drugs are in the phar- macy, there is an up-to-date list of all drugs dispensed, patients receive health educational messages, patient satisfaction is measured, remedial activities to improve patient satisfaction were implemented, there is a local community advisory committee, staff meets at least every 3 months, feedback on performance is given at thosemeetings.WecomputedCronbach’salphaandthe Kaiser–Meyer–Olkin measure of sampling adequacy [27] for these nine items. The resulting coefficients were, respectively, 0.70 and 0.65 (results not shown). We conclude the nine items are tapping a common underlying construct but we note that 0.80 is the con- ventional “gold standard” for both measures [27]. The second measure, EmOC capacity score, is the percent score on a battery of 711 items the evaluators used to assess the technical capacity of a facility to deal with obstetric emergencies. The evaluators grouped the indicators into nine categories: human resources, pre- natal and obstetric equipment, radiology, pharmacy, delivery room equipment, neonatal care unit, maternity ward, operating room and blood bank. As Table 1 shows, treatment facilities scored higher on both the autoevaluacion quality of care index and EOC capacity score. This apparent improvement could be a true difference due to the Proyecto 2000 inputs or it could be an artifact of the non-random match- ing of treatment and control facilities, attrition or other sources of bias. To explore this further we used four of the routinely reported EmOC facility indicators to compute a propensity score for the assignment pro- cess. The aim of propensity scoring is to make assign- ment “strongly ignorable” by blocking observations on observables [28,29]. The outcome is the dummy variable indicating assignment to treatment or control group. The covariates we used are: number of obstetri- cians and gynecologists on staff, number of maternal deathsin2000,numberofcaesariansectionsperformed in 2000 and the proportion of all deliveries performed outside of the facility. We generated a balanced score with matched pairs of facilities falling into eight blocks (results not shown). We then used the propensity score to generate three alternative non-parametric treatment effects estimates for each quality of care measure. 3.4. Household measures We control for several household risk factors in our models. Maternal education is a positive predictor of maternal behaviors in Peru [14]. Other important fac- tors include maternal age, number of births and socioe- conomic status [30]. Maternal educational attainment is coded using terciles, where 1 = no or primary educa- tion, 2 = some secondary and 3 = completed secondary and higher. Another dummy variable is coded one for women who have had three or more live births, zero otherwise. To control for household wealth we use the Filmer–Pritchett method [31], wherein weights from principal components are applied to a list of household assets, scores are summed and ranked and each house- hold is assigned to one of five wealth quintiles. We add additional dummy variables to control for whether the last child was born in 1998, 1999, 2000 or 2001. We use a binary dummy variable to indicate whether or not the household participates in the SMI Program. As shown in Table 2, the characteristics of Proyecto 2000 sample households were broadly comparable across treatment and control areas. Only ethnicity var- ied: treatment area women were less likely to be Span- ish speakers. Delivery patterns also appear similar across the study arms. Four of every five women in both treatment and control areas delivered their last babies in some kind of institution. Though the matching appears adequate, the Proyecto 2000 sample is not a nationally representative sample. Table 2 shows the same indica- tors computed from Peru’s DHS IV survey. The DHS
  6. 6. 226 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 Table 2 Sample characteristics and maternal health indicators, women giving birth in previous 5 years, Peru 1996–2002 Variable Proyecto 2000 t-Test DHS IV Control facilities Treatment facilities Matched to P2000 Full sample Mean S.D. Mean S.D. Mean S.D. Mean S.D. Last birth institutional 0.83 0.37 0.82 0.38 0.77 0.21 0.53 0.50 Last birth prenatal care 0.91 0.29 0.91 0.29 0.87 0.11 0.78 0.42 Mothers characteristics Age (years) 26.64 6.62 27.67 7.10 −5.38a 29.32 1.41 29.21 7.05 No. live births 2.57 1.70 2.66 1.88 2.92 0.60 3.51 2.47 Educational level Primary 0.21 0.40 0.23 0.42 −2.37a 0.31 0.15 0.45 0.50 Secondary 0.44 0.50 0.44 0.50 0.38 0.15 0.32 0.47 Superior 0.35 0.48 0.32 0.47 0.26 0.13 0.14 0.34 Union status Married 0.38 0.48 0.38 0.49 0.42 0.16 0.41 0.49 Consensual 0.48 0.50 0.47 0.50 0.43 0.16 0.47 0.50 Divorced/separated/widow 0.15 0.36 0.15 0.36 0.08 0.05 0.07 0.25 Rural origin 0.39 0.49 0.36 0.48 0.26 0.19 0.40 0.49 Non-Spanish speaker 0.02 0.15 0.08 0.27 −9.30a 0.10 0.17 0.22 0.42 Households Electricity 0.93 0.25 0.90 0.30 −4.05a 0.80 0.20 0.52 0.50 Safe water 0.86 0.35 0.81 0.39 4.64a 0.97 0.12 0.81 0.39 Durable floor 0.53 0.50 0.56 0.50 −2.40a 0.44 0.11 0.53 0.50 Safe toilet 0.59 0.49 0.62 0.49 −2.09a 0.75 0.19 0.60 0.49 n 2514 2821 5826 13832 a Significant at p < 0.05 level. IV sample is a nationally representative weighted sam- ple drawn from 589 of the 1789 distritos enumerated in Peru’s 1993 household census. We used the distrito identifiers to match the DHS IV and Proyecto 2000 data (n = 68 matched distritos). The Proyecto 2000 sample is somewhat better educated, more likely to be Spanish- speaking and living at a slightly higher socioeconomic level than the DHS subsample from the same distri- tos. Compared to the national DHS sample, women in the Proyecto 2000 distritos were more intensive mater- nal health service users, better educated, more likely to speak Spanish and less likely to have households with electricity. Accordingly, all inferences we will make are limited to the Proyecto 2000 sample data. 3.5. Behavioral model As mentioned, we estimate a facility-level Proyecto 2000 treatment effect using propensity scoring. Here we describe our behavioral model, which includes indi- cator variables that control for the effects of both pro- grams. We interpret their slopes as indirect treatment estimates. Given the heteroscedastic treatments and the many suspected unobserved variables that could have affectedmothers’deliverychoices,wefittwo-levelran- dom effects models of the form: yij = πij + εij logit(πij) = β0j + β1Xij + β2Tij + β3Iij +β4Pj + β5IijPj β0j = δ0j + δ01z1j + κj εij ∼ N(0, 1), cov(Xij, Pj, Iij, Tij, εij) = 0 κj ∼ N(0, σ2 κ ), cov(z1j, κj) = 0 cov(εij, κj) = 0 In this model πij is the probability mother i in EmOC facility service area j chose institutional delivery yij, and εij is an individual error term. β1 is a parame-
  7. 7. M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 227 ter measuring individual effects due to household and individualcovariatesXij.β2 measurestheeffectoftime, specified as dummy variables for child i’s birth year Tij. β3 controls for SMI Program participation, indicated by Ii, which is coded one for participants, zero for non- participants. β4 controls for being in a Proyecto 2000 treatment facility area, indicated by Pj, a dummy vari- able coded one for Proyecto 2000 treatment distritos, zero otherwise. We include β5 to capture any cross- level interaction between the two treatments. This term also adjusts for the possibility the insurance program was not uniformly implemented across the Proyecto 2000 areas. β0j is a random facility-level intercept, δ0j and δ01 are parameters, z1j is a dummy variable for facility and kj is a facility-level random effect. If the variance of kj, denoted as σ2 k , is significant, then we know there are unobserved variable effects which might otherwise have biased the fixed effect parameters in a conventional model. To fit the behavioral model we must make several assumptions. We assume that each mother is influenced solely by her own EmOC facility. We further assume that all mothers in the Proyecto 2000 treatment areas were equally exposed to the treatments and that access to the nearest Ministry of Health EmOC facility did not differ between treatment and control areas. This was not the case for the SMI Program, which was not uni- formly implemented and was means-tested. Although we lack any SMI Program data, we assume that all eli- gible households with access did enroll. Because the subsidy was conditioned on use of the nearest Ministry EmOC facility we assume that any SMI Program effect contributed fully to the likelihood of our outcome. We estimate a series of nested multilevel models using Stata’s gllamm program [32]. The program uses a maximum likelihood algorithm with adaptive quadra- ture to model latent variables as random effects. One advantage of gllamm over other multilevel programs is that it generates log-likelihood statistics useful for comparing model fits. All standard errors are estimated using the Huber–White sandwich estimator to adjust for the clustered survey design effect [33]. 4. Results 4.1. Institutional model Our institutional treatment effects are shown in Table 3. The estimates include population-average treatment effects (ATE) produced by radius matching and nearest neighbor matching algorithms. Following Imbens (2003), we also estimate the within-sample ATE. For comparison, we report the slope of a simple OLS model with the treatment dummy the sole regres- sor. The two indicators are measured on different met- rics so their treatment effect estimates are not directly comparable. What we expect are consistent estimates for each indicator. Inferences are based on Wald tests. With the outcome the EmOC capacity score, popula- tion and sample ATE estimates were all significant. With the autoevaluacion factor score as the outcome, Table 3 Facility-level treatment effects estimates, conditioned on propensity scorea, Proyecto 2000 EmOC capacity (n = 52 facilities) Autoevaluation (n = 55 facilities) Coefficient S.D. (Pairs) Coefficient S.D. (Pairs) OLS slope 12.2** 2.8 0.55** 0.28 Radius matching ATTb 11.7** 2.6 (26t,18c) 0.5 0.43 (6t,16c) Nearest neighbor Random draw ATT 12.1** 2.7 (26t,11c) 0.55* 0.32 (26t,11c) Equal weights ATT 12.1** 2.7 (26t,11c) 0.55 0.36 (26t,11c) Sample ATEc 12.8** 2.9 (43t,43c) 0.65** 0.27 (n = 41t,41c) a Propensity score variables: number of ob-gyns, no. maternal deaths 2000, no. caesarian sections 2000, proportion of all cases delivered in facility in 2000. b ATT, average treatment effect on the treated. c ATE, average treatment effect. * Significant at 0.10 < p < 0.05 level. ** Significant at p < 0.05 level.
  8. 8. 228 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 we find a significant sample ATE (0.65) but only one populationaveragetreatmenteffect,anditismarginally significant. The EmOC capacity score is apparently a more sensitive quality of care measure. We conclude the estimates are robust and that Proyecto 2000 inputs did improve the quality of care in the EmOC treatment facilities. Our main interest, however, is in measuring any health behavioral impacts and assessing whether they are linked to facility quality of care improvements, to the provision of MCH insurance or a combination of the two. 4.2. Behavioral model Our behavioral model results are shown in Table 4. We show exponentiated slopes (odds ratios) to ease interpretation. In Model 1, the reference household model, covariate effects are signed as expected. The more educated and wealthier the woman, the more likely she delivered in the EmOC facility. Those who do not speak Spanish and have had three or more live births are less likely to choose institutional delivery. The dummy variables for birth years 1999, 2000 and 2001 capture unmeasured variables that are associated with EmOC delivery. Those net effects are positive compared with those in 1997 and 1998, the omitted categories. There is a significant random effect, indi- cating that women’s decisions to deliver in the public EmOC facility are correlated in some catchment areas more than in others due to omitted variables that jointly affect their behaviors. Model 2 results show that living in a Proyecto 2000 area has no significant effect on delivery choice. Model 3, in contrast, shows that the odds of institutional deliv- ery for women covered by the SMI Program were twice the odds for women not covered. Controlling for insur- ance removes upward biases on the highest education and wealth dummies. The difference in log-likelihoods shows that Model 3 is also a significantly better-fitting model than Models 1 or 2. Model 4 includes an inter- action term between the highest wealth quintile and the insurance dummies. These better-off households were ineligible for coverage and the negative interac- tion term captures this fact. Controlling this interac- tion further decreases the direct effects of being in the wealthiest quintile. The most dramatic effect, however, is a seven-fold increase the odds of EmOC delivery for the insured women. This pattern is consistent with the fact only the poorest households were eligible for the SMI Program. In Model 5 we add a cross-level interac- Table 4 Two-level logistic regression delivery models, exponentiated effects, Proyecto 2000 Variablea Coefficient (S.E.) Model 1 Model 2 Model 3 Model 4 Model 5 Non-Spanish speaker 0.38** (0.07) 0.38** (0.07) 0.36** (0.07) 0.36** (0.07) 0.36** (0.07) Some secondary education 2.69** (0.27) 2.69** (0.27) 2.59** (0.26) 2.61** (0.27) 2.61** (0.27) Complete secondary or more 5.63** (0.83) 5.63** (0.83) 4.81** (0.72) 4.75** (0.71) 4.76** (0.71) Three or more live births 0.66** (0.06) 0.66** (0.06) 0.64** (0.06) 0.64** (0.06) 0.64** (0.06) 60–79th wealth quintile 2.23** (0.31) 2.24** (0.31) 2.10** (0.30) 2.13** (0.30) 2.13** (0.30) 80–100th wealth quintile 3.26** (0.61) 3.27** (0.62) 2.86** (0.54) 2.25** (0.46) 2.26** (0.46) Born 1999 1.30** (0.15) 1.30** (0.15) 1.29** (0.15) 1.29** (0.15) 1.29** (0.15) Born 2000 1.75** (0.22) 1.75** (0.22) 1.76** (0.22) 1.76** (0.22) 1.76** (0.22) Born 2001 1.62** (0.20) 1.62** (0.20) 1.64** (0.20) 1.63** (0.20) 1.64** (0.20) Insured 2.02** (0.25) 15.71** (14.52) 14.96** (13.98) Insured 80–100th quintile* 0.34** (0.16) 0.34** (0.16) P2000 treatment area 0.79 (0.25) 0.80 (0.26) P2000 area insured* 1.08 (0.26) Level-two random effect σ2 k 1.06** (0.27) 1.05** (0.26) 1.11** (0.28) 1.00** (0.28) 1.09** (0.24) Log-likelihood −1797.4 −1797.1 −1780.1 −1777.0 −1776.8 n 5190 5190 5190 5190 5190 a Omitted categories: education secondary and beyond, Spanish speaker, one or two live births, lowest three wealth quintiles, born 1998, no insurance. * Significant at 0.10 < p < 0.05 level. ** Significant at p < 0.05 level.
  9. 9. M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 229 Fig. 2. Real and simulated posterior means Models 1–5. tion term to test whether the two programs interacted to affect EmOC probabilities. The interaction is insignif- icant. Comparisons of the models with their conventional single-equation logit analogs show that, in each case, the random effects specification improved model fit (results not shown). To check whether the normality assumption was met, we standardized and plotted the posterior means from each model. All distributions were near normal but somewhat negatively skewed. The skewness was due to three clinics (two treatment, one control) whose means were more than two standard deviations below the sample means. We used gllamm’s post-estimation command gllasim to resample the pos- teriormeans.ResultsareshowninFig.2.Thesimulated distributions were normal and no longer skewed. Fol- lowing arguments by Longford [34], the three clinics are thus not true outliers; their apparent outlier status is a feature of the realized sample. We leave them in the dataset and conclude that the models are robust. 5. Discussion and conclusions 5.1. Project impacts Our results show that Proyecto 2000 improved the quality of care on offer but did not directly increase the probability of delivery in Ministry of Health EmOC facilities. Nor was there an interaction between the system-level Proyecto 2000 inputs and the household- level SMI Program. Though they targeted the same sub-population, each program operated independently. The only behavioral impact we document is that of the SMI Program. It shows, simply, that reducing out-of- pocket costs increases EmOC utilization. The poorest Peruvian women clearly benefited from the targeted insurance program, however, the household risk fac- tor effects remained consistently negative across the models, indicating that neither program significantly reduced socioeconomic or ethnic disparities in EmOC utilization. Behavioral impacts due to Proyecto 2000 may have been too weak to be detectable or may have occurred after the endline survey. As shown in the DHS data, the share of births delivered in Ministry EmOC facilities rose nationwide during the period. Looking at our sample, we also see increasingly positive period effects, represented by the slopes on the birth year dummies in our models. The forces propelling those increases were likely more decisive than any attributable to Proyecto 2000. A lagged Proyecto 2000 treatment effect would be plausible for two reasons. First, only about 40% of the women surveyed gave birth during the project’s most intensive second phase (2000–2002). Although the dummy variables for
  10. 10. 230 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 birth year capture a rising probability of EmOC use, the majority of women interviewed may have been unaware of any local improvements when they made their birthing decisions, or any improvements made may not have been noticeable. Second, delivery behav- iors may be socially mediated. If so, the observation period may have been too short for social learning or other endogenous social processes to reach some theoretical threshold level of women. The data did not permit us to test this hypothesis, however, social forces are one possible source of the consistent cor- relation of birthing decisions within catchment areas captured by the random effects. Future studies would do well to explore these social aspects of maternal behaviors. 5.2. Limitations There are a number of methodological shortcomings in this study. The relatively rich quasi-experimental Proyecto 2000 data allowed us to estimate a treat- ment effect for that program. However, we lacked any kind of design for evaluating the SMI Program. Strong designs are needed in order to evaluate such tar- geted programs. A recent example was Gertler’s 2000 [35] evaluation of Mexico’s Progresa Program. In that study,Gertleruseddifference-in-differencesestimators and panel data from households in randomly sam- pled treatment and control areas to show the program increased school enrollment and health services utiliza- tion and improved health outcomes. Had panel data or even repeated cross-sectional data from the same catchment areas been available we might have detected household-level Proyecto 2000 treatment effects. Our study also faced obvious sampling problems. Attrition of the Proyecto 2000 facilities during Phase I and the replacement of 14 of the original control facilities with new ones at endline are likely sources of sample selection bias. If the attriting EmOC facil- ities were the stronger institutions then any treatment effect would be underestimated. We lacked the data necessary to assess this. The targeted nature of the two programs presents another potential source of bias in that the characteristics of facilities and households not given treatment are likely to differ from those that did receive treatments. We estimated Proyecto 2000 treat- ment effects using a propensity score balanced on just four observable covariates; many other, unmeasured covariates could differ systematically across the two groups. Regarding the SMI Program, the beneficiaries we observed may differ from other potential beneficia- ries in Proyecto 2000 catchment areas where the insur- ance program had not yet been implemented. A more general problem are background disturbances caused by the constantly evolving mix of EmOC services many Peruvian communities faced as public health services decentralized and to some extent recentralized. In this fluid policy environment, perceptions of EmOC qual- ity, perhaps the legitimacy of public health services in general, were in flux. 5.3. Policy implications Peru’s SMI Program proved an effective means of inducing high-risk women to use public EmOC facil- ities. We document here its short-term impacts. They show that cost is a significant barrier to many women. However, such subsidized programs are generally fis- cally unsustainable, particularly in poor countries. Fur- ther, they may not be efficacious. The subsidies could merely act as side payments for compliance and when the subsidies end, the desired behavior, here use of EmOC, may end too. The long-term sustainability of targeted subsidy programs is an area where more research is needed. Proyecto 2000 sought to induce greater EmOC uti- lization through more elaborate, technical strategies. It theorized that improving institutional quality of care, educating the public and working with communities would be sufficient to induce behavioral change. We lacked data on the latter but the data we did have showed the first goal was achieved. Improved quality, our results suggest, is not sufficient to change delivery behaviors. Something else is needed. Recently, Gilson [36] proposed a theory wherein trust, initially between client and provider and later between community and the state, is a necessary condition for communities to become healthier. For this to happen people must per- ceive the quality of care to be high and the public health services to be legitimate. If out-of-pocket cost is a bar- rier, then targeted subsidies may be warranted as an interim measure. Studies elsewhere have shown even the poorest people are willing to pay for health ser- vices they value [37,38]. Though our interaction term was insignificant, we encourage other researchers to test this hypothesis.
  11. 11. M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 231 Acknowledgements Technical assistance for Proyecto 2000 was pro- vided by Pathfinder International with funding from USAID-Peru, Contract 527-0366-C5049-00. Support for our study was provided by the Bill and Melinda Gates Institute for Population and Reproductive Health, Department of Population and Family Health Sciences, Johns Hopkins Bloomberg School of Public Health. References [1] Arroyo-Laguna J. Greater autonomy for public hospitals in Lima and the consequences of that on the use of health ser- vices,1988–1997.PanAmericanJournalofHealth1999;6:301– 210. [2] Ministry of Health. Lineamientos de pol´ıticas del sector salud 1995–2000: hacia un sector salud con eficiencia, equidad y cal- idad. Lima: Ministerio de Salud; 1996. [3] Pan American Health Organization. Core select data indicators. Peru, 2004. [4] MEASURE DHS+. Country statistics: Peru. Demographic and Health Surveys, 2004. Website: countries/country.cfm?ctry id=33 accessed 4/2004. [5] Hill K, Abou-Zahr C, Wardlaw T. Estimates of maternal mor- tality for 19/95. Bulletin of the World Health Organization 2001;79:182–93. [6] Pan American Health Organization. Regional strategy for maternal mortality and morbidity reduction, Document CSP26/14. Washington, DC: Pan American Health Organiza- tion; 2002. [7] Murray SF, Davies S, Kumwenda Phiri R, Ahmed Y. Tools for monitoring the effectiveness of district maternity referral systems. Health Policy and Planning 2001;16(4):353–61. [8] World Health Organization. Mother-baby package. Implement- ing safe motherhood in countries. Document FHE/MSM/94.11. Maternal health and safe motherhood programme. Geneva: World Health Organization; 1994. [9] Instituto Nacional de Estadistica e Informatica. Encuesta demografica y de salud familiar 1996. Lima: Instituto Nacional de Estadistica e Informatica, 1996. Accessed at: http://www. 23 October 2004. [10] Instituto Nacional de Estadistica e Informatica. Encuesta demografica y de salud familiar 2000. Lima: Instituto Nacional de Estadistica e Informatica; 2000. [11] Ross JA, Campbell OMR, Bulatao R. The maternal and neona- tal program effort index. Tropical Medicine and International Health 2001;6:787–98. [12] Futures Group. Herramienta para la incidencia politica en la salud materna: MNPI (indice de esfuerzo de programa materno y neonatal). Lima: Futures Group; 2002. [13] United Nations Development Programme. Human Develop- ment Reports: Human Development Indicators, 2003. Website: f PER.html accessed 4/2004. [14] Elo IT. Utilization of maternal health-care services in Peru: the role of women’s education. Health Transition Review 1992;2:49–69. [15] Vallenas G. Maternal mortality in Peru. Revista Peruana de Poblacion 1993;3:33–56. Website: English/DD/AIS/cp 604.htm accessed 4/2004. [16] de Meer K, Bergman R, Kusner JS. Socio-cultural determi- nants of child mortality in southern Peru: including some methodological considerations. Social Science and Medicine 1993;36:317–31. [17] Averting Maternal Death and Disability Working Group on Indicators. Using UN process indicators to assess needs in emer- gency obstetric services: Pakistan, Peru and Vietnam. Inter- national Journal of Gynaecology and Obstetrics 2002;78:275– 82. [18] Meentzen A. Estrategias de desarrollo culturalmente adecuadas para mujeres indigenas. Unpublished document. Washington, DC: Unidad de Pueblos Indigenas y Desarrollo Comunitario. Banco Interamericano de Desarrollo, 2000. [19] Pan American Health Organization. Situacion de salud de los pueblos indigenas de Peru. Document OPS/HSP/HSO/98.08. Washington, DC: Pan American Health Organization; 1998. [20] Francke BP, editor. Politicas de Salud 2001–2006. Diagnos- tico y Propuesta 5. Lima: Consorcio de investigacion eco- nomica y social, 2001. Available at: CIES/html/diag5.asp. [21] Donnay F. Maternal survival in developing countries: what has been done, what can be achieved in the next decade. Interna- tional Journal of Gynaecology and Obstetrics 2000;70:89–97. [22] Donabedian A. The definition of quality and approaches to its assessment. Ann Arbor, Michigan: Health Administration Press; 1980. [23] McCarthy J, Maine D. A framework for analysing the deter- minants of maternal mortality. Studies in Family Planning 1992;23:23–33. [24] Benavides B, Seclen JE, Novata J, Velasquez A. Impacto del mejoramiento continuo de la calidad en los servicios materno perinatales del Peru. La experiencia del Proyecto 2000. Unpub- lished document. Lima: Pathfinder Fund, 2000. [25] Seclen J, Benavides B, Jacoby E, Velasquez A, Watanabe E. Existe una relacion entre los programas de mejora de la calidad y la satisfaccion de usuarias de atencion prenatal? Experiencia en hospitals de Peru. Revista Panamericana de Salud Publica 2004;16:149–57. [26] EASURE DHS+. Peru 2000 DHSIV Final Report. FR-120. Calverton, MD, 2000. [27] Cureton EE, D’Agostino RB. Factor analysis: an applied approach. Hillsdale, NJ: Erlbaum; 1983. [28] Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70:41–55. [29] Imben G. Nonparametric estimation of average treatment effects under exogeneity: a review. Technical Working Paper
  12. 12. 232 M.J. McQuestion, A. Velasquez / Health Policy 77 (2006) 221–232 293. Cambridge MA: National Bureau of Economic Research, 2003. [30] AbouZahr CL, Wardlaw T. Antenatal care in developing countries. Promises, achievements and missed opportunities. Geneva: World Health Organization; 2003. [31] Filmer D, Pritchett L. Estimating wealth effects without expenditure data or tears: an application to educational enrollments in states of India. Demography 2001;38:115– 32. [32] Rabe-Hesketh S, Skrondal A, Pickles A. GLLAMM manual. Paper 160. UC Berkeley Division of Biostatistics Working Paper Series. The Berkeley Electronic Press, 2004. http://www. [33] Stata Corporation. Stata statistical software. Release 6.0. Col- lege Station, TX: Stata Corporation; 1999. [34] Longford NL. Simulation-based diagnostics in random- coefficient models. Journal of the Royal Statistical Society A 2001;164(Part 2):259–73. [35] Gertler P. Final report. The impact of Progresa on health. Unpublished paper. International Food Policy Research Insti- tute. Washington, DC: International Food Policy Research Insti- tute, 2000. [36] Gilson L. Trust and the development of health care as a social institution. Social Science and Medicine 2003;56:1453–68. [37] Acharya L, Cleland J. Maternal and child health services in rural Nepal: does access or quality matter more? Health Policy and Planning 2000;15:223–9. [38] Ross G, Zeballos J, Infante A. La calidad y la reforma del sector salud en America Latina y el Caribe. Revista Panamericana de Salud Publica 2002;9:93–7.