Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children: Background and Methods

Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children: Background and Methods Florence Nyangara, PhD MEASURE Evaluation/Futures Group Dissemination Meeting, September 3 rd , 2009 Washington, DC

The number of OVC and their corresponding programs increasing
SSA - 12 million orphans (0-17 yrs), 2003
2010 - over 18 million orphans (O)
Other millions are made vulnerable – HIV/AIDS, dire poverty, war, etc (V)
Response – increased attention to the plight of OVC (funds, programs)
Children on the Brink , 2004 Sub-Saharan Africa’s population of children orphaned by AIDS increasing Background

Response to the OVC Crisis
OVC programs – emergency response to areas most HIV-affected
Strategies used were based on existing cultural support systems, conventional wisdom, and lessons learned from other program areas;
Support community-based responses (capacity & resources)
Household/family support (capacity & resources)
Direct support to families & OVC (access to essential services)
Gap – lack of evidence to guide OVC programs
Call for evidence based programming
2006 - USAID funded MEASURE evaluation to conduct targeted evaluations to fill this evidence gap
Background

Evaluation Goals
Find out “what works” in terms of
intervention models and program components
cost effectiveness, and
outcomes (benefits) for OVC and their caregivers in resource poor settings
Provide evidence to guide program decisions such as;
Scaling-up of best practices (models, strategies), and
Modify & improve interventions - to make them effective

Research Preparation Activities
Funds were made available – USAID/PEPFAR/PHE (4) and USAID/Tanzania mission funded (1) program evaluation.
Research team formed - MEASURE Evaluation
Extensive literature reviews (early 2006)
Consultation meetings with stakeholders
Identified OVC programs to be evaluated
Research protocol developed
Ethical approvals obtained – US, Kenya, and Tanzania
Identified local research partners (PSRI – KE; AXIOS - TZ)

Programs Evaluated
Evaluated Five programs : 2 in Kenya & 3 in Tanzania
They have different intervention models with v aried combinations of child, family/household, and/or community centered approaches (multi-faceted).
*** Although, the approaches vary, the goal for all of these programs is to improve the well-being of OVC and their families.

Programs Evaluated
Kenya (2)
Kilifi OVC Project, Catholic Relief Services (CRS)
Operating in Kilifi District for two years
Community Based HIV/AIDS Prevention, Care and Support Program (COPHIA), Pathfinder & Integrated AIDS Program (IAP)
Operating in Thika District for 4 years
Tanzania (3)
CARE Tumaini Project, Allamano, CARE, FHI (Allamano)
Operating in Iringa Region for five years
Mama Mkubwa & Kids Club, Salvation Army (TSA)
Operating in Mbeya Region for 2-years
Jali Watoto , Pact/SAWAKA ( Jali Watoto ) – Field funded
Operating in Karagwe, Kagera Region for four years

Overview of Programs & Strategies Evaluated

Key Research Questions
Impact of indirect support:
How do efforts targeted at the structural systems surrounding children– household and community– affect:
Children well-being
Caregivers well-being
Community attitudes and support of OVC & families?
Impact of direct support on child outcomes
What is the impact of educational, health, legal support, and other direct services on child & families?

Methods
Case studies (2006 and 2007)
Site visits, interviews, program document review
Provide understanding of program strategies, components, goals, and expected outcomes
Document lessons learned from implementation
Case Studies available http://www.cpc.unc.edu/measure/ovc
Program expenditures (2006)
Expense data collected and social costs estimated
Quantified the costs corresponding to specific intervention (e.g. food supplementation, psychosocial service, educational support)

Methods
Outcome evaluation (2007 and 2008)
Post-test study design with intervention & comparison groups
Exposed Vs. Non-exposed
Surveyed children age 8-14 “or 7-15” & their caregivers
Up to 2 children per household
Four questionnaires were applied in each household:
Q1: Household Questionnaires
Q2: Parent/Guardian/Caregiver Questionnaire
Q3: Parent/Guardian/Caregiver Regarding Child Questionnaire
Q4: Child (age 8-14 “or 7-15”) Questionnaire

Ideal Impact Assessment R

Post-test Study Design Used Exposed Non-exposed X O 1 O 2 The Groups are Not Randomly Assigned ** Jali Watoto – Study compared intact groups of intervention versus comparison

Research Designs and Sampling

Principles Guiding Questionnaires Development
Capture multiple measures for each domain
Use existing standardized scales where possible (PSS, SES)
Intervention Exposure questions to be specific to each program
To facilitate comparison across countries and program models, same survey instruments were used except intervention modules
Multi-faceted programs necessitated sufficient questions across multiple domains
Multiple perspectives on child well-being ( child and caregiver )
Measures of caregiver, household & community well-being

Outcomes Examined
Psychosocial well-being – multiple measures – standard scales used where appropriate ( child and caregivers )
Education – enrollment & attendance ( child )
Health – self-reported health status and access to health services ( child and caregiver )
HIV-prevention – HIV-knowledge ( child ) & HIV-testing ( caregiver )
Legal protection – birth registration, alternate caregiver
Community support – stigma and in-kind support ( child & caregiver )

Study Strengths and Limitations
Strengths:
Yielded immediate data on program effects
Results can be used to improve current programs
Ethical - not withholding services for experiment sake
Limitations:
Post-test design – no baseline data - impossible to make conclusions concerning change in outcomes resulting from program exposure
Selection bias - self-selection to participate and those who did not -makes it difficult to conclude with certainty that the interventions are responsible for the observed differences

Analyses Plan
Who are the OVC/MVC program beneficiaries?
Effects of community level interventions i.e.
Community care and support meetings/sensitization
Effects of household or caregiver level interventions i.e.
Community volunteer or Health Worker home visits
Caregivers participation in OVC care seminars
Effects of child level interventions i.e.
Kids clubs
Basic needs support (e.g., education, health, legal)

Statistical Analyses
Descriptive analysis (Univariate)
Bivariate analysis (ANOVA and Chi-square)
Multivariate (logistics, and linear regression)
Control variable: non-program factors e.g. socio-demographic
Child Level – Age, sex, orphan status, relationship to caregiver, and number of different homes the child had lived in the past year.
Caregiver level – Age, sex, marital status, education, illness, SES, and # children

Description of OVC Program Beneficiaries
Who is enrolled in OVC programs (MVC Profile):-
Although, these programs targeted geographic areas most affected by HIV/AIDS, MVC were identified and assisted regardless of specific causes of vulnerability
Majority of children enrolled in OVC programs are vulnerable in several fronts & not just orphanhood

OVC Profile
Orphans (over 66% across programs)
Living in food insecure HH (over 80% across programs)
P oorest households (< 2assets) – over 40%
Living with chronically ill primary caregiver (over 20%)
Living with caregiver aged 50+ (about 23%)
Lived in two or more households in past year (14%)

Thank YOU!
Key Findings are presented next….

MEASURE Evaluation is funded by the U.S. Agency for International Development through Cooperative Agreement GHA-A-00-08-00003-00 and is implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government. Visit us online at http://www.cpc.unc.edu/measure.

Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children: Background and Methods

by MEASURE Evaluation on Oct 13, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 2

Statistics

Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children: Background and Methods — Presentation Transcript