Assessing Child/Household Needs and Well Being_Senefeld_5.2.12
•Download as PPTX, PDF•
0 likes•672 views
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (9)
Similar to Assessing Child/Household Needs and Well Being_Senefeld_5.2.12
Similar to Assessing Child/Household Needs and Well Being_Senefeld_5.2.12 (20)
More from CORE Group
More from CORE Group (20)
Recently uploaded
Recently uploaded (20)
Assessing Child/Household Needs and Well Being_Senefeld_5.2.12
- 1. OVC Wellbeing Tool (OWT)
- 2. OWT Guiding Principles The OWT is based on the following guiding principles: 1. Wellbeing from the child’s perspective 2. A valid and reliable measure of wellbeing 3. Age-appropriate 4. Applicable to multiple settings 5. Ease of Use 6. Repeated measure
- 3. Background • CRS technical staff recognized the need for a comprehensive measure of OVC wellbeing • Goal: create an instrument which can be used internationally to represent holistic OVC programming that is valid, reliable and practical to administer.
- 4. OWT Development The OWT was developed over a two- year time period between 2006 and 2008 through a multi-step process: 1. CRS technical advisors brainstormed on domains of wellbeing 2. Domains subsequently verified through a process of review and comparison against other tools and definitions of wellbeing 3. Self-reported statements generated for each domain using a free listing methodology as well as adaption from other validated tools 4. Statements shared with expert judges within CRS, drawn from fields such as education, social protection, health, and food security 5. Draft tool piloted through a larger evaluation of PEPFAR-funded OVC programs 6. Advanced statistical analyses, along with feedback from the pilot countries further served to refine the OWT
- 5. Tool Development • Originally 48 Questions • Self-Report Measure • Likert Scale • Used for Children Aged 13-18 • 10 Domains of Wellbeing – Nutrition and food security – Spirituality – Shelter and environment – Mental health – Protection – Education – Family – Economic – Health – Community cohesion
- 6. Tool Development: Piloting the OWT • 5 country evaluation: – Rwanda – Kenya – Zambia – Haiti – Tanzania
- 7. Tool Development: Analysis • Data from 890 OVC from the 5 countries • Compared the OWT data to larger evaluation data • Validated OWT against Hope Scale • Statistical Analysis: – Cronbach’s Alpha – Confirmatory Factor Analysis
- 8. Tool Development: Pilot Results • Cronbach’s Alpha • Confirmatory Factor Analysis – Reduced number of items in the scale – 48 to 36
- 9. Tool Development: Correlating OWT with larger survey • Based on domains of traditional OVC intervention • Similar across all five countries, corresponding to OVC sectoral foci • Generally strong correlation between larger survey and OWT scores
- 10. Tool Development: The Children’s Hope Scale • Stable tool shown to have internal consistency, convergent, discriminant and incremental validity • Children’s hope conceptualized as positive expectations; agency and pathways • Consistent with various definitions of resilience • 6 point Likert scale • Premise • Children are goal directed • Higher “hope” → Increasing levels of agency and pathways thinking • Validated against Children’s Hope Scale, the OWT showed a Spearman’s Rho of p<.01 for both the original long version and the shortened version. 1Snyder et al (1997). The Development and Validation of the Children’s Hope Scale. Journal of Pediatric Psychiatry 22(3), 399-421.
- 11. Tool Development: Challenges • Hope Scale Validation • Translation • Health domain revisited • Data management
- 12. Tool Development Conclusion • OWT now finalized – Reduced number of items in the tool from 48 to 36 items – Required approximately 20 minutes to complete • End Result: A valid, practical tool to monitor OVC wellbeing according to the interventions that are being implemented.
- 13. Tool Use: Where and Who • CRS: Botswana, Kenya, Rwanda, Haiti, Tanzania, Lesotho, Zambia, Malawi, Vietnam, Ethiopia, DRC, Pakistan, India • Other agencies: Western Africa, Kenya, Tanzania, Malawi, India, Haiti, Botswana
- 14. Tool Use: How the OWT information should be used • To monitor OVC programs at an aggregate level to identify patterns of change in OVC wellbeing within projects. – Weaker domains should be explored – Strengths identified • NOT an in-depth assessment tool at the individual level! – However, rapid scoring can highlight need for follow-up and more in-depth assessment with children who report significant problems.
- 15. Strengths and Challenges of such a tool • Strengths: – Child’s perspective – Holistic – Universal tool – Age appropriate – Rapid – Low cost • Challenges: – Universal tool – Translation accuracy – Eliciting information from children on sensitive subjects
- 16. Feedback from the Children • Initial feedback was critical in the adaptation of the tool (e.g. questions eliminated based on their feedback) • Currently=Enthusiastic • Have said they are pleased to be able to give their opinion on matters relating to them • No current reported problems understanding the questions on the OWT
- 17. OWT Application: Kenya • In April 2008, OWT baseline administered for 633 participants in OVC program in Nyanza Province • Program Areas – Education and Vocational Training – Health – Psychosocial Support • The OWT re-administered to 345 of these same OVC in March 2009
- 18. OWT Application: Kenya OWT domain scores before and after one year of OVC programming
- 19. OWT Application: Kenya OWT domain scores before and after one year of OVC programming
- 20. OWT & CSI Some Comparisons Suggest Criterion Related Validity • Measures consistent • Measures seem to assess different constructs 1. Food security and 1. Shelter – CSI ratings seem to refer economic issues to physical structure; child self 2. Health and wellness report seems related to care from 3. Mental health adults 2. Child Protection – OWT child feels safe and treated like others, whereas CSI observes possibility of abuse, legal needs.
- 21. Latest work: adaptation for younger age group • Two OVC programs, in Kenya and Rwanda, administered an adapted version of the OWT to children aged 7-12 (N=593; n=288 Kenya and n=305 Rwanda). • The original OWT used a three-point Likert scale, which was easy for adolescents to understand, but rendered sensitivity detection regarding change over time more difficult. Thus, the current pilot included a five-point Likert scale response set (i.e., never true, rarely true, sometimes true, often true, always true). • To ensure children understood the Likert scale responses, pictorial representations of the responses were developed, along with standardized enumerator instructions for explaining items, item response options, and query responses. • Challenges in developing images that represented measurements in a culturally- appropriate way. Pre-testing demonstrated visual representation of cultural measurements varies from one context to the next; different graphics ultimately were used in Kenya and Rwanda.
- 23. Latest work: adaptation for younger age group (2) • 56 items; stats TBD fewer items for final tool • 87.5% of enumerators (Kenya) & 89.1% in Rwanda reported overall experience with the tool for this age group as either “great” or “good” • 80.1% of enumerators Rwanda and 88.2% in Kenya reported that their experience with the pictorial representations was “great” or “good” • All items were reported as easily understood by at least 70% of surveyed children. The majority of items (~80%) required no additional explanations beyond the initial instructions.
- 24. Thank you Contact: Dr. Shannon Senefeld Catholic Relief Services Shannon.senefeld@crs.org http://www.crsprogramquality.org/ovcwt/ Senefeld, S., Strasser, S., Campbell, J., & Perrin, P. (2011). Measuring adolescent wellbeing: The development of a standardized measure for adolescents participating in orphan and vulnerable children programming. Journal of Vulnerable Children and Youth Studies, 6:4, 346-359. Tool developed by a team of staff, led by Shannon Senefeld, Susan Strasser, and James Campbell. Please note that the photographs in this publication are used for illustrative purposes only; they do not imply any particular health status (such as HIV or AIDS) on the part of the person who appears in the photograph.
Editor's Notes
- A goal of orphan and vulnerable children (OVC) programs is to improve wellbeing. Yet, measuring wellbeing has proven to be an elusive concept for many engaged in OVC programming. Catholic Relief Services (CRS) has placed an agency priority on OVC programming and aimed to find a way to measure the wellbeing of OVC in a holistic manner. The OWT was not designed to provide in-depth information about individual children in the program. Instead the tool should be used as a rapid assessment tool to determine if additional intervention is needed at the program level. Point 1 – While it is standard practice for programs to collect data and reports from caregivers, teachers, and other adults interacting with OVC, at the time the OWT was developed the authors were unable to find any cross-culturally adapted and easy-to-use quantitative tools available that assessed a child’s wellbeing from his or her perspective. To address this gap, an effort was made to design a tool that would be suitable for capturing wellbeing from the child’s perspective. There was also a need for a tool that would move program monitoring beyond commonly collected output indicators, such as the number of children provided with a service, to a higher measure of outcome, such as quality of life or wellbeing. Point 2 - Since wellbeing is a vaguely defined construct, and factors which affect wellbeing are many and complex, the tool needed to be scientifically grounded and based on current best practices in OVC programs. The correlation with a previously validated tool was seen as essential to strengthen the validity and usefulness of the tool. Point 3 - The developers of the tool realized that although it would be ideal to have a self-reported measure for all age groups, it is necessary to develop different tools for different age groups due to their different developmental levels. As the majority of existing research focuses on adolescent self-reports, the developers opted to first focus on developing a tool for OVC aged 13-18. This tool should not be used for children outside this age rangePoint 4 - While the team recognized the usefulness of a tool developed from an ethnographic methodology, ultimately the developers opted for a tool that could be applied in many settings and cultures. As such, the OWT was developed to be appropriate for multiple contexts, which allows for the same tool to be used, and for results to be compared, across various countries. In some cases, the tool may be strengthened by adapting it to the local culture. Such adaptation is encouraged as long as it follows a scientific process and is validated accordingly. Please note, however, that such adaptation will not necessarily allow the results to be compared to those derived from countries that used the standardized OWT. Point 5 - The team desired a tool that was straightforward and easy to administer. In addition, it was necessary to create a tool that could be administered quickly to the children in their natural environments. Point 6 - As this tool will be used with children over time to monitor trends, it was necessary to design a tool that could be easily repeated. Repeated measures strengthen a program’s ability to accurately track children’s wellbeing and monitor and evaluate its effectiveness over time.
- Point 2 – This was done using what was available in the published literaturePoint 3 – For example, for the food and nutrition domain, one statement was “I have enough food to eat.” More than 100 statements were generated for the different domains; many were taken or adapted from other validated tools. Point 4 – Nearly 40 different judges ranked the statements’ relevance as a proxy measure for wellbeing. Forty-eight statements, each answerable using a 3-point Likert scale, were kept based on ≥ 70% agreement (S.D.≤0.75) among judges.Point 5 – Pilot countries included Haiti, Kenya, Rwanda, Tanzania, and ZambiaPoint 6 –A total of 890 children aged 13–18 years participated in the pilot of the tool. To assess validity, the tool was also compared to the previously validated Children’s Hope Scale, and showed strong correlation. However, many country programs expressed concern that, at 48 questions, the tool was too long to administer to large groups of children. In order to reduce the number of items in the tool, confirmatory factor analysis was conducted. The resulting tool consists of 36 statements grouped into ten domains. The OWT has now been used in Ethiopia, Haiti, India, Kenya, Malawi, Rwanda, Tanzania, Vietnam, and Zambia.It is important to note that the OWT was designed as a tool that could have broad use across countries and regions. The OWT was not developed using an ethnographic approach. It is completely possible that communities in certain areas will have a different understanding or definition of what constitutes “wellbeing” for the children in their contexts. As such, it is important to engage with communities to determine if the domains in this wellbeing tool are relevant to the culture before administering and subsequently running the risk of misinterpreting the data from their perspective. Yet, the authors believe that in general each domain represents an important universal component of healthy child development and wellbeing.
- Map with all of this highlighted? Hmmm.
- With an estimated HIV prevalence of 15.3%, Nyanza Province has the highest HIV prevalence in Kenya. It is estimated that there are 500,000 orphans in Nyanza Province, accounting for about 25% of the orphans in Kenya. Program Areas bullet – 1) Education: More than 70% of OVC enrolled in the program received education support in the form of school fees, uniforms, materials and school visits by community health workers. Graduates of primary school were enrolled in secondary school or in polytechnics for vocational training.(2) Health: In partnership with government health facilities, the program payed for treatment of all beneficiaries (OVC and caregivers) who are in need. The program also carried out HIV counseling and testing within the project area and makes referrals for antiretroviral treatment. OVC were trained using “In Charge,” a curriculum for HIV prevention and are guided in training their peers. 3) Psychosocial: All OVC were visited twice a month by a social worker or a community health volunteer. Volunteers are trained to identify and provide or refer counseling for children with special needs.
- At baseline, this sample had the lowest overall scores in the community (1.65), family (1.81), economic (1.82), and protection (1.84) domains. The highest scores were in education (2.12) and faith domains (2.08). The overall average wellbeing score was 18.7 out of 30. One year later, all domains had improved. The lowest domains were protection (1.90) and economic (2.00). The highest domains were education (2.50) and faith (2.49). The overall wellbeing score was 22.46.
- A paired samples t-test between the baseline and post-intervention group revealed statistically significant changes in overall wellbeing score, along with significant changes in 8 of 10 domains. Only protection and economic domains were not statistically significantly changed, although there were positive increases in both domains. Noteworthy, the targeted intervention areas of the program (i.e., education, psychosocial support, and health) all were significantly increased from baseline after the one-year intervention period. The OWT can be used as a repeated measure to assess the impact of OVC programming on child wellbeing. It should be used to monitor OVC programs at an aggregate level to identify patterns of change in OVC wellbeing over time. The results of the OWT assist the program to focus interventions in lowest scored domains. In this particular illustration, the OWT was able to demonstrate program impact in the wellbeing of children. The changes in scores from baseline correspond to the targeted interventions within the program. Those programs that were not directly targeted, but addressed primarily through referral networks, demonstrated the least amount of change from baseline. However, all domain scores increased over time, along with the overall wellbeing score, suggesting that even indirect interventions contribute to the overall wellbeing of children in the program.