Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Leave those surveys behind!
Key issues to consider in evaluating
VYA programs
Rebecka Lundgren and Susan Igras
USAID Globa...
Recent IRH Youth Initiatives
• Evaluation of the New “My Changing Body”
(CRS, Rwanda ), (APROFAM, Guatemala)
• Evaluation ...
Technical Consultation for Advancing Program and
Research/Evaluation Practices for Curriculum-based
Programs Reaching Very...
Social construction of childhood
“Children live in and negotiate
worlds that they create for
themselves,
worlds others cre...
Childhood studies
“Listening to the voices of children themselves
(rather than what adults say about them) reveals
what is...
Methodological Issues
• Research methods advantage
adults in terms of social or
communication skills or
knowledge.
• Need ...
Visual, participatory methods
• Communicate meanings
• Adaptation of methods used
in play therapy (clay, cultural
artifact...
• Protection
• Provision
• Participation
• Voice,
representation and
authenticity
Implications for Research with VYA:
ethi...
Key VYA Program Evaluation Questions
- What effect can we realistically
expect from programs?
- How can these be measured ...
Fertility Awareness/Body Literacy:
•Understanding fertility
•Accepting sexuality
•Understanding changes in puberty
•Self-c...
HIV/STI Prevention:
•Understanding
risky/protective behaviors,
symptoms, routes of
transmission
•Communication (e.g.
condo...
Environmental:
•Availability of peer networks
•Social support
•Safe spaces
•Parental supervision
•Asset building opportuni...
Our Challenge: Develop participatory
methods appropriate for VYAs to evaluate
changes in cultural/social meanings
Particip...
BUT… how to measure change using
these methods?
• Compare with control group
• Triangulate with quantitative
data
• Use st...
Techniques for collecting
information from VYAs on gender
and fertility awareness
•Card games
•Gender role game/pile sort
...
Evaluation Methods
• Stories and advice
(explore gender
attitudes)
• My week (pie chart
activity to find out how
boys and ...
This is the story of Juan who is
14 years old. Last month when he
woke up his underwear was a
little wet and he realised h...
Do you think that Juan needs
to ejaculate each time he has
an erection?
Is it bad for Juan to touch his
genitals (masturba...
These techniques can be used
with groups…
Or individually….
They can be used to collect
qualitative….
...or quantitative data
LET’S TRY A FEW……
Agree or
Disagree? OPINION
CARDS
FACT
CARDS
ADVICE
CARDS
CARD GAME
FREE LISTING
AND PILE SORT
Boys! Girls!
Gender!
Hopes and Dreams
PHOTO
ELICITATION
Life in my
community
Photo Voice
HOW HAVE
WE USED
THESE
TECHNIQUES?
Experimental Child Clubs
(12)
Choices
Control Child Clubs
Data collection
Structured interviews
w/youth (600)
O1 O2
In-dep...
Evaluation Indicators:
Behaviors
Boys and girls…
• talk about their feelings and dreams with each
other
• promote gender e...
Evaluation Indicators:
Attitudes
Boys and girls…
• imagine a life in which men and women have
equal opportunities
• accept...
Study Design: My Changing Body
Phase 1:
Formative Research
• 400 boys and girls and
their parents and teachers
(2 sessions...
RESULTS
Knowledge of male fertility among
youth and parents before and after
MCB
Nocturnal
ejaculation
normal
Fertile every
day
p<...
Youth shift towards less stereotyped
gender attitudes after MCB-Guatemala
(n=57)
Hopes and Dreams of “Choices” Participants
Girls say…
“I would ask my brothers to
talk to my parents to continue
my educat...
“People laugh at a man who cooks food in their home. But from the
day we have taken “Choices” classes, our brothers have s...
“Life for boys and girls is not equal in our community. Most of
the boys go to school while girls have to look after house...
“Life for boys and girls is not equal in
our community. Boys have freedom,
they can go any where they want ,but
our parent...
Final thoughts on
methods for evaluating
programs reaching very
young adolescents
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Upcoming SlideShare
Loading in …5
×

Leave those surveys behind! Key issues to consider in evaluating VYA programs

1,043 views

Published on

Published in: Health & Medicine, Technology
  • Be the first to comment

  • Be the first to like this

Leave those surveys behind! Key issues to consider in evaluating VYA programs

  1. 1. Leave those surveys behind! Key issues to consider in evaluating VYA programs Rebecka Lundgren and Susan Igras USAID Global Health Mini University, October 2010
  2. 2. Recent IRH Youth Initiatives • Evaluation of the New “My Changing Body” (CRS, Rwanda ), (APROFAM, Guatemala) • Evaluation of Save the Children Curriculum “Choices” (Nepal) • New! Gender Forming and Transforming Interventions for Youth in Uganda (Pathfinder, Save the Children)
  3. 3. Technical Consultation for Advancing Program and Research/Evaluation Practices for Curriculum-based Programs Reaching Very Young Adolescents…… • June 2010 • 30 participants • Literature and Program Review document prepared for meeting • Report of meeting (forthcoming)
  4. 4. Social construction of childhood “Children live in and negotiate worlds that they create for themselves, worlds others create for them, and worlds in concert with others.” Myrna Bluegood
  5. 5. Childhood studies “Listening to the voices of children themselves (rather than what adults say about them) reveals what is important to them, such as the patterning of gender in children’s social relationships.” (Alison James, 2007)
  6. 6. Methodological Issues • Research methods advantage adults in terms of social or communication skills or knowledge. • Need for methods that shift balance of power • Move from verbal to visual
  7. 7. Visual, participatory methods • Communicate meanings • Adaptation of methods used in play therapy (clay, cultural artifacts, drawing, role playing) • Children show what they want to communicate, as well as expressing their thoughts verbally.
  8. 8. • Protection • Provision • Participation • Voice, representation and authenticity Implications for Research with VYA: ethics and methods
  9. 9. Key VYA Program Evaluation Questions - What effect can we realistically expect from programs? - How can these be measured in a project time frame? - How can we operationalize concepts such as gender attitudes and norms? - How reliable and valid are our results?
  10. 10. Fertility Awareness/Body Literacy: •Understanding fertility •Accepting sexuality •Understanding changes in puberty •Self-care •Intergenerational communication skills •Self-advocacy of these topics with peers, parents and other adults Selected Outcomes for VYA Programs Gender: •Gender consciousness •Attitudes toward gender roles & equity •Gender equitable behavior Agency: •Self efficacy •Self esteem •Self confidence •Assertiveness
  11. 11. HIV/STI Prevention: •Understanding risky/protective behaviors, symptoms, routes of transmission •Communication (e.g. condom negotiation, seeking advice from providers Sexual Behaviors: •Delay of early marriage •Age at first intercourse/child •Contraceptive use •# adolescent/unintended pregnancies •STI prevention behavior: # partners, abstinence, condom use Selected Outcomes for VYA Programs Gender-based Violence: •Acceptability of GBV •Experience of GBV
  12. 12. Environmental: •Availability of peer networks •Social support •Safe spaces •Parental supervision •Asset building opportunities •Youth friendly services availability •Availability of gender-equitable recreational opportunities, such as sports & other activities Selected Outcomes for VYA Programs
  13. 13. Our Challenge: Develop participatory methods appropriate for VYAs to evaluate changes in cultural/social meanings Participatory Methods Youth play active role in info- gathering. Useful to encourage change based on results. Qualitative Methods Explore meanings, processes, explanations
  14. 14. BUT… how to measure change using these methods? • Compare with control group • Triangulate with quantitative data • Use structured matrices to “process” and compare data • Take photographs of visual data • Capture open-ended answers in quantifiable ways
  15. 15. Techniques for collecting information from VYAs on gender and fertility awareness •Card games •Gender role game/pile sort •Stories/Q&A •Advice letters •Photo elicitation •PhotoVoice •Free listing •Collage/artwork
  16. 16. Evaluation Methods • Stories and advice (explore gender attitudes) • My week (pie chart activity to find out how boys and girls spend their time) • Pile sorts with photos (to explore frequency of gender-equitable behavior)
  17. 17. This is the story of Juan who is 14 years old. Last month when he woke up his underwear was a little wet and he realised he had his first ejaculation. Sometimes his penis gets hard when he sees someone he likes. He is not sure if that is normal, but he is too embarassed to talk about this with anyone. Juan’s Story
  18. 18. Do you think that Juan needs to ejaculate each time he has an erection? Is it bad for Juan to touch his genitals (masturbate) often? Starting now, will Juan be fertile every day or only some days? What do you tell Juan?
  19. 19. These techniques can be used with groups…
  20. 20. Or individually….
  21. 21. They can be used to collect qualitative….
  22. 22. ...or quantitative data
  23. 23. LET’S TRY A FEW……
  24. 24. Agree or Disagree? OPINION CARDS FACT CARDS ADVICE CARDS CARD GAME
  25. 25. FREE LISTING AND PILE SORT Boys! Girls! Gender!
  26. 26. Hopes and Dreams PHOTO ELICITATION
  27. 27. Life in my community Photo Voice
  28. 28. HOW HAVE WE USED THESE TECHNIQUES?
  29. 29. Experimental Child Clubs (12) Choices Control Child Clubs Data collection Structured interviews w/youth (600) O1 O2 In-depth interviews w/youth O2 Focus groups with parents O2 Photovoice with youth O2 Choices Evaluation Design
  30. 30. Evaluation Indicators: Behaviors Boys and girls… • talk about their feelings and dreams with each other • promote gender equity in their lives • take action to improve the lives of their sisters • don’t tease their peers for behaving in non- traditional gender norms
  31. 31. Evaluation Indicators: Attitudes Boys and girls… • imagine a life in which men and women have equal opportunities • accept non-traditional gender roles • value relationships based on equality, respect and intimacy • value the role of men nurturing their family as well as providing financial support • expect to make decisions jointly with their spouses
  32. 32. Study Design: My Changing Body Phase 1: Formative Research • 400 boys and girls and their parents and teachers (2 sessions) Phase 2: Evaluation  Pre/post measurement of control and experimental group (Individual interviews)  Post-test only, control and experimental group (Group activities)
  33. 33. RESULTS
  34. 34. Knowledge of male fertility among youth and parents before and after MCB Nocturnal ejaculation normal Fertile every day p<.01
  35. 35. Youth shift towards less stereotyped gender attitudes after MCB-Guatemala (n=57)
  36. 36. Hopes and Dreams of “Choices” Participants Girls say… “I would ask my brothers to talk to my parents to continue my education and I would study higher and become a successful woman some day.” “To make my dream come true, I should ask for help form my brothers and work hard.” Boys say.. “I want to be a banker and also want my sisters to have a respected job in the future.”
  37. 37. “People laugh at a man who cooks food in their home. But from the day we have taken “Choices” classes, our brothers have started helping us and we help them too. We will teach the same to our friends in our village as well.” - Girl after participating in “Choices”
  38. 38. “Life for boys and girls is not equal in our community. Most of the boys go to school while girls have to look after household chores. Men have more freedom, they don’t have to work at home Girls are married and sent to her husband’s home. However, we have learnt from the child club that life for boys and girls is equal. We have to work together and help each other”. Boy after participating in “Choices”
  39. 39. “Life for boys and girls is not equal in our community. Boys have freedom, they can go any where they want ,but our parents do not allow our sisters to go outside home. Boys can play games, while girls have to look after household chores .Boys should do boyish work not girlish work. If people in our community see boys doing girl’s work they laugh at them, they are believed to be inferior.“ Boy in Control Group
  40. 40. Final thoughts on methods for evaluating programs reaching very young adolescents

×