Your SlideShare is downloading. ×
Leave those surveys behind!
Key issues to consider in evaluating
VYA programs
Rebecka Lundgren and Susan Igras
USAID Globa...
Recent IRH Youth Initiatives
• Evaluation of the New “My Changing Body”
(CRS, Rwanda ), (APROFAM, Guatemala)
• Evaluation ...
Technical Consultation for Advancing Program and
Research/Evaluation Practices for Curriculum-based
Programs Reaching Very...
Social construction of childhood
“Children live in and negotiate
worlds that they create for
themselves,
worlds others cre...
Childhood studies
“Listening to the voices of children themselves
(rather than what adults say about them) reveals
what is...
Methodological Issues
• Research methods advantage
adults in terms of social or
communication skills or
knowledge.
• Need ...
Visual, participatory methods
• Communicate meanings
• Adaptation of methods used
in play therapy (clay, cultural
artifact...
• Protection
• Provision
• Participation
• Voice,
representation and
authenticity
Implications for Research with VYA:
ethi...
Key VYA Program Evaluation Questions
- What effect can we realistically
expect from programs?
- How can these be measured ...
Fertility Awareness/Body Literacy:
•Understanding fertility
•Accepting sexuality
•Understanding changes in puberty
•Self-c...
HIV/STI Prevention:
•Understanding
risky/protective behaviors,
symptoms, routes of
transmission
•Communication (e.g.
condo...
Environmental:
•Availability of peer networks
•Social support
•Safe spaces
•Parental supervision
•Asset building opportuni...
Our Challenge: Develop participatory
methods appropriate for VYAs to evaluate
changes in cultural/social meanings
Particip...
BUT… how to measure change using
these methods?
• Compare with control group
• Triangulate with quantitative
data
• Use st...
Techniques for collecting
information from VYAs on gender
and fertility awareness
•Card games
•Gender role game/pile sort
...
Evaluation Methods
• Stories and advice
(explore gender
attitudes)
• My week (pie chart
activity to find out how
boys and ...
This is the story of Juan who is
14 years old. Last month when he
woke up his underwear was a
little wet and he realised h...
Do you think that Juan needs
to ejaculate each time he has
an erection?
Is it bad for Juan to touch his
genitals (masturba...
These techniques can be used
with groups…
Or individually….
They can be used to collect
qualitative….
...or quantitative data
LET’S TRY A FEW……
Agree or
Disagree? OPINION
CARDS
FACT
CARDS
ADVICE
CARDS
CARD GAME
FREE LISTING
AND PILE SORT
Boys! Girls!
Gender!
Hopes and Dreams
PHOTO
ELICITATION
Life in my
community
Photo Voice
HOW HAVE
WE USED
THESE
TECHNIQUES?
Experimental Child Clubs
(12)
Choices
Control Child Clubs
Data collection
Structured interviews
w/youth (600)
O1 O2
In-dep...
Evaluation Indicators:
Behaviors
Boys and girls…
• talk about their feelings and dreams with each
other
• promote gender e...
Evaluation Indicators:
Attitudes
Boys and girls…
• imagine a life in which men and women have
equal opportunities
• accept...
Study Design: My Changing Body
Phase 1:
Formative Research
• 400 boys and girls and
their parents and teachers
(2 sessions...
RESULTS
Knowledge of male fertility among
youth and parents before and after
MCB
Nocturnal
ejaculation
normal
Fertile every
day
p<...
Youth shift towards less stereotyped
gender attitudes after MCB-Guatemala
(n=57)
Hopes and Dreams of “Choices” Participants
Girls say…
“I would ask my brothers to
talk to my parents to continue
my educat...
“People laugh at a man who cooks food in their home. But from the
day we have taken “Choices” classes, our brothers have s...
“Life for boys and girls is not equal in our community. Most of
the boys go to school while girls have to look after house...
“Life for boys and girls is not equal in
our community. Boys have freedom,
they can go any where they want ,but
our parent...
Final thoughts on
methods for evaluating
programs reaching very
young adolescents
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Leave those surveys behind! Key issues to consider in evaluating VYA programs
Upcoming SlideShare
Loading in...5
×

Leave those surveys behind! Key issues to consider in evaluating VYA programs

887

Published on

Published in: Health & Medicine, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
887
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • COMPLETE TITLE

  • The social constructionist perspective recognizes individuals as active agents in constructing and reconstructing their identity. Current work in the anthropology of children views young people as active agents who construct their own identities within a social space that is structurally determined by a range of social institutions.
  • Understanding childhood as a socio-structural space as well as children’s own perspectives as social actors
  • Visual methods (drawing, photography)
    - unintimidating, enjoyable, minimize power imbalance, help project meaning
  • Visual methods such as photography, videotaping and drawing may be familiar, non intimidating and even enjoyable to children and offer a means of minimizing the power imbalances inherent in traditional methods such as interviews or focus groups.
    According to child-centered researchers,
    Clark, for example, proposes the adaptation of methods used in play therapy such as role playing, drawing and painting, games, clay manipulation and play with cultural artifacts (telephones, dolls, puppets).
    Through these methods,
  • Adult-child power imbalances
    Access to children
    Voluntary and meaningful consent/assent
    Methodologies advantage adults
  • Two youth facilitators led the activities and discussion, at times dividing the group into smaller discussion groups to provide more time for all of the children to participate..
    Two youth observers attended each session, taking notes and photographing the visual output. Matrices were developed for the facilitators and observers to record key points of each discussion, and where appropriate, quantify results. Even so, documentation was challenging.
  • Card Games: opinion, advice, facts (roll colored die, select colored card, answer recording sheet)

    Agree or disagree:
    It’s normal that boys and girls begin to have romantic feelings once their bodies began to develop.

    Opinion:
    A boy can ride the bus to the mall alone, but a girl must be accompanied by an older brother or parent. Is this difference fair to girls? To boys?

    Advice:
    Your classmate tells you that someone in his family touches him in a way that he doesn’t like. He told his aunt, but she told him that he must be making it up. What would you tell him?



  • Group activitires
    Boys! Girls! Gender
    My Universe
    Living my Changing Body

  • During the two phases of this research, we had the opportunity to try out a variety of methods to better understand the lived experience of pre-adolescents (formative research), while at the same time working to develop ways to measure changes in body literacy and gender awareness resulting from interventions.
    We experimented with a number of methods in different settings, including using video clips, puppet (ET from outer space), gender-stereotyped toys, storytelling, collages, mapping, photo elicitation and photo voice. Some of these we abandoned, and other we have refined and are now using in a new study in Nepal with STC.
    For the rest of my presentation today, I will share some of the results we obtained from use of these methods
  • EXAMPLE OF DATA FROM CARD GAME
  • PARTICIPANTS IN CONTROL GROUP DID NOT MENTION BROTHER OR SISTERS.
  • Transcript of "Leave those surveys behind! Key issues to consider in evaluating VYA programs"

    1. 1. Leave those surveys behind! Key issues to consider in evaluating VYA programs Rebecka Lundgren and Susan Igras USAID Global Health Mini University, October 2010
    2. 2. Recent IRH Youth Initiatives • Evaluation of the New “My Changing Body” (CRS, Rwanda ), (APROFAM, Guatemala) • Evaluation of Save the Children Curriculum “Choices” (Nepal) • New! Gender Forming and Transforming Interventions for Youth in Uganda (Pathfinder, Save the Children)
    3. 3. Technical Consultation for Advancing Program and Research/Evaluation Practices for Curriculum-based Programs Reaching Very Young Adolescents…… • June 2010 • 30 participants • Literature and Program Review document prepared for meeting • Report of meeting (forthcoming)
    4. 4. Social construction of childhood “Children live in and negotiate worlds that they create for themselves, worlds others create for them, and worlds in concert with others.” Myrna Bluegood
    5. 5. Childhood studies “Listening to the voices of children themselves (rather than what adults say about them) reveals what is important to them, such as the patterning of gender in children’s social relationships.” (Alison James, 2007)
    6. 6. Methodological Issues • Research methods advantage adults in terms of social or communication skills or knowledge. • Need for methods that shift balance of power • Move from verbal to visual
    7. 7. Visual, participatory methods • Communicate meanings • Adaptation of methods used in play therapy (clay, cultural artifacts, drawing, role playing) • Children show what they want to communicate, as well as expressing their thoughts verbally.
    8. 8. • Protection • Provision • Participation • Voice, representation and authenticity Implications for Research with VYA: ethics and methods
    9. 9. Key VYA Program Evaluation Questions - What effect can we realistically expect from programs? - How can these be measured in a project time frame? - How can we operationalize concepts such as gender attitudes and norms? - How reliable and valid are our results?
    10. 10. Fertility Awareness/Body Literacy: •Understanding fertility •Accepting sexuality •Understanding changes in puberty •Self-care •Intergenerational communication skills •Self-advocacy of these topics with peers, parents and other adults Selected Outcomes for VYA Programs Gender: •Gender consciousness •Attitudes toward gender roles & equity •Gender equitable behavior Agency: •Self efficacy •Self esteem •Self confidence •Assertiveness
    11. 11. HIV/STI Prevention: •Understanding risky/protective behaviors, symptoms, routes of transmission •Communication (e.g. condom negotiation, seeking advice from providers Sexual Behaviors: •Delay of early marriage •Age at first intercourse/child •Contraceptive use •# adolescent/unintended pregnancies •STI prevention behavior: # partners, abstinence, condom use Selected Outcomes for VYA Programs Gender-based Violence: •Acceptability of GBV •Experience of GBV
    12. 12. Environmental: •Availability of peer networks •Social support •Safe spaces •Parental supervision •Asset building opportunities •Youth friendly services availability •Availability of gender-equitable recreational opportunities, such as sports & other activities Selected Outcomes for VYA Programs
    13. 13. Our Challenge: Develop participatory methods appropriate for VYAs to evaluate changes in cultural/social meanings Participatory Methods Youth play active role in info- gathering. Useful to encourage change based on results. Qualitative Methods Explore meanings, processes, explanations
    14. 14. BUT… how to measure change using these methods? • Compare with control group • Triangulate with quantitative data • Use structured matrices to “process” and compare data • Take photographs of visual data • Capture open-ended answers in quantifiable ways
    15. 15. Techniques for collecting information from VYAs on gender and fertility awareness •Card games •Gender role game/pile sort •Stories/Q&A •Advice letters •Photo elicitation •PhotoVoice •Free listing •Collage/artwork
    16. 16. Evaluation Methods • Stories and advice (explore gender attitudes) • My week (pie chart activity to find out how boys and girls spend their time) • Pile sorts with photos (to explore frequency of gender-equitable behavior)
    17. 17. This is the story of Juan who is 14 years old. Last month when he woke up his underwear was a little wet and he realised he had his first ejaculation. Sometimes his penis gets hard when he sees someone he likes. He is not sure if that is normal, but he is too embarassed to talk about this with anyone. Juan’s Story
    18. 18. Do you think that Juan needs to ejaculate each time he has an erection? Is it bad for Juan to touch his genitals (masturbate) often? Starting now, will Juan be fertile every day or only some days? What do you tell Juan?
    19. 19. These techniques can be used with groups…
    20. 20. Or individually….
    21. 21. They can be used to collect qualitative….
    22. 22. ...or quantitative data
    23. 23. LET’S TRY A FEW……
    24. 24. Agree or Disagree? OPINION CARDS FACT CARDS ADVICE CARDS CARD GAME
    25. 25. FREE LISTING AND PILE SORT Boys! Girls! Gender!
    26. 26. Hopes and Dreams PHOTO ELICITATION
    27. 27. Life in my community Photo Voice
    28. 28. HOW HAVE WE USED THESE TECHNIQUES?
    29. 29. Experimental Child Clubs (12) Choices Control Child Clubs Data collection Structured interviews w/youth (600) O1 O2 In-depth interviews w/youth O2 Focus groups with parents O2 Photovoice with youth O2 Choices Evaluation Design
    30. 30. Evaluation Indicators: Behaviors Boys and girls… • talk about their feelings and dreams with each other • promote gender equity in their lives • take action to improve the lives of their sisters • don’t tease their peers for behaving in non- traditional gender norms
    31. 31. Evaluation Indicators: Attitudes Boys and girls… • imagine a life in which men and women have equal opportunities • accept non-traditional gender roles • value relationships based on equality, respect and intimacy • value the role of men nurturing their family as well as providing financial support • expect to make decisions jointly with their spouses
    32. 32. Study Design: My Changing Body Phase 1: Formative Research • 400 boys and girls and their parents and teachers (2 sessions) Phase 2: Evaluation  Pre/post measurement of control and experimental group (Individual interviews)  Post-test only, control and experimental group (Group activities)
    33. 33. RESULTS
    34. 34. Knowledge of male fertility among youth and parents before and after MCB Nocturnal ejaculation normal Fertile every day p<.01
    35. 35. Youth shift towards less stereotyped gender attitudes after MCB-Guatemala (n=57)
    36. 36. Hopes and Dreams of “Choices” Participants Girls say… “I would ask my brothers to talk to my parents to continue my education and I would study higher and become a successful woman some day.” “To make my dream come true, I should ask for help form my brothers and work hard.” Boys say.. “I want to be a banker and also want my sisters to have a respected job in the future.”
    37. 37. “People laugh at a man who cooks food in their home. But from the day we have taken “Choices” classes, our brothers have started helping us and we help them too. We will teach the same to our friends in our village as well.” - Girl after participating in “Choices”
    38. 38. “Life for boys and girls is not equal in our community. Most of the boys go to school while girls have to look after household chores. Men have more freedom, they don’t have to work at home Girls are married and sent to her husband’s home. However, we have learnt from the child club that life for boys and girls is equal. We have to work together and help each other”. Boy after participating in “Choices”
    39. 39. “Life for boys and girls is not equal in our community. Boys have freedom, they can go any where they want ,but our parents do not allow our sisters to go outside home. Boys can play games, while girls have to look after household chores .Boys should do boyish work not girlish work. If people in our community see boys doing girl’s work they laugh at them, they are believed to be inferior.“ Boy in Control Group
    40. 40. Final thoughts on methods for evaluating programs reaching very young adolescents

    ×