Findings from a longitudinal study in conjunction with Act With Her Ethiopia on wellbeing outcomes for very young adolescents.

1. Act With Her Ethiopia:
Short-run findings on programming with Very Young Adolescents
.
Workneh Yadete (GAGE Ethiopia, Quest Consulting), June 2022
Primary school students in Ebenat, Ethiopia © Nathalie Bertrams / GAGE 2020

2. Presentation overview
Introduction to GAGE
Contribution of GAGE impact evaluation
of the Act with Her programme
Research sample and mixed methods impact evaluation design and analytical approach
Short-term findings on primary and secondary outcomes
Reflections on how programming could be further strengthened

3. GAGE timeline, age of cohorts and gender equality milestones
YOUNGER
COHORT
10-12 years 12-14 years 13-15 years 15-17 years
GAGE launched
(2016)
2017 2018 2019 2020 2021 2022 2023
Baseline Midline 1 Midline 2 Endline
OLDER
COHORT
15-17 years 17-19 years 18-20 years 20-22 years
SDGs launched
(2015)
Family
Planning
Summit
London (2017)
BMGF
Gender
Strategy
(2018)
ICPD + 25
(2019)
Beijing + 25
Decade for
Action launch
(2020)
Generation
Equality
Forums
(2021)
Women
Deliver
Conference
(2022)
SDGs + 10
(2025)
Very young adolescent stage Mid-adolescent stage
Youth/ adolescent to early adulthood transition
COVID-19
virtual data
collection

4. UNICEF’sMakani
programme
(Jordan)
Community based
organisations
(Jordan)
Act with Her
(Ethiopia)
Plan International –
Champions of Change (Rohingya) ,
starting 2021 World Bank adolescent education
(Bangladesh)
LOST& Na’baa
(Lebanon)
Room to Read
(Nepal)
Programmes GAGE is evaluating
Update: Makani
programme adapting
to covid 19 and gov.
engagement
Update: engagement
& application of
midline findings
Update: engagement
& application of
participatory findings
Update: engagement &
application of
participatory findings
Update: Programme now
rolling out in Jan 21 due to
covid 19

5. GAGE 3C’s Conceptual Framework

6. Impact evaluation
sample and
methodology
Adolescent pregnant girl, Ethiopia © Nathalie Bertrams / GAGE 2020

7. GAGE longitudinal research sample

8. Context diversity is critical in understanding
programming starting points
Zones selected based on:
 high prevalence of child marriage
(as proxy for conservative gender
norms) and then
 districts that are designated food
insecure (as proxy for econ
vulnerability)
In South Gondar:
 96% of girls and 89% of boys are enrolled
in school
 Girls have completed a half year “more”
school than boys: 4.8 versus 4.3
In East Hararghe
 85% of boys and 69% of girls are enrolled
 Boys have completed a half year “more”
school than girls” 5 vs 4.5
In Zone 5, Afar:
 Only 63% of teens are enrolled
 Young adolescents have missed 24% of
school days in the last two weeks
 Average grade attainment is only 2.3 years
School enrolment varies tremendously—and in different ways
Figures
from GAGE
baseline

9. Context diversity is critical in understanding
programming starting points
In Amhara:
 Age of marriage climbed from 14.4 to
16.2 between 2005 and 2016
 “Only” half of girls are cut in Amhara,
3% are infibulated
In Oromia:
 Age of marriage was comparatively
unchanged, 17.1 to 17.4
 Infibulation is rarer in Oromia (7%),
where girls are cut later
In Afar:
 Age of marriage dropped from 16.7
to 16.4
 Nearly all girls under 15 are cut in
Afar—70% are infibulated
Similar diversity exists in terms of child marriage and FGM/C
Figures
from GAGE
baseline
Figures from
DHS

10. Quantitative sample
4,518 VYAs in 173 kebeles
Kebeles within woredas
selected based on mapping of
access to markets,
infrastructure and services
(high and low)
 Amhara: 1,113 girls and 819
boys (74 Kebeles)
 Oromia: 1,190 girls and 878
boys (79 kebeles)
 Afar: 298 girls and 220 boys
(20 kebeles)
Aged 10-12 during GAGE
baseline data collection
(late 2017; midline late
2019 8 months after
programming start)
Randomly selected from a door-to-door
household listing in communities
Randomly assigned kebeles to different
programming arms (and control) with
implementation starting in early 2019

11. Summary statistics (Oromia and Amhara, control)
Oromia Amhara
n=377 n=423
HH size 7.154 5.388
HH head literate 0.269 0.378
HH currently receives PSNP benefits 0.306 0.355
FIES score(0-8, higher is morefood insecure) 5.173 3.537
HH experiences any food insecurity 0.888 0.766
Panel A: Household level variables

12. Summary statistics (Oromia and Amhara control)
Girlsn=216 Boysn=161 Girlsn=241 Boysn=182
Age 12.921 12.969 13.000 13.054
Enrolled in school during most recent session 0.619 0.819 0.987 0.851
=1 if hungry in thepast 4 weeks 0.229 0.368 0.042 0.138
Number ofmeals with animal protein previous day 0.086 0.097 0.164 0.188
=1 if sourceof information on puberty 0.500 0.693 0.828 0.772
Has begun menstruation 0.106 na 0.050 na
=1 if can namea modern method of contraception 0.053 0.195 0.667 0.561
=1 if Ever Married 0.046 0.006 0.046 0.000
=1 if Has savings 0.686 0.675 0.344 0.260
=1 if Has not experienced/witnessed HH violencein
last 12 months
0.444 0.638 0.451 0.567
=1 if Has an adult s(he)trusts 0.452 0.716 0.697 0.674
Panel B: Individual level variables
Oromia Amhara

13. Qualitative sample
1 kebele per intervention arm and 1 control
per region
(i.e. 5 kebeles in Amhara and Oromia and 2
in Afar)
275 adolescents (179 HS or AWH
participants)
Inclusion of socially disadvantaged adolescents
based on disability (10%) and marital status (10%)
but largely non-programme participants
Amhara:
 112 adolescents
(77 HS or AWH participants)
 73 parents
 10 mentors
 34 key informants
Oromia:
 112 adolescents
(77 HS or AWH participants)
 87 parents
 13 mentors
 33 key informants
Afar:
 51 adolescents
(25 AWH participants)
 48 parents
 7 mentors
 10 key informants

15. Analysis approach for short-term findings
 All adolescents and parents in our study who live in an AWH community are
considered treated, whether they were part of the program or not.
 Generally considered the policy relevant outcome (as uptake of these programs is
always less than 100%, and often much lower)
These are intent to treat estimates:
 Interest in the program was high across regions (much higher than generally seen
with these programs)
 Still manually entering attendance data to give exact numbers on enrollment and
attendance (time consuming!!)—working for better process moving forward
Future quasi-experimental analysis will look at treatment on the treated
(e.g. are there larger impacts for those who actually participated in AWH)
Full pre-analysis plan can be found here. Trial is registered at AEA RCT Registry
(#AEARCTR-0004024), ClinicalTrials.gov (#NCT03890237), and went through the
BMGF review process.

16. Analysis approach for short-term findings
Qualitative data
was thematically
coded in MAXQDA
and disaggregated by
region, gender and
by participant status.
Qualitative data
used to triangulate
findings from quant
survey, to provide
context specificity to
findings, and to
disentangle potential
pathways to explain
positive, negative or
non-effects.
Key informant
interviews
provided additional
understanding of
community, district
and zonal political,
economic and social
dynamics.
Quotes
are used to illustrate
findings; where they
represent the voices
of outliers this is
noted.

17. High-level findings on
short-term outcomes
for girls in Amhara and
Oromia
Adolescent girl in Oromia, Ethiopia © Nathalie Bertrams / GAGE 2019

18. Reminder on Girls’ outcomes
Outcomes for adolescent girls:
• Education
• Bodily integrity
• Physical health, nutrition and SRH
outcomes
• Psychosocial wellbeing
• Voice and agency
• Economic empowerment
• Cross-cutting: attitudes,
knowledge, support systems
Adolescent girl in Oromia © Nathalie Bertrams /
GAGE 2020

19. Girls’ primary indices: Her Spaces vs. AWH (no assets)
 Significant changes generally all in the right direction, with
larger improvements for the AWH arm
 Significant differences in mental health and gendered
attitudes with greater improvement for the AWH group over
Her Spaces
 Significant program impacts on mental health (AWH only),
resilience, voice and agency and knowledge (both AWH and
Her Spaces) provide a pathway to longer-run impacts.

20. Girls’ primary indices: AWH (assets) vs. AWH (no assets)
 Girls in communities
assigned to receive the asset
transfer also showed
improvements in Voice and
Agency, Knowledge and Econ
empowerment, and Violence
 Did not see impacts on
mental health or gender
equitable attitudes
 Improvements in Mental
Health, Resilience, Voice &
Agency, and Knowledge
 No Impacts in Econ
Empowerment, Violence,
Gender Equitable Attitudes
AWH (assets) AWH (no assets)

21. Normalization of menstruation (qual and quant)
‘Thanks to the education, we don’t feel shame
about it currently.’ (Girl, Her Spaces)
‘We learn to manage menstruation with a
clean piece of cloth so that we can attend class
conveniently. The menstrual flow will not spill
down on our legs.’ (Girl, assets)
Girl in East Hararghe © Nathalie Bertrams / GAGE 2020
 Sig. positive impacts on improved menstrual
hygiene and menstrual management
(many girls not menstruating yet)
 Where groups were running well, there was
good learning regarding puberty and
menstrual hygiene management.

22. Room for improvement on knowledge about SRH (control)….
Girlsn=216 Boysn=161 Girlsn=241 Boysn=182
% Correctly identifiied foods that contain iron vs.
not
0.480 0.498 0.607 0.569
Correctly identifies healthy number ofmain meals
per day for an adolescent
0.833 0.839 0.983 0.917
Correctly answers: Approximately how often
does menstruation usually occur? (Once per
month, every four weeks, or once per 28
days=1, any other frequency=0, DK=0)
0.281 0.176 0.584 0.374
Correctly answers: Getting pregnant early in
puberty can be bad for the health of the girl
and the baby. (True=1, False=0, DK=0)
0.641 0.588 0.693 0.641
Correctly identifies the legal age of marriage
for girls 0.068 0.210
Correctly identifies that FGM/C hasrisks 0.157 0.213 0.510 0.389
Knows whereto get help for violence 0.292 0.522 0.269 0.55
Knows whereto keep money 0.734 0.759 0.887 0.93
Knowledge
Oromia Amahra

23. ‘They told us that they’ll give her a pill right away and they’ll
make sure the offender gets proper punishment too. If a girl
is raped, they told us that she has to take examinations at
the clinic.’ (girl, Her Spaces)

24. ‘Since I joined this program, I understand that child marriage is a harmful
tradition. Therefore, I would tell them that I will only marry once I completed
my education and once I am self-sufficient.’ (12-year-old girl, AwH)

25. Mixed effects on parent-adolescent communication (quant and qual)
‘Previously I was shy to express my thoughts
even in the classroom. But now I can express
my feelings and thoughts.’
(13-year-old girl, assets)
‘If they’re planning to make me skip school so
that I’ll do something for them, I’ll tell them
whatever it is they want me to do, I’ll get it
done after school.’ (Girl, assets)
 Girls report being better able to speak up—
at home and at school.
 BUT– the overwhelming theme is
that girls are obeying more specifically
because of what they’ve learned in AWH.
‘The thing is whenever they order us
something we used to disobey them but now
that we took the lessons we don’t do that
anymore….’ (Girl, Her Spaces)
‘I used to say no and be rebellious... But
now…I will always say yes.’
(Girl, curriculum)
 HOWEVER —improved obedience is improving mother-child relationships—and
may be strengthening resilience because girls feel more supported.

26. Shifting gender attitudes is not straightforward
‘The learners listen to the education when they attend it. They forget
it when they go out of the education session. They put it aside when
they meet male adolescents.’ (13-year-old boy, AwH, Oromia, reflecting on
girls’ submissive behaviour around boys)

27. Afar
Adolescent girl with her father, Afar © Nathalie Bertrams / GAGE 2020

28. Afar: context snapshot
Zone 5 is politically marginalized
in the region which exacerbates
service deficits; some very
remote communities which are
seldom visited by district officials
and have no NGO presence.
During baseline and midline
serious clan conflicts as well as
conflict with the Argobba ethnic
group in a neighbouring district
make difficult terrain even less
accessible, and also risky for
adolescents.
Drought conditions
experience at baseline had
much improved, as had local
food security - but rapidly
rising rates of migration of
adolescent girls to Djibouti
and MENA.
Major change in midline in civil service cadre
– new generation, with greater competencies
and vision for change.
Gender norms:
• absuma (mandatory maternal cousin) marriage
system;
• Type 3 FGM and sometimes infibulation carried
out as young as infancy through to early
adolescence

29. Afar: sample
• Smaller sample size. This was always meant as a pilot from the research POV. Can we
successfully conduct longitudinal mixed methods research in remote pastoralist settings.
• Implementers able to contact 84% of GAGE sample; 95% interested in programming.
• Successfully surveyed 81.5% of the sample at midline
• Not differential by treatment, nor based on baseline characteristics
• Just a very hard to reach population!
Research sample:
• Quantitative: 298 girls and 220 boys (20 kebeles)
• Qualitative:
• 51 adolescents, 48 parents, 7 mentors, 10 key informants

30. Baseline characteristics (control)
Overall
HH size 6.320
HH head literate 0.053
HH currently receives PSNP benefits 0.398
Pastoralist HH 0.877
Girls Boys
Age 10.952 10.762
Enrolled in school during most recent
session
0.713 0.536
Did not miss a week of school in the
past year (among enrolled)
0.656 0.756
Has not experienced or witnessed HH
violence in last 12 months
0.575 0.56
Has an adult s(he) trusts 0.464 0.337

33.  45% less likely to disagree
that a ‘woman should obey
her husband’
 20% less likely to agree that
‘boys and girls should share
HH tasks equally’
 No change in knowing where
to keep money
 No change in knowing that
menarche allows for
pregnancy
Positive effects: Negative effects:
No effects:
 29% more likely to
know about the risk
of FGM/C
 82% more likely to
know of a place to go
for help from
violence

34. Mentors
Some positive gains – especially in terms of knowledge and changes in attitudes and practice re
gender division of labour in the household
In Afar, challenges in recruiting mentors from same locale and clan slowed acceptance by
communities of the programme
Concerns expressed by mentors included limited support from supervisors, limited site visits
(due to both distance –tended to live in Harar and challenging to visit districts during unrest),
yet significant resistance and backlash from some communities esp. due to SRH content and
fears about religious conversion.
In Oromia, communication around girl-only asset sparked considerable discontent among boys,
parents, community leaders – in sites with the asset transfer as well as neighbouring communities,
and was exacerbated by over-promising by some mentors

35. Mentors
 Mentors themselves were very engaged,
emphasized that being part of the project was
personally empowering in terms of hard and soft
skills
 However, in Oromia, also high levels of
dissatisfaction reflected in higher turnover than
elsewhere, uneven attendance, some disbanding of
groups prematurely
The change is not only for the trainee girls but
also for myself… When I was in school and also
before I joined Act With Her, I was so shy; I was
not even sharing ideas with others. However,
after I joined this programme I began to
communicate with everyone without fear and
shyness.’ (female mentor, Amhara AWH-E)
Teacher in a girls’ room, Ebenat, Ethiopia © Nathalie Bertrams
GAGE 2020

36. Summary and reflections for
programme strengthening
Adolescent girls in East Hararghe, Ethiopia © Nathalie Bertrams / GAGE 2019

37. Summary
Largest short-run impacts are on:
 adolescent voice and agency,
 girls’ knowledge,
 with suggestive evidence on mental health, resilience, and menstruation
For more detail see these region-specific GAGE knowledge products
For
more
resources
see:
https://www.gage.odi.org/publications/

38. Emerging lessons to further strengthen outcomes for adolescents
More concrete and tailored planning with adolescents about alternative pathways – not just
professional careers but also TVET etc.
Specific curriculum content on GBV prevention and response – and service visits –
especially in terms of awareness of reporting violence/ child marriage to justice sector /
police for adolescents and potentially also parents
More region-tailored approaches to tackling child marriage and FGM/C – e.g. how do
you resist brokers in Oromia, how do you advocate against FGM/C in Amhara that
speaks to the reality of the practice
Less curriculum content but more repetition/ recap in low literacy communities

39. Emerging lessons to further strengthen outcomes for adolescents
Better support for mentors facing backlash and risks
Programming that engages with boys’ challenges around masculinities and norms
More early engagement with religious leaders at woreda levels to secure buy-in to
programme aims
Systems strengthening work that is out of school to test approaches that are school-based vs
out-of-school – esp as those in school are not the most disadvantaged.

40. GAGE
recent
reports
on
mid-line
f
indings
For
more
resources
see:
https://www.gage.odi.org/publications/

41. Recent
journal
publications
on
Ethiopia
https://www.gage.odi.org/publication/b
etwixt-and-between-adolescent-
transitions-and-social-policy-lacunae-in-
ethiopia/
https://www.gage.odi.org/publication/constrai
ned-choices-exploring-the-complexities-of-
adolescent-girls-voice-and-agency-in-child-
marriage-decisions-in-ethiopia/
https://www.gage.odi.org/publication/leave-
no-adolescent-behind-addressing-inequities-
in-educational-attainment-in-rural-ethiopia/
https://www.gage.odi.org/publication/gender-
norms-violence-and-adolescence-exploring-how-
gender-norms-are-associated-with-experiences-of-
childhood-violence-among-young-adolescents-in-
ethiopia/
https://www.gage.odi.org/publication/do-restrictive-
gender-attitudes-and-norms-influence-physical-and-
mental-health-during-very-young-adolescence-evidence-
from-bangladesh-and-ethiopia/
https://www.gage.odi.org/publicati
on/intersecting-inequalities-
gender-and-adolescent-health-in-
ethiopia/
https://www.gage.odi.org/publication/explor
ing-the-role-of-evolving-gender-norms-in-
shaping-adolescents-experiences-of-
violence-in-pastoralist-afar-ethiopia/

42. Contact Us
WEBSITE
www.gage.odi.org
TWITTER
@GAGE_programme
FACEBOOK
GenderandAdolescence
About GAGE:
 Gender and Adolescence: Global Evidence
(GAGE) is a nine-year (2015-2024) mixed-
methods longitudinal research programme
focused on what works to support
adolescent girls’ and boys’ capabilities in the
second decade of life and beyond.
 We are following the lives of 20,000
adolescents in six focal countries in Africa,
Asia and the Middle East.