Seal of Good Local Governance (SGLG) 2024Final.pptx
2 Can Mobile Phones Improve Learning? Jenny Aker
1. Jenny
C.
Aker,
Tu.s
University
and
IFPRI
Lessons
from
a
Decade’s
Research
on
Poverty:
InnovaCon,
Engagement
and
Impact
Pretoria,
South
Africa
March
16-‐18,
2016
Jenny
C.
Aker
Can
Mobile
Phones
Improve
Learning?
Evidence
from
Three
Evalua<ons
of
Adult
Educa<on
Programs
Mobile
Phones
and
Adult
EducaCon
2. Source:
UN
Human
Development
Report,2014
Educa<on
in
sub-‐Saharan
Africa
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
0
10
20
30
40
50
60
70
80
90
100
Botswana
Nigeria
Ghana
Tanzania
DRC
Togo
Central
African
Republic
Ivory
Coast
Niger
Mali
3. • How
can
educaConal
outcomes
be
improved?
• SubstanCal
research
on
educaCon
intervenCons
provides
evidence
on
what
is
effecCve
in
increasing
school
parCcipaCon
o CondiConal
and
uncondiConal
cash
transfers,
scholarships,
deworming,
informaCon
• Yet
evidence
is
mixed
on
how
to
improve
learning
in
a
cost-‐effecCve
way
o Number
of
evaluaCons
have
shown
that
spending
on
certain
inputs
has
not
increased
test
scores
Mo<va<on
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
4. Mobile
Phone
Adop<on
in
Africa
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
16
25
37
52
82
135
200
279
376
455
515
620
754
802
895
-‐
100
200
300
400
500
600
700
800
900
1,000
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
Number
of
Subscribers
(Millions)
Total
Number
of
Mobile
Phone
Subscribers
in
Sub-‐
Saharan
Africa
Source:
Wireless
Intelligence,
Ericsson
2015,
GSMA
2013
5. Research
Goals
• Can
simple
informaCon
technology
(ie,
mobile
phones)
be
used
as
an
educa&onal
input
for
low-‐
literate
adults?
o Does
it
improve
learning
and
other
welfare
outcomes?
o Is
it
cost
effecCve?
Research
Ques<ons
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
6. Source:
Transparency
InternaConal
2013
Teacher
Absenteeism
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
27%
absenteeism
in
some
African
countries
(TI
2013)
7. • In
response
to
high
teacher
absenteeism,
many
governments
have
shi.ed
to
community
teachers
o Such
teachers
have
short-‐term,
flexible
contracts
and
may
be
more
easily
monitored
by
the
community
• Monitoring
teachers
remains
a
significant
challenge,
especially
in
countries
with
high
transport
costs
and
weak
insCtuCons
Teacher
Absenteeism
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
8. Research
Goals
• Can
simple
informaCon
technology
(ie,
mobile
phones)
be
used
as
an
educa&onal
input
for
low-‐literate
adults?
o Does
it
improve
learning
and
other
welfare
outcomes?
o Is
it
cost
effecCve?
• Can
simple
mobile
phones
be
used
as
a
means
to
monitor
teachers’
absenteeism
and
improve
accountability?
o Does
this
improve
learning?
o How
does
it
compare
with
in-‐person
monitoring?
• Could
mobile
phones
serve
as
a
subs&tute
for
teachers
for
certain
types
of
learning?
• Three
randomized
control
trials
of
adult
educaCon
programs
in
Niger
and
the
US
Research
Ques<ons
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
9. 9
Context
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
10. • NGOs
(Catholic
Relief
Services)
and
the
Ministry
provide
ten
months
of
adult
educaCon
classes
over
a
two-‐year
period
• Held
February
through
June
of
each
year
• Taught
reading,
wriCng
and
math
skills
in
indigenous
language
(Hausa
and
Zarma)
• 5
courses
per
week,
three
hours
per
day
• Taught
by
community
literacy
teachers
• Fi.y
literacy
parCcipants
per
village
(25
men,
25
women)
–
separate
classes
by
gender
• Typically
characterized
low
enrolment,
high
drop-‐out
and
rapid
skills
depreciaCon
Jenny
C.
Aker
Adult
Educa<on
Interven<ons:
Niger
Mobile
Phones
and
Adult
EducaCon
11. Bakin
Birgi
(Monday)
Zinder
(Thursday)
Tanout
(Friday)
65
km~2
min
Niamey
(Sunday)
20
km~1
hour
750
km~2
min
Mobile
Phones
as
an
Educa<onal
Tool?
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
12. Alphabe(sa(on
de
Base
par
Cellulaire
(ABC)
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
13. • Students:
o Learned
where
to
find
numbers
and
lejers
on
mobile
phone
handset
o Learned
how
to
make
and
receive
a
call
o Learned
how
to
send
and
read
SMS
o Were
provided
with
access
to
shared
mobile
phones
(5
people
per
mobile
phone)
worth
about
$USD
10
• ABC
module
started
six
weeks
before
the
end
of
classes
• No
extra
class
Cme
for
students
Jenny
C.
Aker
The
ABC
Module
Mobile
Phones
and
Adult
EducaCon
14. • 113
villages
were
randomly
assigned
to
one
of
two
intervenCons:
• T1.
Regular
adult
educaCon
(non-‐ABC)
• T2.
Regular
adult
educaCon
+
ABC
module
(ABC)
• No
pure
control
group,
so
compares
the
relaCve
impact
of
the
ABC
module
on
learning
Research
Design
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
15.
• Learning
outcomes
(wriCng
and
math
test
scores)
• Household
producCon,
markeCng,
migraCon
• Mechanisms
• Cost-‐effecCveness
Impacts
Jenny
C.
Aker
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
16. Raw Writing Test Scores
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Baseline Five months Eleven months
ABC
Non-ABC
Wri<ng
Scores
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
17. 0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
Baseline Five months Eleven months
ABC
Non-ABC
Raw Math Test Scores
Math
Scores
Similar
effects
for
men
and
women
and
younger
and
older
learners
Results
persisted
two
years
a.er
the
program
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
18. • Do
mobile
phones
allow
students
to
pracCce
more
outside
of
class?
Yes
• Do
mobile
phones
affect
students’
effort
within
class?
It
depends
on
teacher
quality.
• Are
teachers
more
likely
to
ajend
classes
because
they
are
moCvated
by
the
presence
of
technology?
No.
18
Jenny
C.
Aker
Why
did
learning
improve?
Mobile
Phones
and
Adult
EducaCon
19. • The
program
cost
$USD
7
more
per
student
as
compared
to
the
tradiConal
adult
educaCon
program,
but
students
had
higher
learning
outcomes
and
earned
more
• But
community
teachers
taught
only
2/3
of
contracted
courses
o Could
mobile
phones
be
used
to
improve
teacher
accountability?
Jenny
C.
Aker
Key
Takeaways
Mobile
Phones
and
Adult
EducaCon
20. • Agents
made
four
weekly
phone
calls
to
the
teacher,
the
village
chief
and
two
randomly-‐selected
students
over
six
weeks
• Short
script
with
five
quesCons:
o Was
there
a
literacy
course
this
week?
o How
many
days
per
week?
o How
many
hours
per
day?
o How
many
students
ajended?
o Is
there
anything
else
you
would
like
to
share?
• General
feedback
provided
to
CRS
on
a
weekly
basis
o No
formal
sancCons
for
less
than
contracted
effort
and
no
follow-‐up
visits
in
the
short-‐term
Mobile
Monitoring
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
21. • 160
villages
in
two
regions
were
randomly
assigned
to
one
of
three
intervenCons:
• T1.
Adult
Educa<on:
TradiConal
adult
educaCon
program
• T2.
Mobile
Monitoring:
Adult
educaCon
plus
mobile
monitoring
• C:
Comparison.
No
adult
educaCon.
21
Research
Design
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
22. • The
NGO
is
unable
to
obtain
informaCon
about
the
teachers’
effort
o High
costs
of
supervision
• If
teachers
believe
they
may
be
fired
or
penalized,
addiConal
monitoring
should
increase
teachers’
effort
and
student
learning
• Yet
monitoring
could
fail
if
wages
are
not
high
enough
or
it
crowds
out
intrinsic
moCvaCon
22
How
would
this
affect
learning?
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
23. • Learning
outcomes
(EGRA/EGMA
Reading
and
Math
Tests)
• Self-‐Efficacy
and
Self-‐Esteem
• Household
welfare
• Mechanisms
23
Impacts
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
24. Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
Reading
Z-‐Scores
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Lejers
Syllables
Words
Phrases
Reading
Z-‐Score
Non-‐Monitoring
Monitoring
Similar
results
for
math
Different
effects
by
teacher’s
“alternaCve
opCon”
25. “These
calls
prevent
us
from
missing
courses.”
“Someone
who
works
must
be
‘controlled’”.
“I
felt
closer
to
the
hierarchy
and
wanted
to
do
well.”
“This
proves
that
our
work
is
important.”
“It
gave
me
courage.”
Why
did
this
improve
learning?
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
26. • Teachers
o Do
monitoring
teachers
teach
more?
A
bit.
o Are
monitoring
teachers
more
moCvated?
Yes.
o Were
they
more
likely
to
be
fired?
No.
• Student
o Are
students
less
likely
to
drop
out?
No.
o Are
students
more
moCvated?
Yes.
Why
did
this
improve
learning?
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
27. Cost
Effec<veness
$0.00
$1.00
$2.00
$3.00
$4.00
$5.00
$6.00
$7.00
Mobile
Monitoring
In-‐Person
Monitoring
Staff
Cme
Call
credit
Gas
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
28. • Mobile
monitoring
increased
student
learning
by
20-‐50%
o Will
this
be
sustained
if
scaled?
• Less
costly
than
in-‐person
monitoring
o In-‐person
monitoring
was
very
light
in
this
context
• Why?
Changes
in
teacher
and
student
moCvaCon
• Can
mobile
phones
be
used
in
other
ways
to
support
teachers?
(Ie,
pedagogy)
• Can
they
be
used
as
subsCtutes
for
teachers
in
areas
where
condiCons
are
difficult
(ie,
conflicts)?
28
Key
Takeaways
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
29. • Mobile
phone-‐based
adult
educaCon
curriculum
using
a
simple
mobile
phone
o Series
of
400
micro-‐modules
via
audio,
calls
and
SMS
• Piloted
in
Spanish
in
Los
Angeles
with
low-‐literate
learners
• Students
increased
their
reading
levels
by
two
years
over
a
four-‐month
period
• Caveats:
Small
sample
size,
high
drop-‐out
rates
29
The
CellEd
Program
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
30. • Can
mobile
phones
be
used
as
an
educaConal
tool?
• Evidence
is
mixed
for
ICT
and
children’s
educaConal
outcomes
in
general,
but
research
in
Niger
suggests
it
can
lead
to
sustained
learning
outcomes
for
adults
• Depends
upon
the
type
of
ICT
and
how
the
ICT
is
incorporated
into
the
learning
curriculum
• ICT
can
be
either
a
complement
or
subsCtute
for
teacher
quality
• Less
evidence
of
the
impact
of
adult
educaCon
programs
on
household
welfare
and
children’s
educaConal
outcomes
o Currently
measuring
these
impacts
in
Niger
30
The
BoVom
Line
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
31. 31
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
Thank
you
33. Source: Evans and Ghosh (2008) 0
5
10
15
20
25
30
35
40
45
50
Blackboards
-‐
Ghana
Furniture
-‐
Philippines
Teacher
training
-‐
Honduras
Workbooks
-‐
Philippines
Remedial
EducaCon
(tutors)
-‐
India
CapitaCon
grant
-‐
Uganda
Mobile
phone
module
wriCng
-‐
Niger
Mobile
phone
module
math
-‐
Niger
Classroom
repair
-‐
Ghana
AddiConal
teachers
with
tracking
-‐
Kenya
Girls'
scholarship
-‐
Kenya
Teacher
incenCves
-‐
India
Teacher
incenCves
-‐
Kenya
Textbooks
-‐
Kenya
Remedial
EducaCon
-‐
India
Preschool
-‐
Philippines
Class
size
reducCon
-‐
Honduras
EducaConal
vouchers-‐Colombia
Cost ($USD) per .1 s.d. increase in test scores
How
does
this
compare
to
other
interven<ons?
Cost
per
.1
s.d.
increase
34. • 70%
of
teachers
thought
the
calls
were
from
CRS,
29%
from
the
Ministry
• 80%
of
teachers
knew
the
village
chief
was
called,
77%
knew
the
students
were
called
• High
intra-‐village
correlaCon
of
responses,
even
if
courses
were
not
held
o Within-‐village
correlaCons
do
not
appear
to
change
over
Cme
o Collusion
or
high
degree
of
informaCon-‐sharing?
Mobile
Monitoring
Experiment
Aker
and
Ksoll
Mobile
Monitoring
Works
35. Approche
de
Recherche
35
DFAP
Villages
500
Outside
evaluation
Part of
evaluation
(160 villages)
Random
Assignment
ABC
Normal
Monitoring
(70)
Pure control
(20)
Non-ABC
Mobile
Monitoring
(70)
ABC
Non-ABC
Experimental
Design
Aker
and
Ksoll
Mobile
Monitoring
Works
2014
and
2015
2016
Geographic
straCficaCon
36. Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon
Math
Z-‐Scores
0
0.1
0.2
0.3
0.4
0.5
0.6
#
IdenCficaCon
QuanCty
Comparison
Add/Subtract
MulCply/Divide
Math
Z-‐Score
Non-‐Monitoring
Monitoring
37. • Geography:
o The
effects
are
primarily
driven
by
one
region
-‐
where
the
costs
of
in-‐person
monitoring
are
higher
• Gender:
o Stronger
monitoring
effects
on
men’s
reading
scores
• Teacher
characterisCcs:
o Stronger
monitoring
effects
for
literacy
teachers
with
a
lower
outside
opCon
(gender,
educaCon,
experience)
o Stronger
effects
when
teachers
know
the
chief
was
called
Heterogeneous
effects
Jenny
C.
Aker
Mobile
Phones
and
Adult
EducaCon