Promoting Equitable
Learning: Changing Teachers
and Systems
Caine Rolleston
UCL
7 November 2018
WHAT IS YOUNG LIVES?
• Established in 2001 with a 15-
year horizon (MDGs)
• 4 Country international
longitudinal & comparative
study (survey)
• Following 12,000 children in
two age cohorts in Ethiopia,
India (Andhra Pradesh and
Telangana), Peru, Vietnam
• Including large scale primary
and secondary school surveys
WHAT IS AN EQUITABLE SYSTEM IN TERMS
OF LEARNING OUTCOMES?
• Students achieve good learning outcomes whatever their
backgrounds
• Teachers play a key role, but work within systems
PROBLEM: NOT ALL EDUCATION SYSTEMS
PROMOTE EQUITABLE LEARNING
 Some systems serve to
“pick winners” in “bad”
exams/curricula
 In some countries
education outcomes are
more unequal than
incomes (e.g. Peru)
 In some systems,
disadvantaged pupils
systematically go to
poorer quality schools
 Teachers CAN mitigate
inequality but systems
are required to
CHANGE it
AT AGE 15 GAPS IN MATHS PERFORMANCE BETWEEN
COUNTRIES ARE VERY LARGE
0
2
4
6
8
10
12
14
16
18
20
India Vietnam
Site-level average maths score at age 14-15
WITHIN COUNTRIES: TRIPLE DISADVANTAGE
In unequal settings,
disadvantaged pupils suffer
3 fold:
1. Home disadvantage
2. Lower school/teacher
quality due to ‘sorting’
3. Within school
disadvantage (including due
to forms of discrimination)
Better outcomes overall in
contexts with less
inequality (Vietnam)
*defined as neither parent has been to school and neither parent can read or write
0
.2
.4
.6
.8
1
400 500 600 700 800
School Average Maths Test Score
Schools Predicted Relationship
Proportionoffirstgenerationlearners
Young Lives school survey Ethiopia 2016-17: upper primary schools
Schools with higher
proportions of FGLs have
lower average maths
scores
HOME DISADVANTAGE CHALLENGE FOR TEACHERS
SORTING IS A MAJOR CHALLENGE
E.G. BY ‘ABILITY TO PAY’ (INDIA)
0
1020304050
High fee Moderate fee Low fee
Mean School VA (unconditional)
Mean School VA (conditional)
-0.18
0.41*
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
Vietnam Peru
Proportion of 1 SD of maths test score distribution
• Compare effect of increasing school
quality on the most and least wealthy
groups in the same schools
• In Vietnam, schools equally effective in
teaching Maths regardless of background
• In Peru, schools significantly less effective
at teaching children from disadvantaged
backgrounds
• Linked to forms of discrimination or
systemic features e.g. language
/curriculum?
• Up to 0.25 SD gain from ‘teacher
matching’ in India – perhaps linked to
forms of discrimination
• Rawal & Kingdon (2010)
WITHIN SCHOOL EFFECTIVENESS CHALLENGE
WHAT IS DIFFERENT IN VIETNAM?
TEACHING STRATEGIES HELP TO NARROW GAPS
(PRIMARY GRADE 5)
Vietnamese Mathematics
• Teachers have what they need: Emphasis on
‘fundamental’ or minimum school quality levels
including teacher quality (especially in disadvantaged
areas)
• Teachers know what to teach: Common centralised
curricula & text books in use matched closely to pupils’
learning levels; focused curriculum (e.g. 6 basic subjects
only at primary level)
• Teachers have the knowledge required: Teacher
knowledge (YL curriculum tests) is similar between more
and less disadvantaged areas
• Students are expected to progress: Commitment to
‘mastery’ by all pupils - use of regular assessment by
teachers emphasis on effort/work not ability
WHAT IS DIFFERENT IN VIETNAM?
Indicator Vietnam India
Mean class size 27.61 16.23
Mean years of teacher experience 17.47 7.71
Mean monthly teacher salary
(USD/Month)
164 226
% of teachers with no formal
teacher training qualification
0% 16.50%
Teacher absenteeism 2.34 days per year 35.12% pupils said ‘
my class teacher often does
not come to school’
All children have access to maths
textbooks
96.16% 60.84%
Teacher always checks/marks
maths homework
41.28% 18.06%
VIETNAM AND INDIA: SCHOOL-SYSTEM QUALITY INDICATORS
(FROM SCHOOL SURVEYS)
400
450
500
550
600
650
1 2 3 4 5 6 7 8 9 10
YoungLivesTestScore
Teacher Reported Test Score
TEACHERS IN VIETNAM KNOW WHAT PUPILS KNOW
(AND NEED TO KNOW)
CLASSROOM OBSERVATION STUDY IN INDIA
THE CLASS OBSERVATION TOOL
• We revisited schools with known ‘value-added’ from earlier
school survey in India (purposive selection) to observe teachers
using CLASS protocol
• CLASS is based on 3 domains of teacher-student interactions:
Emotional Support; Classroom Organisation; and Instructional
Support, each comprising several dimensions (total 11)
• Teachers were observed for 2 lessons by trained fieldworkers,
and scored for each dimension (out of 7). Their score for each
dimension is averaged across these lessons.
• Standardised scores for each dimension have then been added to
create a total CLASS score across the 11 dimensions
A COMPLEX PICTURE EMERGES 1
• Overall scores relating to classroom organisation and management
were high in India (compared to USA for e.g.)
• Students with higher test scores found in classes taught by
teachers who perform better on CLASS (sorting)
• Or…teachers more likely to use pedagogies which are scored
highly by CLASS (i.e. more interactive) in classes with higher
performing children?
• Private school teachers more effective in raising learning
outcomes
• But mixed on whether private school teachers are using high
scoring (interactive) pedagogies
A COMPLEX PICTURE EMERGES 2
• The relationship between CLASS score and learning
progress varies by school management type:
– positive and significant for government school
teachers in both maths and English
– While it is negative and significant for private
unaided school teachers in both subjects
• Perhaps private school teachers’ focus on exam scores
requires strategies less highly rated on CLASS?
• Government teachers have more autonomy, more
training but often teach more disadvantaged students,
much more variation in effectiveness…and pedagogy
CONCLUSIONS
• Equity axiomatic for all policies, incl. teacher policies BUT many routes to same
destination (Finland vs Shanghai) … coherence & relevance of choices is key
1. Possible benefits from better balance of autonomy and accountability (India) PLUS
improvement on CLASS domains via teacher education – e.g. emotional and
instructional support (India)
2. Reduce ‘discrimination’ by improving teacher ‘matching’ OR 3?
3. Improve teacher effectiveness for disadvantaged pupils e.g. targeting minimum
learning outcomes PLUS strategies to recognise heterogeneity
4. Reduce the effects of ‘sorting’ by setting minimum standards relating to teacher
quality
• Success of teacher education reforms likely depends on other equity-oriented
systemic reforms, e.g.
1. Reduce ‘discrimination’ resulting from 'over-ambitious' curricula and high stakes
exams which encourage teachers to 'teach to the top’
2. A “mass basic skills” approach automatically prioritises areas with weaker provision
(as in Vietnam) including regarding teacher quality (this is often not the focus)
END
@yloxford

Promoting Equitable Learning: Changing Teachers and Systems

  • 1.
    Promoting Equitable Learning: ChangingTeachers and Systems Caine Rolleston UCL 7 November 2018
  • 2.
    WHAT IS YOUNGLIVES? • Established in 2001 with a 15- year horizon (MDGs) • 4 Country international longitudinal & comparative study (survey) • Following 12,000 children in two age cohorts in Ethiopia, India (Andhra Pradesh and Telangana), Peru, Vietnam • Including large scale primary and secondary school surveys
  • 3.
    WHAT IS ANEQUITABLE SYSTEM IN TERMS OF LEARNING OUTCOMES? • Students achieve good learning outcomes whatever their backgrounds • Teachers play a key role, but work within systems
  • 4.
    PROBLEM: NOT ALLEDUCATION SYSTEMS PROMOTE EQUITABLE LEARNING  Some systems serve to “pick winners” in “bad” exams/curricula  In some countries education outcomes are more unequal than incomes (e.g. Peru)  In some systems, disadvantaged pupils systematically go to poorer quality schools  Teachers CAN mitigate inequality but systems are required to CHANGE it
  • 5.
    AT AGE 15GAPS IN MATHS PERFORMANCE BETWEEN COUNTRIES ARE VERY LARGE 0 2 4 6 8 10 12 14 16 18 20 India Vietnam Site-level average maths score at age 14-15
  • 6.
    WITHIN COUNTRIES: TRIPLEDISADVANTAGE In unequal settings, disadvantaged pupils suffer 3 fold: 1. Home disadvantage 2. Lower school/teacher quality due to ‘sorting’ 3. Within school disadvantage (including due to forms of discrimination) Better outcomes overall in contexts with less inequality (Vietnam)
  • 7.
    *defined as neitherparent has been to school and neither parent can read or write 0 .2 .4 .6 .8 1 400 500 600 700 800 School Average Maths Test Score Schools Predicted Relationship Proportionoffirstgenerationlearners Young Lives school survey Ethiopia 2016-17: upper primary schools Schools with higher proportions of FGLs have lower average maths scores HOME DISADVANTAGE CHALLENGE FOR TEACHERS
  • 8.
    SORTING IS AMAJOR CHALLENGE E.G. BY ‘ABILITY TO PAY’ (INDIA) 0 1020304050 High fee Moderate fee Low fee Mean School VA (unconditional) Mean School VA (conditional)
  • 9.
    -0.18 0.41* -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 Vietnam Peru Proportion of1 SD of maths test score distribution • Compare effect of increasing school quality on the most and least wealthy groups in the same schools • In Vietnam, schools equally effective in teaching Maths regardless of background • In Peru, schools significantly less effective at teaching children from disadvantaged backgrounds • Linked to forms of discrimination or systemic features e.g. language /curriculum? • Up to 0.25 SD gain from ‘teacher matching’ in India – perhaps linked to forms of discrimination • Rawal & Kingdon (2010) WITHIN SCHOOL EFFECTIVENESS CHALLENGE
  • 10.
    WHAT IS DIFFERENTIN VIETNAM?
  • 11.
    TEACHING STRATEGIES HELPTO NARROW GAPS (PRIMARY GRADE 5) Vietnamese Mathematics
  • 12.
    • Teachers havewhat they need: Emphasis on ‘fundamental’ or minimum school quality levels including teacher quality (especially in disadvantaged areas) • Teachers know what to teach: Common centralised curricula & text books in use matched closely to pupils’ learning levels; focused curriculum (e.g. 6 basic subjects only at primary level) • Teachers have the knowledge required: Teacher knowledge (YL curriculum tests) is similar between more and less disadvantaged areas • Students are expected to progress: Commitment to ‘mastery’ by all pupils - use of regular assessment by teachers emphasis on effort/work not ability WHAT IS DIFFERENT IN VIETNAM?
  • 13.
    Indicator Vietnam India Meanclass size 27.61 16.23 Mean years of teacher experience 17.47 7.71 Mean monthly teacher salary (USD/Month) 164 226 % of teachers with no formal teacher training qualification 0% 16.50% Teacher absenteeism 2.34 days per year 35.12% pupils said ‘ my class teacher often does not come to school’ All children have access to maths textbooks 96.16% 60.84% Teacher always checks/marks maths homework 41.28% 18.06% VIETNAM AND INDIA: SCHOOL-SYSTEM QUALITY INDICATORS (FROM SCHOOL SURVEYS)
  • 14.
    400 450 500 550 600 650 1 2 34 5 6 7 8 9 10 YoungLivesTestScore Teacher Reported Test Score TEACHERS IN VIETNAM KNOW WHAT PUPILS KNOW (AND NEED TO KNOW)
  • 15.
  • 16.
    THE CLASS OBSERVATIONTOOL • We revisited schools with known ‘value-added’ from earlier school survey in India (purposive selection) to observe teachers using CLASS protocol • CLASS is based on 3 domains of teacher-student interactions: Emotional Support; Classroom Organisation; and Instructional Support, each comprising several dimensions (total 11) • Teachers were observed for 2 lessons by trained fieldworkers, and scored for each dimension (out of 7). Their score for each dimension is averaged across these lessons. • Standardised scores for each dimension have then been added to create a total CLASS score across the 11 dimensions
  • 17.
    A COMPLEX PICTUREEMERGES 1 • Overall scores relating to classroom organisation and management were high in India (compared to USA for e.g.) • Students with higher test scores found in classes taught by teachers who perform better on CLASS (sorting) • Or…teachers more likely to use pedagogies which are scored highly by CLASS (i.e. more interactive) in classes with higher performing children? • Private school teachers more effective in raising learning outcomes • But mixed on whether private school teachers are using high scoring (interactive) pedagogies
  • 19.
    A COMPLEX PICTUREEMERGES 2 • The relationship between CLASS score and learning progress varies by school management type: – positive and significant for government school teachers in both maths and English – While it is negative and significant for private unaided school teachers in both subjects • Perhaps private school teachers’ focus on exam scores requires strategies less highly rated on CLASS? • Government teachers have more autonomy, more training but often teach more disadvantaged students, much more variation in effectiveness…and pedagogy
  • 20.
    CONCLUSIONS • Equity axiomaticfor all policies, incl. teacher policies BUT many routes to same destination (Finland vs Shanghai) … coherence & relevance of choices is key 1. Possible benefits from better balance of autonomy and accountability (India) PLUS improvement on CLASS domains via teacher education – e.g. emotional and instructional support (India) 2. Reduce ‘discrimination’ by improving teacher ‘matching’ OR 3? 3. Improve teacher effectiveness for disadvantaged pupils e.g. targeting minimum learning outcomes PLUS strategies to recognise heterogeneity 4. Reduce the effects of ‘sorting’ by setting minimum standards relating to teacher quality • Success of teacher education reforms likely depends on other equity-oriented systemic reforms, e.g. 1. Reduce ‘discrimination’ resulting from 'over-ambitious' curricula and high stakes exams which encourage teachers to 'teach to the top’ 2. A “mass basic skills” approach automatically prioritises areas with weaker provision (as in Vietnam) including regarding teacher quality (this is often not the focus)
  • 21.