Effective schooling in Ethiopia: Who benefits? Caine Rolleston and Zoe James

1. Effective Schooling in
Ethiopia: Who
Benefits?
Caine Rolleston and Zoe James
UCL Institute of Education

2. • Analysis of YL data as part of World Bank
Public Expenditure Review for Education
– YL data alongside EMIS, SDI etc.
– Focus on efficiency and effectiveness of
spending in Ethiopia’s education sector
– Efficiency includes attendance, progression
– Effectiveness focuses on outcomes
INTRODUCTION

3. YOUNG LIVES SITES ETHIOPIA

4. • 2012-2013 school year
• Site-level school census
• All pupils in all G4 & G5 classes
• Start and end of year survey
(W1 & W2)
• Child, class, teacher, principal and
school data
• Assessments in maths and reading
comprehension
• Allows identification of learning progress over school year
ETHIOPIA SCHOOL SURVEY 2012-13

5. SCHOOL SURVEY SAMPLE
• 7 regions
• 30 sites
• 94 schools
• 19 non-government
• 280 classes
• 142 grade 4
• 134 grade 5
• 11,982 pupils at Wave 1
• 10,068 pupils at Wave 2

6. CONTEXT: STRONG PROGRESS IN TERMS OF ON-TIME
ENROLMENT & PROGRESSION
Enrolled in 1st cycle primary by
age 7
Older Cohort
(born 1994/95)
Younger Cohort
(born 2000/01)
Total 27.7 50.0
Location Urban (R2) 43.2 71.2
Rural (R2) 18.2 39.5
Poverty Least poor quintile
(R2)
47.2 74.7
Poorest quintile
(R2)
15.8 32.4

7. BUT…. ISSUES PERSIST IN TERMS OF GRADE
REPETITION, DROP-OUT AND PUPIL ABSENCE
Has dropped
out, %
Absence
Total 17.3 4.6
Pastoralist
livelihood
Pastoralist 24.0 6.9
Non-pastoralist 17.2 4.4
Location Urban 16.1 3.7
Rural 20.4 7.1
Poverty Least poor quintile 13.0 3.8
Poorest quintile 19.2 6.9
Pastoralist children, rural children, and poor children are
particularly disadvantaged, raising serious equity concerns

8. ASSESSMENT DESIGN AND COMPETENCY LEVELS
• Development of test items in relation to MLCs & textbooks (maths
and reading)
• Inclusion of items from grades 1-3 plus 4 & 5 (all MC)
• Competency levels arrived at using empirical difficulty of items in
test data, and grade-level expectations – linking norm-referenced
(IRT) scores to criterion referenced scale
• Items allocated to competency levels 1-4 with level 0 denoting a
score below level 1. Level 4 is ‘grade-expectation’ level
• Pupils who score correctly on 2/3 of items in a particular
competency level are considered to be at that level providing they
also reach required competency of levels below this

9. DISTRIBUTION OF W2 MATHS SCORES IN SELECTED SITES
0
.001.002.003
Density
300 400 500 600 700 800
Score
Addis Amhara Urban (Typical)
SNNP Rural Somali Rural
Somali Jijiga Afar Rural
Maths Scores R2: Selected Sites

10. 0
.002.004.006
Density
300 400 500 600 700 800
x
Below Level 1 Level 1
Level 2 Level 3
Level 4
Maths Scores (Wave 2) by Competency Level
DISTRIBUTION OF NORM-REFERENCED (IRT) SCORES BY
COMPETENCY LEVEL IN MATHS

11. Competency
Level
Grade
Level
Level Description Reading Comprehension Competencies
0
Below Early
Foundational
N/A Below early
foundational reading
comprehension
Pupils have not reached level 1
1
Early
Foundational
1 Early foundational
reading
comprehension
Pupils can read and understand simple familiar words and
very short simple sentences illustrated by pictures.
2
Foundational
2 Foundational reading
comprehension
Pupils can read and understand short sentences without
illustration and to complete the meaning of sentences with
simple missing words.
Pupils can read a short passage with basic understanding of
simple events.
3
Emerging Basic
2-4 Emerging basic
reading
comprehension skills
Pupils can read longer sentences and passages containing
some less familiar words, with an understanding of simple
events and characters. Pupils have a basic and emerging
ability to interpret events and characters.
4
Basic
4-5 Basic reading
comprehension
Pupils can read passages with understanding of the main
points communicated and a basic understanding of
intention and purpose.
Pupils have an emerging ability to use inference and
deduction to interpret texts in terms of message.

12. Competency
Level
Example Mathematics items
0 - Below Early
Foundational
1- Early
Foundational
- How many dots are there?
- Which is a triangle?
- 9 X 2 =
- 2 + 3 =
2
Foundational
- Put numbers in ascending order: 19, 6, 2, 11
- How many minutes in 1 hour?
- Tamiru has 5 Birr. His mother takes 4 Birr. How many Birr does Tamiru
have?
- Which is half of 6?
3
Emerging Basic
Numeracy
- 85 x 5 =
- Which difference is closest to 300,000?
- 30m = ____ cm
- What is the value of the number 2 in 928?
4
Basic Numeracy
- 2.34 +7.65 =
- What is the average of 10, 12, 18, 24?
- Calculate the perimeter of the rectangle
- Fill in the appropriate number in the sequence: 1, 3, ___, 27

13. COMPETENCY LEVELS ARE LOW - MATHS
Competency level Number of Pupils Percentage of
Pupils
Cumulative
Percentage
0 (Below Level 1) 853 8.66 8.66
1 (Early Foundational) (~ G1) 2,121 21.54 30.21
2 (Foundational) (~G2-3) 5,152 52.33 82.53
3 (Emerging) (~G3-4) 1,473 14.96 97.49
4 (Grade level) (~G4-5) 247 2.51 100.00
Total 9,846 100.00
0
20406080
100
percent
Rural Urban
5 4 5 4
Level 0 Level 1
Level 2 Level 3
Level 4

14. COMPETENCY LEVELS ARE ALSO LOW IN READING
0
20406080
100
percent
Rural Urban
5 4 5 4
Level 0 Level 1
Level 2 Level 3
Level 4
Competency level Number of Pupils Percentage of Pupils Cumulative Percentage
0 (Below Level 1) 466 6.05 6.05
1 (Early Foundational) 1,437 18.67 24.72
2 (Foundational) 840 10.91 35.64
3 (Emerging) 4,902 63.69 99.32
4 (Grade level) 52 0.68 100.00
Total 7,697 100.00

15. DO PUPILS IN GRADE 4 AND GRADE 5 MAKE
PROGRESS OVER THE SCHOOL YEAR?
Mathematics Reading Comprehension
October
2012
May
2013
Gain October
2012
May
2013
Gain
Mean 500.0 530.0 30.0 500.0 530.6 30.6
Gender Boy 502.1 532.5 30.4 498.0 527.5 29.5
Girl 497.9 527.6 29.7 501.9 533.6 31.7
Difference 4.2 4.9 0.7 -3.9 -6.1 -2.2
Location Urban 517.6 548.5 30.9 521.1 551.1 30.0
Rural 456.4 481.2 24.8 447.3 476.6 29.3
Difference 61.2 67.3 6.1 73.8 74.5 0.7

16. WHILE SCHOOL AND CLASS FACTORS CONTRIBUTE TO LEARNING, PUPILS
BACKGROUNDS AND HOUSEHOLD CHARACTERISTICS REMAIN VERY
IMPORTANT
VARIABLES Maths T1 Maths T2 Reading T1 Reading T2
Girl -10.2031 *** -4.4330 *** 0.1650 4.8107 ***
Has 3+ meals per day 18.2975 *** 6.6820 *** 12.6032 *** 2.1339
PCA pupil durable assets 4.5648 *** 1.7210 *** 4.9427 *** 0.8929 *
% days absence W1-W2 -3.3064 *** -1.8387 *** -3.1255 *** -1.4661 ***
Orphan (single or double) 2.8872 -3.2279 * 2.6308 -3.3137 **
No-one in household literate -9.1339 ** -5.7685 * -14.3049 *** -1.8481
Attended pre-school 2.6854 -0.3207 6.3172 *** 3.3394 **
Ever repeated a grade -39.6920 *** -5.0135 *** -38.7614 *** -4.2894 ***
Ever dropped-out -6.1516 ** -2.1293 -13.4413 *** -3.7113 **
Reads books at home 17.5734 *** 6.2704 *** 17.3460 *** 3.6314 *
Child learns in home language 3.8083 -2.2792 14.5838 *** 5.4488 *
Pastoralist -24.1437 *** -4.3839 -37.9640 *** -8.0455 **
Pupil spends time on paid work -10.4830 *** 0.2447 -12.1998 *** -1.0128
Pupils backgrounds and household experience continue to
influence their learning progress (T2), in addition to their
achievement at the start of the school year, and overcoming these
factors is a significant challenge to effective schooling.

17. ARE ALL SCHOOLS EQUAL IN ADDING VALUE IN
TERMS OF PUPIL LEARNING IN MATHS?
• In maths urban schools do not add more value than
rural schools
-100
-50
0
50
100
0 20 40 60 80 100
School Rank
Urban Rural

18. IN READING –VARIATION BETWEEN RURAL AND
URBAN SCHOOLS
• In reading, urban schools do have an advantage, likely
linked to languages of instruction
-100
-50
0
50
100
0 20 40 60 80 100
School Rank
Urban Rural

19. SCHOOL, CLASS AND TEACHER FACTORS ASSOCIATED
WITH HIGH AND LOW ‘VALUE-ADDED’ CLASSES
Characteristic Maths Reading
High VA Low VA Sig High VA Low VA Sig
School teaches only shift classes 0.58 0.73 * 0.49 0.87 ***
Class Assets -0.01 -0.40 * -0.06 -0.30
Class has electric light 0.45 0.26 ** 0.60 0.24 ***
Class has radio 0.39 0.12 *** 0.27 0.13 *
Teacher score on maths test % 63.13 55.79 ***
Teacher education Post-Sec Dip 0.46 0.50 0.49 0.30 **
Teacher education University 0.15 0.03 ** 0.21 0.01 ***

20. www.younglives.org.uk
FINDING OUT MORE