This document summarizes research assessing the educational consequences of teacher deficits in content matter and language of instruction. Testing of 185 teacher trainees in Cameroon found low levels of English proficiency and math knowledge. On average, trainees scored below 50% on a reading comprehension test and 38% in math. Analysis suggests teachers' limited content mastery and language skills may negatively impact students' learning through a probabilistic model. The research highlights the need to better understand how to improve teacher competency to support quality instruction in diverse educational systems.

1. Assessing the Educational Consequences
of Teacher Deficits in Content Matter and
Language of Instruction (L2)
Steve Walter - GIAL

2. Some conventional wisdom:
Schools are only as good as their teachers.

3. Research Context
Location (Cameroon, Philippines)
Experimental program testing L1 instruction
Early exit program
12 experimental schools; 12 comparison schools
Testing began in 2008
Testing in Reading, Math, English
All teachers speak L1

4. Key observation about Kom schools
Standard Schools Mean Mean Experimental
Schools
GS Mentang 10.3 31.7 CBC Fujua
CS Kindoh 14.1 31.8 GS Ngwah
GS Yuwi 14.5 33.9 GS Laikom
GS Meli 15 42.1 GS Kitchu
CBC Fundeng 20.4 46.9 GS Bolem
GS Baichu 23.2 47.9 GS Muteff
GS Atondum 24.6 50.5 GS Ameng
GS Fundong Village 24.8 50.5 GS Mboh
CBC Abuh/Wainchia 26.6 54.6 CBC Kikfuini
GS Njinikijem 28.3 64.7 GS Wombong
PS Ngwah 36.3 65.6 GS Ilung
CS Wombong 37.6 72.6 CBC Belo

5. Sources of Variation – some obvious and some less so
Some “obvious” hypotheses
Motivation (avocation)
Commitment to teaching
Training and Education
Ability and creativity
Teacher-centered instruction
Low compensation
Some “less obvious” hypotheses
Low content mastery on the part of teachers
Low mastery of the language of instruction by teachers
Low time on task
Crowded curriculum requirements
Poorly constructed curricula
Erratic attendance; automatic promotion
Clumsy screening and placement practices

6. The Investigation
Venue:
Government Teacher Training Center in Fundong
Participants:
185 students tested (139 female, 44 male)
150 in Form 3 (the last year of training)
29 in Form 2
6 in Form 1
Content of Tests:
English (including reading comprehension)
Basic math

7. Distribution of Participants by Age
18 20 22 24 26 28 30 32
20
15
10
5
0
age
Percent
Histogram of age

8. The Test Instruments

9. The English/reading Assessment
Two texts for reading comprehension
“Tenali and the Thieves” Grade 3 level 668 words 13.36 words per sentence
10 comprehension questions
“Living an Adventurous Life” Grade 9 level 468 words 19.5 words per
sentence
10 comprehension questions
English knowledge and vocabulary
Word meanings and associations – 10 items
ex. 6. crush a. sweet b. cheap c. change d. smash
Synonyms and antonyms – 10 items
ex. 4. Which of the following words is closest in meaning to the word resolve?
a. faint b. require c. weird d. determination
Words in context (naturalness) – 10 items
ex. 5. The teacher praised the girl because her ideas were so __________.
a. unique b. odd c. purple d. foolish

10. Nature of the Comprehension Questions
General comprehension (vocabulary, textual content, etc.) – 7 items
4 items from Tenali
3 items from Adventure
Understanding the plot structure – 2 items
2 items from Tenali
Simple factual comprehension – 4 items
1 items from Tenali
3 items from Adventure
Drawing inferences and conclusions – 7 items
3 items from Tenali
4 items from Adventure

11. Nature of the Vocabulary Section
30 vocabulary items (taken from the Cameroon textbook series for teaching
English)
Level 4 – 10 items ex. oil, respect, good, open
Level 6 – 9 items ex. ambition, estimate, acquire, pursuit
Level 8 – 6 items ex. cunning, resolve, prior, endorse
Levels 10 and 12 – 5 items ex. mutual, crush, unique, append
Categories of vocabulary items
General use vocabulary – 23 items ex. shelf, trick, chance, dead
Academic vocabulary – 7 items ex. structure, resolve, mutual, append

12. The Math Test
Classification by response method
Free response (supply a math fact or do an actual computation)
12 items
Multiple choice
30 items
Classification by manner of presentation
Direct (simple computation or directed solution)
22 items
Story problems (an abstract scenario is created from which information
must be gleaned in order to solve a problem)
20 items

13. The Math Test – Classification by Content
Basic operations – 4 items
Math facts (basic and advanced) – 4 items
Operations with Fractions – 3 items
Basic algebra – 6 items
Conversions and equivalences – 2 items
Probability and statistics – 2 items
Logical and applied analysis – 7 items
Symbolic representation – 6 items
Problem solving with percentages – 2 items
Area and perimeter – 3 items (two of which were discarded)
Graphical representation – 1 item
Ratios – 2 items

14. A Look at the Results

15. Distribution of Overall Performance
10 20 30 40 50 60 70
25
20
15
10
5
0
OVERALL_PER
Percent
Histogram of OVERALL_PER

16. Distribution of Performance on the English Assessment
15 30 45 60 75 90
20
15
10
5
0
eng_PER
Percent
Histogram of eng_PER

17. Distribution of Performance in Math
10 20 30 40 50 60 70
25
20
15
10
5
0
Math_PER
Percent
Histogram of Math_PER

18. Summary Statistics of Performance
Mean Stan. Deviation
Math 37.77 10.51
English/reading 65.18 13.79
Overall 52.41 9.42

19. The Results for English and Reading

20. Suggested Standards for Reading Comprehension
Barr, Blachowitz, Katz, and Kaufman (2002)
Proficiency Level Level of Reading Comprehension
Independent 90 – 100 percent
Instructional 75 – 79 percent
Borderline 50 – 74 percent
Frustration below 50 percent

21. Reading Comprehension – a closer look

22. Comprehension vs. Textual Difficulty

23. Vocabulary Knowledge vs. Textual Difficulty
Predictive probabilities
.92 for Tenali
.37 for Adventure

24. Vocabulary – a closer look

25. Relationship between Academic and General Vocabulary
among GTTC trainees
0 20 40 60 80 100
100
80
60
40
20
0
Vocab_no_ac_PER
Vocab_ac_PER
S 18.2455
R-Sq 29.3%
R-Sq(adj) 28.8%
Fitted Line Plot
Vocab_ac_PER = 9.333 + 0.5839 Vocab_no_ac_PER
Mean AWL = 48.0
Mean GWL = 66.2

26. Best Predictors of Reading Comprehension
For the Level 3 Text
Level 4 vocabulary (F = 18.68; p = 0.000; r-sq. = 10.61%)
For the Level 9 Text
Academic vocabulary (F = 23.33; p = 0.000)
Form (years in GTTC) (F = 8.05; p = 0.000; r-sq. = 17.35%)

27. Vocabulary and Text Comprehension

28. Comparison to Grade 5 Readers
Level 3 Text

29. Distribution of Reading Comprehension among GTTC trainees
relative to the Barr et al. Standards
Barr, Blachowitz, Katz, and Kaufman (2002)
When reading a…
Proficiency Level Percent Comprehension Level 3 Text Level 9 Text
Independent 90 – 100 percent 77.6% 2.4%
Instructional 75 – 79 percent 9.1% 2.4%
Borderline 50 – 74 percent 11.5% 61.2%
Frustration below 50 percent 1.8% 33.9%

30. Implications and Questions raised by the Data
Levels of general proficiency in English are obviously low.
The level of reading comprehension is quite limited.
How effectively can such teachers teach via English?
Is there a self-perpetuating negative spiral in place in terms of
learning and teaching English?
What is the probability of educational success for non English-speaking
children when their teachers have low proficiency in
the language of instruction?
How much improvement in teachers’ proficiency is needed to raise
students’ performance?
What steps could be taken to improve teachers’ proficiency in
English?

31. The Math Results

32. Distribution of Performance in Math
0 10 20 30 40 50 60 70 80 90 100
25
20
15
10
5
0
Math_PER
Percent
Histogram of Math_PER
Mean = 37.77
SD = 10.51

33. Performance by Content Area
No. of
items
Mean
Score
Standard
Dev.
Number who answered
zero items correctly in
this category
Basic operations 4 49.4 22.2 3
Math facts 4 46.8 24.6 11
Fractions 3 33.8 26.9 44
Basic algebra 6 25.3 23.1 47
Percentages 2 23.4 28.6 90
Conversions 2 60.4 34.4 24
Probability and statistics 2 32.9 32.3 69
Logical analysis 7 42.0 16.5 3
Symbolic representation 6 30.9 20.2 17
Geometry (area and perimeter) 3* 24.7 43.3 119
Graphical representation 1 9.5 29.4 143
Ratios 2 62.7 35.2 24

34. GTTC trainees vs. Class 5 students
Class 5 Students GTTC
Skill or knowledge domain Mean SD Mean SD
Basic operations 10.9 17.7 49.4 22.2
The number system 19.9 14.7 52.1 21.6
Fractions 12.6 13.3 22.8 34.1
Algebra -- -- 24.3 20.7
Word Problems
Basic operations 19.5 19.8 31.7 28.4
Analysis -- -- 44.8 14.9
Geometric figures 23.8 23.1 22.2 30.1
Graphs and tables 23.3 21.7 22.2 27.4
Percent, interest, commissions -- -- 30.1 34.3
Basic statistics -- -- 32.9 32.3
Math Overall 16.6 7.7 37.8 10.5

35. Some Inferences for Math Learning and Instruction
Mastery of basic operations is not strong.
It is reported that many have taken no math courses
beyond primary (Grade 6) and the data appear to
support this claim.
Complex numbers including decimals are troublesome.
Mathematical abstractions (models, variables,
symbols) are especially difficult.
If teachers lack knowledge and skill, we must assume
that students will as well.

36. Data from the MaguindanaonMLE
Experience in the Philippines:
Comparing L1 vs. L2 speakers of the
Language of Instruction

37. At the Kindergarten Level

38. At the Grade 1 Level

39. At the Grade 2 Level

40. (Negative) Impact on Instructional Efficiency

41. A Simple Probabilistic Model of Learning
S = student
T = teacher
Prob. X Prob. X Prob. = Prob.
T knows T can explain S will understand S learns

42. A Simple Probabilistic Model of Learning
Suggested probabilities
Teachers’ performance on primary-level math items = 42.24%
Teachers’ performance on English assessment (Grade 3 text plus
vocabulary) = 75.9%
Class 5 students’ performance on English assessment (easiest story plus
vocabulary) = 38.6%
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Prob. X Prob. X Prob. = Prob.
T knows T can explain S will understand S learns
.4224 X .759 X .386 = .124
(actual mean math score = 17.0%)

43. Conclusions and Challenges
Teacher mastery of curricular content, while frequently
considered suspect, has not been widely measured.
Limitations in linguistic competence are widely
recognized in fact, but little understood in terms of
instructional impact.
The weakness of educational systems in linguistically
diverse developing countries involves teachers as well
as students.
How much competence in a language is required to
enable quality instruction?
What kinds of language learning/teaching interventions
are needed to developed the needed competence?

44. Thank you.