Presentation by Hau Kit-Tai - Chinese University of Hong Kong. Presentazione di Hau Kit-Tai in occasione del suo intervento al convegno internazionale "Migliorare la scuola" (Napoli, 14-15 Maggio 2015), organizzato dall'Indire.

1. Kit-Tai Hau
The Chinese University of Hong Kong
1
Assessments of National Educational
Progress, School Improvement and
Students’ Learning –
Dynamics and Practices in Asia

2. Outstanding Asian Performance
Converging evidence:
– International studies: Asian outstanding over widely
diverse representative population
– e.g., 2nd IEA: 13-yr China, Taiwan among the top
– TIMSS: G.4, 8 Hong Kong, Japan, South Korea,
Singapore outperformed others
– PISA: Singapore, Hong Kong, S Korea top 3
– Recent PISA (2012): Shanghai (China), 1st in all 3
subjects (Reading, Maths, Science), Hong Kong 2nd,
2nd, 3rd
2

3. – Hong Kong, Taiwan, China, Japan, South Korea (except
Singapore) have Ed Monitoring systems, some (Hong
Kong, S Korea, part of Japan, some parts in China)
producing individual school reports to the public (league
table)
– Will use Hong Kong as an example to illustrate the
dynamics /issues involved
3

4. Presentation Outline
I. Educational Monitoring in SE Asia/Pacific
II. General Framework
III. Hong Kong Attainment Test (HKAT)
IV. Problems of HKAT
V. Consultancy Report: Can one test serve all?
VI. Divided into: Student + System Assessments
VII. Student Assessment
VIII.System Assessment
IX. Positive Uses of Monitoring Assessment Results
X. Current Problems/Issues
4

5. I. Ed Assessment in SE Asia/Pacific
Country Level Subjects First Yr
Australia G 3, 5, 7, 9 Literacy, Numeracy 2008
G 6, 10 ICT 2005
G 6, 10 Science (Sc) 2003
G 6, 10 Civil Knowledge 2004
Bangladesh G 3, 5 Language (Lang), Math 2006
G 1, 5 Lang, Eng, Math, Sc, Soc Sc 2004
G 8 Lang, Eng, Math 2008
Bhutan G 2, 4 Lang, Eng, Math 2007
G 4, 6, 8 Eng, Math, Sc 2008
G 6 Eng, Math 2003
G 10 Lang, Eng, Math, History 2006 5

6. Country Level Subjects First Yr
Cambodia G 3, 6, 9 Lang, Math 2005
China G 4, 8 Chinese, Eng, Math, Sc, Psycho
health, physical Ed/ Health
2007
Cook Isld G 4, 6 Lang, Eng, Math 1999
G 4, 5, 6 Eng, Lang 1994
G 3, 7 Math 1994
Fuji G 4, 6, 8 Literacy, Numeracy 2007
India G 3 Lang, Math 2004
G 5 Lang, Math, Enviro Sc 2002
G 8 Lang, Math, Sc, Soc Sc 2003
Age 5-16 Literacy, Numeracy 2005
Japan G 6, 9 Jap, Math, Sc 2007
Kiribati G 4, 6 Lang, Eng, Numeracy 2004
Lao G 1, 2, 5 Lang, Math, Sc 1996
Maldives G 4, 7 Lang, Eng, Math, EnvSc, SocSc 2008
G 9 Phys, Chem, Bio, Hist, Geog 2013 6

7. Country Level Subjects First Yr
Myanmar G 3, 5 Lang, Math 2007
Nepal G 3, 5, 6,
8, 10
Nepali, Math, Soc Studies, Eng,
Sc, Health
1999
N Zealand G 4, 8 Sc, Visual Arts, ICT, Lang, Tech,
Music, Math, Soc Studies, Health
1995
Pakistan G 3, 4, 8 Lang, Maths, Sc, Soc Studies 2003
Philippines G 3, 6, 12 Eng, Filipino, Math, Sc, Soc Sc,
Critical thinking
2004
S Korea G 6, 9, 11 Korean, Eng Math, Sc, Soc Studies 2000
Samoa G 4, 6 Samoan, Eng, Numeracy 1996
Singapore G 6, 10,
12
Lang, Math, Sc, Soc Sc, Applied
subjects
1960
Salomon I G 4, 6 Literacy, Numeracy 2005
Sri Lanka G 4, 8, 10 Singalese, Tamil, Eng, Math, Sc,
Tech
2003
7

8. Country Level Subjects First Yr
Thailand G 2, 3, 5,
6, 9, 11,
12
Thai, Foreign Lang, Math, Sc, Soc
Sc, Religion, Health, PE, Art,
Career, Occupational Studies
2005
Tonga G 4, 6 Tongan, Eng, Numeracy 2011
Tuvalu G 4, 6 Literacy, Numeracy 2000
Vanuatu G 4, 6 Literacy, Numeracy 2007
Vietnam G 5, 6, 9,
11
Vietnamese, Math, Eng, Physics,
Bio
2001
8

9. II. General Framework: Monitor + Feedback
9
Level Purpose/
Help……
Comparison targets
Student Student
learn
Own past, classmate, national
norms
Class Teacher
teach
previous yr, other classes,
national norm
School School
Accountable
previous yr, other schools,
national norm
State/
City
Plan,
accountable
previous yr, other states, national
norm
Country Monitor
progress
previous yr, other countries (e.g.,
PISA,TIMSS)

10. III. Hong Kong Attainment test (HKAT)
• 1976-2003
• One test at each grade, G.1 - 9
• Subjects: Chinese, English, Maths
• Operation:
– test/questions left with school
– changed every few years
– marked by own teachers
– school submit 1/30 (e.g., birth day 13th
) for Ed
Bureau to mark (build HK norm, monitor HK)
– later change to 1/3 to monitor school
• Main purpose: to monitor whole Hong Kong
10

11. • Little information provided:
– for students: question paper too short, no
diagnostic value
– for schools: only a relative ranking (cf. HK norm)
– for whole Hong Kong: only a summary total (2015
M=87.6, no other information)
• Once a year, insufficient to help daily learning
• Teachers already have other means (school tests) to
understand own students’ strength/weakness
• Teachers’ heavy load in marking
• Paper too easy for top schools
11
IV. Problems of HKAT

12. V. Consultancy Report (review 2000): Can
one test serve both purposes (Monitor/
Feedback)?
12
• Seek
consultant’s
advice
• Visit ETS, etc.
• Public
consultation
Prof W. J. van der Linden

13. Can one test serve all purposes?
Student School Country
Once a year Not enough Too much, has to rotate
subjects as in PISA
paper long
short
Good, but expensive
Coverage
not enough
Each student answers
part of the questions
Difficulty:
key competence
mid difficulty
Not enough discrimination, low
quality
Best psychometric properties, but
students, parents feel too difficult
High Stake Narrowing of curriculum, teachers
cheat  results not valid
13

14. Ideal Monitoring System
• To Student: test daily, auto-marking, low stake, can
retake, in line with self-regulation
• To school/city/country: should attempt to reduce the
unavoidably high stake, can change focus each year
(academic subject on rotation, as in PISA)
• Schools have to supervise test administration, to
reduce incentive in cheating, should not allow school
to publicize results
• If cheating too serious/pressure on students too
large, do not report school results, only provide
higher level summaries
• But too costly if the whole scheme is just to monitor
the country and does not produce school reports 14

15. VI. Divided into Student + System
Assessments
Decision of the HK government in 2000:
•Will monitor Chinese, English, Maths basic competence
•To help learning + monitor school/provide support
15
(A)Secured test (G.3, G.6, G.9)
System Assessment (slightly high stake)
(B)Web based adaptive test(G.1-G.9)
(Student Assessment)
Hong Kong
Attainment
test

16. VII. Student Assessment
• Web based
• Purpose: help teachers in assessing their students
• Supplement and adapt to students’ need,
difficulty appropriate to students’ ability
• Can fully serve the daily need throughout the
school term
16

17. 17

18. 18

19. 19

20. 20

21. Problems of Student Assessment
21
• In-charge by the Examination Authorities (HK
Examinations and Assessment Authorities): high
quality but too few items
• Not enough promotion of usage, self-regulation, self-
monitoring function
• Student mark recording system too complicated (does
not facilitate widespread use)
• Does not link to learning or remedial packages, thus
cannot be a fully standalone learning system

22. VIII. System Assessment
• Mandatory tests for all students in all government
funded schools
• To understand the achievement of basic competence
• Only report % students (e.g., 85%) meeting basic
competence in each school, cannot publicize to
avoid building league table and creating additional
pressure to students/schools
• On Chinese/English/Maths, 30-90 min G.3, 65-1 – 2
hour G.9
• Only at G.3, 6, 9 (no G.1, cannot calculate value
added for primary schools)
22

23. • One external teacher help invigilation
• Centrally marked
• Include even oral exam for Chinese, English (only
randomly select 10%-20% of students)
• Example report at system level
(% meeting Basic Competence
23
G.3 G.6 G.9
Chinese 85% 77% 76%
English 79% 71% 69%
Maths 87% 84% 80%

24. IX. Positive uses of Monitoring
Assessment Results
• Analyses on item/domain performance on student +
system assessment can generate useful feedback on
school performance (on their strength/weakness)
• Exam pressure, over-drilling already serious in Asia 
Singapore no monitoring tests, other countries careful
not add pressure through school reports, league tables
• In PISA, Hong Kong 1st
in equity - Social Economic
Status (SES) has least effect on students’ achievement
24

25. • Help for new migrants, low SES students through
– Extra funds when schools have (i) new migrants,
(ii) low SES, (iii) SEN (special ed need) students –
but must allow migrants to join System
Assessment (for accountability)
– Ed Bureau (government) special team to provide
school based professional support: jointly prepare
teaching/ assessment materials
– University-school partner teams (supported by
government funds) help needy schools
25

26. X. Current Problems and Issues
System Assessment
•Only % pass/fail basic competence, no fine grade
•After a few years, as these figures are quite stable, no
one is interested, no one care (good, no pressure on
students), but little diagnostic information as well
•Items focus basic competence too easy  low
discrimination  low quality in monitoring whole Hong
Kong  more difficult items added recently
•To enrich the reports, now providing item performance
information for schools in System Assessment  but
schools drill on identical item types (possible solution
generate item analyses from Student Assessment)
26

27. • Not rotating academic subjects across years yet
• questionnaire now added on students’ attitude,
family background, etc.
• Low monitoring in test administration  schools may
cheat, not totally cheating-proof
• Item types too narrow, not creative enough, not
having a positive effect on teaching
• Population declines  primary schools great
pressure to drill to get good results  asked
government not to conduct test/or release results
 now stop releasing results to primary schools
• Secondary schools (G.9-12) have a high stake
university entrance exam anyway  results
continued to be released to them 27

28. • Student Assessment
– Not widely used by students
– Insufficient coverage of fully curriculum
– Item types not attractive
– No linkage to remedial teaching packages (to
enable self-learning)
– Insufficient convenience for students’ self-usage
• Conclusion
– A good system is a compromise of educational,
psychometrical (measurement), and political
considerations
28