Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Kit-Tai Hau
The Chinese University of Hong Kong
1
Assessments of National Educational
Progress, School Improvement and
Stu...
Outstanding Asian Performance
Converging evidence:
– International studies: Asian outstanding over widely
diverse represen...
– Hong Kong, Taiwan, China, Japan, South Korea (except
Singapore) have Ed Monitoring systems, some (Hong
Kong, S Korea, pa...
Presentation Outline
I. Educational Monitoring in SE Asia/Pacific
II. General Framework
III. Hong Kong Attainment Test (HK...
I. Ed Assessment in SE Asia/Pacific
Country Level Subjects First Yr
Australia G 3, 5, 7, 9 Literacy, Numeracy 2008
G 6, 10...
Country Level Subjects First Yr
Cambodia G 3, 6, 9 Lang, Math 2005
China G 4, 8 Chinese, Eng, Math, Sc, Psycho
health, phy...
Country Level Subjects First Yr
Myanmar G 3, 5 Lang, Math 2007
Nepal G 3, 5, 6,
8, 10
Nepali, Math, Soc Studies, Eng,
Sc, ...
Country Level Subjects First Yr
Thailand G 2, 3, 5,
6, 9, 11,
12
Thai, Foreign Lang, Math, Sc, Soc
Sc, Religion, Health, P...
II. General Framework: Monitor + Feedback
9
Level Purpose/
Help……
Comparison targets
Student Student
learn
Own past, class...
III. Hong Kong Attainment test (HKAT)
• 1976-2003
• One test at each grade, G.1 - 9
• Subjects: Chinese, English, Maths
• ...
• Little information provided:
– for students: question paper too short, no
diagnostic value
– for schools: only a relativ...
V. Consultancy Report (review 2000): Can
one test serve both purposes (Monitor/
Feedback)?
12
• Seek
consultant’s
advice
•...
Can one test serve all purposes?
Student School Country
Once a year Not enough Too much, has to rotate
subjects as in PISA...
Ideal Monitoring System
• To Student: test daily, auto-marking, low stake, can
retake, in line with self-regulation
• To s...
VI. Divided into Student + System
Assessments
Decision of the HK government in 2000:
•Will monitor Chinese, English, Maths...
VII. Student Assessment
• Web based
• Purpose: help teachers in assessing their students
• Supplement and adapt to student...
17
18
19
20
Problems of Student Assessment
21
• In-charge by the Examination Authorities (HK
Examinations and Assessment Authorities):...
VIII. System Assessment
• Mandatory tests for all students in all government
funded schools
• To understand the achievemen...
• One external teacher help invigilation
• Centrally marked
• Include even oral exam for Chinese, English (only
randomly s...
IX. Positive uses of Monitoring
Assessment Results
• Analyses on item/domain performance on student +
system assessment ca...
• Help for new migrants, low SES students through
– Extra funds when schools have (i) new migrants,
(ii) low SES, (iii) SE...
X. Current Problems and Issues
System Assessment
•Only % pass/fail basic competence, no fine grade
•After a few years, as ...
• Not rotating academic subjects across years yet
• questionnaire now added on students’ attitude,
family background, etc....
• Student Assessment
– Not widely used by students
– Insufficient coverage of fully curriculum
– Item types not attractive...
Upcoming SlideShare
Loading in …5
×

Assessments of National Educational Progress, School Improvement and Students’ Learning. Dynamics and Practices in Asia

916 views

Published on

Presentation by Hau Kit-Tai - Chinese University of Hong Kong.

Presentazione di Hau Kit-Tai in occasione del suo intervento al convegno internazionale "Migliorare la scuola" (Napoli, 14-15 Maggio 2015), organizzato dall'Indire.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Assessments of National Educational Progress, School Improvement and Students’ Learning. Dynamics and Practices in Asia

  1. 1. Kit-Tai Hau The Chinese University of Hong Kong 1 Assessments of National Educational Progress, School Improvement and Students’ Learning – Dynamics and Practices in Asia
  2. 2. Outstanding Asian Performance Converging evidence: – International studies: Asian outstanding over widely diverse representative population – e.g., 2nd IEA: 13-yr China, Taiwan among the top – TIMSS: G.4, 8 Hong Kong, Japan, South Korea, Singapore outperformed others – PISA: Singapore, Hong Kong, S Korea top 3 – Recent PISA (2012): Shanghai (China), 1st in all 3 subjects (Reading, Maths, Science), Hong Kong 2nd, 2nd, 3rd 2
  3. 3. – Hong Kong, Taiwan, China, Japan, South Korea (except Singapore) have Ed Monitoring systems, some (Hong Kong, S Korea, part of Japan, some parts in China) producing individual school reports to the public (league table) – Will use Hong Kong as an example to illustrate the dynamics /issues involved 3
  4. 4. Presentation Outline I. Educational Monitoring in SE Asia/Pacific II. General Framework III. Hong Kong Attainment Test (HKAT) IV. Problems of HKAT V. Consultancy Report: Can one test serve all? VI. Divided into: Student + System Assessments VII. Student Assessment VIII.System Assessment IX. Positive Uses of Monitoring Assessment Results X. Current Problems/Issues 4
  5. 5. I. Ed Assessment in SE Asia/Pacific Country Level Subjects First Yr Australia G 3, 5, 7, 9 Literacy, Numeracy 2008 G 6, 10 ICT 2005 G 6, 10 Science (Sc) 2003 G 6, 10 Civil Knowledge 2004 Bangladesh G 3, 5 Language (Lang), Math 2006 G 1, 5 Lang, Eng, Math, Sc, Soc Sc 2004 G 8 Lang, Eng, Math 2008 Bhutan G 2, 4 Lang, Eng, Math 2007 G 4, 6, 8 Eng, Math, Sc 2008 G 6 Eng, Math 2003 G 10 Lang, Eng, Math, History 2006 5
  6. 6. Country Level Subjects First Yr Cambodia G 3, 6, 9 Lang, Math 2005 China G 4, 8 Chinese, Eng, Math, Sc, Psycho health, physical Ed/ Health 2007 Cook Isld G 4, 6 Lang, Eng, Math 1999 G 4, 5, 6 Eng, Lang 1994 G 3, 7 Math 1994 Fuji G 4, 6, 8 Literacy, Numeracy 2007 India G 3 Lang, Math 2004 G 5 Lang, Math, Enviro Sc 2002 G 8 Lang, Math, Sc, Soc Sc 2003 Age 5-16 Literacy, Numeracy 2005 Japan G 6, 9 Jap, Math, Sc 2007 Kiribati G 4, 6 Lang, Eng, Numeracy 2004 Lao G 1, 2, 5 Lang, Math, Sc 1996 Maldives G 4, 7 Lang, Eng, Math, EnvSc, SocSc 2008 G 9 Phys, Chem, Bio, Hist, Geog 2013 6
  7. 7. Country Level Subjects First Yr Myanmar G 3, 5 Lang, Math 2007 Nepal G 3, 5, 6, 8, 10 Nepali, Math, Soc Studies, Eng, Sc, Health 1999 N Zealand G 4, 8 Sc, Visual Arts, ICT, Lang, Tech, Music, Math, Soc Studies, Health 1995 Pakistan G 3, 4, 8 Lang, Maths, Sc, Soc Studies 2003 Philippines G 3, 6, 12 Eng, Filipino, Math, Sc, Soc Sc, Critical thinking 2004 S Korea G 6, 9, 11 Korean, Eng Math, Sc, Soc Studies 2000 Samoa G 4, 6 Samoan, Eng, Numeracy 1996 Singapore G 6, 10, 12 Lang, Math, Sc, Soc Sc, Applied subjects 1960 Salomon I G 4, 6 Literacy, Numeracy 2005 Sri Lanka G 4, 8, 10 Singalese, Tamil, Eng, Math, Sc, Tech 2003 7
  8. 8. Country Level Subjects First Yr Thailand G 2, 3, 5, 6, 9, 11, 12 Thai, Foreign Lang, Math, Sc, Soc Sc, Religion, Health, PE, Art, Career, Occupational Studies 2005 Tonga G 4, 6 Tongan, Eng, Numeracy 2011 Tuvalu G 4, 6 Literacy, Numeracy 2000 Vanuatu G 4, 6 Literacy, Numeracy 2007 Vietnam G 5, 6, 9, 11 Vietnamese, Math, Eng, Physics, Bio 2001 8
  9. 9. II. General Framework: Monitor + Feedback 9 Level Purpose/ Help…… Comparison targets Student Student learn Own past, classmate, national norms Class Teacher teach previous yr, other classes, national norm School School Accountable previous yr, other schools, national norm State/ City Plan, accountable previous yr, other states, national norm Country Monitor progress previous yr, other countries (e.g., PISA,TIMSS)
  10. 10. III. Hong Kong Attainment test (HKAT) • 1976-2003 • One test at each grade, G.1 - 9 • Subjects: Chinese, English, Maths • Operation: – test/questions left with school – changed every few years – marked by own teachers – school submit 1/30 (e.g., birth day 13th ) for Ed Bureau to mark (build HK norm, monitor HK) – later change to 1/3 to monitor school • Main purpose: to monitor whole Hong Kong 10
  11. 11. • Little information provided: – for students: question paper too short, no diagnostic value – for schools: only a relative ranking (cf. HK norm) – for whole Hong Kong: only a summary total (2015 M=87.6, no other information) • Once a year, insufficient to help daily learning • Teachers already have other means (school tests) to understand own students’ strength/weakness • Teachers’ heavy load in marking • Paper too easy for top schools 11 IV. Problems of HKAT
  12. 12. V. Consultancy Report (review 2000): Can one test serve both purposes (Monitor/ Feedback)? 12 • Seek consultant’s advice • Visit ETS, etc. • Public consultation Prof W. J. van der Linden
  13. 13. Can one test serve all purposes? Student School Country Once a year Not enough Too much, has to rotate subjects as in PISA paper long short Good, but expensive Coverage not enough Each student answers part of the questions Difficulty: key competence mid difficulty Not enough discrimination, low quality Best psychometric properties, but students, parents feel too difficult High Stake Narrowing of curriculum, teachers cheat  results not valid 13
  14. 14. Ideal Monitoring System • To Student: test daily, auto-marking, low stake, can retake, in line with self-regulation • To school/city/country: should attempt to reduce the unavoidably high stake, can change focus each year (academic subject on rotation, as in PISA) • Schools have to supervise test administration, to reduce incentive in cheating, should not allow school to publicize results • If cheating too serious/pressure on students too large, do not report school results, only provide higher level summaries • But too costly if the whole scheme is just to monitor the country and does not produce school reports 14
  15. 15. VI. Divided into Student + System Assessments Decision of the HK government in 2000: •Will monitor Chinese, English, Maths basic competence •To help learning + monitor school/provide support 15 (A)Secured test (G.3, G.6, G.9) System Assessment (slightly high stake) (B)Web based adaptive test(G.1-G.9) (Student Assessment) Hong Kong Attainment test
  16. 16. VII. Student Assessment • Web based • Purpose: help teachers in assessing their students • Supplement and adapt to students’ need, difficulty appropriate to students’ ability • Can fully serve the daily need throughout the school term 16
  17. 17. 17
  18. 18. 18
  19. 19. 19
  20. 20. 20
  21. 21. Problems of Student Assessment 21 • In-charge by the Examination Authorities (HK Examinations and Assessment Authorities): high quality but too few items • Not enough promotion of usage, self-regulation, self- monitoring function • Student mark recording system too complicated (does not facilitate widespread use) • Does not link to learning or remedial packages, thus cannot be a fully standalone learning system
  22. 22. VIII. System Assessment • Mandatory tests for all students in all government funded schools • To understand the achievement of basic competence • Only report % students (e.g., 85%) meeting basic competence in each school, cannot publicize to avoid building league table and creating additional pressure to students/schools • On Chinese/English/Maths, 30-90 min G.3, 65-1 – 2 hour G.9 • Only at G.3, 6, 9 (no G.1, cannot calculate value added for primary schools) 22
  23. 23. • One external teacher help invigilation • Centrally marked • Include even oral exam for Chinese, English (only randomly select 10%-20% of students) • Example report at system level (% meeting Basic Competence 23 G.3 G.6 G.9 Chinese 85% 77% 76% English 79% 71% 69% Maths 87% 84% 80%
  24. 24. IX. Positive uses of Monitoring Assessment Results • Analyses on item/domain performance on student + system assessment can generate useful feedback on school performance (on their strength/weakness) • Exam pressure, over-drilling already serious in Asia  Singapore no monitoring tests, other countries careful not add pressure through school reports, league tables • In PISA, Hong Kong 1st in equity - Social Economic Status (SES) has least effect on students’ achievement 24
  25. 25. • Help for new migrants, low SES students through – Extra funds when schools have (i) new migrants, (ii) low SES, (iii) SEN (special ed need) students – but must allow migrants to join System Assessment (for accountability) – Ed Bureau (government) special team to provide school based professional support: jointly prepare teaching/ assessment materials – University-school partner teams (supported by government funds) help needy schools 25
  26. 26. X. Current Problems and Issues System Assessment •Only % pass/fail basic competence, no fine grade •After a few years, as these figures are quite stable, no one is interested, no one care (good, no pressure on students), but little diagnostic information as well •Items focus basic competence too easy  low discrimination  low quality in monitoring whole Hong Kong  more difficult items added recently •To enrich the reports, now providing item performance information for schools in System Assessment  but schools drill on identical item types (possible solution generate item analyses from Student Assessment) 26
  27. 27. • Not rotating academic subjects across years yet • questionnaire now added on students’ attitude, family background, etc. • Low monitoring in test administration  schools may cheat, not totally cheating-proof • Item types too narrow, not creative enough, not having a positive effect on teaching • Population declines  primary schools great pressure to drill to get good results  asked government not to conduct test/or release results  now stop releasing results to primary schools • Secondary schools (G.9-12) have a high stake university entrance exam anyway  results continued to be released to them 27
  28. 28. • Student Assessment – Not widely used by students – Insufficient coverage of fully curriculum – Item types not attractive – No linkage to remedial teaching packages (to enable self-learning) – Insufficient convenience for students’ self-usage • Conclusion – A good system is a compromise of educational, psychometrical (measurement), and political considerations 28

×