Erickson: Large-scale assessment of oral language proficiency - A Swedish example

965 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
965
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Erickson: Large-scale assessment of oral language proficiency - A Swedish example

  1. 1. LukSuS project Helsinki, 30 November 2013 Helsinki, 30 November 2013 LukSuS project Large-scale Assessment of Oral Language Proficiency — A Swedish Example Gudrun Erickson University of Gothenburg, Sweden Gudrun.Erickson@ped.gu.se Overview Context – “Facts about Sweden” Why? – Purpose(s) What? – Construct How? – Methods & Models When? – Timing and Sequencing Who? – Actors And…? – Uses, Consequences, ‘Issues’… EALTA Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 1
  2. 2. LukSuS project Helsinki, 30 November 2013 One of the protagonists about oral testing It’s always good to be able to pronounce words. If you’re in England for example, you can’t go around and write notes to everybody you see… Swedish 15-year-old about the oral test of English for grade 9 The Swedish School System • 9-year compulsory school; 98 % continue to three-year upper secondary education (c. 75 % graduate) • National curricula and syllabuses for subjects • Grading from year 6 (students 11-12 years of age) • Goal/criterion referenced grading system; six grade levels (A-E + F) • Teachers responsible for marking national tests, and for final grading • Grades high stakes – used for selection for higher education • Extensive national testing programme – advisory function (however, quantitatively undefined) – No exams • 4 mandatory tests of EFL: ages 12, 16 // 17, 18 (approx. A2, B1, B1+, B2) Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 2
  3. 3. LukSuS project Helsinki, 30 November 2013 [Current] Aims of national tests • To enhance fairness and equity in assessment and grading • To provide data for local, regional and national quality control “The system may also contribute to” • clarifying and operationalizing the national syllabus • enhancing students’ learning and attainment of national objectives A collaborative approach to test development • Analyses of relevant literature and research • Continuous work in broad groups of experts (teachers, teacher educators, special needs teachers, researchers, L1 speakers…) • Small-scale piloting > Adjustments (an iterative process) • Large-scale pre-testing in randomly selected groups of students in the country (n ≈ 400) • Analyses of results and of students’ and teachers’ views and suggestions • Compilation of tests (reference groups) • Standard setting and benchmarking in broad groups • Analyses / Research / Reporting Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 3
  4. 4. LukSuS project Helsinki, 30 November 2013 Focal areas in the national syllabuses of FL A functional, action oriented approach (cf CEFR) • Receptive competence (listening and reading) • Productive competence (speaking and writing) • Interactive competence (speaking and writing) • Intercultural communicative competence • Strategies; Adaptation (to purpose, context & receiver) The typical national FL test Four subtest (LC & RC aggregated into one score) •  Oral production and interaction •  Reception: Listening comprehension • Reception : Reading comprehension •  Written production (and interaction) ———————————————— • Strategies and Adaptation (to purpose, content and receiver) integrated • “Culture”: an integrated aspect of test content and design • Reflection, e.g. self assessment, focused upon in separate sections/materials and/or in student profiles • Results / Sub-scores presented in profiles • Model provided for aggregation into total score Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 4
  5. 5. LukSuS project Helsinki, 30 November 2013 Oral proficiency in the national syllabuses (upper secondary level) • • • • Clear distinction between production and interaction Speaking and Writing combined “CENTRAL CONTENT”: …Oral and written production and interaction of different kinds, including more formal contexts, where students instruct, tell about things, summarise, explain, comment, evaluate, justify their views, discuss and argue. – Strategies for contributing to and actively participating in discussions related to societal and working life. – Revision and modification of their own and others' oral and written communication in order to vary, clarify and specify, as well as to create structure and adapt it to purpose and situation. PERFORMANCE STANDARDS: …In oral and written communication of different genres, students can express themselves in relatively varied ways, relatively clearly and relatively coherently. Students can express themselves with some fluency and in a way that is to some degree adapted to purpose, recipient and situation. Students revise and make improvements to their communications. In oral and written interaction in different, including more formal, contexts, students can express themselves clearly and with some fluency and some adaptation to purpose, recipient and situation. In addition, students can choose and use broadly functional strategies that will, to some extent, solve communication problems and improve interaction. The Development of oral tests of EFL in Sweden • Offered as a complement to the national tests since the late 1980s • Mandatory in grade 9 since 1998, in upper sec. since 2002 *** • One to one (teacher + student) 1991-1993 • Student interactive (pairs) + one to one 1994-1997 • Student interactive (paired orals) 1998- Production + Interaction; Often three parts: Warm up > Production – Argue – Discuss (10-20 mins per pair, depending on age, level etc.) *** Interrogation… – Interview… – Interaction… Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 5
  6. 6. LukSuS project Helsinki, 30 November 2013 Typical themes of oral tests Year 5/6 (≈ A2) • What Do You Think? • The International Talk Show Year 9 (≈ B1) • The World Around Us • Quality of Life English 5 (≈ B1+) • Doing the Right Thing • Skills for Life English 6 (≈ B2 ) • Integrity and Technology • Life Balance Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 6
  7. 7. LukSuS project Helsinki, 30 November 2013 Holistic rating supported by analytical factors • CONTENT Clarity – Complexity and variation (different examples and perspectives) – Coherence and structure – Adaptation to purpose, recipient, situation and genre • LANGUAGE AND /SELF/ EXPRESSION Communicative strategies (to develop and advance a conversation; to solve linguistic problems, e.g. by rephrasing, explaining and clarifying) – Fluency and ease – Range, variation, complexity, clarity and accuracy (vocabulary, phraseology and idiomatic expression; pronunciation and intonation; grammatical structures) – Adaptation to purpose, recipient, situation and genre Materials to support teachers • General guidelines • Performance standards • Analytical factors • Authentic, graded benchmarks with extensive comments related to the syllabus, standards and factors [Year 6: Extensive training material] Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 7
  8. 8. LukSuS project Helsinki, 30 November 2013 Some results ( % ) Grade level Year 6 Year 9 En 5 En 6 F 5 3 4 1 >E 78 81 79 86 A 18 18 16 18 Teacher Questionnaire (oral test) Good/Ok test 98 95 95 99 Recording 46 46 49 50 Correlations between raters NB: Based on benchmarking data Average Lowest Highest Year 6 (11) .75 .53 .95 Year 9 (12) .90 .82 .97 E 5 (10) .85 .59 .98 E 6 (8) .80 .53 .93 Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 8
  9. 9. LukSuS project Helsinki, 30 November 2013 Co-Rating ? The teacher as the only rater Subtest Year 6 Year 9 En 5 En 6 A (oral) 58 52 55 57 B (rec) 44 42 57 57 C (writ) 17 10 13 11 Opinions about oral tests & testing • Topics good and relevant / Nice things to talk about • Positive reflection of the syllabus / Important to know/learn • Students really enjoy doing it / Fun to do! **** • Practical difficulties; time consuming • No ‘visible’ time for co-rating • They never stop talking! Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 9
  10. 10. LukSuS project Helsinki, 30 November 2013 ISSUES… • Practicalities (incl. time) • Teacher present or absent? • Recording? • Co-rating – how? • Interlocutor/Partner effects • Preparation time? • Type of language – tasks – content… • Balance production – Interaction • Can teachers’ ratings be trusted – in general + of their own students’ performances…? EALTA* Guidelines for Good Practice in Language Testing and Assessment Three categories/“audiences”: • Teacher pre- and in-service training • Classroom assessment • Test development in national or institutional testing units or centres Basic principles the same for all categories: Respect for students/examinees, responsibility, fairness, reliability, validity, collaboration among the parties involved * European Association for Language Testing and Assessment www.ealta.eu.org Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se 10
  11. 11. LukSuS project Helsinki, 30 November 2013 European student 2005: “I think that a good language test/ assessment should get students chance to show what they know and devolope their skills. It should contain of listening and reading comprehations, as well as writing and grammar part. But the most important thing is to speak in that language, so I think there should be also oral exams. Speaking is the most important, because without it, we wouldn’t communicate. Because of that students should also learn pronunciation.” “ Now have I not so much more to tell you… ” ! # Gudrun Erickson University of Gothenburg, Sweden gudrun.erickson@ped.gu.se   # ! THANK YOU ! TACK FÖR MIG !# 11

×