Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Eaquals Training for Excellence: Assessment, Elaine Boyd

90 views

Published on

Malta 2017

Published in: Education
  • Be the first to comment

Eaquals Training for Excellence: Assessment, Elaine Boyd

  1. 1. Evaluating and designing language assessments Elaine Boyd University College London elaine.boyd@ucl.ac.uk ©Eaquals Eaquals – Training for Excellence | Malta | 17-18 November 2017 www.eaquals.org 1
  2. 2. Objectives • testing principles • types of assessment • applying principles • mapping levels of assessments • writing assessment tasks • judging performance • individual challenges ©Eaquals www.eaquals.org 2Eaquals – Training for Excellence | Malta | 17-18 November 2017
  3. 3. Friday • Why are principles important? • What are the principles of assessment? • What are we measuring? • How do principles apply to your tasks? • How do we level tasks? ©Eaquals www.eaquals.org 3 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  4. 4. Why are principles important? • Ethics • Fairness • Consequences • Fit for purpose ©Eaquals www.eaquals.org 4Eaquals – Training for Excellence | Malta | 17-18 November 2017
  5. 5. What are the principles of assessment? VALIDITY • Is the test testing what it claims to be testing? • Does the test represent the test taker’s real life language use? • Does the test represent learning activities? ©Eaquals www.eaquals.org 5Eaquals – Training for Excellence | Malta | 17-18 November 2017
  6. 6. What are the principles of assessment? RELIABILITY • Is every version of the test the same? • Is the test the same across time? • Is every test taker’s performance assessed in the same way? How do you check? ©Eaquals www.eaquals.org 6 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  7. 7. What are the principles of assessment? PRACTICALITY • time to deliver • complexity of instructions • time to assess • resources needed What can go wrong? ©Eaquals www.eaquals.org 7 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  8. 8. What are the principles of assessment? IMPACT • What effect does the test have on teaching? • What effect does the test have on the learner? • How wide is the impact of the test? When do you design for impact? ©Eaquals www.eaquals.org 8Eaquals – Training for Excellence | Malta | 17-18 November 2017
  9. 9. Principles of assessment Interlocking or is there a priority? What is the effect of getting one wrong? Who benefits? ©Eaquals www.eaquals.org 9Eaquals – Training for Excellence | Malta | 17-18 November 2017
  10. 10. Communicative competences (Bachman & Palmer, 1997) ©Eaquals www.eaquals.org 10 What do we want to measure? Eaquals – Training for Excellence | Malta | 17-18 November 2017
  11. 11. Example 2: speaking Format: test taker & test taker Level A2 + B1 Length: 5 - 7 minutes Candidate A: You organize and trip with an English friend. You make suggestions and ask your partner if he/she agrees. Candidate B: You organize and trip with an English friend. You make suggestions and ask your partner if he/she agrees. ©Eaquals www.eaquals.org 11 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  12. 12. Example 3: writing Level A2 A Write a description of your best friend. Say: - what she looks like - what she likes - why you like her B Write a description on a visit to the zoo. Say: - what the zoo was like - what animals you like best and why ©Eaquals www.eaquals.org 12 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  13. 13. How do principles apply to your tasks? • What is the purpose of your test? • Which principle is more evident in your test? • Which principle is the most relevant for your context? Why? • What feedback do you give from the test? • What would you change? Why? ©Eaquals www.eaquals.org 13Eaquals – Training for Excellence | Malta | 17-18 November 2017
  14. 14. How do we level tasks? • CEFR • Syllabus in Course Book • School curriculum • Other validated tasks • Collaboration ©Eaquals www.eaquals.org 14Eaquals – Training for Excellence | Malta | 17-18 November 2017
  15. 15. Level: texts & tasks • What level would you test with this text? • Which competences would you test? • What kind of task would you use to elicit those competences? • Could you adapt level? How? Risks? ©Eaquals www.eaquals.org 15Eaquals – Training for Excellence | Malta | 17-18 November 2017
  16. 16. Saturday • Rules for writing assessment tasks • How do we judge performance? • Issues in formative assessment • What are your individual challenges? ©Eaquals www.eaquals.org 16Eaquals – Training for Excellence | Malta | 17-18 November 2017
  17. 17. Writing objective tasks Multiple choice questions What is the speaker’s opinion? a) He supports the project. b) He does not support the project. c) He thinks they need to do more work on the project. ©Eaquals www.eaquals.org 17Eaquals – Training for Excellence | Malta | 17-18 November 2017
  18. 18. Writing objective tasks: Rules for writing M/C § Item must be appropriate level. § Question must be clear on its own. § Question must identify where to listen or read. § Options must capture genuine distraction. § Options must be similar length and structure. § Options all have the same focus e.g. positive or negative. § Key must be true and only the key is true. § Distractors must be plausible. § Must test language knowledge not general knowledge. ©Eaquals www.eaquals.org 18 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  19. 19. Writing objective tasks Write TWO 3-option M/C items to test understanding of the listening text. People prefer nuclear families because
 a. there is more freedom.
 b. there are fewer members.
 c. it is the latest trend in many countries. 2. According to the speaker, young people don’t listen to elders because they a. don’t want to be restricted.
 b. think they know better.
 c. learn from their mistakes. ©Eaquals www.eaquals.org 19 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  20. 20. Writing productive tasks You are teaching a class of A2 level students. The students are 10 years old. Write a writing task for these students. The topic must be Free Time. ©Eaquals www.eaquals.org 20 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  21. 21. How do we judge performance? • How do we decide WHAT to grade? • How do we decide HOW to grade? • How do we decide on balance/weighting? • What is the ‘judgement’ difference between summative and formative tests? • Can we use criteria as Objectives? How? ©Eaquals www.eaquals.org 21Eaquals – Training for Excellence | Malta | 17-18 November 2017
  22. 22. Write assessment criteria Write criteria for Writing task: • use competences & CEFR • decide categories • write top performance descriptor • decide scale with rationale • decide weighting with rationale ©Eaquals www.eaquals.org 22 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  23. 23. Formative assessment • What are the different ways we can do this? • What are the challenges to validity? ©Eaquals www.eaquals.org 23Eaquals – Training for Excellence | Malta | 17-18 November 2017
  24. 24. Your tests • What challenges do you face in your own interaction in tests? • Can you use any of the principles to help you? • What processes could you set up to resolve? • What checks could you set up to review? • What remains? ©Eaquals www.eaquals.org 24Eaquals – Training for Excellence | Malta | 17-18 November 2017
  25. 25. Fairness Check • What am I trying to test? • How do I know that I’m testing what I think I’m testing? • Does the test task reflect the real world and/or classroom practice? • Can I replicate this reliably? • What is are reasonable conditions for the test? • What will happen when someone fails? • What effect will this test task have on classroom practice? ©Eaquals www.eaquals.org 25 Eaquals – Training for Excellence | Malta | 17-18 November 2017
  26. 26. Remember …..... vYOU ARE ALWAYS MEASURING POSITIVE ACHIEVEMENT vNEVER DO IT ALONE! ©Eaquals www.eaquals.org 26Eaquals – Training for Excellence | Malta | 17-18 November 2017
  27. 27. References • Fulcher, G. (2010). Practical language testing. London: Hodder. • Bachman, L., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press. • Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-104). New York, NY: American Council on education and Macmillan. • International Language Testing Association (ILTA) guidelines: https://moodle.ucl.ac.uk/pluginfile.php/4227841/mod_resource/content/1/Week%209 %20-%20ILTA%20guildelines%20for%20practice.pdf • Dianne Wall (2012) Examining washback: What do we know and what is there left to explore? http://www.alte.org/attachments/pdfs/files/dianne_wall_jl7so.pdf • Language Testing site has expert videos, podcasts, articles and references: • http://languagetesting.info • Manual for Language Test Development and Examining. ALTE Council of Europe. 2011 • http://www.coe.int/t/dg4/linguistic/ManualLanguageTest-Alte2011_EN.pdf • EALTA good practice: http://www.ealta.eu.org/documents/archive/guidelines/English.pdf • Council of Europe (2009) Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR). A Manual. https://rm.coe.int/1680667a2d ©Eaquals www.eaquals.org 27Eaquals – Training for Excellence | Malta | 17-18 November 2017

×