KAYSERI, 29-31 JANUARY 2014

Tests of receptive skills
developing tests for Reading and Listening
Aims of the session
! challenges in testing for comprehension
! what sub-skills are to be tested
! selecting texts
! writi...
What makes testing
comprehension challenging?
What makes testing
comprehension challenging?
Pros

Cons

!  relatively easy to create if
texts are available

!  no obser...
receptive skills can only
be tested indirectly

language performance ! inferences to comprehension
abilities
The process of writing a test

set
aims

select
texts

write
tasks
Sub-skills for reading
Sub-skills for listening
Selecting texts
!  a representative sample of text types learners are
likely to encounter
!  appropriate extent
!  texts w...
Item writing checklist
"  text and item grammatically correct
"  English natural and appropriate
"  item achieves test obj...
Common task types for testing
receptive skills
Common task types for testing
receptive skills
! multiple choice
! short answers
! gap fill
! information transfer
! multi...
Common task types for testing
receptive skills
! short answers
! multiple choice
! gap fill
! multiple matching
! informat...
Short answers

Pros
!  each student can
complete an item
according to their
abilities
!  no distractors
!  links reception...
Multiple choice

Pros
!  very familiar format
!  quick to complete
!  easy to mark
!  very objective
!  high discriminatio...
Tips for writing good distractors
"  Make sure there is only ONE correct answer.
"  They should be the same form as the ke...
Gap fill

Pros
!  familiar format
!  easy to create
!  relatively easy to mark

Cons
!  difficult to predict all
alternati...
Multiple matching

Pros
!  clear task, easy to do
!  easy to mark

Cons
!  errors duplicate as
items aren’t
independent of...
Information transfer

Pros

Cons

!  integrated ! real-life
skill

!  two or more skills
tested at once

!  involving and
...
Note-taking, dictation,
transcription
Pros

Cons
Open comprehension
questions
Pros
!  requires detailed output
! better differentiation
between students?
!  integrated: te...
Thank you!
zoltan.rezmuves@digimedia.hu
consonantvoiced.blogspot.com
2.3 tests of receptive skills: workshop CTS-Academic
Upcoming SlideShare
Loading in …5
×

2.3 tests of receptive skills: workshop CTS-Academic

858 views

Published on

The ppt. from the workshop on day 2 (TEAM 2014.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

2.3 tests of receptive skills: workshop CTS-Academic

  1. 1. KAYSERI, 29-31 JANUARY 2014 Tests of receptive skills developing tests for Reading and Listening
  2. 2. Aims of the session ! challenges in testing for comprehension ! what sub-skills are to be tested ! selecting texts ! writing items ! task types: pros and cons
  3. 3. What makes testing comprehension challenging?
  4. 4. What makes testing comprehension challenging? Pros Cons !  relatively easy to create if texts are available !  no observable learner behaviour !  input can control learner output !  familiarity and interest may affect success !  objective marking is normally possible !  if a learner fails, difficult to know why they failed !  are we only testing what we intend to test?
  5. 5. receptive skills can only be tested indirectly language performance ! inferences to comprehension abilities
  6. 6. The process of writing a test set aims select texts write tasks
  7. 7. Sub-skills for reading
  8. 8. Sub-skills for listening
  9. 9. Selecting texts !  a representative sample of text types learners are likely to encounter !  appropriate extent !  texts with high information content !  clear structure !  neutral but interesting themes !  avoid relying on general knowledge !  avoid culturally loaded texts !  avoid texts learners have already read
  10. 10. Item writing checklist "  text and item grammatically correct "  English natural and appropriate "  item achieves test objective "  the intended sub-skills needed to answer item "  one correct key (or a limited number of responses) "  open answers within student abilities "  open answers with valid and reliable marking "  distractors all work "  item economical and practical "  key complete and correct
  11. 11. Common task types for testing receptive skills
  12. 12. Common task types for testing receptive skills ! multiple choice ! short answers ! gap fill ! information transfer ! multiple matching ! for both reading and listening ! note-taking, dictation or transcription ! for listening only
  13. 13. Common task types for testing receptive skills ! short answers ! multiple choice ! gap fill ! multiple matching ! information transfer ! for both reading and listening ! note-taking, dictation or transcription ! for listening only What are the pros and cons of each task type?
  14. 14. Short answers Pros !  each student can complete an item according to their abilities !  no distractors !  links reception to controlled production Cons !  need to decide if answer length is specified !  unpredictable alternative answers !  difficult to phrase stem to determine answer !  encourages verbatim quoting – comprehension?
  15. 15. Multiple choice Pros !  very familiar format !  quick to complete !  easy to mark !  very objective !  high discrimination index Cons !  difficult to write good distractors !  need to produce one correct answer only !  temptation to use for every purpose !  may encourage guessing
  16. 16. Tips for writing good distractors "  Make sure there is only ONE correct answer. "  They should be the same form as the key. "  They should be related logically to the key. "  Key should not stand out – similar length and level. "  Avoid synonyms so they can’t be eliminated at once. "  Keep distractor extent and repetition to a minimum. "  Don’t pair up two distractors (these focus attention). "  Check for unintended clues (e.g. an … ! vowel). "  Avoid illogical or obviously impossible distractors. "  If nobody chooses a distractor, it isn’t a distractor!
  17. 17. Gap fill Pros !  familiar format !  easy to create !  relatively easy to mark Cons !  difficult to predict all alternative answers !  some items can be answered without reading the whole item
  18. 18. Multiple matching Pros !  clear task, easy to do !  easy to mark Cons !  errors duplicate as items aren’t independent of one another !  can be solved by elimination (where there are no distractor items) !  encourages guessing
  19. 19. Information transfer Pros Cons !  integrated ! real-life skill !  two or more skills tested at once !  involving and challenging task !  difficult to write tasks that limit the number of variables !  difficult to mark
  20. 20. Note-taking, dictation, transcription Pros Cons
  21. 21. Open comprehension questions Pros !  requires detailed output ! better differentiation between students? !  integrated: tests comprehension AND production Cons !  answer extent is unspecified
  22. 22. Thank you! zoltan.rezmuves@digimedia.hu consonantvoiced.blogspot.com

×