This document discusses procedures for effective test design and writing. It covers defining test objectives, writing specifications, writing test items, editing items, and providing effective instructions. Effective instructions should be simple, brief, clear, unambiguous, and familiar to students. They should not test comprehension and should include examples for less familiar tasks. The key ingredients for good tests are clear aims, balanced assessment, thorough specifications, items that work as intended, and useful feedback.

1. Test design, test writing
KAYSERI, 29-31 JANUARY 2014

3. Aims
!
Procedures for test design
!
Writing specifications
!
Writing tests
!
Effective instructions
!
Good tests: the key ingredients

4. Procedures for test design

5. Objectives
Specifications
Item writing
Item trialling
Item revision
Assessment
scales
Validating
Handbook
Training

6. Defining our test objectives
1)
What kind of test is it going to be?
2)
What decisions do we need information for?
3)
What abilities are tested?
4)
How much detail do we need?
5)
How accurate should our results be?
6)
Is there any washback effect?
7)
What are the practical constraints?

7. Writing specifications

8. Content
Structure
Timing
Medium
Techniques
Pass / fail levels
Marking and scoring
Administration procedure

9. Writing tests

10. Test writing vs exercise writing
WHAT IS THE DIFFERENCE?
!
no support for students ! clearer
!
simple, unambiguous instructions
!
students familiar with task format and expectations
!
if possible, more than one method to test a skill
!
validation

11. Test writing: where do you begin?
!
Your own specifications, of course!
! NOT
past papers
! NOT
coursebook exercises
! NOT
other people s tests
! NOT
tests for completely different objectives

12. Test writing
specs
texts
items
edit
key

13. Test writing: feedback and editing

14. Effective instructions
Let s look at an example of test instructions.

15. Effective instructions
Read the article about the future of electric cars. Then read the statements
carefully.
Find the information about the statements in the article, and write the number of
the paragraph (1-6) which contains the information.
There are statements which are confirmed by information in the article. Circle a
for ‘true’ for these. There are statements which say something that is not
confirmed by the article. Circle b for ‘false’ for these. There are statements about
which there is information in the article. Circle c for ‘not given’ for these.
example from the draft syllabus for a B1-level
secondary school-leaving examination

16. Effective instructions
Which five of the following adjectives would you use to
describe effective test instructions?

17. detailed
simple
clear
authentic
familiar
unambiguous
brief
challenging

18. Effective instructions
!
simple, brief, clear and unambiguous
! should
! use
!
not test comprehension of instructions
Turkish if you can’t make it simple and brief
that s four adjectives - one more?

19. Effective instructions
!
simple, brief, clear and unambiguous
! should
! use
!
not test comprehension of instructions
Turkish if you can’t make it simple and brief
task should be familiar to students
! use
standardised instructions during practice

20. Familiar?
extract from the
Hungarian schoolleaving examination
2009

21. Effective instructions
!
simple, brief, clear and unambiguous
! should
! use
!
Turkish if you can’t make it simple and brief
task should be familiar to students
! use
!
not test comprehension of instructions
standardised instructions during practice
include an example where necessary
! with
less familiar formats
! where
students know different varieties of format
! where
there is a danger of misunderstanding

23. What s the task here?
Read the sentence and decide which of the options: a, b, c, or d
cannot be used to complete it.
Tourist can access the site of the castle by …
a) ferry from the sea terminal.
b) the regular local bus service.
c) foot across the bridge.
d) hired fishing boats from the village.
Will learners realise this in time?

24. Effective instructions
!
simple, brief, clear and unambiguous
! should
! use
!
Turkish if you can’t make it simple and brief
task should be familiar to students
! use
!
not test comprehension of instructions
standardised instructions during practice
include an example where necessary
! with
less familiar formats
! where
! where
!
students know different varieties of format
there is a danger of misunderstanding
use no terminology if possible

25. Effective instructions
TO SUM UP
!
simple, brief, clear and unambiguous
!
familiar
!
example
!
no terminology
AND
CHECK your instructions with a colleague!

26. clear aims
balanced
assessment
thorough
specs
GOOD
TESTS
items that
work
useful
feedback
effective
instructions

27. Thank you!
zoltan.rezmuves@digimedia.hu
consonantvoiced.blogspot.com