The document provides guidelines for designing tests. It discusses important considerations when writing a test such as determining the purpose and scope of the test. It also covers different types of test items like multiple choice questions, matching, true/false, and open questions. For each type of item, it provides examples and discusses what skills the items can be used to test. The document emphasizes writing clear, unambiguous questions and avoiding clues that make the correct answer obvious without real knowledge of the subject matter.

1. Guidelines
in
Designing Test

2. Guidelines in Designing
Test
What are some of the
considerations before writing a test?
1. What is the purpose of the test?
2. What do we want to test?

3. Most teachers waste their time
by asking questions which are
intended to discover what a pupil
does not know whereas the true
art of questioning has for its
purpose to discover what the
pupil knows or is capable of
knowing.
-- Albert Einstein

4. What do we want to test?
 Proficiency ?
 Achievement ?
 Diagnosis ?
 Prognosis ?

5. Test types in language
assessment
Multiple Choice
Matching
Cloze Transformation
Open questions
Gap-filling
Error Correction
True/False Etc…etc

6. Multiple Choice
Choose the correct word to complete the sentence.
Cook is _______________ today for being one of
Britain’s most famous explorers.
(A) recommended
(B) reminded
(C) recognised
(D) remembered
Can be used to test:
grammar, vocabulary, reading, listening etc

7. Complete the sentence.
Check the exchange _____________ to see how
much your money is worth.
Can be used to test:
Grammar and vocabulary
Very effective at testing listening for specific words.
Gap-filling

8. Complete the second sentence so that it has the same
meaning as the first.
“Do you know what the time is, John?” asked Dave
Dave asked John _______(what)______ it was.
Effective for testing:
Grammar and understanding of form.
But not appropriate to test skills like reading
or listening.
Transformation

9. Match the word on the left to the word with the opposite meaning.
fat . . old
young . . tall
dangerous . . thin
short . . Safe
Most often used to test vocabulary
Matching

10. Complete the text by adding a word to each gap.
Ours was the marsh (1) ________, down by the river, (2) ________, as the
river wound,(3) _________ miles of the sea. (4) _______ first most vivid and
(5) _______ impression of the identity (6) ________ things, sees to me (7)
_______have been gained on (8) ________ memorable raw afternoon
towards (9) _________. At such a time (10) _______ found out for certain,
(11) ________ this bleak place overgrown (12) _______ nettles was the
churchyard ...
These are the words to choose from:
I were that My to with a of broad twenty and evening country ...
More integrative
- have to process the components of the language simultaneously
- a good indicator of overall language proficiency
Very effective for testing grammar, vocabulary and intensive
reading.
Cloze

11. Decide if the statement is true or false.
1. England won the World Cup in 1996. T / F
2. ……... T / F
3. ……… T / F
Mostly used to test:
listening and reading comprehension.
True/False

12. Answer the questions (after reading or listening …)
1. Why did John steal the money?
Very useful for testing grammar and vocabulary as
well as listening and reading.
Open Questions

13. Find the mistakes in the sentence and correct them.
Manchester United was the better team on the night.
Useful for testing any of the four skills.
But less useful for testing grammar or vocabulary.
Error corrections

14. • Translation
• Dictations
• Essays
• Sequencing / ordering of
words / phrases
• Sentence construction
Etc. …Etc

15. Types of Test Items
ϕ AlternateAlternate
ChoiceChoice
ϕ Multiple ChoiceMultiple Choice
ϕ Matching
ϕ Completion/Cloze
ϕ Short-answer
ϕ Structured Essay
ϕ Extended Essay
SELECT TYPE SUPPLY TYPE
Objective Subjective

16. Goals for Our Session
 Understand what makes for
“good” test items & how to write
them.
 Focus: Multiple-Choice
Questions (MCQ)

17. MULTIPLE-CHOICE
ITEMS
i. Terminology …
ii. Types of MCQ …
iii. Writing MCQ

18. On the map, each contour line represents
20 feet of elevation. What is the elevation
at point X ?
A. 240 feet
B. 250 feet
C. 280 feet
D. 300 feet
Anatomy of a MCQAnatomy of a MCQ

19. 1. Single Correct Answer variety
2. Best Answer variety
3. Negative variety
4. Combined Response variety
Types ofTypes of
MMCQCQ

20. Types of Multiple-Choice ItemsTypes of Multiple-Choice Items
The Single Correct Answer VarietyThe Single Correct Answer Variety
Example 1:
6 + 3 =
(A) 2
(B) 3
(C) 9
(D) 18
Key is (C)
Example 2:
Reliability refers to the
(A) variation in scores.
(B) accuracy of scores.
(C) interpretation of scores.
(D) consistency of scores.
Key is (D)

21. Types of Multiple-Choice ItemsTypes of Multiple-Choice Items
The Best Answer VarietyThe Best Answer Variety
Example 3
The best title for today’s session is
A. “Assessment in the Classroom.”
B. “Introduction to Measurement.”
C. “Tips for Developing Quality Multiple-Choice Tests.”
D. “Strategies for ImprovingClassroom Testing.”
Key is C.

22. Example:
Which of the following is not a benefit derived from
preparing a test blueprint?
A. Exam questions will cover relevant course material.
B. Improved reliability
C. Improved validity
D. Emphasis of topics on exam is appropriate.
Key is B
Types of Multiple-Choice ItemsTypes of Multiple-Choice Items
The Negative VarietyThe Negative Variety

23. Types of Multiple-Choice ItemsTypes of Multiple-Choice Items
The Combined-Response VarietyThe Combined-Response Variety
Example:
For a test to be valid, it must
I. be internally consistent.
II. accurately measure the construct.
III. include items of various difficulty levels.
A. I only
B. III only
C. I and II only
D. II and III only
Key is C.

24. WRITING
MULTIPLE-CHOICE ITEMS

25. How to write a good MCQ?
1. WHAT DO YOU WANT TO TEST?
– you should be able to answer this question clearly
Example:
 I want the students to recognize that present
perfect form is required when a phrase with “since”
shows that an action is still going on.
 I also want to make sure that the students can
produce the correct form of the present perfect for
the 3rd
person singular

26. 2. FRAMING THE QUESTION
– question must test exactly what you want to test
and only what you want to test.
– you have to make sure that answering the question
correctly doesn’t require some other extraneous
knowledge that is not part of what you want to test
(can be difficult!)
Eg: Choose the correct form for the verb in brackets:
He ________(live) in Chile since 1994.
– Problem! This could be an acceptable answer:
He had lived in Chile since 1994
– Solution?
• do not give this answer as an option, or
• make the stem more appropriate and unambiguous, like
He’s very happy in Chile. He ______(live) in Chile
since 1994.

27. 3. SPECIFYING THE CORRECT ANSWER
– This is only easy if the stem has been framed
correctly.
– Ideally the correct answer should be the only
possible answer.
– You need to list all the possible correct
answers and then choose the clearest and
most obvious
– eg: He has lived in Chile since 1994.
He’s lived in Chile since 1994

28. 4. CREATING DISTRACTORS
– Creating “wrong answers” – EASY?
– Creating good distractors is the hardest but most
important part of writing a good MCQ.
• Remember the two issues that you want to test
i. recognize the present perfect is required.
ii. form the 3rd
person present perfect correctly
– You have to think/assume what the student will answer
if he/she does not recognize (i) ie misses the
importance of the word “since”, and/or (ii) unable to
form correctly.
– Then some of the possible distractors are:
– eg: He lives in Chile since 1994.
He have lived in Chile since 1994.
He live in Chile since 1994.

29. 5. THE COMPLETE QUESTION
Choose the correct form for the verb in brackets:
He’s very happy in Chile. He _____(live)
in Chile since 1994.
(A) He have lived in Chile since 1994.
(B) He has lived in Chile since 1994.
(C) He lives in Chile since 1994.
(D) He live in Chile since 1994.

30. GUIDELINES
FOR
WRITING MCQ

31. Are you test wise?
– Are you skilled at looking for clues from
item construction flaws?
– You are test wise if you can select the
item keys based on clues given in the
item without knowing the content.

32. Test Wiseness
Try this
A content-free test!
-10 nonsense questions-
Refer to Handout A

33. • Please work on your own.
• Try out your exam technique to pick the most
likely option for each of the following questions
• Then try to work out why you made your selection.
Please jot down numbers 1-10
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Answers
Next slide

34. How do you defeat the “test-wise”
strategies of students who don’t
study?
Be aware of the general
“rules of thumb”
that students use to
guess on MC exams.

35. Some Rules of Thumb
• Pick the longest option
• When in doubt, pick ‘C’ or ‘B’
• Never pick an answer which uses the words ‘always’
or ‘never’ in it
• If in doubt, guess
• Pick the scientific-sounding answer
• Do not pick an answer which is too simple or
obvious
• Pick a word which you remember was related to the
topic
• If there are two answers which express opposites,
pick one or the other and ignore other options

36. A Better Way …
• design test items that are not
easily guessed without good
studying
-NO clues!

37. WRITING
MULTIPLE-CHOICE ITEMS
DOs and DON’Ts
Refer to Handout B

38. WRITING MCQ
Keep it simple
Choose one central
relevant problem
Write the stem to ask a
question (a meaningful
stem)
Use positive stems
Arrange options
logically
Keep it simple
Choose one central
relevant problem
Write the stem to ask a
question (a meaningful
stem)
Use positive stems
Arrange options
logically
 Make sure distractors are
incorrect yet plausible
Make sure only one
correct or best response
to stem.
Write independent items
Write items at the proper
level of difficulty and
appropriate reading level.
Item laid out in a clear
and consistent manner
 Make sure distractors are
incorrect yet plausible
Make sure only one
correct or best response
to stem.
Write independent items
Write items at the proper
level of difficulty and
appropriate reading level.
Item laid out in a clear
and consistent manner
DO!DO!

39. Х Write a longer or more
detailed correct answer
Х Use clues linking the stem
and answer, e.g., are/is or
a/an
Х Use give-away clues,
e.g., always or never
Х Mix concepts in distractors,
Х Write a longer or more
detailed correct answer
Х Use clues linking the stem
and answer, e.g., are/is or
a/an
Х Use give-away clues,
e.g., always or never
Х Mix concepts in distractors,
Х Write mutually inclusive
options
Х Allow more than one right
answer
Х Use multiple T-F, or
all/none of the above
Х Write mutually inclusive
options
Х Allow more than one right
answer
Х Use multiple T-F, or
all/none of the above
DON’T! DON’T!
WRITING MCQ

40. Identifying Flawed MCQ
…MCQ…
Refer to Handout C

41. Summary:
Poorly designed questions may:
• Give away clues to the answer
• Fail to test the skills required by the intended
learning outcomes
• Contain “implausible distracters” (obviously
wrong answers) which can be eliminated by
students with only limited knowledge
• Encourage rote learning
• Confuse or frustrate students with sound
understanding

42. Suggested procedures for writing a test
• Decide what kind of test it is going to be (achievement, proficiency)
• Write a list of what the test is going to cover
• Think about the length, layout and the format
• Find appropriate texts
• Weigh the sections according to importance/time spent etc
• Write the questions
• Write the instructions and examples
• Decide on the marks
• Make a key
• Write a marking scheme for the less objective questions
• Pilot the test
• Review and revise the test and key
• After the test has been taken, analyse the results and decide what
can be kept/rejected

43. Review of Test Items
– to check if an item is worth retaining -
1. Consider the item as a whole and whether
a. it measures knowledge or a skill component
which is worthwhile and appropriate for the
examinees who will be tested;
b. there is a markedly better way to test what
this item tests;
c. it is of the appropriate level of difficulty for the
examinees who will be tested.

44. 2. Consider the stem and whether it
a. presents a clearly defined problem or task
to the examinee;
b. contains unnecessary information;
c. could be worded more simply, clearly or
concisely.

45. 3. Consider the options and whether
a. they are parallel in structure;
b. they fit logically and grammatically with the
stem;
c. they could be worded more simply, clearly
or concisely;
d. any are so inclusive that they logically
eliminate another more restricted option
from being

46. 4. Consider the key and whether it
a. is the best answer among the set of
options for the item;
b. actually answers the question posed in the
stem;
c. is too obvious relative to the other
alternatives (i.e., should be shortened,
lengthened, given greater numbers of details,
made less concrete).

47. 5. Consider the distractors and whether
a. there is any way you could justify one or
more as an acceptable correct answer;
b. they are plausible enough to be
attractive to examinees who are misinformed
or ill-prepared;
c. any one calls attention to the key (e.g.,
no distractor should merely state the reverse
of the key or resemble the key very closely
unless another pair of choices is
similarly parallel or involves opposites).

48. •“I made my own
assessment of my life,
and I began to live
It . That was freedom.” –
Fernando Flores

49. “To teach - it is to learn twice.”