The document outlines effective strategies for creating evaluation questions, emphasizing the importance of question clarity and cognitive level. It provides guidance on various question types, like multiple choice and true/false, highlighting common pitfalls such as unclear wording and patterned responses. The discussion includes tips for improving distractor quality and ensuring tests are fair and reliable for students.

1. Writing Good Evaluation
Questions

2. Types of Questions
Selected Response Constructed Response
True/False Fill-in-the-blank
Matching Essay
Multiple
Choice/Response

3. e ss o f this
L
Part figuring out assessment
Part figuring out answer
More of
th is

4. (True or False)
The physical relationship
between most petroleum
products and purely (True or False)
aqueous solutions is As a general rule, oil and
generally such that water do not mix.
physical interaction and
diffusion of the two is
severely limited.
Which is easier to process for the student? Why?

5. (True or False)
The physical relationship
between most petroleum
products and purely (True or False)
aqueous solutions is As a general rule, oil and
generally such that water do not mix.
physical interaction and
diffusion of the two is
severely limited.
(A) keeps the language simple. Unless the purpose of the test is to survey the extent of
the students’ vocabulary, there is no point in using words that are unfamiliar to students
or in phrasing questions so that they are difficult to understand. Students will be
justifiably angry and frustrated if they get answers wrong because they could not
understand what was being asked rather than because they did not know the right
answer.

6. (True or False)
(True or False)
The area of a rectangle is
A 2’ x 4’ rectangle has an
found by multiplying the
area of 8ft2.
length by the width.
Which asks the student to demonstrate at a higher cognitive level? Why?

7. (True or False)
(True or False)
The area of a rectangle is
A 2’ x 4’ rectangle has an
found by multiplying the
area of 8ft2.
length by the width.
(B) Ask students to apply, rather than simply recall, information.
If students can apply the information they learned, it is a safe bet they have committed it
to memory. It does not follow, however, that simply because students have memorized
information they can also apply it. This being the case, it is better to aim questions at
application rather then simple recall.

8. 5) The number of square feet in a room nine feet long and twelve
feet wide is:
A) 3. B) 21. C) 81. D) 108. E) 144.
6) At $2.00 per square foot, what would it cost to carpet the room
described in question #5?
A) $6 B) $42 C) $162 D) $216 E) $288
Here are two successive questions on an assessment.
What is our opportunity for improvement?

9. 5) The number of square feet in a room nine feet long and twelve
feet wide is:
A) 3. B) 21. C) 81. D) 108. E) 144.
6) At $2.00 per square foot, what would it cost to carpet the room
described in question #5?
A) $6 B) $42 C) $162 D) $216 E) $288
Make sure that each item is independent.
Check questions to be sure that one question does not provide a clue to some other
question or that the answer to one question is not crucial to another.
Any student missing #5 probably missed #6. Nothing gained by linking questions…
combine.

10. A B C D
1)
2)
3)
4)
5)
6)
7)
8)
What is the answer to #8?

11. A B C D
1)
2)
3)
4)
5)
6)
7)
8)
(D) Do not establish or follow a pattern for correct responses. Regardless of
how clever an answer pattern is, some student will eventually discover it and
compromise the test results.
(You know someone has done it, or it wouldn’t be a rule!)

12. The acronym USDA stands for:
A) United States Dairy Association.
B) United States Department of Agriculture.
C) United States Department of Administration.
D) United States Dietitians’ Association.
What is our opportunity for improvement?

13. The acronym USDA stands for:
A) United States Dairy Association.
B) United States Department of Agriculture.
C) United States Department of Administration.
D) United States Dietitians’ Association.
(B) Do not include trick or trivial questions. Sometimes evaluators are tempted to ask
questions that require extended effort for correct interpretation that deal with unimportant
points. Trick or trivial questions not only reduce the validity and reliability of tests, but they
may have a powerful negative effect if they antagonize students.
This should be something important and relevant… something the participant should know
rather than looking up in a reference document.

14. How should you respond to a question from a student
once the assessment has begun?

15. Do not answer questions after the test has started unless it is done publically.
Sometimes individual students will seek further clarification of a question as they work through
the test. If additional information is provided to that student, the test results may be biased,
since the student will have had access to direct help when others did not. It is wise to make a
general announcement prior to each test concerning your reluctance to answer questions
during a test. This, plus careful proofreading of test prior to their administration, should
forestall most questions. IF it does become necessary to answer a question about some test
item, assume that other students may be equally confused about it and call everyone’s
attention to the clarification. Remember however, that interrupting students during a test
destroys their train of thought and thus should be avoided whenever possible.

16. 12) John Adams was:
A) the second president of the United States.
B) the third president of the United States.
C) the fourth president of the United States.
D) none of the above.
13) The second president of the United States was:
A) John Adams.
B) Thomas Jefferson.
C) James Madison.
D) none of the above.
How are these two questions different?

17. 12) John Adams was:
A) the second president of the United States.
B) the third president of the United States.
C) the fourth president of the United States.
D) none of the above.
13) The second president of the United States was: Stem
A) John Adams. Answer
B) Thomas Jefferson.
C) James Madison. Distractors
D) none of the above.
If the stem does its job properly, it gives the student an idea of what is sought before reading
the options. In #12, the student doesn’t know what is sought until they’ve read each option.
Furthermore, more time is consumed because the student must read the repetitive wording
– inefficient testing.
In Question #13, the stem is clear and concise and gives the student sufficient time to begin
thinking about the answer.

18. 13) The second president of the United States was:
A) John Adams.
B) Thomas Jefferson.
C) James Madison.
D) none of the above.
What could be done to improve this question?

19. 13) The second president of the United States was:
A) John Adams.
B) Thomas Jefferson.
C) James Madison.
D) none of the above.
Make distractors reasonable and plausible.
Distractors should be equally appealing to someone who is purely guessing. Someone
vaguely familiar with the early presidents would recognize the names and eliminate
option D, thereby increasing their odds of guessing a right answer from 1 in 4 to 1 in 3.

20. The area of a circle with a radius of 3 feet
is:
3’ A) 9.42 ft2.
B) 18.85 ft2.
C) 28.27 ft2.
D) 29.6 ft2.
Why are these distractors plausible?

21. The area of a circle with a radius of 3 feet
is:
= π*3
3’ A) 9.42 ft . 2
= 2π*3
B) 18.85 ft2.
= π*32
C) 28.27 ft2.
= π2*3
D) 29.6 ft2.
One way to construct good distractors is based on the type of errors most students are
likely to make. In this case, each distractor is an incorrect calculation based on different
errors.

22. 15) To make an effective presentation, the presenter should:
A) identify the audience’s needs in advance.
B) meet everyone in person after the presentation.
C) summarizes the main points in a title slide.
D) looks at the slide to focus attention on it.
What is our opportunity for improvement?

23. 15) To make an effective presentation, the presenter should:
A) identify the audience’s needs in advance.
B) meet everyone in person after the presentation.
C) summarizes the main points in a title slide.
D) looks at the slide to focus attention on it.
Make sure that unintentional clues are not provided.
Sometimes a student can eliminate an option simply because it is grammatically incorrect.
For example a stem ending with … “An” vs “a”
Incorrect tenses or forms of words can provide clues…

24. 16) “Lake effect snow storms” are most likely to occur when:
A) the temperature drops.
B) snow is created when a cold wind blows over the warmer
water in the lake causing the evaporating water to crystalize.
C) it gets cold.
D) it gets warm.
E) it has recently rained.
What is our opportunity for improvement?

25. 16) “Lake effect snow storms” are most likely to occur when:
A) the temperature drops.
B) snow is created when a cold wind blows over the warmer
water in the lake causing the evaporating water to crystalize.
C) it gets cold.
D) it gets warm.
E) it has recently rained.
Another example of unintentional clues. Distractors that are consistently
longer or shorter than the other options.
The use of “always” or “never” is usually a big giveaway, as is the user of “all
of the above” or “none of the above”

26. 18) Who was the second president of the United States?
A) John Adams
B) Warren Harding
C) Thomas Jefferson
D) James Madison
19) The second president of the United States was:
A) John Adams.
B) Warren Harding.
C) Thomas Jefferson.
D) James Madison.
Which punctuation is correct?

27. 18) Who was the second president of the United States?
A) John Adams
B) Warren Harding
C) Thomas Jefferson
D) James Madison
19) The second president of the United States was:
A) John Adams.
B) Warren Harding.
C) Thomas Jefferson.
D) James Madison.
If the stem is a complete question, it should end with a question mark. The answer and
the distractors would be capitalized, but not have periods unless they were complete
sentences.
If the stem is part of a statement that will be completed by one of the choices, it should
end with a colon. Colons precede lists and the choices constitute a list. If the choices
complete the statement, each should begin with a lower case letter and end with a
period because the stem plus the choice will make a complete statement.

28. 19) Who was the second president of the United States?
A) John Adams
B) Warren Harding
C) Thomas Jefferson
D) James Madison
What is our opportunity for improvement?

29. 19) Who was the second president of the United States?
A) John Adams
B) Warren Harding
C) Thomas Jefferson
D) James Madison
Format… Skip a line between the stem and the choices to make it easier for students to
visually separate the two components. Also keep both the stem and the choices on the
same page, if there is not enough room, move the whole question to a new page.

30. (True or False)
If every teacher were given his or her own computer,
most of the difficult problems currently limiting wider
use of computer-assisted instruction would be solved.
What is the opportunity for improvement?

31. (True or False)
If every teacher were given his or her own computer,
most of the difficult problems currently limiting wider
use of computer-assisted instruction would be solved.
“Most of the difficult problems”? How many is that? 99%? 52%?
True False items are most appropriate for lower-level cognitive skills. (Keep in mind it is a
50/50 guess) Although True-False items are often misused, it is possible to construct true-
false items that cover the analysis level of Bloom’s Taxonomy.
Other points to keep in mind when writing True/False items:
•Be sure every item is wholly true or wholly false.
•Whenever possible, avoid such terms as ‘generally’ or ‘usually’. (Although not as much as a
giveaway as “always” or “never”, they are still open to varying interpretations.)
•Be sure that items are not dependent upon insignificant facts. Make sure each item asks
something of importance and worth remembering.
•Be sure that correct items are not consistently longer or shorter than incorrect items.
•Avoid the use of double negatives, but if you use any negative at all, emphasize it.

32. Connect the state with its state bird.
___Connecticut A) Robin
___Nevada B) Mountain Bluebird
___Idaho C) Bluebird
___Ohio D) Cardinal
___Oregon E) Western Meadowlark
___Mississippi F) Mockingbird
___Texas G) Chickadee
___Wisconsin H) Lark Bunting
___Minnesota I) Common Loon
___North Dakota J) Brown Thrasher
___South Carolina K) Great Carolina Wren
___Iowa L) Eastern Goldfinch
What is our opportunity for improvement?

33. Connect the state with its state bird.
___Connecticut A) Robin
___Nevada B) Mountain Bluebird
___Idaho C) Bluebird
___Ohio D) Cardinal
___Oregon E) Western Meadowlark
___Mississippi F) Mockingbird
___Texas G) Chickadee
___Wisconsin H) Lark Bunting
___Minnesota I) Common Loon
___North Dakota J) Brown Thrasher
___South Carolina K) Great Carolina Wren
___Iowa L) Eastern Goldfinch
Matching Items are used most easily to measure low-level cognitive skills such as recall or
comprehension. Link People to dates, capitals to states, labels for charts, etc.
In this question, neither list is in alphabetical order so students must continually scan the list
if they know the answer. If the list is 1-1 match, student can guess right answers by
eliminating some options. So…

34. Connect the state with its state bird.
___Connecticut A) Bluebird
___Idaho B) Brown Thrasher
___Iowa C) Cardinal
___Minnesota D) Chickadee
___Mississippi E) Common Loon
___Nevada F) Eastern Goldfinch
___North Dakota G) Great Carolina Wren
___Ohio H) Lark Bunting
___Oregon I) Mockingbird
___South Carolina J) ) Mountain Bluebird
___Texas K) Robin
___Wisconsin L) Western Meadowlark
1. Sort both lists chronologically
2. Add 1-many options… in this case, some birds are used multiple times for different
states, and some not at all. Alternatively, have more choices than matches. This prevents
guessing by elimination. (Note this in the instructions too!)
3. Ensure all items are around one topic… (seems obvious, but again…)
4. Shorten list… usually no more than 10

35. Fill-in-the-Blank vs. Essay
How is grading different for these two items?

36. Fill-in-the-Blank vs. Essay
For completion items, write items that can be completed with a single word or short phrase.
When students are required to ‘fill in’ more than a few words, the grading of the item is no
longer black and white, and it ceases to be a completion item.

37. The first World War began in ____________.
What is our opportunity for improvement?

38. The first World War began in ____________.
State? Country? Continent? Year? Decade?
Be sure that only one word or phrase can correctly complete the sentence.

39. 21) ______________ was the second president of the United States.
22) The second president of the United States was ______________.
Which completion item is easier for the student?

40. 21) ______________ was the second president of the United States.
22) The second president of the United States was ______________.
Put blanks near the end of the sentence so the student is
guided toward the correct response.

41. The letter from the 1776 General Congress to King
George that severed all political connections between
America and Britain is known as the
_________________ ___ ___________________.
What is our opportunity for improvement?

42. The letter from the 1776 General Congress to King
George that severed all political connections between
America and Britain is known as the
_____________ ______________ _______________.
Make all blanks the same length. Making the length of the lines correspond to
the words, gives away hints for students who are guessing.

43. __________ __________ wrote _________ ___________.
What is our opportunity for improvement?

44. __________ __________ wrote _________ ___________.
No more than 2 blanks in an item. The more blanks in an item, the greater the
chance the student will be unable to determine the question.

45. Benefits and Risks of Essays
Advantages Disadvantages
1. In-depth sampling of students’ 1. Teacher fatigue,
knowledge of a specific topic. subconscious biases, and
2. Students must interpret, other extraneous variables
evaluate, and organize data; can affect grade.
draw conclusions; make 2. Essay tests inherently biased
inferences; and express in favor of students who can
thoughts coherently. write quickly, neatly, and
3. Emphasizes higher-level effectively.
cognitive skills such as 3. Reduced reliability and
synthesis and evaluation. validity because only a few
4. Easier to construct than questions are asked.
objective tests. 4. Longer time to grade.
5. Increased possibility of
bluffing.

46. M. A. Lorber and W. D. Pierce (1990). Objectives, Methods, and
Evaluation for Secondary Teaching (3rd Edition). Englewood Cliffs, NJ:
Prentice Hall.