education evaluation test construction

2. ANDRAGOGICAL SKILLS
CONSTRUCTING TESTS
PRESENTED BY
QAZI ABDUL GHAFOOR
M.Phil Edcuation
Secondary School Teacher
0092-332-1764856

3. TEST
A test is an instrument that follows a systematic
procedure for measuring a sample of behavior by
posing a set of questions.

4. TWO MAJOR TYPES OF TESTS
1. Structured Response / Select items/ objective
items
a. Alternate Response,
b. Multiple-choice,
c. Matching
2. Constructed Response / Supply items
a. Essay,
b. Short Answer,
c. Completion

5. EXAMPLES
A.KNOWLEDGE LEVEL
1. Which of the following does not belong with the others?
a. Aptitude tests
b. Personality tests
c. Intelligence tests
d. Achievement tests
2. Which of the following measures involve nominal data?
a. the test score on an examination
b. the number on a basketball player’s jersey
c. the speed of an automobile
d. the class rank of a college student

6. EXAMPLES
B. COMPREHENSION LEVEL
1. A psychologist monitors a group of nursery-school children, recording
each
instance of altruistic behavior as it occurs. The psychologist is using:
a. case studies
b. The experimental method.
c. naturalistic observation
d. The survey method.
2. In a study of the effect of a new teaching technique on students’
achievement test scores, an important extraneous variable would be
the students:
a. Hair color.
b. athletic skills
c. IQ scores.
d. sociability

7. C. APPLICATION LEVEL
1. The results of Milgram’s (1963) study imply that:
a. In the real world, most people will refuse to follow orders to
inflict harm on a stranger.
b. Many people will obey an authority figure even if innocent
people get hurt.
c. Most people are willing to give obviously wrong answers when
ordered to do so.
d. Most people stick to their own judgment, even when group
members unanimously disagree.
2. What is your chance of flipping 4 head or 4 tails in a row with a fair
coin (i.e., one that comes up heads 50% of the time)?
a. .0625
b. .125
c. .250
d. .375

8. D. PROBLEM SOLVING:
active efforts to discover an underlying process leading to
achievement of a goal,
for example,
the series completion,
analogies, and
transformation problems.
Examples
1. The water lilies on the surface of a small pond double in area
every 24 hours. From the time the first water lily appears until
the pond is completely covered takes 60 days. On what day is
half of the pond covered with lilies?

9. Characteristics of Good Tests
 Validity – the extent to which the
test measures what it intends to
measure
 Reliability – the consistency with
which a test measures what it is
supposed to measure
 Usability – the test can be
administered with ease, clarity
and uniformity

11. Examples of Student Activities and
Verbs for Bloom’s Cognitive Levels
Table 2.1 in Jacobs & Chase (1992:19)
Bloom’s Cognitive
Level
Student Activity Words to Use in Item
Stem
Analysis Breaking material down
into its component parts to
see interrelationships/
hierarchy of ideas
Differentiate,
compare/contrast,
distinguish ____from
____, how does
____relate to ___, why
does ____work
Synthesis Producing something new or
original from component
parts
Design, construct, develop,
formulate, imagine,
create, change, write a
poem or short story
Evaluation Making a judgment based
on a pre-established set of
criteria
Appraise, evaluate,
justify, judge, which would
be better?

12. Tips in Preparing the Table of
Specifications (TOS)
 The following array shows the most common
questions types used at various cognitive levels.
Factual Knowledge Application Analysis and
Evaluation
Multiple Choice
True/False
Matching Type
S. Completion
Short Answer/RRT
Multiple Choice
Short Answer
Problems
Essay
Multiple Choice
Essay

13. CONSTRUCTING OBJECTIVE TEST ITEMS
Alternative Response Items:
is the one that offers two options to choose from.
Examples
a. True or false,
b. Right or wrong,
c. Correct or incorrect,
d. Yes or no, Agree or disagree,
Common use
a. To test the knowledge of grammar
b. Appropriate use of ‘tenses’
c. Contextual meaning of words or spelling mainly
of words that sound alike.

14. TRUE-FALSE
measuring the examinee’s ability to identify the
a. correctness of statements of fact,
b. definitions of terms,
c. statements of principles,

15. SUGGESTIONS FOR CONSTRUCTING
TRUE-FALSE ITEMS
1. Avoid unimportant statements.
2. Avoid broad general statements.
3. Avoid the use of negative statements (When a
negative word used, it should be underlined or put
in italics form)
4. Avoid complex sentences.
5. Avoid including two ideas in one statement
6. Statements are written concisely without more
elaboration than necessary.
7. Statements are NOT quoted exactly from text.

16. 8. Avoid using of specific determiners which
usually gives a clue to the answer.
False = all, always, never, every, none, only
True = generally, sometimes, usually, maybe, often
Example
1. Repetition always strengthens the tendency for a response
to occur. (Using "always" usually means the answer is
false.)
2. The mean, median, and mode are measures of central
tendency, whereas the standard deviation and range are
measures of variability. (Express a single idea in each
statement.) e.g.“The mean and standard deviation are
measures of central tendency.”

17. MULTIPLE CHOICE QUESTIONS
1. Stem: presents the problem
2. Keyed Response: correct or best answer
3. Distracters: appear to be reasonable answers to the
examinee who does not know the content
4. Options: include the distracters and the keyed
response.

18. CHARACTERISTICS OF MULTIPLE
CHOICE QUESTIONS
 MCQ considered better than all items that can be
scored objectively
 Most flexible of the objective type items
 An ingenious and talented item writer can
construct an MCQ.
 measure a variety of educational objectives from
rote learning to more complex learning outcomes
 chances of guessing are significantly
 Desirable difficulty level
 Ability to discriminate between high and low
performers
 Effective distracters

19. ADVANTAGES OF MCQ
 Wide sampling of content
 The problem or the task is well structured or clearly
defined.
 Flexible Difficulty Level
 Efficient scoring of items
 Objective scoring
 Provide scores easily understood and transformed
as needed

20. LIMITATIONS OF MCQ
 It takes time to construct MCQ.
 susceptible to guessing (by rejection method)

21. RULES FOR CONSTRUCTING MULTIPLE
- CHOICE ITEMS
 Be sure that the stem clearly formulates a problem.
 Stem should be written either in direct question form or
in an incomplete statement form.
 The stem of the item should present only one problem.
 keep options as short as possible
 Avoid the use of negative words in the stem
 Do not focus on rote memory of the text
 Use reasonable distracters
 No clues
 The correct answer should appear at each position (a or
b or c or d)
 Avoid using ‘none of the above’, ‘all of the above’, both a
and b etc

22. Example:
What is the effect of releasing a ball in positive gravity?
a) It will fall “down.” correct
b) It will retain its mass. true but unrelated
c) It will rise. false but related
d) Its shape will change. false and unrelated

23. 1. Sheldon developed a highly controversial theory of
personality based on body type and temperament of the
individual. Which of the following is a criticism of
Sheldon's work?
a. He was influenced too much by the Freudian psychoanalysis.
b. His rating of physique and temperament were not independent.
c. He failed to use empirical approach
d. His research sample was improperly selected.
MCQ-EXAMPLE
Better: (Eliminate excessive wording and irrelevant
information)
1. Which of the following is a criticism of Sheldon's theory of
personality?

24. The receptors for the vestibular sense are located
a. in the fovea.
b. in the brain.
c. in the middle ear.
d. in the inner ear.
Better: (Include in the stem any word(s) that might
otherwise be repeated in each option.)
The receptors for the vestibular senses are located in the
_______.
a. fovea
b. brain
c. middle ear
d. inner ear

25. Which is not a major technique for studying brain
function?
a. Accident and injury
b. Cutting and removing
c. Electrical stimulation
d. Direct phrenology
Better: (Use negatively stated stems sparingly.
When used, underline and/or capitalize the
negative word.)
Which is NOT a major technique for studying
brain function?

26. MATCHING COLUMN
Matching exercise consists of two parallel columns
with each word, number, or symbol in one column
being matched to a word, sentences, or phrase in
the other column.
CONSIST ON
premises :
Items in the column for which a match is
sought are called………… and
responses :
the items in the column from which the
selection is made are called responses.

27. Parts of the Matching Type Test
Column A
(Premise)
Column B
(Response)

28. USES OF MATCHING EXERCISES
 When you have a number of questions of the
same type (homogeneous), it is advisable to
frame a matching item.
 When you emphasize the ability to identify the
relationship between two things and a
sufficient number of homogeneous premises
and responses can be obtained, a matching
exercise seems most appropriate.
FOR EXAMPLE
 Inventions and inventers
 Authors and Books
 Scientists and their contribution

29. ADVANTAGE
to measure a large amount of related factual
material in a relatively short time.
LIMITATIONS
• Restricted to the measurement of factual
information.
• It is difficult to find out homogeneous material

30. SUGGESTIONS FOR CONSTRUCTING
MATCHING EXERCISES
• Use only homogeneous material in a single matching
exercise.
• Include an unequal number of responses and
premises
• Instruct the student that responses may be used
once, more than once.
• Keep the list of items to be matched brief on left, and
place the shorter responses on the right.
• Arrange the list of responses in logical order. Place
words in alphabetical order and numbers in
sequence.
• Ambiguity and confusion will be avoided.
• Place all of the items for one matching exercise on
they same page.

31. Examples:
Directions: Match the following.
1. Food A. Primary reinforcer
2. Psychoanalysis B. Sigmund Freud
3. B.F. Skinner C. Operant conditioning
4. Standard deviation D. Measure of variability
5. Schizophrenia E. Hallucinations

32. Examples:
Better: (Use homogenous material in matching items, and if
responses are not to be used more than once, include more
responses than stimuli.)
Match the theories in Column A with their proponents in
Column B. Write the letter of the correct answer.
Column A
1. Psychodynamic
Theory
2. Trait Theory
3. Behaviorism
4. Humanism
5. Social Learning
Theory
Column B
A. Albert Bandura
B. B. B.F. Skinner
C. Carl Rogers
D. Gordon Allport
E. Karn Horney
F. Raymond Cattell
G. Sigmund Freud

33. CONSTRUCTING
SUBJECTIVE TEST ITEMS

34. SHORT-ANSWER
 can be answered by a word, phrase, number, or
symbol.
 uses a direct question
 useful for measuring problem-solving ability
 used to measure the ability to interpret diagrams,
charts, graphs, and pictorial data.
 the question must be stated clearly and concisely.
 It require an answer that is both brief and definite.

35. ADVANTAGES OF SHORT-ANSWER ITEMS
1. easiest to construct.
2. This is used almost exclusively to measure the recall of
memorized information.
3. reduces the possibility
LIMITATIONS OF SHORT-ANSWER
i. not suitable for measuring complex learning outcomes
ii. difficult to score.

36. CONSTRUCTING SHORT-ANSWER ITEMS
• Word the item so that the required answer is both
brief and specific.
• Do not take statements directly from textbooks
• If the answer is to be expressed in numerical units,
indicate the type of answer wanted.

37. ESSAY QUESTION?
…a test item which requires a response composed by
the examinee, usually in the form of one or more
sentences, of a nature that no single response or
pattern of responses can be listed as correct, and the
accuracy and quality of which can be judged
subjectively only by one skilled or informed in the
subject.“
PURPOSES
i. to assess students' understanding of subject-matter
content.
ii. to assess students' writing abilities.

38. AN ESSAY QUESTION SHOULD MEET
THE FOLLOWING CRITERIA:
1. Essay questions require students to construct their
own answer.
2. Student responses must consist of one or more
sentences.
Example
 How do you feel about the removal of prayer from public
school system? (Not a good question)
 What do you think about the removal of prayer from
public school system? (A good question)

39. 3. No single response or single response pattern is correct.
Example
 What was the full name of the man who assassinated President
Abraham Lincoln? (Not a good question)
 State the full name of the man who assassinated President Abraham
Lincoln and explain why he committed the murder. (A good
question)
4. The accuracy and quality of students’ responses to essays must
be judged subjectively by a competent specialist in the subject.
Example
 Explain how Arabs would treat women before advent of Islam. (Not a
good question)
 As mentioned in class/textbook, list main ways women were treated
before advent of Islam. (A good question)

40. ADVANTAGES
i. An effective way of assessing complex learning
outcomes.
ii. To assess students' ability to synthesize ideas, to
organize, and express ideas and to evaluate the
worth of ideas.
iii. Allow students to demonstrate their reasoning.
iv. Allows teachers to gain insights into a student's
way of viewing and solving problems.
v. Constructed responses are closer to real life than
selected responses.(Problem solving and decision-
making are vital life competencies).

41. LIMITATIONS
i. Reduce content validity.
ii. Essay questions have limitations in
reliability.
iii. Essay questions require more time for
scoring student responses
iv. Involve greater subjectivity in scoring,

42. CONSTRUCTING ESSAY QUESTIONS
1. Clearly define the intended learning outcome to be assessed by
the item.
2. Avoid to measure those learning outcomes that are better
assessed with other kinds of assessment.
3. Focus the question.
Example
1. Less focused essay question:
Evaluate the impact of the Industrial Revolution on England.
2. More focused essay question:
Evaluate the impact of the Industrial Revolution on the family in
England.
4. Helpful Instructions: Specify the relative point value and the
approximate time limit in clear directions.

43. 5. Helpful Guidance:
State the criteria for grading
Example
 All of your responses to essay questions will be graded based on the
following criteria:
 The content of your answer will be evaluated in terms of the accuracy,
completeness, and relevance of the ideas expressed. The form of your
answer will be evaluated in terms of clarity, organization, correct
mechanics (spelling. punctuation, grammar, capitalization), and
legibility.
 Essay question, they should specify the relative point value for the
content and the relative point value for the form.

44. 6. Use several relatively short essay questions rather than
one long one.
7. Avoid the use of optional questions
Reasons:
 Students may waste time deciding on an option.
 Some questions are likely to be harder
 The use of optional questions makes it difficult to evaluate
8. Improve the essay question through preview and
review.
Preview (before handing out the essay question to the
students)
a. Predict student responses.
b. Write a model answer.
c. Ask a knowledgeable colleague to critically review the essay
question,
Review (after receiving the student responses)
a. Review student responses to the essay question.