The document provides guidance on creating effective exam questions. It discusses the importance of asking higher-order thinking questions that go beyond simple recall. Various types of questions are examined, including their suitability for different cognitive levels. Best practices for writing multiple choice, essay, and other exam questions are outlined. These include using plausible distractor options, a direct question format, emphasizing higher-level thinking, and ensuring questions have a single correct answer. The document also provides examples of lower and higher-order thinking questions and guidelines for creating novel essay questions that require students to assess problems and derive solutions.

1. JAMILAH HAMID

2. “Asking students to think at
higher levels, beyond simple recall is
an excellent way to stimulate
students’ thought processes.
Different types of questions require
us to use different levels of
thinking”
-- Teacher Vision 2004

3.  Identify level of Bloom taxonomy of a
question
 Create an effective questions
 Evaluate question based on Bloom taxonomy

4. Writing effective exam questions includes:
 identifying what to assess based on the
topics;
 selecting the appropriate type of question to
evaluate the objectives;
 creating a clear question;
 Setting an appropriate time for each
question (eg. 90sec for each obj questions).
 Giving an appropriate marks for each
question.
 Arranging the question from easy to difficult
 Editing the final product.

5. Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Remembering
Understanding
Applying
Analyzing
Evaluating
Creating
1956 - Original 2001-Revised
High level
thinking
order
Low level
thinking
order

6. Problem Solving (HOTS)
Analyze
Appraise
Argue
Assemble
Assess
Compose
Conclude
Construct
Create
Criticize
Debate
Defend
Discriminate
Distinguish
Estimate
Evaluate
Examine
Formulate
Illustrate
Infer
Inspect
Interpret
Judge
Justify
Manage
Modify
Organize
Plan
Predict
Prepare
Propose
Question
Rate
Relate
Recognize
Score
Select
Solve
Support
Test
Value
Write

7. Question Types verses Cognitive Levels of
Learning
Knowledge
Comprehension
Application Analysis
Evaluation
Creating
Multiple Choice
(MC)
True/False (TF)
Structure ( short
essay)
MC
Short Answer
Structure
Essay
MC
Structure
Essay

8. It’s time for mental exercise

11.  Use Plausible Distractors (wrong-response options)
 Only list plausible distractors, even if the number of options
per question changes
 Write the options so they are homogeneous in content. Eg
“Learning is a mental operation that takes place when
information enters through the senses, undergoes mental
manipulation, is stored and is finally used”
The above statement belong to which perspective of learning
theory ?
A cognitive.
B constructive.
C objective.
D behavior.

12.  Use a Question Format
 Experts encourage multiple-choice items to be
prepared as questions (rather than incomplete
statements)
Incomplete Statement Format:
The capital of California is in ________
Direct Question Format:
In which of the following cities is the capital of
California?

13.  Emphasize Higher-Level Thinking
 Use memory-plus application questions. These
questions require students to recall principles, rules
or facts in a real life context.
 The key to preparing memory-plus application
questions is to place the concept in a life situation or
context that requires the student to first recall the
facts and then apply or transfer the application of
those facts into a situation.

14.  Memory Only Example
Which description best characterizes whole foods?
a. orange juice
b. toast
c. bran cereal
d. grapefruit
 Memory-Plus Application Example
Sally’s breakfast this morning included one glass of orange juice (from
concentrate), one slice of toast, a small bowl of bran cereal and a
grapefruit. What “whole food” did Sally eat for breakfast?

15. Ability to Interpret Cause-and-Effect Relationships – HOTS
Example : Why does investing money in common stock protect against loss
of assets during inflation?
a. It pays higher rates of interest during inflation.
b. It provides a steady but dependable income despite economic
conditions.
c. It is protected by the Federal Reserve System.
d. It increases in value as the value of a business increases.
Ability to Justify Methods and Procedures - HOTS
Example: Why is adequate lighting necessary in a balanced aquarium?
a. Fish need light to see their food.
b. Fish take in oxygen in the dark.
c. Plants expel carbon dioxide in the dark.
d. Plants grow too rapidly in the dark.

16.  Keep Option Lengths Similar
 Avoid making your correct answer the long or
short answer
 Avoid Clues to the Correct Answer
 Avoid answering one question in the test by
giving the answer somewhere else in the test
 Avoid extremes – never, always, only
 Avoid nonsense words and unreasonable
statements

17.  Avoid Negative Questions
 31 of 35 testing experts recommend avoiding negative questions •
 Students may be able to find an incorrect answer without knowing
the correct answer
Eg.
Which of the following is INCORRECT objective based on the Dynamic
Instructional Design (DID) model?
A. The student can be able to list down at least three
phases in TIP model correctly.
B. The student can be able to choose the most appropriate
teaching strategies when designing instructional
technologies.
C. The student can be able to discuss advantages of e-
learning correctly.
D. The student can be able to explain briefly three types of
teaching strategies correctly.

18. If wants to use, the negative word should be at the
end
Eg :
The conditions below are essential for effective
technology uses EXCEPT
A. Adequate hardware, software & media
B. Time to plan resources
C. Special needs of students
D. Planning for technology use

19.  Use Only One Correct Option (Or be sure the best option is
clearly the best option)
 The item should include one and only one correct or
clearly best answer
 With one correct answer, alternatives should be
mutually exclusive and not overlapping
 Using MC with questions containing more than one right
answer lowers discrimination between students

20.  Use Only a Single, Clearly-Defined Problem and
Include the Main Idea in the Question
 Students must know what the problem is without having
to read the response options
 Avoid the “All the Above” Option
 Students merely need to recognize two correct options
to get the answer correct
 Avoid the “None of the Above” Option
 You will never know if students know the correct answer
 Don’t Use MC Questions When Other Item Types
Are More Appropriate
 limited distractors or assessing problem-solving and
creativity

21. Create one (1) high level objective
question for your subject. ( 5 mins)

23.  Criteria essay question based on Stalnaker's
(1951) definition:
1. Requires students to compose rather than select
their response.
2. Elicits student responses that must consist of more
than one sentence.
3. Allows different or original responses or pattern of
responses.
4. Requires subjective judgment by a competent
specialist to judge the accuracy and quality of
responses.

24. Example A:
List the 7-step path to making “ethical decisions.” List them
in their correct progressive order.
Is it a good essay question?
Does the question meets the above criteria?
Is it a high level thinking question?

25.  Example A does not meet the criteria for effective
essay questions for the following reasons:
 The question does not require students to use
complete sentences or more than one sentence.
Students may respond to Example A by simply listing
the name and number for each step.
 Depending on where students learned the steps, all
the responses they give to the essay question may be
identical. Students will basically repeat the 7-step
path word for word. The question does not allow for
original responses or response patterns.

26. Explaination
 Although the question does require a list of steps, it
does not require systematic and complex thinking
about the ideas.
 The most effective essay questions require deep and
original thinking of a student. Effective essay
questions elicit a depth of thought from students
that can only be judged by someone with the
appropriate experience and expertise in the content
matter.

27. Same topic but asked in a different way
Example A
List the 7-step path to
making “ethical decisions.”
List them in their correct
progressive order.
Explain B
In what ways a person’s
failure to apply step 5 of
the seven-step path for
making ethical decisions
will impact his or her ability
to make ethical decisions.
Provide an example that
illustrates this impact.

28. Compared to example A, example B
 Not only do students need to compose a response, but they
also need to write several sentences in order to provide an
answer that sufficiently meets the demands of the given
task.
 Moreover, the task given to students in the essay question
is constructed in such a way that it allows for different
responses and response patterns.

29. Compared to example A, example B
 Students can provide a wide variety of examples to
illustrate the impact, and they can structure their
response in various ways.
 They could give the example first and then provide an
explanation, or they could state the impact first and then
provide an example.
 Students could also go back and forth between their
explanation and their example as they write their response

30.  Not all essay questions assess higher-order thinking skills.
Sometimes essay questions often simply assess recall.
What are the major advantages
and limitations of
implementing blended learning
in STEM (Science, Technology &
Mathematics) subject?
• assesses recall of factual
knowledge.
Given their advantages and
limitations, should blended learning
implemented in STEM subjects? In
answering this question provide
brief discussion of the major
advantages and limitations blended
learning. Clearly state whether or not
you think blended learning should be
implemented in these subjects and
justify your answer.
• recall facts, but also to make an
evaluative judgment, and to
explain the reasoning for the
judgment

31. Guideline 1: Clearly define and situate the task within a
problem situation.
 vague or open essay question can lead students to write
response on an indefinite number of pages due to
numerous interpretations.
Eg:
In designing educational games, Malone and Lepper
proposed a guideline to motivate and aid learners in
experiencing well deserved satisfaction. Discuss the
guideline in designing educational games.

32.  Improved version
In designing educational games, Malone and Lepper proposed a
guideline to motivate and aid learners in experiencing well
deserved satisfaction. One of the elements is to provide
challenges in the games. Propose two challenges that can be
embedded in the educational game courseware for the following
objectives:

33.  In essence, an effective essay question produces valid
evidence of the degree to which students have achieved
the intended learning outcome.
 However, limiting the subject matter for a given essay
question depends on different factors
 what material was introduced to the students in the
course.
 hierarchy of knowledge related to the essay question
 What parts of the knowledge were emphasized in class.

34.  Guideline 2. Create a novel problem situation to confront
students with a new context requiring them to assess the
situation and derive an acceptable solution by using
knowledge of the relevant subject matter, and reasoning
skills.

35. Eg. GreenLive is a software house company which focuses on
producing health and fitness software. For 2017, GreenLive
aims to release a series of mobile apps named GreenFit for
flexible fitness program. The completed version is expected to
incorporate the following modules:
 Fitness diagnosis
 Exercise program based on fitness level
 Fitness monitoring
 Diet program
 Calories monitoring
You have been assigned as a team leader for GreenFit
development project and Extreme Programming (XP) was
chosen as a process model. Due to the popularity of the health
& fitness topic these days, and many other applications are
expected to be released in the market, you need to plan the
production of your product wisely so that it can compete and
beat the market sale.

36. a. Based on the description above, illustrate your
development plan based on XP activities and principles
using appropriate diagram, table or chart. Your
illustration must include release plan which consist of
number of release, activities involved, cycle or iteration,
time-frame and release date.
b. State modules or functions for every release which you
had planned on your answer in question (1) and explain
why those modules was divided in that way.
c. Based on your opinion, is XP model suitable to be used as
a process model for GreenFit development or not?
Justify your reason.

37. Guideline 3: Don't give students a choice of questions to
answer.
 Disadvantages of giving students a choice.
 Some students will waste time trying to decide which
questions to answer.
 will not know whether all students are equally
knowledgeable about all the topics covered on the test.
 The exam could be unfair due to some questions are
likely to be harder than others thus, decreases the
validity and reliability of the exam.

38. Guideline 3: Give adequate time for students to answer.
 How to estimate?
 Write out the correct answer yourself.
If you can answer the question in ten minutes, students
will probably need twenty to thirty minutes. Use these
estimates in determining the number of questions to ask
on the exam.

39. Task : 20 minutes (groups of four) .
Referring to exam paper of Software Project Management, you
are required to
a. Identify the level of bloom taxonomy for each question
b. Evaluate question 6 whether it is an effective question or
not. Justify your answer

40. Task : 20 minutes
Get into groups of four, create 3 essay questions (HOT) for
your subject based on Bloom’s taxanomy.

41. “A good teacher makes you
think even when you don’t
want to.”
(Fisher, 1998, Teaching Thinking)

42. Thank You