table of specification

1. Item development for
university students 2023
national exit exam
MoE
March, 2023

2. Test construction
• Tests are important and commonly used assessment
instruments used in education.
• For tests to be valid and reliable they have to be
developed based on carefully designed plans.
• They also have to be arranged on principles of test
construction.
• A good test reflects the goals of the course.
• It is congruent with the cognitive or psychomotor
skills that the instructor wants the students to
develop

3. Writing Test Items
• The basic steps/phases in writing test items
are:
-Identifying purpose of the test
-outlining the content of the lesson
-Developing Table of Specification,/Test-Blue
Print
- Selecting appropriate test item formats, and
-Writing/Constructing relevant test items

4. General Tips About Testing
1. Length of Test
• In theory, the more items a test has, the
more reliable it is.
• On a short test a few wrong answers can
have a great effect on the overall results.
• On a long test, a few wrong answers will not
influence the results as much.
2. Clear, Concise Instructions
• It is necessary to give clear, concise
instructions.

5. Tips…
3. Mix It Up!
• It is often advantageous to mix types of items
(multiple choice, true-false, essay)
• Weaknesses connected with one kind of item
or component or in students’ test taking skills
will be minimized.
4. Test Early
• It is helpful for instructors to test early in the
term and consider discounting the first test if
results are poor.

6. Tips…
5. Test Frequently
• provides instructors with multiple sources of
information to use
• minimizing the effect of “bad days”), and gives
students regular feedback.
6. Check For Accuracy
• Instructors should be cautious about using
tests written by others.

7. Tips…
7. Proofread Exams
• On written exams, it is important to proofread
exams carefully and, when possible, have
another person proofread them.
8. Special Considerations
• It is important to anticipate special considerations
that learning disabled students or non-native
speakers may need.
9. A Little Humor
• Placing less difficult items or tasks at the
beginning of an exam can help students with test
anxiety to reduce their preliminary tension.

8. Planning
1. Cognitive Complexity
• The test questions will focus on appropriate
intellectual activity ranging from simple
recall of facts to problem solving, critical
thinking, and reasoning.
2. Content Quality
• The test questions will permit students to
demonstrate their knowledge of challenging
and important subject matter.

9. Planning…
3. Meaningfulness
The test questions will be worth students’ time
and students will recognize and understand
their value.
4. Language Appropriateness
• The language demands will be clear and
appropriate to the students.
5. Transfer and Generalizability
• Successful performance on the test will allow
valid generalizations about achievement to be
made.

10. Planning…
6. Fairness
• Student performance will be
measured in a way that does not give
advantage to factors irrelevant to
school learning; scoring schemes will
be similarly equitable.

11. Learning objectives

12. What are Learning
outcomes/Objectives
•clear statements of what a learner is expected to be
able to do, understand/think about and/or value by
the completion of a lesson.
•Are statements that describe what students will
know and be able to do after completing a course/a
program.
• Qualities of good learning outcomes
•specify the audience, behavior, context and degree
of proficiency (ABCD),

13. Learning Domains
1. The cognitive domain
• Represents intellectual abilities and skills.
• What we know
2. The Affective domain
• Represents attitudes, feeling, interest, beliefs and values.
•How we feel about what we know and do
3. Psychomotor domain
•Represents bodily movements and physical performance.
•What we do or perform

15. Levels of Cognitive domain
Original Terms New Terms
• Evaluation
• Synthesis
• Analysis
• Application
• Comprehensi
•Creating
•Evaluating
•Analyzing
•Applying
•Understanding
•Remembering
(Based on Pohl, 2000, Learning to Think, Thinking to Learn, p. 8)

16. Levels of Cognitive domain
• Remembering: Recall information such as facts,
terminology, problem solving strategies, or rules.
• Understanding: Comprehending (ability to describe and
explain the material)
• Applying: Ability to use knowledge in a new situation
• Analyzing: Ability to break down into component parts
and identify relationships
• Evaluating: Judgments about value based on criteria
• Creating: Ability to synthesize or develop a new product

17. Cognitive Domain
Sample action verbs:
• Remembering : arranges, defines, duplicates, labels,
lists, memorizes, name, order, recognize, relate, recall,
repeat, reproduce state, record, identifies, matches,
names, outlines
• Comprehension/Understanding: classify, describe,
discuss, explain, express, identify, indicate, locate,
recognize, report, restate, review, select, translate.
• Application: apply, choose, demonstrate, dramatize,
employ, illustrate, interpret, operate, practice, schedule,
sketch, solve, use, write.
• Analysis: analyze, calculate, categorize, compare,
contrast, criticize, differentiate, discriminate, distinguish,
examine, experiment, question, test.
• Evaluation: appraise, argue, assess, attach, choose,
compare, defend, estimate, judge, predict, rate, select,
support, value, evaluate.
• Synthesis/creating: arrange, assemble, collect,
compose, construct, create, design, develop, formulate,
manage, organize, plan, prepare, propose, set up, write.

18. The affective Domain
• This deals with developing learners’ feelings,
emotions, appreciations, interests, preferences,
values, and modes of adjustment.
• Affective behaviours are rarely assessed formally.
• Teachers constantly describe their students
affective characteristics, based on their informal
observations and interactions.

19. Affective Domain …..
Action verbs
• Characterization: Display, exhibit, internalize, review, revise,
conclude, resolve, judge, avoid, resist, require, manage,
resolve,
• Organization: Select, compare, decide, define, balance,
formulate, examine, clarify, systematize, create, integrate
• Valuing: Help, act, express, prefer, debate, express,
convince, argue, organize, display, believe, seek, justify,
respect, search, persuade, appreciate, initiates, invites, joins,
proposes, shares
• Responding: Conforms, allows, cooperates, contributes,
enjoys, satisfies, complies, discusses, applaud, follow,
practice, participate, volunteer, play, obey
• Receiving (Attending): Feel, sense, capture, experience,
pursue, attend, perceive, listen, notice, look, attend, be aware
of, hear, share, control, recognize
• Level of
affection

20. Psychomotor domain/skill
• It involves physical manipulation, movement,
coordination, and use of motor skills.
• It encompasses the skills that require the use of
eye-hand coordination of skeletal muscles.
• Are easier to measure, observe, and describe than
cognitive or affective behaviours.
• It has five hierarchically arranged levels.

21. Psychomotor Domain …
• Level of
skills/psychomot
or domain
Action verbs
• Naturalization: Performing the behaviors:
Naturally, effortlessly, professionally, with
ease, automatically, with poise, routinely,
spontaneously, with perfection, without strain,
• Articulation: Coordinate a series of actions
and performing the behaviors with: Harmony,
speed, confidence, Coordination, timing,
stability, Integration, smoothness, proportion
• Precision (accuracy): perform the behavior
Accurately, precisely, with control,
proficiently, independently, without error, with
balance
• Manipulation: same verbs but from
instructions
• Imitation: copy, follow, mimic, repeat,
replicate, reproduce, trace, align, hold, place,
rest (on), step, grasp, balance

22. The FACTS of good test items
• Face valid
F
• Accurate
A
• Consistent with current practice
C
• Test blueprint representative
T
• Statistically appropriate
S

23. Creating good test items
Right
Expertise
Sound
Processes
Efficient
Tools xxx

24. Right
Expertise
Sound
Processes
Efficient
Tools
xxx
Subject matter expertise
• ensures test is valid
Content development expertise
• ensures test is superiorly crafted
Psychometric expertise
• ensures test is reliable and fair
Right
Expertise

25. Write items
Review &
approve items
Field test
items
Review of field
test results
Gap analysis
of item bank
Right
Expertise
Sound
Processes
Efficient
Tools
xxx
Sound
Processes

26. Item Bank
A repository of items with their associated metadata & historical
record
Right
Expertise
Sound
Processes
Efficient
Tools
xxx
Efficient
Tools

27. Item Bank – Data
• The item itself
• Item author
• Item type
• Content area reference from test blueprint
• Cognitive level
• Revision history
• Item status
• Performance statistics
Right
Expertise
Sound
Processes
Efficient
Tools
xxx
Efficient
Tools

28. Guidelines for an effective item development process
An effective item development process begins
well before the first test item is written with
development of test blueprints.
Right
Expertise
Sound
Processes
Efficient
Tools
Effective
Item
Development

29. Guidelines for an effective item development process
Use item performance analysis to guide further
review of the items and to provide feedback to item
authors.
Right
Expertise
Sound
Processes
Efficient
Tools
Effective
Item
Development

30. Test blue print

31. Blueprint …
• What is test blueprint?
•It is a master plan or framework that
–Outlines/specifies the goals, learning outcomes and
contents necessary to set items for a test,
–Aligns the assessments to learning goals, outcomes
and contents,
–Ensures a balanced representation of learning
outcomes (by level) and contents in the assessment,
–Serves as a base for the determination of the emphasis
given to contents, assessment weight, and test format.

32. Issues to
consider in
test
blueprint
preparation
• Curriculum details
• Course outlines/guidebook details
• Course contents pertinent to the respective learning
outcomes
• Time devoted to cover the content with in a course
(course credit hour)
• Total test length (i.e., total number of test items)
• Number of questions allocated to each content area
using the proportion calculated
• Weight for each item
• Relevance of the content to students future career

33. Blueprint. . .
• Required
documents
to prepare
the test
blueprint
• To prepare a test blueprint, we need to
have the following documents in our
hand.
•Identified core competencies of the
program
•Curriculum of the program

34. Test Blueprint Template Contents
1. Cover Page (Contains Band, Name of the Program , …) -
Please see the hard copy document
2. Introduction
3. Objective/ Significance of Test Blueprint Preparation
4. Expected profile of the Graduates
1. List the expected profile of the graduate mentioned in the
identified core competency with clear demarcation
between knowledge, skill and attitude.
5. General objective
1. List the general objectives

35. Test Blueprint Template Contents …
6. Specific objectives
List the specific objectives from the curriculums which are parallel
to the general objectives identified in the core competency
document of the program.
7. Themes and list of courses
List the themes and courses from the core competency identified
8. Total credit hour of the selected courses
Indicate the credit hour of the courses from the curriculum
Themes and list of courses with credit hour can be done in a table

36. Test Blueprint Template Contents …
9. Share of the Themes/Courses/Items in Percentage (%)
(Please see Table 2 in the appendix section)
•Share of themes (T) =
𝑎
𝑏
𝑥100, Where “a” is the credit hour
of a theme and “b” is the total credit hour of the program.
Credit hour of a theme is the sum of credit hours of courses
in the theme.
•Share of courses(C) =
𝐶𝑟𝑒𝑑𝑖𝑡 ℎ𝑜𝑢𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑐𝑜𝑢𝑟𝑠𝑒
𝐶𝑟𝑒𝑑𝑖𝑡 ℎ𝑜𝑢𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑃𝑟𝑜𝑔𝑟𝑎𝑚
𝑥100
•Share of items per course =
𝑆ℎ𝑎𝑟𝑒 𝑜𝑓 𝑡ℎ𝑒 𝑐𝑜𝑢𝑟𝑠𝑒 𝑋 𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑖𝑡𝑒𝑚𝑠

37. Example
Presentation Title runs here l 00/00/00
36

38. Sample test blueprint
Themes Name of
Courses
Credit
hour
Share of the
themes in
%
General objective Content Specific objectives
Learning outcomes
No.
of
items
Cognitive
Affective
Psychomotor
Remembering
Understanding
Application
Analysis
Evaluation
Creation/Synthesi
s
Fundamentals
of
psychology
General
Psychology
3
𝑎
𝑏
𝑥100
Where “a” is
the credit
hour for a
theme and
“b” is the
total credit
hour for the
program
Use psychological
principles and
theories to understand
and explain their
behavior and others’
Learning,
Motivation
and
Emotion
Apply theories of learning,
motivation and emotion in
explaining behavior
Personality Apply personality theories to
understand human behavior
Learning,
Sensation
and
Perception
Apply principles of Learning,
sensation and perception and
learning in understanding behavior
Personality
Psychology
3 Assist others to
understand how the
psychological factors
influence their day to
day life experiences
Learning,
motivation,
emotion. &
personality
Assess how the psychological
factors affect behavior
Apply descriptors to identify
psychological factors that affect
behavior
Lifespan
Human
Development
Psychology of
Childhood
3
Understand theories
of child and
adolescent
development
Psychology of
Adolescence
3
Distinguish the
different aspects and
stages of
development
Consult various
groups on child and
adolescent
development
Total X

39. Writing Good Multiple-
Choice Test Items

40. What Makes a Content-Appropriate Item
• Reflects specific content
• Able to be classified on the test blueprint
• Does not contain trivial content
• Independent from other items
• Not a trick question
• Vocabulary suitable for candidate
population
• Not opinion-based
Source: Haladyna, T.M. (1996). Writing Test Items to Evaluate Higher Order Thinking. Boston, USA: Allyn & Bacon.

41. Anatomy of a Multiple-Choice Item
Which city has been awarded
the 2016 Summer Olympics?
A. Chicago
B. Madrid
C. Rio de Janeiro
D. Tokyo
Stem
Answer Options
Key
Distractors

42. Not opinion based
Avoid the use of opinion based material in
items
• Poor Stem:
Which are the best football team in the world?
a. Barcelona
b. Manchester United.
c. Bayern Munich
d. Athletico Madrid
• Better Stem:
Which football team won the European championship in
1999?

43. Guidelines for Writing the Stem
Place most of the phrasing in the stem.
• Poor Stem:
Type II diabetes is
a. also called juvenile-onset.
b. characterised by insulin dependency.
c. primarily seen in adults over 40.
d. often managed by drug therapy.
• Better Stem:
What is the most frequent age of onset for Type II
diabetes?

44. Guidelines for Writing the Stem
Avoid excessive, unnecessary wording.
• Poor Stem:
Certified individuals are required to undertake continuing
education. What is the minimum number of hours
required in a two-year period?
a. 10
b. 15
c. 20
d. 25
• Better Stem:
How many hours of continuing education must a
certified individual take in a two-year period?

45. Guidelines for Writing the Stem
Clearly describe the problem or what is being
asked of the candidate.
• Poor Stem:
Continuing education is
a. required every two years.
b. important to life-long learning.
c. an optional activity.
d. undertaken at certified centers.
• Better Stem:
How many hours of continuing education is a
certified individual required to undertake in a 5-year
cycle?

46. • Maintain similarity in the answer options.
• Keep the answer options relatively equal in
length.
• Keep all answer options grammatically correct
with the stem and parallel.
• Avoid cueing the right answer.
• Avoid words such as “always” and “never”.
• Avoid repetitive wording in the answer
options.
• Vary location of key.
• Place answer options in logical order.
Guidelines for the Answer Options

47. Four is most common.
• Recent research indicates three may be
sufficient if the two distractors are
strong and plausible.
• Rarely have three functional
distractors.
• “Using more options does little to
improve item and test score statistics
and typically results in implausible
distractors.”*
Number of Answer Options

48. Constructing multiple
choice items against
levels of domain

49. Remembering
In the area of physical science, which one of the
following definitions describes the term “polarization”?
A. The separation of electric charges by friction.
B. The ionization of atoms by high temperatures.
C. The interference of sound waves in a closed chamber.
D. The excitation of electrons by high-frequency light.
E. The vibration of transverse waves in a single
plane.
Simple recall of the correct definition of
polarization is required.
48

50. Comprehending
Which one of the following describes what takes place in the so-
called PREPARATION stage of the creative process, as applied to
the solution of a particular problem?
A. The problem is identified and defined.
B. All available information about the problem is collected.
C. An attempt is made to see if the proposed solution to the
problem is acceptable.
D. The person goes through some experience leading to a general
idea of how the problem can be solved.
E. The person sets the problem aside, and gets involved with some
other unrelated activity.
The knowledge of the five stages of the creative process
must be recalled (knowledge), and the student is tested for
an understanding (comprehension) of the meaning of each
term, in this case, “preparation.”
49

51. Applying
Which one of the following memory systems
does a piano-tuner mainly use in his occupation?
A. Echoic memory
B. Short-term memory
C. Long-term memory
D. Mono-auditory memory
E. None of the above
This question tests for the application of
previously acquired knowledge (the
various memory systems).
Presentation Title runs here l 00/00/00
50

52. Analyzing:
Read carefully through the paragraph below, and decide which of the
options A-D is correct.
“The basic premise of pragmatism is that questions posed by
speculative metaphysical propositions can often be answered by
determining what the practical consequences of the acceptance of a
particular metaphysical proposition are in this life. Practical
consequences are taken as the criterion for assessing the relevance
of all statements or ideas about truth, norm, and hope.”
A. The word “acceptance” should be replaced by “rejection.”
B. The word “often” should be replaced by “only.”
C. The word “speculative” should be replaced by hypothetical.”
D. The word “criterion” should be replaced by “measure.”
This question requires prior knowledge of and understanding about
the concept of pragmatism.
The student is tested on his/her ability to analyze whether a
word fits with the accepted definition of pragmatism.
51

53. Evaluating:
Judge the sentence in italics according to the criteria given below:
• “The United States took part in the Gulf War against Iraq BECAUSE
of the lack of civil liberties imposed on the Kurds by Saddam
Hussein’s regime.”
A. The assertion and the reason are both correct, and the reason is
valid.
B. The assertion and the reason are both correct, but the
reason is invalid.
C. The assertion is correct but the reason is incorrect.
D. The assertion is incorrect but the reason is correct.
E. Both the assertion and the reason are incorrect.
A knowledge and understanding of Middle East politics are
assumed. The student is tested in the ability to evaluate
between cause and effect in the sentence in terms of
predefined criteria.
Presentation Title runs here l 00/00/00
52