COVENANT UNIVERSITY SHORT COURSE

ENHANCING THE QUALITY OF
TEACHING, TESTING & LEARNING
WITH TEST BLUEPRINT, ITEM ANALYSES & ITEM BANKING
Dayo Odukoya, Ph.D [Psychometrics]
dayoodukoya@gmail.com
2349096505735

CONTENT
•Preamble
•Entry Knowledge
•Test Blueprint
•Item Writing
•Trial Testing
•Item Analyses
•Item Moderation
•Item Banking
•Other Pertinent
Psychometric Issues
•Way Forward

PREAMBLE
•It is applied knowledge
that wields great power,
not mere knowledge.
•Knowing is not enough, it is
what you do with what
you know that makes all
the difference.
•See the pathway of
learning
Information
Knowledge
[accumulation of theories,
principles and laws]
Wisdom
[Application of Knowledge
to create solutions]
Understanding
Deeper
Understanding
Source: Odukoya (2014) – Pathway of Learning

EDUCATION &
PSYCHOMETRICS
EDUCA
TION
TEACH
ING
LEAR
NING
COUNSE
LLING
PRODU
CTIVITY
TESTING
Odukoya (2015) – The Domains of Education

ENTRY KNOWLEDGE
Calculate: 40 X 15
A.0
B.1
C.2
D.3
CD
B
A
Q1-4 What are the correct
labels for letters A, B,C and D?
5. The proponent of the taxonomy of
educational objectives is _______
6. The 6 levels of educational
objectives, in correct sequence, are
______________________
7. Three vital types of item analysis are:
1) ____ 2) _____ 3) _____
8. Give the formula for computing one
of the 3 types of item analyses.
9. The real purpose of trial testing is
_______________
10.List 3 vital conditions or attributes
used in item banking

© 2017 – Dayo Odukoya [dayoodukoya@gmail.com; 09096505735
WE ARE IN THE
ERA OF CBT

DEFINITIONS OF
TERMS
Summative Test –
A test or examination that
covers the entire years of
study of the student [e.g.
JSS1 to 3 or SS1 to 3]. This
kind of examination is often
conducted by an
independent external
examining body like WAEC,
NECO etc
Achievement Test –
These are tests
administered after a
period of teaching,
training or instruction.
The objective is to
assess the depth of
learning achieved
Formative Test – These
are the progressive tests
often conducted within
a school system to
monitor the progression
of students’ learning.
The continuous
assessment tests are part
of formative tests.

DEFINITIONS OF
TERMS
Validity –
The degree to which
the test is accurately
or correctly assessing
what it purports to
measure
Reliability –
The degree of
consistency of
measurement.
Norm –
Test norms consist of
data that make it
possible to determine
the relative standing of
an individual who has
taken a test. They are
mean values for groups
of people used for
interpretation.

DEFINITIONS OF
TERMS
Options –
The possible answers
to a question. For a
one-correct answer
multiple choice
question, there is
one correct answer
with 3 or 4
distracters.
Stem or Prompt –
The main question
to which people
respond. The Stem
& Options constitute
an item
Distracters –
Incorrect options
meant to distract
testees from easily
identifying the
correct option. The
right answer is the
Key

TEST BLUEPRINT
•This is basically the overall plan of a test.
•Though it can be used for various types of tests
and test formats, it is more suited for multiple
choice questions [MCQ] in achievement tests.
•It incorporates the overall educational
objectives, course/subject objectives and
students’ level of cognitive development in
derivation of item weightings.

Can you imagine constructing a
high rising building without a
structural building plan or using
a plan developed by Quacks?
The result is what is in this
picture.
The pathetic reality is that many
teachers and instructors, as
people/nation builders, have
unconsciously been committing
the same havoc

AS BUILDING PLAN IS TO URBAN
DEVELOPMENT, SO IS TEST BLUEPRINT
TO EDUCATIONAL DEVELOPMENT,
AND BY EXTENSION, TO NATIONAL
DEVELOPMENT

WHAT ARE THE ELEMENTS OF
A TEST BLUEPRINT?

TEST BLUEPRINT
[DEMONSTRATION WITH PSY 221:PSYCHOBIOLOGY]
Topics K C Ap An Sy C TOT
Psychology Definitions 1 ? ? ? ? ? 5
Cell & Movements ? ? ? ? ? ? 10
Nervous System & Behavior ? 12 ? ? ? ? 20
Endocrine System & Behavior ? ? ? ? ? ? 15
TOTAL 10 30 5 2 2 1 50
Keys: K-Knowledge; C-Comprehension; Ap-Application; An-Analysis; Sy-Synthesis; C- Creativity
5/50=0.1; 0.1x10=1; 20/50=0.4; 0.4 x 30 = 12
Complete the table

TEST BLUEPRINT [RAW DERIVATIONS]
Psychology Definitions 1 3 0.5 0.2 0.2 0.1 5
Cell & Movements 2 6 1 0.4 0.4 0.2 10
Nervous System & Behavior 4 12 2 0.8 0.8 0.4 20
Endocrine System & Behavior 3 9 1.5 0.6 0.6 0.3 15
TOTAL 10 30 5 2 2 1 50
5/50=0.1; 0.1x10=1; 20/50=0.4; 0.4 x 30 = 12
Completed Table – Actual Derivations

TEST BLUEPRINT REQUIRES
QUANTITATIVE &
QUALITATIVE ANALYSES

TEST BLUEPRINT [ADJUSTED]
Psychology Definitions 1 3 1 - - - 5
Cell & Movements 2 6 - 1 - 1 10
Nervous System & Behavior 4 12 2 1 1 - 20
Endocrine System & Behavior 3 9 2 - 1 - 15
TOTAL 10 30 5 2 2 1 50
Completed Table
Adjusted Conditions: Cognitive level; Course objective; Balance

THE NEXT LOGICAL STEP AFTER
PREPARATION OF THE TEST
BLEUPRINT IS ITEM WRITING

Understandin
g of the core
theories of
cognitive
development
[e.g. Piaget’s]
Deep
understanding
of the subject
matter within
the scope of
the syllabus or
compact.
Knowledge of
the common
verbs linked
with each level
of the
educational
objectives
A sound
understanding
of the core
objectives of
the subject
and how to
achieve them
Knowledge of
the rules guiding
the writing of
multiple choice
questions
[MCQ]
Understandi
ng of the
attributes of
Bloom’s
taxonomy of
educational
objectives
ITEM WRITING
Item writing is the art of
writing questions and
response options
spanning the scheme,
syllabus or compact for
a subject/course.
Professional Item
writing requires the
following:

THE CONFERENCE ITEM WRITING
APPROACH IS RECOMMENDED
[WITH A PSYCHOMETRICIAN & SUBJECT EXPERTS IN
ATTENDANCE. THIS WAY QUESTIONS THAT CAN BE
USED FOR THE NEXT 3 YEARS CAN BE GENERATED]

SAMPLE TEST QUESTIONS FOLLOWING BLOOM’S TAXONOMY
1. Recall
2. Comprehension
3. Application
4. Analysis
5. Synthesis
6. Creativity
1. Who is the President of USA?
2. How did the person become the
President of USA?
3. Show how you can adopt the
USA President’s method to
become the President of Nigeria?
4. What are the essential steps
involved in becoming the
President of Nigeria?
5. Illustrate the flow of events that
led to the election of the current
President of USA
6. Create a better procedure for
selecting the President of Nigeria.

PIAGET
THEORY OF
COGNITIVE
DEVELOPMENT

Have
diagrams
and tables
above the
stem
Problem
posed
should be
clear and
unambigu
ous.
Format
the items
(including
the
options)
vertically
Group
items of
similar
format
together
Give ample
time for item
analyses and
moderation.
Have your
questions
peer-reviewed
Questions
should
progress
from easy to
complex
SOME RULES FOR WRITING MCQ

Use good grammar, punctuation, and spelling
consistently. Use the active voice in your statements.
Minimize the time required to read each item. Ideally it should
not exceed one minute to respond to a question.
The ideal question should be answered correctly by
40 to 60% of the testees. Requires item analysis.
Avoid giving unintended cues – such as making the
correct answer longer than the distractors.

Place options in sequential or logical order.
List options with letters [A, B, C, D] rather than with
numbers [1. 2, 3, 4]
Avoid overlapping options
Options should be homogeneous in content
All the options should be fairly the same in length

Phrase
options
positively,
not
negatively
Avoid
using the
phrase, all
of the
above.
excessively
Avoid
distractors
that can
give clue to
the correct
answer
Avoid
using the
phrase,
none of
the above,
excessively
Avoid the
use of
words like
never and
always.
Avoid
using the
phrase, I
don’t
know,
excessively
Ensure that
there is
only one
correct
answer

APART FROM ENSURING RELEVANCE TO
SYLLABUS CONTENT, THE CONTENT
VALIDITY OF AN ACHIEVEMENT TEST IS
GREATLY ENHANCED BY FOLLOWING THE
RULES GUIDING THE WRITING OF MCQ

WHAT ARE THE SHORTCOMINGS OF THESE QUESTIONS?
1. The connection between
a pregnant woman and her
developing foetus is an
[Biology]
A. placenta
B. umbilical cord
C. fallopian tube
D. navel
2. What is the lacuna
between numbers 3 and 10?
[Maths question]
A. 8
B. 4
C. 3
D. 5

SAMPLE STANDARD QUESTIONS
1.
Fig.
1,
called
power-‐house,

is
a

structure
that
is
found
inside
a

typical
living
cell.
What
is
its
name?
A.

Nucleus
B.
Vesicle
C.
Lysosome
D.
Mitochondrion
2.
Mitosis
is
a

A.
reduction
division
B.
multiplication
division
C.
expansion
division
D.
gonadic
division
3.
Which
of
the
flowcharts
below
best
illustrate

the
correct
order
of
'organisation
of
life’?
a)

cell
-‐-‐>
organelle
-‐-‐>
organ
-‐-‐>
organism
b)

cell
-‐-‐>
tissue
-‐-‐>
organ
-‐-‐>
organ
system
c)

cell
-‐-‐>
organ
-‐-‐>
organism
-‐-‐>
ecosystem
d)

cell
-‐-‐>
organ
-‐-‐>
organelle
-‐-‐>
organism

Fig 1

SAMPLE DEVELOPMENT-ORIENTED QUESTIONS?
Physics:
1) A micrometer is used to
measure the diameter of a
uniform wire [Diagram of
Micrometer gauge gripping a
rod displayed], what is done to
obtain an accurate answer?
[Source: Cambridge May 2013 Paper 1 Q 2];
2). Irregular power supply is a
major problem in Nigeria today,
from your knowledge of
electricity generation, proffer a
sustainable solution to this
problem. [Source: Odukoya (2014)]
Mathematics
1). The exchange rate is $1 to €0.72.
Eddie travels from the USA to
Germany. He changes $300 into
euros (€). How many euros does he
receive? [Source: Cambridge May 2013
Paper 2 – Q 5ai];
2). Thinking of investing 1M naira you
won in lottery. If Treasury Bill offers 3%
interest within a period of 3 months
and your bank offers a fixed deposit
interest of 11% per annum, decide on
the best line of investment. Support
with clear mathematical calculations
[Source: Odukoya (2014)]

THINK ABOUT IT – IF OUR FORMATIVE
AND SUMMATIVE TESTS CONSISTENTLY
FEATURE THESE DEVELOPMENT-ORIENTED
QUESTIONS TO OUR STUDENTS IN THE
NEXT 5 YEARS, WHAT WOULD BE THE
LIKELY EFFECT?

TRIAL TESTING
• Trial testing is imperative for item analysis. The data
required in item analyses is obtained via trial testing.
• Trial testing, also called pilot test, further serve to provide
necessary data for ascertaining the reliability & validity
and normative data for the test undergoing
standardisation.
• Trial testing is conducted with typical testees following
real life test administration rules and regulations.
• Testees are normally not aware of the purpose of trial
testing.

ITEM ANALYSES
• Item analyses are the computations done using
frequency of responses to the items of a test. Such
analyses are often done to determine the relative
level of difficulty of the items, the distractive powers
of the distractors and the power of the test to
discriminate between low and achievers.
• The primary goal of item analyses is to empirically find
out questions and distractors that are not performing
well, and attempt to moderate or discard them.

ITEM ANALYSES: DIFFICULTY INDEX
Item Difficulty: The Proportion [PROP] Statistic
• The item difficulty is the proportion of
subjects in a sample who correctly answer
the item.
𝑝 =
𝑁𝑢𝑚𝑏𝑒𝑟
𝑜 𝑓
𝑟 𝑒𝑠𝑝𝑜𝑛𝑑𝑒𝑛𝑡𝑠
𝑤ℎ𝑜
𝑎 𝑛𝑠𝑤𝑒𝑟𝑒𝑑
𝑡ℎ𝑒
𝑖 𝑡𝑒𝑚
𝑐 𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦
Total
number
of
respondents
who
attempted
the
question

DIFFICULTY INDEX: PRACTICE QUESTION
• 1. The word NEWS is
• A) plural
• B) singular [key] - 10
• C) compound
• D) simple
• The number of respondents who selected the correct answer [the key]
is indicated in front of option B. Fifty [50] students responded to
question. Calculate the difficulty index. Interprete your result.

DISTRACTIVE INDEX: PRACTICE QUESTION
• 1. The word NEWS is
• A) plural - 35
• B) singular [key]
• C) compound - 3
• D) simple - 2
• The number of respondents who selected the distracters is indicated in
front of each distracter. Fifty [50] students responded to question.
Calculate the distractive indices. Interprete your results.
di =
JKLMNO
PQ
ONRSPTUNTVR
WXP
YXPRN
Z
U[RVOZYVNO
]^_`
abcdef
]g
fehi]ajea^h
kl]
_^^eci^ej
^le
mbeh^n]a

ITEM ANALYSES: DISCRIMINATORY INDEX
Discriminatory Index (D) = U – L
‘U’ infers the proportion of respondents who got the question right among the top (Upper)
scorers;; while ‘L’ infers the proportion right of respondents who got the question right
among the low scorers.
U =
JKLMNO
PQ
ONRSPTUNTVR
[T
VXN
KSSNO
oOPKS
W[VX
YPOONYV
ONRSPTRN
pPVZq
TKLMNO
PQ
ONRSPTUNTVR
[T
VXN
KSSNO
oOPKS
L =
JKLMNO
PQ
ONRSPTUNTVR
[T
VXN
qPWNO
oOPKS
W[VX
YPOONYV
ONRSPTRN
pPVZq
TKLMNO
PQ
ONRSPTUNTVR
[T
VXN
qPWNOoOPKS

DISCRIMINATORY INDEX: PRACTICE QUESTION
Assume 5 out of 10 student
in the upper quartile got an
item right and 8 out of the
10 students in the lowest
quartile got the same item
right. Calculate the
discriminatory index and
use your answer to
determine what should be
done to the item under
scrutiny.
𝑈 =
5
10
= 0.5
L =
w
xy
= 0.8
D = 0.5 - 0.8 = - 0.3

POINT
BISERIAL
OTHER DISCRIMINATORY INDEX TECHNIQUES
Pearson Product Moment correlation
between student responses to a particular
item and total scores on all other items on
the test. This index is the equivalent of a
point-biserial coefficient. It provides an
estimate of the degree to which an
individual item is measuring the same thing
as the rest of the items. Assuming that the
total test score accurately discriminates
among individuals in the group tested, then
high positive RPBI's for the correct responses
would represent the most discriminating
items. That is, students who answered the
correct response scored well on the
examination and vice versa.

Mean &
Standard
Deviation
The mean is the
“average” student
response to an item.
The standard deviation, or
S.D., is a measure of the
dispersion of student
scores on that item from
the mean.

Standard
Error of
Measure
ment
The standard error of
measurement is directly
related to the reliability of
the test. It is an index of
the amount of variability in
an individual student’s
performance due to
random measurement
error.

ITEM MODERATION
Items with the following outcomes are eligible for moderation:
• High difficulty index
• Low difficulty index
• Average difficulty index [if the objective of the test is stringent,
e.g. of medical doctors and pilots
• When distracters are attracting significantly more than the key
• When a distracter is dismally attracting the respondents
• Can you mention more? Disc

TRIAL TESTING 2
WITH EXTENSIVE MODERATION,
SECOND ROUND OF TRIAL TESTING
MAY BE NECESSARY

ITEM BANKING
• All items that passed content validation, item writing
rules, item analyses, and moderation requirements
are eligible for item banking.
• Item banking saves time and the rigorous demands
of writing valid MCQs. Consequently it enhances
the overall validity of objective tests, and ultimately
catalyses national development, particularly with
more injection of development-oriented questions.

ITEM BANKING: PARAMETERS FOR
ITEM BANKING
•Group questions by subject.
•Group questions by class or grade.
•Group questions by topic.
•Group questions by difficulty level.
•Automate the entire system
•Can you mention more?

Conductin
g Empirical
Affective
Assessment
Establishing
the
Reliability &
Validity of
Tests
Conducting
Diagnostic
Testing
Using
Testing
Tools to
Provoke
Developm
ent
Computer
Based
Testing
Understanding
the Categories
of Psychological
Instruments &
Tests and How to
Develop them
OTHER PERTINENT PSYCHOMETRIC ISSUES WORTH
PAYING ATTENTION
Conducting
Professional
Scoring,
Grading,
Interpretation
& Reporting

WAY FORWARD & SPECIAL INFO
Covenant University Exam Board will soon take off. The need
for Certified Examiners for CBTs is becoming more pertinent.
From this forum, we shall begin to collate list of Certified
Examiners.
We conduct DIAGNOSTIC MOCK WASSCE and DIAGNOSTIC MOCK
UTME for schools with detailed report showing topical areas requiring
students and teachers special attention and remedial action before
actual examination
CUFOC … lifelong learners are lifelong leaders. Some of our online
courses about to be flagged off are: Satellite Communication;
Basic Writing Skill; Attitudinal Change for Enhanced Productivity;
Python Programming; Java Programming etc

TEACHERS ARE NATION BUILDERS
LET’S DO IT RIGHT TO GET IT RIGHT
THANK YOU!

EXIT KNOWLEDGE
Calculate: 40 X 15
A.0
B.1
C.2
D.3
CD
B
A
Q1-4 What are the correct
labels for letters A, B,C and D?
5. The proponent of the taxonomy of
educational objectives is _______
6. The 6 levels of educational
objectives, in correct sequence, are
______________________
7. Three vital types of item analysis are:
1) ____ 2) _____ 3) _____
8. Give the formula for computing one
of the 3 types of item analyses.
9. The real purpose of trial testing is
_______________
10.List 3 vital conditions or attributes
used in item banking

COVENANT UNIVERSITY SHORT COURSE

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to COVENANT UNIVERSITY SHORT COURSE

Similar to COVENANT UNIVERSITY SHORT COURSE (20)

Recently uploaded

Recently uploaded (20)

COVENANT UNIVERSITY SHORT COURSE