Test Construction

kheru2006@yahoo.com ppfd5105/07 1
Test Construction
(Instrument of Assessment)
Week 6
21 May 2007

What is In The Pro-Forma
• Basic Concept of Learning Assessment
• Basic Measurement Theory and Traditional Assessment
• Quality Assurance and Reliability and Validity Issue
• Discussion with examples of Assessment used in
classroom
• Alternative Approach to Learning Assessment
• Implementation to Alternative Assessment
• Discussion on the Strength of Alternative Assessment
• Affective Domain Assessment
• Psychomotor Domain Assessment
• School-Based Assessment : Issues and Problems
• Grading and Students Report

At the end of the section, students will be able to
1. define what is the table of test specifications,
2. state the purpose of preparing the table of test
specifications,
3. construct a sample table of test specifications.

Table of Specification
• Often test are done spontaneously and
not planned. Test that not well planned
usually have very low validity and
reliability. One of the best way is by
preparing a Table of Specification which is
question/item preparation plan.
• In a table of specification the following is
determined :

Table of Specification
• Form and Time Frame
• Contents of Topics
• Level of Skills
• Allocation of percentage of questions/item
• No of questions
• Level of Difficulty

Importance of Table of Specification
• Test can be constructed well and not
haphazardly
• Can enhance the validity and reliability of the
test
• Serve as a guide and deciding factor for
question planners
• Can ensure a good spread of topics, lesson
objectives and as well as the level of difficulty of
questions by giving appropriate emphasis

Steps in Preparing Table of Specification
Decide The Objective of Test
Decide the Content of Test
Fill in The Content Column
Fill in The Skills Column
Decide The Types and No of Questions
Check The Distribution and % of Questions

Example of Table of Specification (the orthodox)
Levels of Cognitive Domain
Title/Topic Evaluation Understanding Analysis Application Synthesis Knowledge Total
1 Matter 2 3 2 1 1 1 10
2 Chemical 1 3 2 2 2 1 10
3 Acid and
Base
1 2 1 2 2 1 10
4 Material in
Industry
1 2 2 2 1 1 9
5 Chemical
In
Agricultural
2 3 2 1 1 2 11
Total 50

Steps In Test Construction
Type of Test
Objective (MCQ) Subjective (Essay)
Type of
Unlimited
Response
Type of
Response
Objective
Type
Short
Answer
question
sentence
completion
connections
true & false
multiple choice
classifying
combination
cloze objective
sequencing

Types of Tests ~ Objectives
• Objective test is a type of test which
requires the candidate to give correct and
short answers so that it can be examined
easily and correctly. The answers are
short and usually in a few words or a
phrase only
• Can be divided into 2 main groups i.e.
short answer item and multiple choice item

Types of Tests
• Short answer item
• Multiple choice item
• a) objective short answer type test is a
type of test that needs short answers from
the candidate
• Short answers can be divided into :
• i) Type of questions
• E.g. How many credits does a
matriculation students need to take in a
semester?

ii) Sentence completion type
E.g. The father of vision for Malaysia is______________
iii) Connectors Type
E.g. Name the places of worship for the following religions :
Islam _______________________________________
Hinduism ____________________________________
Christianity ___________________________________
Sikhism ______________________________________
b) For the multiple choice type, candidates need to make choices
from the number of answers given. The following are some
examples of objective type of questions

i) True/False Item
E.g. Petronas Twin Tower is the highest tower in the world
(True / False)
iii) Multiple Choice Item
E.g. What is the main crop in Malaysia?
A Coconut B Pineapple
C Cocoa D Rubber
iii) Matching Item
E.g. Match the leaders with their titles
Tun Abdul Razak Father of Vision
Tunku Abdul Rahman Father of Independence
Tun Dr Mahathir Father of Development

iv) Classifying Type Item
E.g. You are required to choose a country which is usually
related to the names given. Write your answers in the space
given
A Melaka B Singapore C Penang
1) ____ St Francis Xavier 3 ____ Sir Francis Light
2) ____ Sir Stamford Raffles 4 ____ St Francis Assisi
v) Sequencing type
E.g. You are required to read the sentences carefully and
rearrange them in an orderly manner
1 Ali fell in the river
2 Ali went for fishing
3 Felt a jerk in the fishing hook
4 A crocodile attacked Ali

vi) Combination type
e.g. Among the following plants, which one does not bear flowers?
I Fern
II Padi
III Grass
A I only C I and III
B I and II D II and III
vii) Cloze Item
e.g. _____________ is the chief producer of padi in Peninsular
Malaysia

Types of Tests ~ Subjective
• Subjective test is a type of test which
requires the candidate to answer
questions using their own words and may
have more than one single answer
• Can contained items that require essay
types of many paragraphs, short, precise
answers of one or two sentences or a
single paragraph

Subjective items can be divided into two
basic types :
a) Limited-response type
b) Unlimited-response type
a) Limited-response type limits the content and
style of answer. Requires candidates to
answer precisely, in one phrase, sentence or
short statement. Usually items will start with
words like What …,Give …,List ……,Explain
…….,Define ….etc.
e.g. “In one sentence, define subjective test.”

b) Unlimited-response type
Unlimited-response type gives candidate to
choose the direction and scope of answer.
Candidates are free to extend answers, using
own thoughts and opinions, style of writing but
keeping to relevant topic. Usually items will
start with words like “ Discuss the probable
causes of ……”, “Compare and contrast ……”,
“In your opinion, ………..”
e.g. “Discuss the importance of learning the
Assessment in Education”

Steps Involved In Item Building
1 Specify the Test Objective
2 Construct Table of Specification
4 Pre-Test
3 Write The Test Item
5 Modify Test Item
6 Specify Allocation of Marks
7 Keep test materials and items
In secure place

Type Advantages Disadvantages
Multiple-choice
•Can measure all levels of student ability.
•Enables wide sampling of subject content .
•Quick and easy to score.
•Enables objective scoring.
•Can be analyzed for effectiveness.
•Difficult to construct good
items.
•Tendency to measure simple
recall.
Matching
•Relatively easy to construct.
•Conserves examinees’ reading time .
•Enables efficient and objective scoring.
•Generally unsuitable for
testing higher-order abilities.
•Tendency to measure simple
recall.
True/False •Efficient for testing large sample of information.
•Enables efficient and objective scoring.
•Permits high guess factor.
•Difficult to construct effective
items.
Completion/Short
Answer
•Minimizes guessing
•Enables coverage of fairly wide content.
•Relatively easy to construct.
•Measures limited range of
abilities.
•Cannot be machine-scored.
•Scoring is highly dependent on
judgment.
Essay
•Can be quickly and easily constructed.
•Eliminates guessing .
•Can test higher order of thinking .
•Limits amount of content
sampled.
•Time-consuming to score.
•Results in low scoring reliability.

Steps Involved In Item Building
• Proper and systematic procedure is important in
order to design a quality test.
• The first step in item building is to determine the
objective of the test. If the objectives is to detect
student’s weakness in a specific skills, the most
suitable is a diagnostic test.
• Therefore the Table of Specification must be
able to help test designer to build a test
diagnostic purpose. Clear objectives help raise
the validity of the test

• A test designer must adhere to the Table
of Specification, changing any detail in the
table will cause the mismatch of the test
and teacher’s objectives.
• e.g. if the test calls for 40 MCQ, the
designer must design the test following the
principles involved.
• Every item must observe the following
guidelines :

• The stem of the item must have only one
focus
• Specific task is clearly stated
• Answer choice must relate to the stem of
the question
• Effective distracter
• Have one correct answer
• Clear sentence structure without any
ambiguity
• Must be ethical and not touch on sensitive
issues

• When the test is complete, it should be
pre-test before administering to students
• Any mistakes should be corrected
• Unsuitable and wrong items should be
removed and replaced with new one
• A set of marking scheme is necessary so
that marking is standardized and hence
meet the objectives of test
• Must be kept in a secure and safe place
until administered to ensure confidentiality
so as the test is reliable

Guideline on Writing Test Items
Before writing test items, refer to Table of Specification
1) Stem of question
should only have one stem, avoid negative questions
e.g. “Describe a teaching method which is not suitable to
be used to promote thinking among students”
2) The Task of the question
the task required for students to answer should be
clearly and explicitly stated. Students should focus on a
specific task

3) Level of Difficulty
items should present intended level of difficulty as stated
in the Table of Specification.
e.g. low level cognitive domain (knowledge), average
level (understanding, application, analysis), high level
(synthesis, evaluative)
4) Proof Reading
make sure that the structure, content and language of
the items are free from mistakes
5) Analysis of Difficulties
before using the item, do analyzing and pre-test to
ensure high level of reliability and validity

Norm-Referenced and Criterion-Reference Test
• The two types of referenced test are the norm-
referenced and criterion-referenced test
• A norm-referenced test is a type of test, assessment,
or evaluation in which the tested individual is compared
to a sample of his or her peers (referred to as a
"normative sample").[1] The term "normative
assessment" refers to the process of comparing one
test-taker to his or her peers.[1]
• A criterion-referenced test is one that provides for
translating the test score into a statement about the
behavior to be expected of a person with that score or
their relationship to a specified subject matter.[1] Robert
Glaser originally coined both terms.[2] Therefore, the
term "criterion-referenced test" is somewhat of a
misnomer, as it more accurately refers to the
interpretation that is made of the score and not the test
itself.[3]

Norm-reference Test
• People taken the test provides the norm of achievement
• It compare own score to group score
• Test item cover all abilities or topics
• Intended for measuring general ability, assessing range
of ability in large group, selecting top candidates
• NRT does not tell whether students ready to move
• NRT not suitable for affective and psychomotor domains
• NRT tend to encourage competition and score
comparisons
• E.g. end of term , UPSR, PMR, SPM, centralized exam
* NRT looks similar to summative evaluation

Criterion-reference Test
• Test scores are compared no to those of others
but to a given criteria or standard performance
• Students are hope to achieve the standard
determined
• Criteria are clearly defined
• Work best when measuring mastery skills
• Determine if students can proceed
• Assessing affective and psychomotor domain
• Suitable for measuring achievement of special
education
• Grouping students to instructions
• e.g. diagnostic test, driving test, typing test …..

Summary NRT & CRT
Dimension
Criterion-Referenced
Tests
Norm-Referenced
Tests
Purpose
To determine whether each student has
achieved specific skills or concepts. To find
out how much students know before
instruction begins and after it has finished.
To rank each student with respect to
the achievement of others in broad
areas of knowledge. To discriminate
between high and low achievers.
Content
Measures specific skills which make up a
designated curriculum. These skills are
identified by teachers and curriculum
experts. Each skill is expressed as an
instructional objective.
Measures broad skill areas sampled
from a variety of textbooks, syllabi,
and the judgments of curriculum
experts.
Item
Characteristics
Each skill is tested by at least four items in
order to obtain an adequate sample of
student performance and to minimize the
effect of guessing. The items which test
any given skill are parallel in difficulty.
Each skill is usually tested by less than
four items. Items vary in difficulty.
Items are selected that discriminate
between high
and low achievers.
Score
Interpretation
Each individual is compared with a preset
standard for acceptable achievement. The
performance of other examinees is
irrelevant. A student's score is usually
expressed as a percentage. Student
achievement is reported for individual
skills.
Each individual is compared with other
examinees and assigned a score--usually
expressed as a percentile, a grade
equivalent score, or a stanine. Student
achievement is reported for broad skill
areas, although some norm-referenced
tests do report student achievement
for individual skills.
The following is adapted from: Popham, J. W. (1975). Educational evaluation. Englewood Cliffs, New Jersey: Prentice-Hall,
Inc.

Summary of NRT and CRT
Dimension Norm-Referenced Tests Criterion-Referenced Tests
Concept
Relative-achievement evaluation
showing comparative
performance of groups in the
form of normal graph
Objective referenced evaluation
determine individual achievement
based on minimum criteria pre-
determined earlier
Aim
Compare and distinguish
performance among candidates
or among groups
Determine ability to master
learning based on certain pre-
determined criteria
Uses
Summative test to distinguish
candidates in performance level
of distinction, credit, pass or
fail
Formative test to improve
teaching-learning based on the
test result
Target
Comparison of performance
among candidates
Determination of performance
individually or in small groups
Question-
characteristics
Questions arranged from easy
to difficult. with discrimination
among candidates
Questions have almost the same
level of difficulty based o learning
objectives
Grading
Passing mark or grading is
determined after the test
result
Passing mark or grading is
determined before the test is
carried out
Coverage
Questions covered wide range of
learning skills
Question content concentrates on
limited learning skills
Examples
Public examinations such as
UPSR, PMR, SPM and STPM
Short test, course work, written
exercises and portfolio project

PPFD5105 Short-Test
02 Jun 07
Q1. In our first meeting, you had been
exposed to the learning assessment.
i) List down the learning assessment you
had been informed.
ii) Which of the learning assessment is
meant for you as a student ?
iii) Briefly describe the learning assessment
you had chosen. [ 3,1,2 ]
.

PPFD5105 Short-Test
02 Jun 07
• Q2. In our second meeting, we talked
about the importance of assessment for
teachers. As such, why do you think
teachers need to know about
assessment? [ 5 ]

PPFD5105 Short-Test
02 Jun 07
Q3. We had also discussed on the
importance of assessment format for
use. Should you be assigned to be a
Ketua Panatia for your school, design a
simple assessment for use accordingly.
[10]

PPFD5105 Short-Test
02 Jun 07
Q4. The alphabet A, E, M, T is synonymous
to assessment terms.
i) What does the alphabet stands for ?
ii) Describe briefly what you had
mentioned. [4,4]

PPFD5105 Short-Test
02 Jun 07
Q5. Create one test item each from each
topics we had discussed and state the
difficulty level of each stand? [ 10 ]

SUBJECT
SCHOOL-BASED
ASSESSMENT
CENTRALIZED-BASED ASSESSMENT
Types of Instrument
E.g.Project,
Portfolio
Type of Test e.g. Objective, Subjective ...
Type of Item
Based on
Project/Activity
Type of Questions e.g. closed-ended,open-ended
Number of Item    Section A Section B Section C
10 3 2
Score Competent/Not-Competent
Total Score
Based on number of
competencies
assessed in subject
Section A Section B Section C
50 30 20
100
Length of Test
Throughout  P&P
process in form 4 &
5
50 min 40 min 60 min
2 hours  30 min
Construct Assessed
COMPETENCY
Competency based
on evidence process
and product
Competency based on knowledge evidence
Knowledge 1 Application 1 Creativity
(experiential) (one situation to
another situation)
(Innovation)
Context Coverage Context covered to all modules  in ...... form 4 and form 5
Difficulty level and
Weightage
Based on standard
in criterion
Section A Section B Sectioan C
Level of Difficulty
5
Level of Difficulty
3
Level of Difficulty
2
Accessories Allowed e.g scientific calculator, handphone
Example of Assessment Format for Malaysian Exam (English Version)
ASSESSMENT FOR (NAME OF SUBJECT e.g.Maths 7200/1))
NAME OF EXAMINATION (e.g.UPSR,PMR,SPM)

Test Construction

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Test Construction

Similar to Test Construction (20)

More from Azizi Ahmad

More from Azizi Ahmad (20)

Recently uploaded

Recently uploaded (20)

Test Construction