This document outlines the key steps in constructing a language test: 1. Developing test specifications that describe the test purpose, test takers, skills assessed, and other details. 2. Writing and revising test items based on the specifications. 3. Trying out the test on students and analyzing results to evaluate item difficulty and effectiveness. 4. Validating the test through expert review and comparing scores to other measures of student ability. 5. Conducting a public trial with future test users to get feedback and approval before finalizing the test.

1. TEST SPECIFICATION
TEST CONSTRUCTION
MODERATION (REVISION)
TRY-OUT
TEST ANALYSIS
VALIDATION
PUBLIC AND USER TRIAL
•Baso Jabu
•Universitas Negeri Makassar
1TEST CONSTRUCTION PROCEDURES

2. Test Specifications
A test’s specifications provide the official statement
about what the test tests and how it tests it
 What is the purpose of the test?
 What sort of learner will be taking the test?
 How many sections should the test have?
 What text types should be chosen – written/spoken?
 What language skills should be tested?
 What language elements should be tested?
 What sort of tasks are required
2TEST CONSTRUCTION PROCEDURES

3. Test Specifications
 How many items are required for each section?
 What test methods are to be used?
 What rubrics are to be used as instructions?
 Which criteria will be used for assessment by markers?
3TEST CONSTRUCTION PROCEDURES

4. Test Specifications
Test specifications should include all or some of the following:
 The test’s purpose
 Description of the test taker
 Test level
 Construct (theoretical framework for test)
 Description of suitable language course or textbook
 Number of sections/papers
 Time of each section/paper
 Target language situation
 Text-types
 Text length
 Language skills to be tested
4TEST CONSTRUCTION PROCEDURES

5. Test Specifications
Test specifications should include all or some of the following:
 Language elements to be tested
 Test tasks
 Test methods
 Rubrics
 Criteria for marking
 Descriptions of typical performance of each level
 Descriptions of candidates at each level can do in the
real world
 Sample papers and Samples of students’ performance
on tasks
5TEST CONSTRUCTION PROCEDURES

6. Test Specifications
Three Domains
of Taxonomy
(Bloom):
Cognitive
Knowledge
Affective
Attitudes
Psychomotor
Skills
6TEST CONSTRUCTION PROCEDURES

7. Test Specifications
Cognitive
 Cognitive domains are objectives which emphasize
remembering or reproducing something which has
presumably been learned, as well as objectives which
involve the solving of some intellectual tasks for which
the individual has to determine the essential problem
and then reorder given material or combine it with
ideas, methods, or procedures previously learned.
7TEST CONSTRUCTION PROCEDURES

8. Test Specifications
Cognitive Domain
Six Major Areas
Evaluation
Synthesis
Analysis
Application
Knowledge
Comprehension
New Forms (Verbs)
Creating
Evaluating
Analyzing
Applying
Understanding
Remembering
8TEST CONSTRUCTION PROCEDURES

9. Test Specification
Cognitive Domain
 Knowledge refers to the recall of specific information;
 Comprehension refers to an understanding of what was
read;
 Application refers to the converting of abstract content to
concrete situations;
 Analysis refers to the comparison and contrast of the content
to personal experiences;
 Synthesis refers to the organization of thoughts, ideas, and
information from the content; and
 Evaluation refers to the judgment and evaluation of
characters, actions, outcome, etc., for personal reflection and
understanding
9TEST CONSTRUCTION PROCEDURES

10. Test Specification
Cognitive Domain
 Remembering indicates recalling information
(recognising, listing, describing, retrieving, naming, finding).
 Understanding signifies explaining ideas or concepts
(interpreting, summarising, paraphrasing, classifying, and
explaining).
 Applying conveys using information in another familiar situation
(implementing, carrying out, using, executing).
 Analysing denotes breaking information into parts to explore
understandings and relationships
(comparing, organising, deconstructing, interrogating, and
finding).
 Evaluating implies justifying a decision or course of action
(checking, hypothesising, critiquing, experimenting, and judging).
 Creating refers to generating new ideas, products, or ways of
viewing things
(designing, constructing, planning, producing, inventing).
10TEST CONSTRUCTION PROCEDURES

11. Test Specification
Affective Domain
 Affective domains are objectives, which emphasize a
feeling tone, an emotion, or a degree of acceptance
or rejection. Affective objectives vary from simple
attention to selected phenomena to complex but
internally consistent qualities of character and
conscience. A large number of such objectives in the
literature expressed as
interests, attitudes, appreciations, values, and
emotional sets or biases.
11TEST CONSTRUCTION PROCEDURES

12. Test Specification
Affective Domain
 Affective domains:
 Spiritual Attitude:  pious, virtuous
 Social Attitude:
1. Honest  to oneself and others
2. Disciplined  punctual and orderly
3. Responsible  take risk and apologize
4.Tolerance  respect others
5. Cooperative  work together
6.Polite  respect and behave well
7. Self-confident  brave and courageous
12TEST CONSTRUCTION PROCEDURES

13. Test Specification
Psychomotor Domain
 Psychomotor domains are objectives, which
emphasize some muscular or motor skill, some
manipulation of materials and objectives, or some
acts that requires a neuromuscular coordination.
13TEST CONSTRUCTION PROCEDURES

14. Test Specification
Example
Subject: _________________________________________
Semester/Class: __________________________________
School: __________________________________________
Test description: _________________________________
No
Basic
Competence
Objective/
Indicators
Domain Techniques Items
1. Listening … C1, C2 MC 1 – 10
2. Speaking C3 … …
3. Reading …
4. Writing
14TEST CONSTRUCTION PROCEDURES

15. Test Specification
Example: Reading
Subject: _________________________________________
Semester/Class: __________________________________
School: __________________________________________
Test description: _________________________________
Item Cognitive Techniques Objectives Item Types Remarks
1. C2 MC Finding main
idea
Topic
sentence
2. C4 MC Comparing
Paragraphs
Difference …
3. C1 MC Finding
explicit info
Facts or
Realities
…
4. … … … … …
15TEST CONSTRUCTION PROCEDURES

16. Test Construction and Moderation
Item Writing and Revision
 In writing test items, one should ideally combine both necessary
formal professional
 The item writing must be based on the test specifications
 The method used for testing a language ability may itself affect
the students’ score, which is called the method effect
 It is likely that particular test methods will lend themselves to
testing some abilities, and not be so good at testing others
 In terms of test editing or moderation, each item and the test as
a whole are considered for the degree of match with the test
specifications, likely level of difficulty, possible unforeseen
problems, ambiguities in the wording of items and of
instructions, problems of layout, match between stems and
choices, and overall balance of the subtest or paper.
16TEST CONSTRUCTION PROCEDURES

17. Try-out
 However well designed a test may be, and however carefully
it has been edited, it is not likely to know how it will work
until it has been tried out on students.
 We do not only need to know how difficult the test items
are, but we also need to know whether they work.
 The number of students on whom a test should be trialed
depends on the importance and type of test, and also the
availability of suitable students. The only guiding rule is
the more the better, since the more students there are, the
less effect change will have on the results.
17TEST CONSTRUCTION PROCEDURES

18. Test Analysis
 This analysis will show us the extent to which each item works.
For objective test items, traditionally there are two measures of
calculation – the facility value and the discrimination index.
 For subjectively marked tests although item analysis is
inappropriate, such as summaries, essays, and oral
interviews, these tests still need to be tried out to see whether
the items elicit the intended sample of language; whether the
marking system, which should have been drafted during the
item writing stage, is usable; and whether the examiners are able
to mark consistently.
 Once the papers or interviews have been administered, there
should be trial marking sessions to see whether the test item
prompts have produced the intended kinds of responses, and
whether the marking guidelines and criteria are working
satisfactorily.
18TEST CONSTRUCTION PROCEDURES

19. Validation
 The most important question of all in language testing
is validity.
 The validation process involves the terms internal and
external validity, with the distinction being that
internal validity relates to studies of the perceived
content of the test and its perceived effect, and
external validity relates to studies comparing students’
test scores with measures of their ability gleaned from
outside the test.
 The commonest types of external validity are
concurrent validity and predictive validity.
 Expert Validation
19TEST CONSTRUCTION PROCEDURES

20. Public or User Trial
 The tests that have been constructed, tried out, and
analyzed should also be evaluated by public, especially
the future users of the tests. The tests are presented to
the future users and they analyze the tests, give
comments or suggestions to the improvement of the
tests, and approve the tests.
20TEST CONSTRUCTION PROCEDURES

21. Exercises
 What are the procedures in constructing a test?
 What should be included in the test specifications?
 Explain the three domains of taxonomy.
 What are the new terms in cognitive domains?
21TEST CONSTRUCTION PROCEDURES