To assert that a test has construct validity; empirical
evidence is needed.
Subordinate forms of validity.
A valid test
it is intended to
The content of the test constitutes a respresentative sample of
the skills it is supossed to measure = Content Validity
Specification of the skills or structures that the test is meant to
The test must include a
proper sample of the
Content validation should
be carried out while a test
is being developed.
•The test and the criterion are administer at the same time.
•Example: Oral exam. Long vs. Short version of the exam.
•Levels of agreement = correlation coefficient.
•Perfect agreement = coefficient of 1. Lack of agreement =
coefficient of zero.
•The degree to which a test can predict candidates’ future
•Example: Profiency test to predict a student’s ability to cope
with a graduate course at a British university. Criterion
measure: student’s English perceived by his supervisor or the
outcome of the course.
Validity in scoring
Items and the way in which they are scored
Example: A reading test.
(Should we consider grammar and spelling
mistakes in the responses?)
A test is said to have face
For example: A test to
How to make tests more valid
The scoring must
be related to what
is being testing.
for the test.
We have to: construct, administer and score items in
a way that we will obtain similar results in different
The reliability coefficient
To quantify the realibility of a test.
coefficient = 1 Would always
give the same
Sets of results
with each other.
It is required
to have two
sets of scores
A group of students
take the same test
1. Too soon
2. Too late
Split half method =
If the scoring of a test is not reliable, then the test results cannot
be reliable either.
The scorer reliability coefficient on a composition writing
test = .92
The reliability coefficient for the test = .84
Variability in the performance of individual candidates
accounted for the differece between the two coefficients.
How to make tests more reliable
Do not allow
tests are well
laid out and
1. More items = more reliability
2. Too easy and too difficult items
3. Choice of questions
4. Unclear meaning of the items
5. The supposition that the students all understand the instructions
6. Institutional tests are badly typed
7. Unfamiliar aspects of the test
8. Precautions must be taken
Ways of obtaining scorer reliability
Use items that
which is as
direct as possible
Provide a detailed
at outset of scoring
by number, not
Reliability and validity
To be valid a test must provide consistently accurate
A reliable test may not be valid at all.
Example: writing test
To make tests reliable, we must be wary of reducing