This presentation discusses the concepts of validity and reliability as they relate to examination scores. Validity refers to whether a test measures what it intends to measure, and there are three types: content validity, construct validity, and criterion validity. Reliability refers to the consistency and stability of test scores and is concerned with minimizing errors. A test can be reliable but not valid, as reliability is a necessary but not sufficient condition for validity. Both concepts are important for determining the quality and usefulness of examination scores.

1. This presentation was developed for the
exclusive use of students enrolled in:
Educational Testing & Grading,
Professor Gregory E. Stone.
© 2004 Gregory E. Stone. All rights reserved. This presentation may not be
reproduced in any form, in part or as a whole, without the express written permission
of the author.

2. Exam Score
Quality
Validity & Reliability

3. No matter how well the
objectives are written,
or how clever the items,
the quality and usefulness
of an examination is
predicated on Validity
and Reliability
Validity & Reliability

4. Evaluative
Examination Score
Quality
Statistical
Validity & Reliability

5. Validity
Examination Score
Quality
Reliability
Validity & Reliability

6. Validity
BOTH CONCEPTS
RELATE TO THE
SCORE NOT
THE TEST!
Reliability
Validity & Reliability

7. Validity Reliability
We don’t say “an exam is valid and reliable”
We do say “the exam score is reliable and
Valid for a specified purpose”
KEY ELEMENT!
Validity & Reliability

8. Reliability
“Was the measure of student
ability reached by accident or
chance, or was it reached through
a clear and stable, meaningful
examination?”
Validity & Reliability

9. Reliability
Answers questions relative to
the stability and clarity of an
examination.
Validity & Reliability

10. Reliability
Reliability is a STATISTIC.
Specifically, a correlation
ranging from
0 = Completely Unreliable
1 = Perfectly Reliable
Validity & Reliability

11. Reliability
Reliability is also a measure of
error within an examination.
Poor wording, ambiguity,
correct answer issues, failure
to link with objectives, etc.
Validity & Reliability

12. Content/
Action + Error
Validity & Reliability

13. The information we seek and our
best hope for obtaining it.
Content/
Action + Error
Validity & Reliability

14. The information we seek and our
best hope for obtaining it.
Content/
Action + Error
Our human frailty and inability to
write effective questions.
Validity & Reliability

15. Reliability
Reliability comes in multiple
forms, including:
Test-Retest Reliability
Split-Half Reliability
Validity & Reliability

16. Reliability
Test-Restest
Administer to students.
After time, administer again.
Are the students performing in
a consistent manner?
Validity & Reliability

17. Reliability
Split-Half
Create 2 smaller content
Balanced tests and administer.
Do students perform similarly
on each test half?
Validity & Reliability

18. Validity
“Did the test we created really
measure what we hoped it
would measure?”
Validity & Reliability

19. Validity
Relates to questions about the
reasonability of content,
criterion and construct
associated with the learning
objectives and assessment.
Validity & Reliability

20. Validity & Reliability

21. Validity & Reliability

22. Validity
Content Related Validity
Does the content we have selected
appropriately represent our
construct (top level domain)?
Validity & Reliability

23. Validity
Construct Validity
Is there really such a concept as
what we created as a construct?
(e.g. Does “reading ability” really
exist?)
Validity & Reliability

24. Validity
Criterion Related Validity
Does our test adequately link the
content to the construct?
Is the test a good measure of the
construct?
Validity & Reliability

25. Validity
Examination Quality
Reliability
Validity & Reliability

26. Validity & Reliability

27. RELIABILITY
is a necessary but
not sufficient condition for
VALIDITY
Validity & Reliability