How to evaluate a useful language test

1. What is a good
test?
How can someone
design a good test?

2. Test Joke
• Mother: Why did you get such a
low mark on that test?
• Junior: Because of absence.
• Mother: You mean you were
absent on the day of the test?
• Junior: No, but the kid who sits
next to me was.

3. Designing a good
classroom test

4. Some Basic Steps in Designing a
Good Classroom Test
• Determine the purpose of the test and other
basic info about it.
• Identify the appropriate test type/s
• Write the test
• Evaluate the test in terms of : practicality,
validity, and reliability
• Revise the test

5. What are three
important
characteristics
of a good test?

6. Three Important Characteristics of a
Good Test (Chitravelu, Sithamparam, Choon, 2005)
• Practicality
• Validity
• Reliability

7. How can one assess practicality?

8. Practicality
• Is the test worth the time,
money and effort?
• Is the test easy to
construct, administer, and
interpret?

9. What is validity?

10. What is validity?
“the truth of the test in relation to
what it is supposed to evaluate”

11. Validity
“Test validity involves three concerns: first,
one wants to know that a test is measuring
what it is supposed to measure; second, one
wants to know as fully as possible what the
score obtained from the use of the test
means, and third, one wants to know how
an individual’s score on a test relates to
other observable facts about the individual”
(Wolf, 1982).

12. Validity
Davies (1968) identified 5 types of validity:
1. Does it look like a good test?
2. Does it accurately reflect the
objectives of the syllabus it is
based on?
3. Can it accurately predict future
performance?
4. Are the results similar with other
tests?
5. Does it reflect the valid theory of
language learning that it takes as its
model?
Construct Concurrent Face Content Predictive
Construct
Concurrent
Face
Content
Predictive

13. Validity
What can you say
about a written test on
the oral proficiency of
students?

14. What is reliability?

15. What is reliability?
“Consistency of a test’s judgment or results”
Bynom (2001)

16. Reliability
“Consistency of measurement of individuals
by a test usually expressed in a reliability
coefficient” (p. 6).
- McNamara (2000))

17. Reliability
Inter-rater reliability = 0 to 1
Test to measure the oral proficiency of students.
If 23 out of the 30 raters have the same evaluation, we can
report that 23/30 or 77 % agree.
Reliability coefficient = .8

18. Should teachers aim for a perfect test?

19. Should teachers aim for a perfect test?
No, they shouldn’t because
there is no such thing as a
perfect test. It is simply a
matter of improving reliability,
practicality and validity. As
such, variety and frequency are
important aspects of language
testing.

20. What should teachers aim for?

21. What should teachers aim for?
• Validity
• Reliability
• Practicality

22. Task 1:Comment on the validity, reliability and
practicality of the following tests.
1. A group of waiters being assessed while they are working
in a restaurant after they have completed a course on English
for Waiters.
2. Unscramble the following letters to form English words:
AECMIN, AEUGG, TAC, & EKHCC
3. Reading aloud as a test of oral proficiency.
4. Write an essay on one of the following:
a. Describe what you did yesterday.
b. Write about your most memorable experience.
c. Write about your first date.

23. What should teachers aim for?

24. What should teachers aim for?
• Validity
• Reliability
• Practicality

25. What should teachers aim for?
• A Useful Test

26. On Test Usefulness
(Bachman & Palmer, 1996)
Test Usefulness = reliability,
construct validity,
authenticity, interactiveness,
impact, and practicality.

27. Three Principles
1. It is the overall usefulness of the test that is
to be maximized rather than the individual
characteristics that affect usefulness
2. The individual test qualities cannot be
evaluated independently, but must be
evaluated in terms of their combined effect
on the overall usefulness of the test

28. Three Principles
3. Test usefulness and appropriate balance
among the different qualities cannot be
prescribed in general, but must be
determined for each specific testing
situation.

29. Practicality
• Demands of a given test on existing resources.

30. Construct Validity
• Meaningfulness and appropriateness of
interpretations.

31. Reliability
• Consistency of measurement

32. Authenticity
• Correspondence between TLU task and test
tasks

33. Interactiveness
• Type and extent of test takers involvement

34. Impact
• Effects of tests on individuals, educational
institutions, and society.

35. What makes
your sample
tests useful?