The importance of testing Distinctions among test, evaluation, and assessment Qualities of a language test: reliability, validity, etc.

1. FUNDAMENTAL CONCEPTS
AND PRINCIPLES IN
LANGUAGE TESTING
Subject: Language Testing
Instructor: Nguyễn Thanh Tùng, Ph.D.
Class: TESOL 2014B
1. Phạm Phúc Khánh Minh 4. Võ Thị Thanh Thư
2. Nguyễn Trần Hoài Phương 5. Đỗ Thị Bạch Vân
3. Nguyễn Ngọc Phương Thành 6. Ngô Thảo Vy

2. 1. The importance of testing
2. Distinctions among test,
evaluation and measurement
3. Qualities of a language test
CONTENTS

3. 1. The importance of testing

4. 1.1. The relationship of testing and
teaching
 “Testing and teaching are closely interrelated
that it is impossible to work in either field
without being constantly concerned with the
other.” (Heaton, J. B. 1988)

5. Good tests of
grammar,
translation or
language
manipulation
Good
communicative
tests of
language
1.1. The relationship of testing and
teaching

6. 1.2. The elements of a good
classroom test
A good test should:
 enable teachers to increase their effectiveness by
making adjustments in their teaching
 help to locate the precise areas of difficulty
encountered by the class or by the individual student
 enable the teacher to ascertain which parts of the
language programme have been found difficult by
the class
 provide the students with an opportunity to show their
ability to perform a certain task

7. 1.3. Aspects to be tested
What
should
be
tested?
Four skills in communicating:
listening, speaking, reading, and
writing
The language areas learnt:
grammar and usage, vocabulary,
and phonology
Language elements: nouns, verbs,
adjectives, and so on

8. 1.4. Testing the language skills
 It is important to concentrate on types of
test items which are relevant to the ability to
use language for real-life communication,
especially in oral interaction.
 Ways of assessing performance in the four
major skills may take the form of tests of:

9. listening (auditory)
comprehension (short
utterances, dialogues, talks
and lectures are given to
the learners);
speaking ability, usually in
the form of an interview, a
picture description, role play
and a problem-solving task
involving pair work or group
work;
reading comprehension
(questions are set to test the
students' ability to
understand the gist of a text
and to extract key
information on specific
points in the text); and
writing ability, usually in the
form of letters, reports,
memos, messages,
instructions, and accounts
of past events, etc.

10.  It is the test constructor's task to assess the
relative importance of these skills at the various
levels and to devise an accurate means of
measuring the student's success in developing
these skills.

11. 1.5. Testing language areas
 In an attempt to isolate the language areas
learnt, a considerable number of tests include
sections on:
 Grammar usage
 Vocabulary (concerned with word meanings,
word formation and collocations)
 Phonology (concerned with phonemes, stress
and intonation)

12. •to measure students' ability
to recognize appropriate
grammatical forms and to
manipulate structures
grammar
and usage
• to measure students' knowledge
of the meaning of words and the
patterns and collocations in which
they occur.
• may test their active or their
passive vocabulary
vocabulary
•might attempt to assess the 3 sub-skills:
ability to recognise and pronounce
the significant sound contrasts, ability
to recognise and use the stress
patterns, and ability to hear and
produce the melody or patterns of the
tunes (i.e. the rise and fall of the voice)
phonology

13. 1.6. Language skills and
language elements
 Testing students' ability to handle the elements
of the language or testing the integrated skills
depends both the level and the purpose of
the test.
 At all levels but the most elementary, it is
generally advisable to include test items which
measure the ability to communicate in the
target language.

14. 1.7. Main item types of tests
Recognition
to test the
recognition
of correct
words and
forms
Example: Choose
the correct
answer and write
A, B, C or D.
I've been
standing here ___
half an hour.
A. since B. during
C. while D. for
Production
to test if
students
can
produce
the correct
answer
Example:
Complete each
blank with the
correct word.
I've been standing
here ___ half an
hour.

15. 1.8. Sampling problems and
avoiding traps
 The test must cover an adequate and
representative section of those areas and skills
which it is desired to test.
 A good test should never be constructed in
such a way as to trap the students into giving an
incorrect answer.

16. 2. Distinctions among test,
evaluation and
measurement

17. 2. Distinctions among test, evaluation
and measurement
- Often used synonymously
- For example: Giving a test to
evaluate students’ language
proficiency
- Being essential to the
development and use of
language tests

18. 2.1. Measurement
 The process of quantifying the characteristics of persons
according to explicit procedures and rules
Features Quantification
Characteristics
Rules and procedures

19. 2.1.1. Quantification
- Assigning numbers
- Differing from qualitative descriptions such as visual
presentation, verbal or non-verbal accounts…

20. 2.1.1. Quantification
1 2
• Scales of measurement:
+ Assigning numbers
+ Non-numerical categories, etc.

21. 2.1.2. Characteristics
 Whatever attributes or abilities we measure, it is these attributes or abilities
and not the people themselves that we are measuring
- Both physical and mental characteristics
- Mental attributes: aptitude, intelligence, motivation, attitude, fluency in
speaking, etc.
- Mental abilities: being able to do something , performance on a set of mental
tasks
 The higher degrees of a given ability, the higher probability of correct
performance on tasks of lower difficulty or complexity

22. 2.1.3. Rules and procedures
Quantification must be done according to explicit rules
and procedures
The observation of an attribute must be replicable for
other observers, in other contexts and with other
individuals
Many types of measures: rankings, rating scales and
tests

23. 2.2. Test
A psychological or educational test is a procedure designed to elicit
certain behavior from which one can make inferences about certain
characteristics of an individual.
(Carroll, 1968:46)
For example: The Interagency Language Roundtable (ILR) oral interview – a
speaking test:
+ A set of elicitation procedures (activities, questions & topics)
+ A measurement scale of language proficiency (0 5)

24. 2.2. Test
 Designed to obtain a specific sample of behavior
 Provide the means for more focusing on the specific language
abilities that are of interest
 Viewed as supplemental to other methods of measurement
The best means of assuring the
sufficiency of the sample of
language obtained
 For example: the ILR oral interview, the TOELF, etc.

25. 2.3. Evaluation
requires
The ability of the
decision maker
The quality of the
information: reliable
and relevant
The systematic
gathering
information for
the purpose of
making decisions
For example:
+ Education decisions will be based on rumor
+ Sex and motivation are relevant to learning strategies

26. 2.3. Evaluation
- Not be exclusively quantitative information (verbal
descriptions, overall impressions, ratings, test scores, etc.)
- Not necessarily entail testing
- Tests can be for purely descriptive purposes - not evaluative
 It is important to distinguish the information-providing
function of measurement from the decision-making function of
evaluation

27. 2.4. Relationship among measurement,
tests, and evaluation

28. 2.4. Relationship among measurement,
tests, and evaluation
1. An evaluation excludes tests and measures
Ex: Qualitative descriptions of student performance
2. A non-test measure for evaluation
Ex: Teacher ranking used for assigning grades
3. A test for purposes of evaluation
Ex: Using achievement test to determine student progress
4. A test not used for evaluation
Ex: Using proficiency test as a criterion in SLA research
5. A non-test measure not used for evaluation
Ex: Assigning code numbers to school subjects

29. 3. Qualities of a language
test

30. 3. Qualities of language tests
Usefulness
of the test
Reliability
Construct
validity
Authenticity
Interactiveness
Impact
Practicality

31. 3.1. Reliability
Reliability
Consistency of
measurement
Consistent across
different characteristics
of the testing situation

32. 3.1. Reliability
Example:
If the score of for the first student given by 3 examiners is
10/10. However, the score for the second students is just 2/10.
 The scores is not consistent and would be considered to be
unreliable indicators of the ability we want to measure.

33. VALIDITY
the extent to which the test measures what it is
supposed to measure
Content validity
Construct validity
Face validity
3.2. Validity

34. CONTENTVALIDITY
The extent to which a test represent all
facets of tasks within the domain being
tested
Example: One teacher gives
students the final test. However,
the test only covers the material for
the last 3 weeks
 Low content validity
3.2.1. Content validity

35. 3.2.2. Construct validity
Construct validity
pertains to the meaningfulness of and
appropriateness of the interpretations that
we make on the basis of test scores
the characteristics of the
test task
construct definition

37. 3.2.3. Face validity
FACEVALIDITY
the extent to which a test is subjectively
viewed as a covering the concepts it
purports to measure
Example: After a group of students
sat a test, the teacher asked for
feedback., particularly if they
thought the test was a good one.

38. 3.3. Authenticity
the degree of
correspondence of the
characteristics of a given
language test task to the
characteristics of a TLU task
provide a link between
test performance and
the TLU tasks and
domain we want to
generalize
the way test takes
perceive the relative
authentic of test task
can facilitate their
test performance

39. 3.4. Interactiveness
Interactiveness is the extent and type of involvement
of the test take’s individual characteristics in
accomplishing a test task
Interactiveness is the heart
of many current views of
language teaching and
language leaning
Interactiveness is a
function of the extent and
type of involvement of the
test takes' language ability
and affective schemata

40. 3.5. Impact
Impact
Micro level:
individual
Macro level:
society,
education
system
Washback
(Backwash):
influence of
testing on
teaching and
learning.

41. 3.6. Practicality
 Practicality is the relationship between the resources
that will be required in the design, development,
and the use of the test and the resources that will be
available for these activities.
 A practical test is one whose design, development,
and use do not require more resources than are
available.
 Types of resources : human resources, material
resources, and time.

42. References
Heaton, J. B. (1988). Writing English language tests (New ed.).
London: Longman.
Bachman, L. F. (1997). Fundamental considerations in language
testing. Oxford: Oxford University Press.
Bachman, L. F. & Palmer, A. S. (1996). Language testing in Practice:
design and developing useful language tests. Oxford: Oxford
University Press.

43. Thank you!