1. Testing grammar and vocabulary
(Arthur Hughes)
Daniel Díez Lorenzo
Assessment and Foreign Language Teaching According to the CEFRL
Second Language Learning and Teaching - University of Cantabria
2. Introduction: main concepts
● Achievement test: specific objectives of a language course.
● Proficiency test: overall use of the foreign language in general.
● Placement test: setting students at a particular stage.
● Diagnostic test: identify strengths and weaknesses.
● Backwash effect: “the direct or indirect effect of examinations on teaching
methods” (Prodromou, 1995).
3. Why test grammar?
● Grammar as the core of language ability. Language skills are the
focus. Grammar is an ability that underlies these skills.
○ Language skills are directly tested.
○ The backwash effect is preferable.
● Proficiency tests still have a grammar section.
○ More items to be administered and scored in a short period of time.
○ Content validity: it covers a wide range of grammatical elements.
● Grammar should be included in achievement tests too.
○ Since the development of skills remains the primary objective, grammar should not be given
too much prominence in order to avoid the backwash effect.
● Grammatical ability is necessary to perform in language skills.
○ Therefore, information on grammar ability is very useful.
○ Diagnostic tests of grammar are necessary.
SHIFT
4. Writing specifications & sampling
● Achievements tests:
○ The syllabus includes grammatical structures on the teaching objectives.
○ Inferring grammatical structures from textbooks and other materials.
● Placement tests:
○ Identifying all the structures and those of lower levels.
● Proficiency and diagnostic tests: notional-functional approach.
● Sampling: selecting most important structures to test content
validity
5. Writing items
● Grammatically correct-
structures and natural
language (not always the
case)
○ Corpus-based examples.
● Four techniques:
○ Production:
■ Gap filling.
■ Paraphrase.
■ Completion.
○ Recognition:
■ Multiple choice.
Objective Proficiency - Cambridge University Press
6. Gap filling
● Filling gaps in one text.
● Just one correct answer per gap.
○ Many correct answers if the meaning is
the same.
■ Example: which/that
● Every possibility of answer needs
to be clear.
■ Example: contractions
● An example of answer should be
given at the beginning.
Cambridge English: Proficiency 1 with answers
7. Paraphrase
● Writing a sentence
equivalent in meaning
to one that is given.
● It is helpful to give part
of the paraphrase.
Cambridge English: Proficiency 1 with answers
8. Completion
● A variety of structures can
be tested.
● Specific structures are
elicited, which students will
have to complete according
to the context.
FCE sample from Testing Grammar and Vocabulary
(Hughes, 2007)
9. Multiple choice
● Instead of gap filling, this
can be used to reduce
the possible answers of
the gap.
● Adding context helps also
reduce the possible
answers.
● Distractors have a key
role.
My Grammar Lab B1/B2 - Pearson
10. Scoring production grammar tests
● We should be clear about what each
item is testing.
● Points are awarded for the correct
use of structures we are testing.
○ No points should be deducted from non-
grammatical errors or grammatical errors
from structures we are not testing.
● We need to make clear how scoring
goes = reliable scoring.
○ Preparation of the scoring key.
Objective Proficiency - Cambridge University Press
11. Why test vocabulary?
● Teaching vocabulary is devoted less time than teaching grammar.
● Little call for achievements tests of vocabulary only?
○ It is hoped that vocabulary learning is taking place.
○ If vocabulary is explicitly taught, vocabulary achievements tests are appreciated for
their backwash effect.
● No need of general diagnostic tests of vocabulary.
● Placement tests:
○ We just look for the adequacy of the student’s vocabulary.
12. Writing specifications & sampling
● Achievement tests:
○ If vocabulary is consciously taught, the items should be included in the specifications.
○ We can test other items that students have met in other activities.
● Vocabulary placement test = proficiency test.
○ Specifying the lexical items in reference to published word lists.
● Sampling:
○ Words are grouped according to:
■ frequency.
■ usefulness
13. Writing items: testing recognition ability
● Multiple-choice questions: guessing the meaning.
● No harmful backwash effect.
RECOGNISE SYNONYMS/DEFINITIONS RECOGNISE APPROPRIATE WORD IN
CONTEXT
● It is important to use distractors that
are not too common and have the
same lenght.
● Context helps.
● Providing context in an item makes
the task more authentic.
● Context helps activate memory.
Which of the following items means “in fact”?
a) therefore
b) because of
c) actually
d) otherwise
“___ rain is likely to occur in the north of Spain.You’d
better take your umbrellas.”
a) damp
b) heavy
c) acid
d) powerful
14. Writing items: testing production ability
● Normally, production is not tested, since it is too difficult. Receptive ability is
considered to be sufficient.
PICTURES DEFINITIONS GAP FILLING
● Restricted to concrete nouns that
the tester have in mind.
● Not all words can be defined in
simpler words.
● The possible answers should be
provided in a box so that there are
no alternative answers.
Testing Grammar and Vocabulary (Hughes, 2007)
15. Conclusions
● Grammar and vocabulary are not an end themselves; they contribute to the
development of communicative skills.
● It is important to provide items that test both recognition and production of
grammar structures and vocabulary, although receptive skills are thought to
be sufficient regarding the last one.
● Scoring features in grammar may be applied to vocabulary as well.
● We need to make the teaching method agree with the testing method so that
there is no negative backwash effect.
16. References
● Cambridge English: Proficiency 1 with answers. (2012). Cambridge University
Press.
● Capel, A., Sharp, W. (2013). Objective proficiency: student’s book with
answers (Second Edition). Cambridge University Press.
● Foley, M., Hall, D. (2014). MyGrammarLab Intermediate B1/B2 (Fourth
Edition). Pearson Education Limited.
● Hughes, A. (2007). Testing grammar and vocabulary. En Testing for language
teachers (pp.172-185). Cambridge: CUP.
● Prodromou, L. (1995). The backwash effect: from testing to teaching. ELT
Journal, Volume 49, Issue 1, 1 January 1995, pp. 13–25,
https://doi.org/10.1093/elt/49.1.13
