Corpora Terminology

1. Understanding Corpora Tables
By, Jamie Watanabe

2. • Counting words
• Tokens
• Types
• Lemmas
• Word families
• Frequency levels
Agenda/Topics to Be Covered

3. According to Nation, there are several ways of counting words, that is,
deciding what will be counted. (Nation, 2013)
Counting words

4. e.g. Its not easy to say it correctly
A simple way to count the preceding example is to count every word
form in a spoken or written text and if the same word form occurs more
than once, then each occurrence is counted. So, the example sentence,
would contain eight words, even though two of them are the same word
for, it. (Nation, 2013)
I.S.P. Nation- Learning vocabulary in another language, second edition
Tokens

5. e.g. Its not easy to say it correctly
Another way to count would be that if the same word occurs again, we
do not count it again. So the sentence of eight tokens consists of seven
different words or types. (Nation, 2013)
Types

6. e.g. cook and cooks
Counting the preceding example as two different words to be learned, is
strange. So, instead of counting different types as different words, closely
related words could be counted as members of the same word or
lemmas.
A lemma consists of headword and its inflected forms and reduced forms.
(Nation, 2013)
Lemmas

7. A word family consists of a headword, its inflected forms and its closely
related derived forms. (Nation, 2013)
Word families

8. Three kinds of vocabulary based on frequency levels.
For example, looking at academic text and examine the different
frequency levels of vocabulary it contains. The vocabulary is divided into
three groups according to frequency lists of word families.
• High frequency words
• Mid-frequency words
• Low frequency words
Frequency-based word lists

9. High frequency vocabulary
2,000 word families
(with proper nouns etc.-90% coverage)
Mid-frequency vocabulary
7,000 word families
(9%) coverage
Low-frequency vocabulary
(1% coverage) around 50,000 words
British National Corpus

10. Small group of high-frequency words which are very important because
these words cover a very large proportion of the running words in
spoken and written texts and occur in all kinds of uses of the language
According to Nation, there are 2,000 word families in high-frequency
vocabulary.
Michael West’s (1953) A General Service List of English Words, which
contains around 2,000 word families. About 165 word families in this list
are function words, such as a, some, two, because, and to.
High frequency words

11. A large group of generally useful words that occur rather infrequently,
but frequently enough to be sensible learning goal after the high-
frequency and specialized vocabulary is known.
Mid-frequency words consist of 7,000 word families from the third to the
ninth 1,000.
Depends on the Corpus
Mid-frequency words

12. There is a very large group of words that occur very infrequently and
cover only a small portion of any text.
Around 3% running words in the British National Corpus are like Carl,
Johnson, and Ohio.
Low-frequency words

13. Nation,I.S.P.(2013). Learning vocabulary in another language(2nd ed.).New
York: Cambridge University Press.
Reference