Andrew Bax & Bruce Bax: Using an online tool to benchmark texts to the CEFR: Textinspector.com
1. TextInspector.com: An online tool for benchmarking
texts to the CEFR
Andrew & Bruce Bax
bruce.bax@newcastle.ac.uk
textinspectorhelpteam@gmail.com
How can I
measure the lexis
in my students’
essays?
Is this text
too hard for
my class?
I’m preparing a
TEST – which text
is best?
8. Thanks to…
Text Inspector Team
Canterbury Christ Church University
CRELLA – University of Bedfordshire
The Open University
NILE - Norwich Institute for Language
Education
Users – 250,000 since 2015
10. As we seek to learn and teach in this
uncertain, troubled, brave new digital
world….
how can we blend technology with
human intervention in the most
productive and painless way?
11. Normalisation
“the stage when the technology becomes
invisible, embedded in everyday practice”
“the stage when a technology is ….. hardly
even recognised as a technology, taken for
granted in everyday life.”
(Bax 2003:23)
12.
13.
14. How can you use it?
Checking student writing (e.g. to help with mass
testing)
Checking texts for classroom use
Researching texts
15. But…
Text Inspector is not a complete, automated
marking system
Text Inspector cannot measure everything (e.g.
relevance)
The teacher still needs to use judgement
So we should see it as a valuable tool, not a
panacea
16. Why vocabulary?
“64% of variance in the reading score is
accounted for by vocabulary”.
(Laufer & Ravenhorst-Kalovski 2010:26,
emphasis added).
17. Last night the earth trembled. It was half past three and everybody was sleeping. Before going to bed I
felt a little bit strange but I couldn’t imagine that something like that would have happened. Suddenly I
felt my bed quaking, I began calling my parents because I was completely shocked, the dogs started
barking, it seem to me to be the character of an action film. It lasted few seconds but it seemed to last
forever.
After that I tried to sleep but it was impossible for me so I turned on the TV and I will never forget
what I’ve seen. There was rubble everywhere, people who was digging with bare hands, who were
looking for their friends and their relatives. I stayed in front of the TV the whole day and the news was
getting worse. Many people were still scared and they couldn’t realize what had happened, somebody
has lost a son, a niece, a uncle and somebody else has lost everything they owned. The homeless began
to get in touch with their relatives to know if they were still alive, they started to find a place where
they could stay because their house was completely destroyed like Onna inhabitants, a small village
that was entirely reduced to rubble.
I think that if the houses had been built according to the anti- seismic system, more people would have
been saved, there wouldn’t have been as many people without a place to live. The government said that
many buildings were made of stone and this is not possible in a seismic area like Abruzzo. Some
people said that an event like that can’t be predict but a geologist called Giuliani studied the previous
quakes and he had imagined that something terrible would have happened but he was accused of
creating panic.
In front of that I felt a bit useless, I would like to help these people but I don’t know what to do. In my
opinion, one of the most important things that the government should do after this catastrophe is to
rebuild these villages in a safe way. The only thing the Italian could do at this moment is collect money,
clothes, blankets to help them to come back to their everyday life.
TextInspector.com
18.
19.
20.
21.
22.
23. Example: reading data
Large dataset
Aim: to identify criterial distinguishing
features
(e.g. words per sentence)
24. Result: a set of metrics (16)
Base statistics
Average Syllables per sentence
Average words per sentence
Readability
Flesch Kincaid Reading Ease
Flesch Kincaid Reading Grade
Gunning Fog
Propositional density
Verbal elements per sentence
Nominal elements per sentence
Lexical sophistication
EVP: % of words (types) at A1 level
EVP: % of words (types) at B2 level
EVP: % of words (types) at C1 level
BNC 50th percentile (Types)
BNC 60th percentile (Types)
COCA 50th percentile (Types)
COCA 60th percentile (Types)
Lexical sophistication (academic)
Academic Word List: % of all AWL words in
the text
Metadiscourse
Percentage of Logical Connectives (tokens)
25.
26. Text Inspector uses datasets:
- Tagged in terms of CEFR levels
- (Ideally) tagged grammatically (e.g. N, V)
or with Treetagger
- Not in lemma form, but in inflected form
(e.g. says, saying, said)
- E.g. B1,say,said,VP
A2,decir,dicho,PP
- Other languages please!
27. So datasets in other languages with:
- Researched frequency data
- CEFR tags (ideal but not essential)
- Grammatical tags (ideal but not essential)
And we can try to make a free text analysis
tool in your language.
TextInspectorhelpteam@gmail.com
28. Discourse and text analysis
www.textInspector.com/francais
/espanol
etcetera
29. Summary: Text Inspector is
Based on meticulous research
Easily accessible / cheap
As detailed as you want it to be
Very straightforward to use
30. But…
Text Inspector is not a complete, automated
marking system
Text Inspector cannot measure everything (e.g.
relevance)
The teacher still needs to use judgement
So we should see it as a valuable tool, not a
panacea
31. Thank you
Bruce Bax & Andrew Bax
bruce.bax@newcastle.ac.uk
textinspectorhelpteam@gmail.com
Discount code: EnglishAgenda7
32. How others have used Text Inspector
• “To inform you a bit about myself, I would like to tell you that I am a teacher of English with over 17
years of teaching experience.
• However, for the past 7 years, I have been working specifically as an exams teacher helping candidates
prepare for English proficiency tests, especially IELTS and TOEFL.
• As for my job, I sometimes have to prepare materials for my lessons. To that end, I always try to use
authentic resources such as journals, weblogs, newspapers, etc. as extracts, either in audio or written
forms, to use for my lessons.
• "Text Inspector" proved to be a very useful tool in determining the difficulty of passages which I have
to choose to use in my lessons so as to help my students develop their language skills, especially
reading and listening.
• One good experience I had with this tool when I first used it was that I realised why students tend to do
better with some passages than with others. Then, I began to use this tool to find materials suited to my
students' CERF level, where they could cope with the material more confidently and they were less
likely to feel overwhelmed and disappointed by the materials the difficulty of which fell way beyond
their ability.
• I suppose this tool is a must for exams teachers as well as professional ESL teachers, especially those
who also need to design and develop their own materials and curriculums, respectively. ”
Mohammed, Turkey
33. “ My current research centers around lexical development and diversity,
and in particular longitudinal lexical development.
I have collected scripts of learners and tried to see if there was an upward
trend in longitudinal Type Token Ratio (TTR) data within the period of a
year. Text inspector has helped me analyze the trend, which throws great
light on second language acquisition.
I teach in a South Korean context, where vocabulary is emphatically
emphasized in the syllabus and curriculum, so I thought that an
investigation regarding whether productive work from learners correlates
to the input they receive would be both insightful and interesting.”
Peter, South Korea
How others have used Text Inspector
(continued…)
