This document summarizes research on how knowledge of multiword expressions affects reading comprehension. It discusses how previous studies have focused on how formulaic language affects second language production and processing, but less is known about its effect on comprehension. One study found that participants' comprehension of an idiomatic text was significantly lower than their reported comprehension, showing the "deceptive transparency" of idioms. The document then describes the development of a Phrasal Vocabulary Size Test to measure knowledge of multiword expressions in a similar way to single-word vocabulary tests, and findings that this knowledge correlates with better reading comprehension, especially of idiomatic texts. More research is still needed to fully understand this puzzle.

1. How does (a lack of) knowledge
of multiword expressions affect
reading comprehension?
Ron Martinez
AAAL, Dallas,TX - March 17th, 2013
(and Candy Palma)

3. Still putting together ‘the
(formulaic language) puzzle’
• “the puzzle of nativelike selection” (Pawley & Syder,
1983: 194)
• estimates of over 50% written text (Erman &
Warren, 2000)
• studies tend to focus on effect of formulaic language
(FL) on L2 production or processing (e.g. Boers, et
al, 2006; Conklin & Schmitt, 2008; Millar, 2011)
• still not a lot known about how FL affects
comprehension (perhaps more relevant?)

4. Martinez & Murphy (2011)
Let me tell you about my home. It’s on this little hill out in
the country. But I’m not far from the city (I don’t like the city
– do you?) – not much time to get here. I can’t wait to show
you a photo… or you can call me to come over to see in
person! 07786 237 679
I don’t get out much – it’s about time I do. I’m not from here
– this country or city. (But I like this country.) I’m far from
home. I’m a little over the hill, let me tell you, but you can’t
tell! (I can show you my photo, or wait to come see me in
person!) Call me on 07786 554 0978
exact
same
words
all very
frequent
words (top
2,000)

5. 1.
2.
3.
4.
5.
6.
7.
!He wants to go out but has a problem with time.
!He is foreign.
!He lives in a remote area.
!He wants to keep his location a secret.
!He thinks he looks younger than his age.
!He probably lives in an area with hills.
!He lives on a hill, or near one, but not on top of it.
My comprehension of this text: 5% 25% 50% 75% 100%
I don’t get out much – it’s about time I do. I’m not from here –
this country or city. (But I like this country.) I’m far from home.
I’m a little over the hill, let me tell you, but you can’t tell! (I can
show you my photo, or wait to come see me in person!) Call
me on 07786 554 0978

6. The results
Min. Max. Mean SD
Transpar.
Text
18 28 24.09 2.44
Idiomatic
Text
6 25 14.76 3.93
t = 24.10 (p ≤ 0.001), eta squared = 0.828

7. Reported Comprehension vs.Actual
Comprehension
• No statistically significant difference for Part 1
(87.38% reported vs 86.03% actual).
• Reported comprehension significantly
overestimated in Part 2 (t = 3.95, p≤ 0.001, eta
squared = 0.07) – 60.29% reported vs 52.58%
actual.
• "Deceptive transparency" (Laufer, 1989) extended.

9. ‘on occasion’
INTERMEDIATE
HIGHER

10. Questions about ‘frequency’
1. Is there a testable difference between the
high-frequency single-word vocabulary L2
learners know and multiword vocabulary of
the same frequency range?
2. Does that difference have any impact on
reading comprehension?
11

11. 20
Vocabulary Size Test (Nation & Beglar, 2007)

12. A tricky question...
How can a test be devised that assesses
knowledge of multiword expressions in the
same or similar way as current widely-used
vocabulary tests?
Ron Martinez

13. at all times at all costs at all
More compositional? Less compositional?
Meaning still retained when each
lexical word replaced with its own
definition (Grant & Bauer, 2004)

14. 14
transparent opaque
frequent
infrequent
Frequency-
Transparency
Framework (FTF)
(Martinez, 2013)
at all
at all costs
at all times

15. 38
PHRASE List (Martinez & Schmitt, 2012)

16. single word – multiword expression
frequency matching
39
BEFORE AFTERintegratedwordlist

17. 45
Phrasal Vocabulary Size Test
(PVST)
45

18. Does knowledge of the PHRASE List make a
difference?
• 108 (from 130+) adult Brazilian university students
(major: English)
• all students took both Martinez & Murphy (2011)
reading comprehension test and Nation & Beglar
(2007) Vocabulary SizeTest (VST) and Phrasal
Vocabulary SizeTest (PVST)
• 156 test items total (28 + 28 on Reading
Comprehension, 50 + 50 onVocabulary)
• Also administered questionnaire (e.g. age,AoO,
motivation, strategic investment, etc.)
18

19. How does PVST compare toVST?
• Candidates knew significantly (t = 7.67, p<.001)more items
onVST, but tests highly correlate (r = .90, p<.001) and both
reliable (Chronbach’s Alpha: VST = 0.92, PVST = 0.96, BOTH
= 0.97)
19
0.0
2.0
4.0
6.0
8.0
K1 K2 K3 K4 K5
VST PVST
25.0
26.5
28.0
29.5
31.0
Total
M SD
VST 30.49 9.16
PVST 25.78 13.1

20. 20
number incorrect = 69.45%

21. How does PVST (andVST) knowledge
relate to reading?
• Correlations all significant p<.001
• But...
• with Reported Comprehension (x2), effect observed only with
VST x IdiomaticText (F (2,92) = 33.19, p<0.05, eta squared
0.05)
• Participants who performed relatively well on single-word
vocabulary tended to think they understood more than they
did when texts contained opaque idiomaticity.
21
Transp.
Reading
Idiom.
Reading
VST r = 0.68 r = 0.60
PVST r = 0.64 r = 0.63

22. What does PVST add?
• PVST cut-off set at 25 (mean was 25.78)
• If VST >25 and PVST ≤ 25 (n=36)
• If VST >25 and PVST >25 (n=49)
22
M (28 max.) SD
Transp. Text 25.88 1.12
Idiom. Text 12.52 2.41
M (28 max.) SD
Transp. Text 26.16 1.07
Idiom. Text 16.22 4.22
• PVST seems to add some predictive validity.

23. What about the questionnaire?
23

24. 24

25. • Among reported time (per week) reading,
listening to, and viewing media in English...
• only reading significantly impacted bothVST and
PVST performance (Kruskal-Wallis: H(4) =22.02/19.04,
p <0.05, respectively)
• and TV viewing only significantly interacted with
PVST (H(4) =11.30, p<0.05)
• Length of Study seemed to matter forVST
(F(27,76) =1.80, p <0.05, eta squared 0.09), but not the
PVST.
25

26. Vespoor, M.H., de Bot, K., and van Rein, E. (2011). English as a foreign language:
The role of out-of-school language input. In A. De Houwer and A. Wilton (Eds.),
English in Europe Today: Sociocultural and educational perspectives (pp. 147-166).
Amsterdam: John Benjamins.

27. Discovering America?
• Vocabulary size important.
• Vocabulary helps reading.
• Reading helps vocabulary.
•Measuring one kind of vocabulary
perhaps provides an incomplete
picture.

28. Still plenty of the puzzle
missing...
• Working memory?
• Time abroad?
• Effect on production?
• What about less "polarized" reading
measure?
• Other L1s, more interview data, more
analysis, less back surgery...
28

29. Thank you
ronmartinez@sfsu.edu