The effectiveness of multimedia lexical knowledge representation as a learning tool This study examined the effectiveness of using schematic visual representations of phrasal verbs (PVs) in an online dictionary compared to a traditional textual dictionary. 121 Japanese university students were randomly assigned to study 12 English PVs using either a multimedia application with visual schemas or a printed handout. Both groups showed improved comprehension from pre- to post-test, but the application group performed significantly better on figurative senses of PVs. While both representations helped with concrete senses, the visual schemas more effectively supported understanding metaphorical uses. The findings suggest multimedia dictionaries can enhance L2 vocabulary learning compared to traditional formats, especially for complex multi-word units like phrasal verbs

1. The effectiveness of
multimedia lexical
knowledge representation
as a learning tool
Takeshi SATO (Tokyo Univ. of Agri. & Tech., Japan)
Tyler BURDEN (Meisei Univ., Japan)
Presentation for AsiaLex 2017 on 11th of June
@ Guangdong University of Foreign Studies

2. Overview
1. Introduction
2. Research Questions
3. Research Procedures
4. Findings & Discussion
5. Conclusion

3. Is an electronic dictionary merely
a reference tool?
Genius English-Japanese Dictionary (3rd version)

4. Advantage of computational
knowledge representation
•Multimodality: the combination of text,
images and sound
•Longer knowledge retention
(Lindstromberg & Boers 2008, Sato, Lai & Burden
2014, Yoshii & Flaiz, 2002 etc.)
•Better text comprehension
(Sato, Matsunuma, & Suzuki 2013)

5. Advantage of online dictionaries
•On-screen presentation can make
linguistic features of a lexical item
salient.
from a reference tool to a
learning tool (Pachler, 2001)

6. Which lexical items
to focus on?
Which linguistic features
to make salient?

7. Which lexical items
to focus on?
English Phrasal Verbs

8. Phrasal verbs (PVs)
• Multiword unit consisting of a verb and a
preposition (adverb)
• Both words are polysemic.
• “[O]ne of the most challenging features of the
English language” (Garnier & Schmitt 2016,
p.30)
• Cannot acquire PVs by memorizing as an idiom
(Lindstromberg, 2001b)
• Not only language teachers but also cognitive
linguists are interested in PVs (ex. Dirven, 2001;
Rice 2003; Rudzka-Ostyn, 2003)

9. Complicated semantic network
Take Over
(Norbig & Lakoff, 1987) (Tyler & Evans, 2003)

10. “Go off”
1. explode
2. start making a noise
3. electricity, etc.: not work
4. leave
5. happen
6. become angry
7. food/drink: not fresh
8. become worse
9. stop liking
someone/something
10. begin sleeping
(from Macmillan Dictionary)
•Professor Buden
went off to get a
drink.
(concrete sense)
•Professor Burden
went off Mr. Sato
after an
argument.
(figurative sense)

11. Drawbacks of a dictionary
(Lindstromberg, 2001a)
1. jumbled or fragmented positioning of
information
2. lack of attention to metaphor
3. lack of information about paradigmatic
semantic contrasts
4. lack of pictorial illustration
5. flawed sense information
6. difficult language
7. omitted usages

12. Which linguistic features
to make salient?
Schema & Prototype

13. Schema-extension model (Langacker,
1998, 2000, Tayler, 2002)
C. Schema
B. ExtensionA. Prototype
Semantic extension cannot be done without schematization
abstract commonality
between A and B
schematization

14. Process to understand L2 PVs with
schema
C. Schema
The fire alarm went
off this morning
The meeting went
off well.
Prof. Burden went
off Mr. Sato after an
argument .
Prof. Burden always
goes off at a tangent
Prof. Burden
went off to get a
drink.
Prof. Burden
went off with all
the money.
Relates the
senses with
each other
Metaphorical comptence is crucial
for L2 learners (Boers, 2000, Littlemore, 2001)

15. Examples of schema
Dewell (1994)

16. Suggestions to improve the
drawbacks (Lindstromberg, 2001)
• pictorial representation of meaning
• explicit division between literal and
metaphorical uses
• clear division between a preposition and
other related prepositions
• stop treating uses in order of frequency
• standardization of the organization of entries
according to semantic views such as
prominent highlighting of succinct rubrics.

17. Suggestions to make a dictionary
a learning tool
•Representing schemata of the
target words
•Showing a prototypical sense first
•Dividing literal and metaphorical
senses
•Dividing clearly a preposition/verb
and other related
prepositions/verbs

18. 2. Research Questions

19. 1. Do schematic visual glosses
enhance L2 text comprehension
with L2 PVs compared with verbal
glosses?
2. Which senses of L2 PVs do the
schematic visual glosses enhance,
concrete senses or figurative
senses?

20. 3. Research Procedures

21. 1. 121 students randomly divided into
two groups (handout/application)
2. Pre PV test (24 questions within 10
minutes)
3. Self-study with the treatments (5
minutes)
4. Post PV test (Same as the pretest)
5. Analysis (t-test)

22. Participants
• 121 Freshmen from
two Japanese
universities
• Technology
• Commerce
• Randomly divided into
two groups
1. Handout (PVs, core
sense, index of the
meanings, example
and its translation)
2. Application with
schematic images of
the core sense
Cited 8th of June from
https://goo.gl/1T4pJp

23. Target PVs
give off
give in
give out
give over
go in
go off
go out
go over
keep in
keep off
keep on
keep out

24. Application

25. Handout
語句 基本イメージ 意味 例文 和訳
1 give off
give:与える＋off:～から
離れて
【他】(1)＜熱・におい・光・煙
＞を発する (2)＜木などが＞
（小枝などを）出す
I switched off the heater 20
minutes ago, but it is still
giving off heat.
20分前にヒーターを止
めたのだが、まだ熱が
出ている
2 give in give:与える＋in:～の中に
【自】＜…に＞屈服する・降参
する 【他】(1)＜書類・答案な
どを＞（…に）提出する・手渡
す[to] (2)（候補者などとして）
［…に］＜名前を＞届ける[to]
He gave in without a fight. 彼は戦う前に降参した.
3 give out
give:与える＋out:～の外
に
【他】(1)（…に）＜商品・用
紙・ビラなどを＞配る・配布す
る [to] (2) ＜…を＞発表する
【自】(1)＜供給物・力などが＞
尽きる・なくなる
There are people at Shinjuku
station who give out tissues.
新宿駅でティッシュを
配っている人がいる
4
give
over
give:与える＋over:～を
超えて
【他】(1) ＜物・人など＞を
（…に）渡す・預ける・譲る、
（犯人などを）［警察などに］
引き渡す [to] 【自】（命令法
で）＜英口語＞やめる
She gave her all property
over to her daughter.
自分の全財産を娘に
譲った.

26. Self-study
After the
instructor’s
explanation of the
core sense of each
word, the
participants were
asked to learn the
PVs by connecting
the core senses
with each sense

27. 24 Questions (12 concrete & 12
figurative senses)

28. Analyses
•T-test between the two groups
•T-test between the pre and post
tests in each group
•Questions about
concrete/figurative senses

29. Pre & post tests (overall)
6.92
8.92
7.56
9.33
0
2
4
6
8
10
12
14
16
18
20
pretest posttest
software
handout

30. T-test for the pretest
6.92
8.92
7.56
9.33
0
2
4
6
8
10
12
14
16
18
20
pretest posttest
software
handout
p = 0.21

31. T-test for the posttest
6.92
8.92
7.56
9.33
0
2
4
6
8
10
12
14
16
18
20
pretest posttest
application
handout
p = 0.21
p = 0.46

32. T-test results for the handout group
4.09
2.83
5.13
3.34
0
1
2
3
4
5
6
handout-concrete handout-figurative
pretest
posttest
p = 0.10
p = 0.00

33. T-test results for the app group
4.52
3.04
5.59
3.91
0
1
2
3
4
5
6
application-concrete application-figurative
pretest
posttest
p = 0.01
p = 0.00

34. Answers of our RQs
• RQ1: Do schematic visual glosses enhance L2 text
comprehension with L2 PVs compared with verbal
glosses?
No significant difference between the
treatments
• RQ2: Which senses of L2 PVs do the schematic
visual glosses enhance, concrete senses or
figurative senses?
Both aids enhanced the concrete senses
of the PVs
Schematic aids enhanced their figurative
senses

35. Conclusion and limitations
•Traditional and multimodal PV
representation facilitated its
comprehension.
•Multimodal PV representation
facilitated improved comprehension
of the figurative senses of the PVs
compared to the traditional one.
•Large-scaled research required.

36. References
• Boers, F., & Lindstromberg, S. (2008). How cognitive linguistics can foster effective vocabulary teaching. In F. Boers, & S.
Lindstromberg (Eds.), Cognitive linguistic approaches to teaching vocabulary and phraseology (pp.1-64). Berlin: Mouton de
Gruyter.
• Dewell, R. B. (1994). Over again: Image-schema transformations in semantic analysis. Cognitive Linguistics, 5(4), 351-380.
• Garnier, M. & Schmitt, N. (2016). Picking up polysemous phrasal verbs: How many do learners know and what facilitates this
knowledge?, System, 59, 29-44.
• Lakoff, G.(1987) Woman, fire and dangerous thing. Chicago: Chicago University Press.
• Langacker, R, W.(1987) Foundations of Cognitive Grammar, Volume I, Theoretical Prerequisites. Stanford, California: Stanford
University Press.
• Lindstormberg, S. (2001a). Preposition entries in UK monolingual learner’s dictionaries: Problems and possible solutions. Applied
Linguistics 22(1), 79-103.
• Lindstromberg, S. (2001b). (Sometimes) Against the grain. Humanising Language Teaching Magazine, 3(3). Retrieved 12th of
November, 2016 from http://www.hltmag.co.uk/may01/lind.htm
• NORVIG P., & LAKOFF G. (1987). Taking: A study in lexical network theory, A. Jon, B. Natasha, M. Laura A. & F. Hana (Eds), Proceedings
of the Thirteenth Annual Meeting of the Berkeley Linguistics Society (pp. 195-206).
• Rudzka-Ostyn, B. (2003). Word power phrasal verbs and compounds: A cognitive approach. Berlin, Germany: Walter de Gruyter.
• Sato, T., Lai, Y., & Burden, T. (2014). Examining the Impact of Individual Differences of Information Processing Styles in Technology-
Enhanced Second Vocabulary Learning. Proceedings of CLaSIC 2014. p. 432-440.
• Yoshii,M., & Fraitz, J.(2002). Second Language Incidental Vocabulary Retention: The Effect of Text and Picture Annotation Types.
CALICO Journal, 20 (1), 33-58.
• Yeh, Y., & Wang, C. (2003). Effects of Multimedia Vocabulary Annotations and learning styles on vocabulary learning. CALICO
Journal, 21(1). 131-144.

37. This work is supported by JSPS KAKENHI
Grant Number 26370658

38. ご清聴ありがとうございました
Thank you for listening
Takeshi SATO ( tsato@cc.tuat.ac.jp )
Tyler BURDEN ( tylerbuden1@gmail.com )