The document summarizes an eye tracking study that investigated how literal and figurative language descriptions of scenes affect visual processing. The study found that figurative language descriptions incorporating fictive motion evoked longer looking times and more eye movements along paths within scenes compared to semantically matched literal descriptions. This suggests figurative language activates dynamic mental simulations that interact with how the visual system interprets and represents the world.

1. Introduction
Experiment part
Conclusion
The integration of figurative language and
static depictions:
An eye movement study of fictive motion
(Richardson, D. and Matlock, T. 2007)
Danish Nadeem
Seminar Language and Embodiment SS07, 12 June, 2007
Danish Nadeem Simulation III Figurative Language

2. Introduction
Research Motivation
Experiment part
Approach
Conclusion
Outline of talk
1 Introduction
Research Motivation
Approach
2 Experiment part
Hypothesis
Method
Results
3 Conclusion
Discussion
Danish Nadeem Simulation III Figurative Language

3. Introduction
Research Motivation
Experiment part
Approach
Conclusion
Research Motivation
How figurative language affects the process through which
we perceive the world ?
How do visual and verbal processes interact ?
How scene would be perceived when described by literal
and figurative language ?
Assumption is that differences in "eye movement patterns"
could tell us about the distinct mental representation due to
figurative language and the integration of visual and verbal
processing.
Danish Nadeem Simulation III Figurative Language

4. Introduction
Research Motivation
Experiment part
Approach
Conclusion
Research Motivation
How figurative language affects the process through which
we perceive the world ?
How do visual and verbal processes interact ?
How scene would be perceived when described by literal
and figurative language ?
Assumption is that differences in "eye movement patterns"
could tell us about the distinct mental representation due to
figurative language and the integration of visual and verbal
processing.
Danish Nadeem Simulation III Figurative Language

5. Introduction
Research Motivation
Experiment part
Approach
Conclusion
Research Motivation
How figurative language affects the process through which
we perceive the world ?
How do visual and verbal processes interact ?
How scene would be perceived when described by literal
and figurative language ?
Assumption is that differences in "eye movement patterns"
could tell us about the distinct mental representation due to
figurative language and the integration of visual and verbal
processing.
Danish Nadeem Simulation III Figurative Language

6. Introduction
Research Motivation
Experiment part
Approach
Conclusion
Research Motivation
How figurative language affects the process through which
we perceive the world ?
How do visual and verbal processes interact ?
How scene would be perceived when described by literal
and figurative language ?
Assumption is that differences in "eye movement patterns"
could tell us about the distinct mental representation due to
figurative language and the integration of visual and verbal
processing.
Danish Nadeem Simulation III Figurative Language

7. Introduction
Research Motivation
Experiment part
Approach
Conclusion
Approach
Figurative spatial descriptions (aka FM sentences) and their
"semantically" equivalent non-FM sentences were choosen:
Fictive Motion (FM) sentences
1a: The road goes through the desert
1b: The fence follows the coastline
Non-FM sentences
2a: The road is in the desert
2b: The fence is next to the coastline
Danish Nadeem Simulation III Figurative Language

8. Introduction Hypothesis
Experiment part Method
Conclusion Results
Hypothesis
Study showed that eye movements correspond to the
spatial content in the stories (Spivey et al. 2000) for
instance:
vertical eye movement when hearing about "someone
repelling down a canyon wall"
horizontal eye movement when hearing about "a train
pulling out of station"
The underlying hypothesis is that FM descriptions would
activate representations of motion, which implies:
longer looking times to the path (gaze)
sequences of eye movements that correspond to motion
Danish Nadeem Simulation III Figurative Language

9. Introduction Hypothesis
Experiment part Method
Conclusion Results
Method
participants: 63 Stanford psychology students with normal
vision
stimuli:
visual stimuli consisted of 32 pictures of spatial scenes.
verbal stimuli consisted of 64 sentences recorded in 16
block of 4 sentences each.
Experimental trial: Difficult/Easy + FM/Non-FM
Difficult + FM: "The desert is hilly" + Road goes through the desert (FM).
Difficult + Non-FM: "The desert is hilly" + Road is in the desert (non-FM).
Danish Nadeem Simulation III Figurative Language

10. Introduction Hypothesis
Experiment part Method
Conclusion Results
Results
Eye movement data were categorized into two dependent
variables:
the total looking time (gaze)
the frequency of path scanning fixations
Figure: Total looking time and frequency of path scanning fixations
Danish Nadeem Simulation III Figurative Language

11. Introduction
Experiment part Discussion
Conclusion
Discussion
Comparision of literal and figurative spatial descriptions of
the same scene.
Figurative descriptions evoke distinct spatial
representation, unlike literal ones.
Inspection times and eye movements scanning along the
path increased during FM descriptions.
Eye movements were affected because FM language
evokes a dynamic mental simulation which interacts with
the ways in which visual system interprets the world.
Danish Nadeem Simulation III Figurative Language

12. Introduction
Experiment part Discussion
Conclusion
Discussion
Comparision of literal and figurative spatial descriptions of
the same scene.
Figurative descriptions evoke distinct spatial
representation, unlike literal ones.
Inspection times and eye movements scanning along the
path increased during FM descriptions.
Eye movements were affected because FM language
evokes a dynamic mental simulation which interacts with
the ways in which visual system interprets the world.
Danish Nadeem Simulation III Figurative Language

13. Introduction
Experiment part Discussion
Conclusion
Discussion
Comparision of literal and figurative spatial descriptions of
the same scene.
Figurative descriptions evoke distinct spatial
representation, unlike literal ones.
Inspection times and eye movements scanning along the
path increased during FM descriptions.
Eye movements were affected because FM language
evokes a dynamic mental simulation which interacts with
the ways in which visual system interprets the world.
Danish Nadeem Simulation III Figurative Language

14. Introduction
Experiment part Discussion
Conclusion
Discussion
Comparision of literal and figurative spatial descriptions of
the same scene.
Figurative descriptions evoke distinct spatial
representation, unlike literal ones.
Inspection times and eye movements scanning along the
path increased during FM descriptions.
Eye movements were affected because FM language
evokes a dynamic mental simulation which interacts with
the ways in which visual system interprets the world.
Danish Nadeem Simulation III Figurative Language

15. Introduction
Experiment part Discussion
Conclusion
Thank you for your attention !
Danish Nadeem Simulation III Figurative Language