Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Kutas Lab LatArt
1. Hemispheric Differences in
Linguistic Prediction Given
High Constraint Contexts
Kiana Miyamoto
Kutas Cognitive Electrophysiology Lab
April 23rd, 2016
2. Prediction and the Brain’s Two
Hemispheres
• Prediction in Language
• Hemispheric Differences of Language
• What are the hemispheric roles in predictive
language comprehension?
3. Methodology
• Electroencephalography (EEG)
– Noninvasive method to visualize electrical changes in the brain
• High temporal resolution affords collection of event-related
brain potentials (ERPs)
LCE Cognitive and Social Systems
4. dog
sugar
N400 Component
• Kutas & Hilyard (1980):
I take my coffee with cream and
• Negative peak around 400ms
• Sensitive to semantic fit of item in context
sugardog
5. Changing Perspectives
• Integration:
– Words are processed only once they are received
• Prediction:
– Use context to unconsciously pre-activate
linguistic information prior to input
6. … airplane?
The day was breezy.
The boy went outside to fly…
Complete the sentence:
… a kite?
an…
7. Expectancy on N400 Amplitude
DeLong et al 2005
… a kite
... an airplane
N400
High Expectancy
Low Expectancy
N400
Compatible Article
Incompatible Article
8. Federmeier and Kutas 1999
• Used Visual Hemifield
(VF) paradigm to infer:
– Left hemisphere (LH)
predictive
– Right hemisphere (RH)
integrative
9. Right Hemisphere:
more integrative
Left Hemisphere:
more predictive
+ aan +
Current Study:
LVF RVF
The boy went
outside to fly….
How do the hemispheres independently contribute to
predictive language comprehension?
…airplane
…kite
10. Experimental Design
• Independent Variables:
– Word Expectancy : High or Low
– Visual Field Presentation : Left (LVF) or Right(RVF)
• Record ERPs to lateralized articles (a/an)
– 26 scalp electrodes
– 208 sentence pairs
• 24 right handed, native English speakers
Experimental
Conditions
Right VF
Left Hemi
Left VF
Right Hemi
High Expectancy 32 High-RVF 32 High-LVF
Low Expectancy 32 Low-RVF 32 Low-LVF
11. +
The day was breezy.Theboywentoutsidetofly++++airplane an
12. ① Article N100: Lateralization Manipulation Check
② Noun N400: Semantic Fit within Context
③ Article N400: Prediction effect
Current Study: ERP Predictions
LVF/RH articles
RVF/LH articles
N1N1
N400
Central presentation
article N400Predicted LH
article N400
Predicted RH
article N400
14. Predictions Checklist
①N100 Visual Perception
– Lateralization Manipulation
②N400 at the Centrally Presented Noun
③N400 at the Lateralized Article
15. Expected vs Unexpected Nouns
N400
N400
High Expectancy LVF Nouns
High Expectancy RVF Nouns
Low Expectancy LVF Nouns
Low Expectancy RVF Nouns
16. Noun N400 Following
Lateralized Articles
High Expectancy Nouns
Low Expectancy Nouns
Nouns Following
RVF/LH Articles
Nouns Following
LVF/RH Articles
Central
Presentation
17. Predictions Checklist
①N100 Visual Perception
②N400 at the Centrally Presented Noun
– Noun Semantic Fit Manipulation
③N400 at the Lateralized Article
19. Lateralized Article N400
Compatible Articles
Incompatible Articles
Right Visual Field
Left Hemisphere
Central
Presentation
Left Visual Field
Right Hemisphere
21. Why not?
• Prediction may require collaboration from both
hemispheres
• Effect at article is smaller than noun because
“a/an” are short words with little semantic
weight
– Not strong enough to withstand the lateralization
manipulation
– Visual Hemifield Paradigm is prone to eye movements
• No observed effect in the analyzed time window.
– Further analyses in different locations/time intervals
23. Acknowledgements
• Marta Kutas
• Tom Urbach
• Reina Mizrahi
• Sara Brooke
• Janey Xie
• Claudio Hartmann
• All of our Participants
• Katherine DeLong
• Yoana Vergilova
• Diego Gomes
• Hilda Parra
Editor's Notes
Do you ever find yourself ending the line of a song or finishing your friend’s sentences before they themselves have even finished saying it? Although this is our intuition, there has only recently been evidence to support the brain predicting language through the activation of words or features of those words, before the words themselves are presented.
Historically, language comprehension and production have been solely considered left hemispheric processes. Further research has indicated that the right hemisphere may also contribute but in sometimes different ways. Here we investigate the cerebral hemispheres in language prediction.
Transition:
To do this, we used a technique that allowed us to track brain activity in real time. This is important because language occurs very quickly and prediction studies need to find evidence for activation of words that haven’t yet occurred or may never occur.
Electroencephalography is a non-invasive neuroimaging method that measures neural-related, electrical activity at the scalp. The is done using an electrode embedded cap placed on the participant’s head, as shown in the left image. EEG’s high temporal resolution allows us to detect rapid fluctuating brain activity, as seen in various stages of language processing. By recording activity corresponding to specific stimuli, we can average these data across many trials from like conditions and examine what is called the event-related brain potentials.
The ERPs are also averaged across many experiment participants to ensure the observation is generalizable. Looking at the averaged waveform, you can see time across the x-axis in milliseconds and amplitude on the vertical axis in microvolts, by convention positive plotted down and negative up. Often, the deflections in the waveform are often labeled as components according to the polarity, positive or negative, and when in time they occur in milliseconds.
Transition:
Cognitive neuroscientists have been able to attribute specific brainwave components in the ERP signal to particular types of neural processing.
In 1980, Kutas and Hilyard discovered the N400 brainwave component. This ERP component is a negative deflection peaking at approximately 400ms and responds to the semantic fit of an item within a given context.
If I were to say, “I take my coffee with cream and sugar”, sugar fits the context and is highly expected. The N400 amplitude is reduced.
Now, if I were to say, “I take my coffee with cream and dog”, dog is not expected because it does not fit the context and would instead elicit a large N400.
Transition:
The N400 component has been used to study language comprehension and can also be used to study prediction as we have done.
Transition:
Despite our intuitions about “feeling “ like we predict while listening or reading sentences, it was long thought that language could only be processed once a stimulus was received. This is the integration perspective. We can think about integration as waiting to receive an input that then activates stored meanings and related features to be pieced together. This mental representation is constructed in short term memory.
Integration dominated the language field until the prediction perspective surfaced. Prediction is a highly automatic neural process in which the brain pre-activates stored information from the preceding context to generate possible expected continuations. When the context leads to an expected end word, the word is easier to process.
Transition: Keeping this idea of pre-activation in mind, try completing the following sentence…
The day was breezy. The boy went outside to fly…
‘A kite’ has the highest expectancy in this context.
So let’s try this again….
The day was breezy. The boy went outside to fly an…
Most of you probably completed with “an airplane”.
This time you are constrained with the article “an”. This limits your word choice because English has phonological grammar rules in that words beginning with a consonant sound are preceded by ‘a’ and words beginning with a vowel sound are preceded by ‘an’.
Transition:
In highly constraining sentences, meaning when the context cues for a particular item, the phonologically compatible articles can be used as an early test of prediction.
DeLong and colleagues in 2005 used the article N400 ERP component to test if people are predicting in language comprehension. This study greatly influenced the design of our current study.
In this slide we can see the results from the DeLong et al 2005 study. When looking at the expected and unexpected nouns, we see a large effect in the N400 component. Kite elicits a reduced N400. The N400 pattern is as expected based on previous studies, indicating the nouns were being processed semantically. But there is no way of distinguishing whether the brain is integrating or predicting the word because of the differing word meanings. The large N400 effect could be a result from the greater difficulty to integrate the word airplane into the sentence or because airplane does not fit the expectation of the pre-activated word kite.
To resolve this issue, this study also looked at the N400 component at the preceding article a/an, which showed the same trend, although with a smaller (but reliable) effect. Unlike “airplane” and “kite”, the articles ‘a’ and ‘an’ share the same semantics. There should be no reason for the N400 amplitude to differ unless the readers were already forming a prediction at the article.
The words were presented one word at a time in central fixation. So when hemispheres were allowed to cooperate, there is strong evidence for pre-activation of upcoming words during sentence processing.
Transition: But what are the hemispheres individuals roles?
In 1999, Federmeier and Kutas used the visual hemifield paradigm in a sentence processing study, and theorized that the left hemisphere processes language more predictively while the right hemisphere is more integrative.
Transition:
This brings us to our question of interest for our current study, how do the hemispheres independently contribute to predictive language comprehension?
We used the Visual Hemifield technique to lateralize the a/an articles. Banich describes the Visual Hemifield technique as: presenting stimuli a few degrees to the right or left of fixation in order to expose only the contralateral hemisphere to that stimulus for approximately 10 ms (Banich, 2003). This means when the stimulus an is presented to the left visual field, crosses into the primary visual cortex in the right hemisphere, and vice versa.
The consequence of this short headstart in apprehending the stimulus results in hemispheric differences that carry over even into later stages of processing.
So by placing the a/an articles in the left or right visual fields, we can observe if there are differences in how the two hemispheres may be predicting the upcoming nouns.
The independent variables in our study were word expectancy and visual field in which the articles were presented. We used ERPs to measure the effect of lateralized articles in 24 right handed, native English speakers.
This is an example of a stimulus shown to the participants. The participant was allowed to read the first context sentence, The day was breezy.
Then, the next sentence is presented using Rapid Serial Visual Presentation(RSVP) to time-lock to the brain’s response to each word with the critical article lateralized to the left or right of fixation and assess the brain’s response to each word.
In this study, we had three main predictions:
Number 1: The first step was to ensure that participants were processing the lateralized articles. Based on previous visual hemifield studies, this early sensory processing can be observed in the N100 ERP component in the contralateral visual area.
Number 2: We also expect to see a large N400 amplitude for the low expectancy nouns and a reduced N400 for the high expectancy nouns.
Number 3: When looking at the N400 component for the articles, the hypothesis for this study is If the LH is processing more predictively, we should observe a N400 reduction to the expected articles presented to the RVF/LH but not the LVF/RH.
These are the averaged brainwaves at each electrode in the time interval -100 to 400 ms post stimulus to focus on the N1 effect peaking shortly after 100ms. The red line shows the brain activity when the article was lateralized to the RVF and the black line is when the article was lateralized to the LVF. We see that there is increased N100 amplitude in the hemisphere contralateral to the visual field of presentation.
This result confirms that our lateralization manipulation worked as expected.
Let’s take a look at the N400 Component for the nouns.
For this analysis, we focused on the 9 posterior electrodes where N400s are usually found.
1500ms time window from 500ms before the onset of stimulus to 1000ms after
This is a view of the Midline Parietal electrode. In the center is the result from the DeLong et al central presentation experiment. In both hemispheres, we see the same ERP difference between the high and low expectancy nouns, with high expectancy nouns showing more reduced N400 amplitude than low expectancy nouns.
Again, we observed the expected result of the noun expectancy manipulation. Check!
Given what we know from Federmeier and Kutas as well as DeLong et al, we expected to see the predictive N400 reduction at the article but only in the left hemisphere.
We focused on the same 9 posterior electrodes as we did for nouns. We analyzed a 600ms timewindow from 100ms before to 500ms after the onset of the article.
Without focusing in, you can already see that the ERPs to the articles do not appear to differ as a result of expectancy and visual field.
In the center is the result from the DeLong et al central presentation experiment.
We do not observe a difference in the N400 to the articles presented to the RVF/LH to support our Predictive Left Hemisphere hypothesis.
While the first two experimental predictions were upheld, we were not able to observe an N400 reduction to articles preceding high expectancy nouns.
There are a few possible reasons for why we did not observe our predicted LH article ERP effect.
1) The first is that it could be that predictive language processing of the sort that we were trying to detect, requires the contributions of both cerebral hemispheres.
But other possibilities that we have to consider are that:
2) Articles and nouns are simply very different kinds of words. Nouns have rich semantic content whereas articles do not. N400s to nouns are typically larger than those to words like articles. In fact, even in the original DeLong et al central presentation study, the articles showed a smaller N400 effect than the nouns. So the addition of the lateralization component of the experiment may have added other processing noise to the signal, washing out any article N400 effect, which would have been small to begin with.
3) Another possibility is that the ERP signal could show evidence for LH-biased predictive processing, but it could be that these differences could be reflected in other ERP components, over different time-windows and scalp electrodes. This possibility will require further exploration and ERP analyses.
The next sentence is presented using Rapid Serial Visual Presentation(RSVP) to isolate the brain’s response to each word.
The critical word airplane fits the sentence just as well as kite but given the context, most people choose kite.
