Brain-Based Translation Reveals Language-Independent Semantic Representations in ATL
•Download as PPTX, PDF•
1 like•357 views
The study used fMRI and multi-voxel pattern analysis to examine the neural representations of semantic concepts in bilingual participants. The researchers found that abstract semantic representations of words like "horse" could be decoded from brain activity patterns in anterior temporal lobe regions, independently of the language used. This provides evidence that some ATL regions construct language-independent semantic representations and act as a semantic hub in the brain. The study demonstrates the ability of MVPA to examine fine-grained semantic representations across languages with higher sensitivity than traditional univariate analysis.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Similar to Brain-Based Translation Reveals Language-Independent Semantic Representations in ATL
Similar to Brain-Based Translation Reveals Language-Independent Semantic Representations in ATL (20)
Recently uploaded
Recently uploaded (20)
Brain-Based Translation Reveals Language-Independent Semantic Representations in ATL
- 1. Brain-Based Translation: fMRI Decoding of Spoken Words in Bilinguals Reveals Language- Independent Semantic Representations in Anterior Temporal Lobe Correia et al (2014). The Journal of Neuroscience
- 2. Statement of the problem What are the neural mechanisms underlying the representation of language-independent semantic concepts in the brain? “Horse” in English “Paard” in Dutch
- 3. General findings Several regions in the ATL (Anterior Temporal Lobe) seem to be responsible for organizing language-independent semantic concepts. Specifically, at the fine-grained level of within-semantic-category discriminations (e.g “animal” category). “Horse” “Paard”
- 4. Claims Evidence Patients with ATL brain lesions demonstrate selective deficits of semantic knowledge, also known as semantic dementia. A crucial function of the ATL is to act as a “semantic hub.” That is, language- independent representations of words and concepts are constructed in some regions of the ATL. Previous work (Damasio et al., 1996; Pattersonet al., 2007)
- 5. The knowledge gap in the literature How this paper attempts to address it Using bilingual participants, fMRI, a unique experimental paradigm, and a whole lot of complicated data analyses that I hope we can figure out together. Lesion correlation evidence is insightful, but by no means abundant or consistent across patients (e.g. what counts as a lesion to ATL, what counts as semantic dementia). Ideally, we’d want converging evidence from several kinds of data (behavioral, lesion, neurolinguistic...etc.). Some work has been done in this regard, but there exist some challenges to find reliable ATL activation in neuroimaging. These include “susceptibility artifacts” or “limited field of view.” Recent studies have devised creative new techniques to overcome these challenges. This paper is one of them.
- 6. Important terms and concepts ● Within-language discrimination ● Across-language generalization ● Block experiment design ● Univariate Analysis ● Multi-Voxel Pattern Analysis ● Cortex-based alignment ● Repetition Time (RT)? ● Acquisition Time (AT)? ● Baseline? ● Classification ● Training algorithms ● Searchlight method ● Support vector machine (SVM)
- 7. Experimental design In separate Dutch and English blocks, bilingual participants were asked to listen to individual animal nouns (e.g. horse) and to detect occasional non-animal target nouns (e.g. bike). The goal of the task was to help maintain a constant attention level throughout the experiment to promote speech comprehension at every word presentation. Importantly, each word was spoken by 3 different female speakers, which allowed for speaker invariant word discrimination.
- 8. Methodology Participant criteria Ten bilinguals university students. All L1 Dutch, proficient L2 English (3 males, 7 females). Why bilinguals? Allows us to tease apart language- specific & language-independent representations in a way that monolingual data can’t. Stimuli Consisted of Dutch and English spoken words representing 4 different animals (bull, duck, horse, shark) and 3 inanimate object words (bike, dress, suit) Task Listen & detect non-animal words. Data collection fMRI Data analysis It’s complicated
- 9. Data analysis 1. Functional and anatomical image acquisition 2. fMRI data preprocessing 3. MRI data analysis univariate statistics 4. fMRI data analysis MVPA 5. Classification (Feature extraction, Feature selection, Cross- validation) 6. Discriminative maps
- 10. Data analysis 1. Functional and anatomical image acquisition 2. fMRI data preprocessing 3. MRI data analysis univariate statistics 4. fMRI data analysis MVPA 5. Classification (Feature extraction, Feature selection, Cross- validation) 6. Discriminative maps Whole-brain analysis was achieved with high-resolution (voxel size, 1x1x1 mm3) anatomical images.
- 11. Data analysis 1. Functional and anatomical image acquisition 2. fMRI data preprocessing 3. MRI data analysis univariate statistics 4. fMRI data analysis MVPA 5. Classification (Feature extraction, Feature selection, Cross- validation) 6. Discriminative maps They obtained an anatomically-aligned, group-averaged 3D surface representation so that subject data could be compared. They did this by preprocessing and analyzing fMRI data using Brain Voyager and custom-made MATLAB routines. ● Functional data were corrected for (3D motion, slice scan time differences, removing noise) ● Anatomical data were corrected for (intensity inhomogeneity, transformed into Talairach space). ● Functional & anatomical data then aligned to create 4D volume time courses (cortex-based alignment).
- 12. Data analysis 1. Functional and anatomical image acquisition 2. fMRI data preprocessing 3. fMRI data analysis univariate statistics 4. fMRI data analysis MVPA 5. Classification (Feature extraction, Feature selection, Cross- validation) 6. Discriminative maps Functional contrast maps were constructed in order to identify the cortical regions involved in the processing of spoken words. This was done by: (1) comparing activation to all animal words vs. baseline across subjects (2) combining all possible binary contrasts within nouns of the same language (3) grouping all equivalent nouns into single concepts and contrasting all possible binary combinations of concepts. Functional contrast maps were submitted to a whole-brain correction criterion based on the estimate of the spatial smoothness of the map. Univariate analysis is the simplest form of data analysis. It means that your analysis is considering only one variable. It doesn’t deal with causes or relationships (like MVPA does). Its major purpose is to describe, summarize and find patterns in the data with respect to that one variable.
- 13. fMRI data Univariate analysis Here we’re looking at the fMRI responses elicited by all animal words across subjects, as computed by a functional contrast map (t statistics) comparing all animal words versus baseline.
- 14. Data analysis 1. Functional and anatomical image acquisition 2. fMRI data preprocessing 3. fMRI data analysis univariate statistics 4. fMRI data analysis MVPA 5. Classification (Feature extraction, Feature selection, Cross- validation) 6. Discriminative maps What can Multi-voxel pattern analysis (MVPA) do that Univariate analysis can’t? Detect differences between conditions with higher sensitivity than conventional univariate analysis by focusing on the analysis and comparison of distributed patterns of activity. Data from individual voxels within a region are jointly analyzed. Furthermore, MVPA is often presented in the context of "brain reading" applications reporting that specific mental states or representational content can be decoded from fMRI activity patterns after performing a "training" or "learning phase. In this context, MVPA tools are often referred to as classifiers or, more generally, learning machines. The latter names stress that many MVPA tools originate from a field called machine learning, a branch of artificial intelligence. BrainVoyager introduces a comprehensive set of MVPA tools for locally distributed as well as more extended (sparse) patterns of activation. The tools include a multivariate searchlight mapping approach, which is used both for analyzing patterns in ROIs and for discriminating patterns that are potentially spread out across the whole brain. BrainVoyager QX v2.8 | 2014 Rainer Goebel.
- 15. fMRI data analysis MVPA Here we’re looking at the statistical maps of searchlight selections for which the word discrimination and the language generalization analyses yielded accuracies significantly above chance level (50%).
- 16. Data analysis 1. Functional and anatomical image acquisition 2. fMRI data preprocessing 3. fMRI data analysis univariate statistics 4. fMRI data analysis MVPA 5. Classification (Feature extraction, Feature selection, Cross-validation) 6. Discriminative maps ?
- 17. Data analysis 1. Functional and anatomical image acquisition 2. fMRI data preprocessing 3. fMRI data analysis univariate statistics 4. fMRI data analysis MVPA 5. Classification (Feature extraction, Feature selection, Cross- validation) 6. Discriminative maps Accuracy maps of within-language word discrimination and across-language word generalization were constructed. Accuracy maps were averaged within each subject across binary comparisons and cross-validation folds. Thereafter, individual averaged accuracy maps were projected onto the group-averaged cortical surface and anatomically aligned using cortex-based alignment. In the within-language discrimination analysis, one map was produced per language and subsequently combined into a single map by means of conjunction analysis. The resulting discrimination map thus depicts regions with consistent sensitivity in English and Dutch. For the across-language word generalization, one map was produced from all possible binary language generalizations.
- 20. Conclusions ● That abstract horse representation exists in the brain and can be located. Brain-based decoding of individual spoken words at the fine-grained level of within- semantic category is possible within and across the first and second languages of bilingual adults. ● That representation exists somewhere in the ATL. Specifically, localized clusters in the left anterior temporal lobe, the left angular gyrus and the posterior bank of the left postcentral gyrus, the right posterior superior temporal sulcus/superior temporal gyrus, the right medial anterior temporal lobe, the right anterior insula, and bilateral occipital cortex. ● MVPA is a good approach for this kind of research question. Results indicate the benefits of MVPA based on the generalization of the pattern information across specific stimulus dimensions. This approach enabled examining the representation of spoken words independently of the speaker and the representation of semantic– conceptual information independently of the input language.
- 21. Questions ● So what? Why should we care where representations are or where processes happen in the brain? ● Could we translate this study (an fMRI study examining the location of abstract semantic representations in the brain) with other neurolinguistic tools (MEG, EEG...etc.). If so, how? Benefits or advantages of doing that? ● Do you feel their experimental design tested what they thought they were testing? ● If you were to change something about their experimental design or data analysis, what would it be?
Editor's Notes
- Does that abstract representation exist? If so, where? How can we find out?
- In this approach, each voxel is visited, as in a standard univariate analysis, but instead of using only data of the visited voxel for analysis, several voxels in the neighborhood are included forming a set of features for joined multivariate analysis. Support Vector Machines (SVM) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other,
- The difference between the Acquisition Time (TA) and Repetition Time (TR) introduced a silent gap used for the presentation of the auditory stimuli.
- unctional data were 3D motion corrected (trilinear interpolation), corrected for slice scan time differences,and were temporally filtered by removing frequency components of five or less cycles per time course. Anatomical data were corrected for intensity inhomogeneity and transformed into Talairach space. Functional data were then aligned with the anatomical data and transformed into the same space to create 4D volume time courses. Individual cortical surfaces were reconstructed from gray–white matter segmentations of the anatomical data and aligned using a moving target-group average approach based on curvature information (cortex-based alignment)
- Univariate effects were analyzed using a random-effects general linear model (GLM). A single predictor per stimulus condition was convoluted with a double gamma hemodynamic response function.
- Univariate effects were analyzed using a random-effects general linear model (GLM). A single predictor per stimulus condition was convoluted with a double gamma hemodynamic response function.
- The idea of this step was to investigate speech content information in the fMRI responses. To do this, they used a supervised machine learning algorithm combined with single-trial multivoxel classification. Classifications were performed to evaluate whether patterns of voxels conveyed information on the identity of spoken words (within-language word discrimination) and their language-invariant representations (across-language word generalization). Within-language word discrimination entailed training a classifier to discriminate between two words of the same language (e.g., horse vs duck) and testing in the same words spoken by a speaker not included in the training phase. Across-language word generalization was performed by training a classifier to discriminate between two words within one language (e.g., horse vs duck) and testing in the translational equivalent words within the other language (e.g., paard vs eend), thus relying on language-independent information of spoken words representing equivalent concepts in Dutch and English (Fig. 1A).
- Univariate effects were analyzed using a random-effects general linear model (GLM). A single predictor per stimulus condition was convoluted with a double gamma hemodynamic response function.
- The idea of this step was to investigate speech content information in the fMRI responses. To do this, they used a supervised machine learning algorithm combined with single-trial multivoxel classification. Classifications were performed to evaluate whether patterns of voxels conveyed information on the identity of spoken words (within-language word discrimination) and their language-invariant representations (across-language word generalization). Within-language word discrimination entailed training a classifier to discriminate between two words of the same language (e.g., horse vs duck) and testing in the same words spoken by a speaker not included in the training phase. Across-language word generalization was performed by training a classifier to discriminate between two words within one language (e.g., horse vs duck) and testing in the translational equivalent words within the other language (e.g., paard vs eend), thus relying on language-independent information of spoken words representing equivalent concepts in Dutch and English (Fig. 1A).
- The idea of this step was to investigate speech content information in the fMRI responses. To do this, they used a supervised machine learning algorithm combined with single-trial multivoxel classification. Classifications were performed to evaluate whether patterns of voxels conveyed information on the identity of spoken words (within-language word discrimination) and their language-invariant representations (across-language word generalization). Within-language word discrimination entailed training a classifier to discriminate between two words of the same language (e.g., horse vs duck) and testing in the same words spoken by a speaker not included in the training phase. Across-language word generalization was performed by training a classifier to discriminate between two words within one language (e.g., horse vs duck) and testing in the translational equivalent words within the other language (e.g., paard vs eend), thus relying on language-independent information of spoken words representing equivalent concepts in Dutch and English (Fig. 1A).