Search

Reading the mind’s eye: Decoding object information during mental imagery from fMRI patterns

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1


    I don’t think I need to convince anyone here in this audience that it is possible to reliably readout the category information of visually presented stimuli.
    Here the question we are trying to address is whether or not such readout is still possible in the COMPLETE absence of bottom-up inputs.
    For instance, if I ask you to close your eyes and imagine as vividly as possible specific objects or people can I readout the content of your mental images?
    If this readout is at all possible we would like to characterize the nature and reliability of these patterns...

    I don’t think I need to convince anyone here in this audience that it is possible to reliably readout the category information of visually presented stimuli.
    Here the question we are trying to address is whether or not such readout is still possible in the COMPLETE absence of bottom-up inputs.
    For instance, if I ask you to close your eyes and imagine as vividly as possible specific objects or people can I readout the content of your mental images?
    If this readout is at all possible we would like to characterize the nature and reliability of these patterns...

    At the same time, some of these studies also showed a number of qualitative differences between I and P
    For instance we know that imagery is associated with a strong negative bold activation ie de-activation of several brain areas
    In general studies have showed that I activates a smaller number of voxels and with lower magnitude of the BOLD response
    Finally it has been shown that the spatial overlap between I and P is larger in frontal areas than ventral temporal areas.
    YET almost all the studies mentioned above relied on univariate data analysis methods

    At the same time, some of these studies also showed a number of qualitative differences between I and P
    For instance we know that imagery is associated with a strong negative bold activation ie de-activation of several brain areas
    In general studies have showed that I activates a smaller number of voxels and with lower magnitude of the BOLD response
    Finally it has been shown that the spatial overlap between I and P is larger in frontal areas than ventral temporal areas.
    YET almost all the studies mentioned above relied on univariate data analysis methods

    There are at least two limitations associated with the use of univariate analysis:
    First we cannot conclude anything about the patterns of fMRI activity that underly the associated object representation:
    2 ROIs can have the same exact average bold activity, yet the underlying patterns of fMRI activity could be quite different as illustrated here on the right
    So the degree of similarity between I and P could have been overestimated
    Conversely, a voxel can be visually selective yet not carry any information about the underlying category of the stimulus understudy. The use of pattern matching and classification techniques here allows us to only focus on the voxels which are category-related. In particular, it is possible that in previous studies some of the differences between P and I were due to non-category selective voxels and therefore the degree of similarity could have been underestimated.

    There are at least two limitations associated with the use of univariate analysis:
    First we cannot conclude anything about the patterns of fMRI activity that underly the associated object representation:
    2 ROIs can have the same exact average bold activity, yet the underlying patterns of fMRI activity could be quite different as illustrated here on the right
    So the degree of similarity between I and P could have been overestimated
    Conversely, a voxel can be visually selective yet not carry any information about the underlying category of the stimulus understudy. The use of pattern matching and classification techniques here allows us to only focus on the voxels which are category-related. In particular, it is possible that in previous studies some of the differences between P and I were due to non-category selective voxels and therefore the degree of similarity could have been underestimated.

    There are at least two limitations associated with the use of univariate analysis:
    First we cannot conclude anything about the patterns of fMRI activity that underly the associated object representation:
    2 ROIs can have the same exact average bold activity, yet the underlying patterns of fMRI activity could be quite different as illustrated here on the right
    So the degree of similarity between I and P could have been overestimated
    Conversely, a voxel can be visually selective yet not carry any information about the underlying category of the stimulus understudy. The use of pattern matching and classification techniques here allows us to only focus on the voxels which are category-related. In particular, it is possible that in previous studies some of the differences between P and I were due to non-category selective voxels and therefore the degree of similarity could have been underestimated.

    There are at least two limitations associated with the use of univariate analysis:
    First we cannot conclude anything about the patterns of fMRI activity that underly the associated object representation:
    2 ROIs can have the same exact average bold activity, yet the underlying patterns of fMRI activity could be quite different as illustrated here on the right
    So the degree of similarity between I and P could have been overestimated
    Conversely, a voxel can be visually selective yet not carry any information about the underlying category of the stimulus understudy. The use of pattern matching and classification techniques here allows us to only focus on the voxels which are category-related. In particular, it is possible that in previous studies some of the differences between P and I were due to non-category selective voxels and therefore the degree of similarity could have been underestimated.

    To address these issues we scanned 10 participants in two fMRI conditions:
    In a visual perception (P) condition subjects viewed different exemplars of 4 categories of objects (famous faces, famous buildings, tools and food (common fruits and vegetables)). To make sure subjects attended to the stimuli they were asked to perform a 1-back task on the color of the stimuli.
    In the visual imagery (I) condition subjects were given auditory instructions with the names of the stimuli and asked to generate vivid and detailed mental images corresponding to these names. Subjects performed a 1-back task on the color of the imagined stimuli.
    2 sec visual presentation or 2 sec for mental imagery
    This stimuli were not completely new to the participants: subjects were provided with the set of 32 images (4 categories x 8 exemplars) that would be used during the visual presentation blocks, and their associated names, and asked to familiarize themselves with the stimuli. Additionally, 15 minutes prior to the scan, subjects were again asked to examine the stimuli.
    Before the scan session, subjects were instructed to try to generate vivid and detailed mental images as similar as possible to the corresponding images seen in the visual presentation condition.

    To address these issues we scanned 10 participants in two fMRI conditions:
    In a visual perception (P) condition subjects viewed different exemplars of 4 categories of objects (famous faces, famous buildings, tools and food (common fruits and vegetables)). To make sure subjects attended to the stimuli they were asked to perform a 1-back task on the color of the stimuli.
    In the visual imagery (I) condition subjects were given auditory instructions with the names of the stimuli and asked to generate vivid and detailed mental images corresponding to these names. Subjects performed a 1-back task on the color of the imagined stimuli.
    2 sec visual presentation or 2 sec for mental imagery
    This stimuli were not completely new to the participants: subjects were provided with the set of 32 images (4 categories x 8 exemplars) that would be used during the visual presentation blocks, and their associated names, and asked to familiarize themselves with the stimuli. Additionally, 15 minutes prior to the scan, subjects were again asked to examine the stimuli.
    Before the scan session, subjects were instructed to try to generate vivid and detailed mental images as similar as possible to the corresponding images seen in the visual presentation condition.

    To address these issues we scanned 10 participants in two fMRI conditions:
    In a visual perception (P) condition subjects viewed different exemplars of 4 categories of objects (famous faces, famous buildings, tools and food (common fruits and vegetables)). To make sure subjects attended to the stimuli they were asked to perform a 1-back task on the color of the stimuli.
    In the visual imagery (I) condition subjects were given auditory instructions with the names of the stimuli and asked to generate vivid and detailed mental images corresponding to these names. Subjects performed a 1-back task on the color of the imagined stimuli.
    2 sec visual presentation or 2 sec for mental imagery
    This stimuli were not completely new to the participants: subjects were provided with the set of 32 images (4 categories x 8 exemplars) that would be used during the visual presentation blocks, and their associated names, and asked to familiarize themselves with the stimuli. Additionally, 15 minutes prior to the scan, subjects were again asked to examine the stimuli.
    Before the scan session, subjects were instructed to try to generate vivid and detailed mental images as similar as possible to the corresponding images seen in the visual presentation condition.

    To address these issues we scanned 10 participants in two fMRI conditions:
    In a visual perception (P) condition subjects viewed different exemplars of 4 categories of objects (famous faces, famous buildings, tools and food (common fruits and vegetables)). To make sure subjects attended to the stimuli they were asked to perform a 1-back task on the color of the stimuli.
    In the visual imagery (I) condition subjects were given auditory instructions with the names of the stimuli and asked to generate vivid and detailed mental images corresponding to these names. Subjects performed a 1-back task on the color of the imagined stimuli.
    2 sec visual presentation or 2 sec for mental imagery
    This stimuli were not completely new to the participants: subjects were provided with the set of 32 images (4 categories x 8 exemplars) that would be used during the visual presentation blocks, and their associated names, and asked to familiarize themselves with the stimuli. Additionally, 15 minutes prior to the scan, subjects were again asked to examine the stimuli.
    Before the scan session, subjects were instructed to try to generate vivid and detailed mental images as similar as possible to the corresponding images seen in the visual presentation condition.

    classify blocks

    7-8 fMRI scanning runs:
    5 fixation blocks
    8 P blocks
    8 I blocks
    16s/block, i.e. 5.6 min/run

    classify blocks

    7-8 fMRI scanning runs:
    5 fixation blocks
    8 P blocks
    8 I blocks
    16s/block, i.e. 5.6 min/run

    classify blocks

    7-8 fMRI scanning runs:
    5 fixation blocks
    8 P blocks
    8 I blocks
    16s/block, i.e. 5.6 min/run

    classify blocks

    7-8 fMRI scanning runs:
    5 fixation blocks
    8 P blocks
    8 I blocks
    16s/block, i.e. 5.6 min/run

    classify blocks

    7-8 fMRI scanning runs:
    5 fixation blocks
    8 P blocks
    8 I blocks
    16s/block, i.e. 5.6 min/run

    classify blocks

    7-8 fMRI scanning runs:
    5 fixation blocks
    8 P blocks
    8 I blocks
    16s/block, i.e. 5.6 min/run

    To test how classification performance depended on category and classification test, we conducted a 2-way repeated measures ANOVA of category X classification. The ANOVA revealed a significant main effect of category (F(3,27) = 10.27; p<.001), a significant main effect of classification test (F(3,27) = 11.51; p < .0001), and a significant interaction effect (F(9,81)=3.93; p<0.005). Post-hoc tests, Bonferroni
    corrected for multiple comparisons, revealed that classification performance for faces and buildings was significantly higher than for food and tools.

    To test how classification performance depended on category and classification test, we conducted a 2-way repeated measures ANOVA of category X classification. The ANOVA revealed a significant main effect of category (F(3,27) = 10.27; p<.001), a significant main effect of classification test (F(3,27) = 11.51; p < .0001), and a significant interaction effect (F(9,81)=3.93; p<0.005). Post-hoc tests, Bonferroni
    corrected for multiple comparisons, revealed that classification performance for faces and buildings was significantly higher than for food and tools.

    To test how classification performance depended on category and classification test, we conducted a 2-way repeated measures ANOVA of category X classification. The ANOVA revealed a significant main effect of category (F(3,27) = 10.27; p<.001), a significant main effect of classification test (F(3,27) = 11.51; p < .0001), and a significant interaction effect (F(9,81)=3.93; p<0.005). Post-hoc tests, Bonferroni
    corrected for multiple comparisons, revealed that classification performance for faces and buildings was significantly higher than for food and tools.

    To test how classification performance depended on category and classification test, we conducted a 2-way repeated measures ANOVA of category X classification. The ANOVA revealed a significant main effect of category (F(3,27) = 10.27; p<.001), a significant main effect of classification test (F(3,27) = 11.51; p < .0001), and a significant interaction effect (F(9,81)=3.93; p<0.005). Post-hoc tests, Bonferroni
    corrected for multiple comparisons, revealed that classification performance for faces and buildings was significantly higher than for food and tools.

    To test how classification performance depended on category and classification test, we conducted a 2-way repeated measures ANOVA of category X classification. The ANOVA revealed a significant main effect of category (F(3,27) = 10.27; p<.001), a significant main effect of classification test (F(3,27) = 11.51; p < .0001), and a significant interaction effect (F(9,81)=3.93; p<0.005). Post-hoc tests, Bonferroni
    corrected for multiple comparisons, revealed that classification performance for faces and buildings was significantly higher than for food and tools.

    To test how classification performance depended on category and classification test, we conducted a 2-way repeated measures ANOVA of category X classification. The ANOVA revealed a significant main effect of category (F(3,27) = 10.27; p<.001), a significant main effect of classification test (F(3,27) = 11.51; p < .0001), and a significant interaction effect (F(9,81)=3.93; p<0.005). Post-hoc tests, Bonferroni
    corrected for multiple comparisons, revealed that classification performance for faces and buildings was significantly higher than for food and tools.


    Post-hoc tests revealed that performance for P/P classification was significantly higher than for the other three types. Importantly, the posthoc test showed no significant difference between the I/I classification and both the P/I and I/P classification tests. In other words, classification performance across the P and I conditions was just as good as performance within the I condition. Performance in the I/I classification test serves as an upper bound for the expected performance of the P/I and I/P classifications – this is because classification within each condition must theoretically be better than, or just as good as, classification across conditions. Thus, the finding that across-condition classification was not significantly different from classification within the I condition indicates that, overall, the activation patterns obtained on perception and imagery runs are at least as alike as patterns obtained on different runs of visual imagery.


    Post-hoc tests revealed that performance for P/P classification was significantly higher than for the other three types. Importantly, the posthoc test showed no significant difference between the I/I classification and both the P/I and I/P classification tests. In other words, classification performance across the P and I conditions was just as good as performance within the I condition. Performance in the I/I classification test serves as an upper bound for the expected performance of the P/I and I/P classifications – this is because classification within each condition must theoretically be better than, or just as good as, classification across conditions. Thus, the finding that across-condition classification was not significantly different from classification within the I condition indicates that, overall, the activation patterns obtained on perception and imagery runs are at least as alike as patterns obtained on different runs of visual imagery.


    Post-hoc tests revealed that performance for P/P classification was significantly higher than for the other three types. Importantly, the posthoc test showed no significant difference between the I/I classification and both the P/I and I/P classification tests. In other words, classification performance across the P and I conditions was just as good as performance within the I condition. Performance in the I/I classification test serves as an upper bound for the expected performance of the P/I and I/P classifications – this is because classification within each condition must theoretically be better than, or just as good as, classification across conditions. Thus, the finding that across-condition classification was not significantly different from classification within the I condition indicates that, overall, the activation patterns obtained on perception and imagery runs are at least as alike as patterns obtained on different runs of visual imagery.


    Post-hoc tests revealed that performance for P/P classification was significantly higher than for the other three types. Importantly, the posthoc test showed no significant difference between the I/I classification and both the P/I and I/P classification tests. In other words, classification performance across the P and I conditions was just as good as performance within the I condition. Performance in the I/I classification test serves as an upper bound for the expected performance of the P/I and I/P classifications – this is because classification within each condition must theoretically be better than, or just as good as, classification across conditions. Thus, the finding that across-condition classification was not significantly different from classification within the I condition indicates that, overall, the activation patterns obtained on perception and imagery runs are at least as alike as patterns obtained on different runs of visual imagery.


    Post-hoc tests revealed that performance for P/P classification was significantly higher than for the other three types. Importantly, the posthoc test showed no significant difference between the I/I classification and both the P/I and I/P classification tests. In other words, classification performance across the P and I conditions was just as good as performance within the I condition. Performance in the I/I classification test serves as an upper bound for the expected performance of the P/I and I/P classifications – this is because classification within each condition must theoretically be better than, or just as good as, classification across conditions. Thus, the finding that across-condition classification was not significantly different from classification within the I condition indicates that, overall, the activation patterns obtained on perception and imagery runs are at least as alike as patterns obtained on different runs of visual imagery.

    These results are to be contrasted with results we obtained in retinotopic areas.

    Meridian mapping: Horizontal + vertical flickering checkerboard pattern for 18 sec at each location

    Here we failed to find such behavior in retinotopic areas. This is to be contrasted with other studies (eg done in Kosslyn’s group) that have shown activation during imagery in V1. Here we think that this is due to the fact that beside the 1 back task our participants did not have any real task to perform associated with the imagery. What this suggests though is that activation from lower retinotopic areas is not necessary for imagery.

    These results are to be contrasted with results we obtained in retinotopic areas.

    Meridian mapping: Horizontal + vertical flickering checkerboard pattern for 18 sec at each location

    Here we failed to find such behavior in retinotopic areas. This is to be contrasted with other studies (eg done in Kosslyn’s group) that have shown activation during imagery in V1. Here we think that this is due to the fact that beside the 1 back task our participants did not have any real task to perform associated with the imagery. What this suggests though is that activation from lower retinotopic areas is not necessary for imagery.

    These results are to be contrasted with results we obtained in retinotopic areas.

    Meridian mapping: Horizontal + vertical flickering checkerboard pattern for 18 sec at each location

    Here we failed to find such behavior in retinotopic areas. This is to be contrasted with other studies (eg done in Kosslyn’s group) that have shown activation during imagery in V1. Here we think that this is due to the fact that beside the 1 back task our participants did not have any real task to perform associated with the imagery. What this suggests though is that activation from lower retinotopic areas is not necessary for imagery.

    These results are to be contrasted with results we obtained in retinotopic areas.

    Meridian mapping: Horizontal + vertical flickering checkerboard pattern for 18 sec at each location

    Here we failed to find such behavior in retinotopic areas. This is to be contrasted with other studies (eg done in Kosslyn’s group) that have shown activation during imagery in V1. Here we think that this is due to the fact that beside the 1 back task our participants did not have any real task to perform associated with the imagery. What this suggests though is that activation from lower retinotopic areas is not necessary for imagery.

    These results are to be contrasted with results we obtained in retinotopic areas.

    Meridian mapping: Horizontal + vertical flickering checkerboard pattern for 18 sec at each location

    Here we failed to find such behavior in retinotopic areas. This is to be contrasted with other studies (eg done in Kosslyn’s group) that have shown activation during imagery in V1. Here we think that this is due to the fact that beside the 1 back task our participants did not have any real task to perform associated with the imagery. What this suggests though is that activation from lower retinotopic areas is not necessary for imagery.

    These results are to be contrasted with results we obtained in retinotopic areas.

    Meridian mapping: Horizontal + vertical flickering checkerboard pattern for 18 sec at each location

    Here we failed to find such behavior in retinotopic areas. This is to be contrasted with other studies (eg done in Kosslyn’s group) that have shown activation during imagery in V1. Here we think that this is due to the fact that beside the 1 back task our participants did not have any real task to perform associated with the imagery. What this suggests though is that activation from lower retinotopic areas is not necessary for imagery.

    Another way to measure the overlap between I and P is to look at the correlation between the classifiers trained for every category and for each condition.

    If the two tasks are related we would expect a strong correlation on the diagonal suggesting that the face classifier trained on the P block is very similar to the classifier trained on the I blocks.


    FOR SINGLE SUBJECTS






    To test how classification performance depended on category and classification test, we conducted a 2-way repeated measures ANOVA of category X classification. The ANOVA revealed a significant main effect of category (F(3,27) = 10.27; p<.001), a significant main effect of classification test (F(3,27) = 11.51; p < .0001), and a significant interaction effect (F(9,81)=3.93; p<0.005). Post-hoc tests, Bonferroni
    corrected for multiple comparisons, revealed that classification performance for faces and buildings was significantly higher than for food and tools.

    This figure illustrates the average overlap of representations during perception and imagery in OR across subjects on the average brain. The average OR ROI in both hemispheres is shown in the leftmost column (red-yellow color map; corresponding color bar shown on the left). Note that this color map indicates the number of subjects who shared a given voxel -- voxels that were activated in just one subject were not included when displaying the average weight maps. In the right four columns we plot the average zscore of the SVM weight maps across subjects (see Methods), obtained for the perception and imagery classifiers trained on each category. Positive weights are shown in red and negative weights in green. Note that this figure is for illustrative purposes only. We separately quantified the similarity of weight maps by considering the overlap individually for each subject (i.e., not on the average brain), and then averaging across subjects. The significantly larger overlap between the perception and imagery weight maps within each category versus between categories is shown in Figure 6.

    This figure illustrates the average overlap of representations during perception and imagery in OR across subjects on the average brain. The average OR ROI in both hemispheres is shown in the leftmost column (red-yellow color map; corresponding color bar shown on the left). Note that this color map indicates the number of subjects who shared a given voxel -- voxels that were activated in just one subject were not included when displaying the average weight maps. In the right four columns we plot the average zscore of the SVM weight maps across subjects (see Methods), obtained for the perception and imagery classifiers trained on each category. Positive weights are shown in red and negative weights in green. Note that this figure is for illustrative purposes only. We separately quantified the similarity of weight maps by considering the overlap individually for each subject (i.e., not on the average brain), and then averaging across subjects. The significantly larger overlap between the perception and imagery weight maps within each category versus between categories is shown in Figure 6.

    This figure illustrates the average overlap of representations during perception and imagery in OR across subjects on the average brain. The average OR ROI in both hemispheres is shown in the leftmost column (red-yellow color map; corresponding color bar shown on the left). Note that this color map indicates the number of subjects who shared a given voxel -- voxels that were activated in just one subject were not included when displaying the average weight maps. In the right four columns we plot the average zscore of the SVM weight maps across subjects (see Methods), obtained for the perception and imagery classifiers trained on each category. Positive weights are shown in red and negative weights in green. Note that this figure is for illustrative purposes only. We separately quantified the similarity of weight maps by considering the overlap individually for each subject (i.e., not on the average brain), and then averaging across subjects. The significantly larger overlap between the perception and imagery weight maps within each category versus between categories is shown in Figure 6.



    Favorites, Groups & Events

    Reading the mind’s eye: Decoding object information during mental imagery from fMRI patterns - Presentation Transcript

    1. Reading the mind’s eye: Decoding object information during mental imagery from fMRI patterns T. Serre*1, L. Reddy*2, N. Tsuchyia3, T. Poggio1, M. Fabre-Thorpe2, C. Koch3 * equal contribution 1 McGovern Institute, MIT, Cambridge, MA 2 CerCo-CNRS, Universite Paul Sabatier, Toulouse, France 3 Computation & Neural Systems, California Institute of Technology, Pasadena, CA
    2. Category readout from MVPs of fMRI activity
    3. Category readout from MVPs of fMRI activity Robust category readout from MVPs of fMRI activity for visually presented objects in ventral temporal cortex Haxby et al ’01; Cox & Savoy ‘03; Carlson et al ’03; Mitchell et al ’03; Kamitani & Tong’05; Haynes & Rees ’05; Davatzikos et al ‘05; LaConte et al ‘05; Mourao-Miranda et al ’05; Kriegeskorte et al ’06 ’07; Reddy & Kanwisher ’07
    4. Category readout from MVPs of fMRI activity Robust category readout from MVPs of fMRI activity for visually presented objects in ventral temporal cortex Haxby et al ’01; Cox & Savoy ‘03; Carlson et al ’03; Mitchell et al ’03; Kamitani & Tong’05; Haynes & Rees ’05; Davatzikos et al ‘05; LaConte et al ‘05; Mourao-Miranda et al ’05; Kriegeskorte et al ’06 ’07; Reddy & Kanwisher ’07 Is readout still possible in the complete absence of bottom-inputs during mental visual imagery?
    5. I and P share the same representation?
    6. I and P share the same representation? I and P activations overlap: Early visual areas for orientation (Kosslyn et al ’95; Ganis et al ’04) FFA and PPA during imagery of faces and houses (O’Craven & Kanwisher ’00) LOC for simple letter shapes (Stokes et al ’09)
    7. I and P share the same representation? I and P activations overlap: Early visual areas for orientation (Kosslyn et al ’95; Ganis et al ’04) FFA and PPA during imagery of faces and houses (O’Craven & Kanwisher ’00) LOC for simple letter shapes (Stokes et al ’09) I and P activations differ: Negative BOLD differentiates I and P (Amedi et al ’05) Spatial overlap between I and P much larger in frontal than in ventral temporal cortex (Ganis et al ’04) I elicits weaker BOLD response (O’Craven & Kanwisher ’00; Ishai et al ’00)
    8. Univariate vs. Multivariate pattern analysis
    9. Univariate vs. Multivariate pattern analysis Same ROI average but different patterns of activity: Overestimate similarity of I and P mechanisms
    10. Univariate vs. Multivariate pattern analysis Same ROI average but P I different patterns of activity: average Overestimate similarity of I bold and P mechanisms
    11. Univariate vs. Multivariate pattern analysis Same ROI average but P I different patterns of activity: average Overestimate similarity of I bold and P mechanisms Visually responsive vs. category selective: Underestimate similarity of I and P mechanisms
    12. Univariate vs. Multivariate pattern analysis Same ROI average but P I different patterns of activity: average Overestimate similarity of I bold and P mechanisms Visually responsive vs. P I category selective: Underestimate similarity of I and P mechanisms XX XX XX XX
    13. Experimental paradigm n=10
    14. given region of interest (ROI) were indistinguishable between perception and imager f a complete spatial overlap of activated regions was observed, these measures coul Experimental paradigm be consistent with the two processes evoking different patterns of representation a level of individual voxels. Indeed, as several recent studies have shown, two activatio erns that are very different on a voxel-by-voxel basis can produce similar averag LD signal responses [1, 6]. Perception blocks (P) n=10 re 1: Experimental Design. The experiment consisted of two conditions. A). In the visual perception (P ition subjects viewed different exemplars of 4 categories of objects (famous faces, famous building
    15. given region of interest (ROI) were indistinguishable between perception and imager f a complete spatial overlap of activated regions was observed, these measures coul Experimental paradigm be consistent with the two processes evoking different patterns of representation a level of individual voxels. Indeed, as several recent studies have shown, two activatio erns that are very different on a voxel-by-voxel basis can produce similar averag LD signal responses [1, 6]. Perception blocks (P) Faces Buildings Food Tools n=10 re 1: Experimental Design. The experiment consisted of two conditions. A). In the visual perception (P ition subjects viewed different exemplars of 4 categories of objects (famous faces, famous building
    16. given region of interest (ROI) were indistinguishable between perception and imager f a complete spatial overlap of activated regions was observed, these measures coul Experimental paradigm be consistent with the two processes evoking different patterns of representation a level of individual voxels. Indeed, as several recent studies have shown, two activatio erns that are very different on a voxel-by-voxel basis can produce similar averag LD signal responses [1, 6]. Perception blocks (P) n=10 re 1: Experimental Design. The experiment consisted of two conditions. A). In the visual perception (P ition subjects viewed different exemplars of 4 categories of objects (famous faces, famous building
    17. given regionregion of interest (ROI)indistinguishable between perception and imageri in a given of interest (ROI) were were indistinguishable between perception and f aor if a complete spatial overlap of activated regions was observed, measures coul complete spatial overlap of activated regions was observed, these these measure Experimental paradigm bestill be consistentthe two processes evoking different patterns of representation a consistent with with the two processes evoking different patterns of represent level of individual voxels. Indeed, as severalseveral recent studiesshown, two activatio the level of individual voxels. Indeed, as recent studies have have shown, two ac erns that are very different on a voxel-by-voxel basis can produce similarsimilar patterns that are very different on a voxel-by-voxel basis can produce averag LD signal responses [1, 6]. [1, 6]. BOLD signal responses Perception blocks (P) Imagery blocks (I) n=10 re Figure 1: Experimental Design. The experiment consisted conditions. A). In the visual perception (P 1: Experimental Design. The experiment consisted of two of two conditions. A). In the visual perce ition subjectssubjects viewed different exemplars of 4 categories of objects (famousfamous famous b condition viewed different exemplars of 4 categories of objects (famous faces, faces, building
    18. MVPA of fMRI data
    19. MVPA of fMRI data Independent localizers object responsive (OR) voxels
    20. MVPA of fMRI data Independent localizers object responsive (OR) voxels Linear SVM classifier (OVA, 4AFC)
    21. MVPA of fMRI data Independent localizers object responsive (OR) voxels Linear SVM classifier (OVA, 4AFC) Leave-one-run-out
    22. MVPA of fMRI data Independent localizers object responsive (OR) voxels Linear SVM classifier (OVA, 4AFC) Leave-one-run-out run 1 ... run n-1 run n voxels TRAINING TEST
    23. MVPA of fMRI data P-P P P I-I I I run 1 ... run n-1 run n voxels TRAINING TEST
    24. MVPA of fMRI data P P-I P Generalization between I and P? see also (Stock et al ’09) I I-P I run 1 ... run n-1 run n voxels TRAINING TEST
    25. predicted actual 1 0
    26. 67% P P chance: 25%
    27. 67% 50% P P I I chance: 25%
    28. 67% 50% 47% 52% P P I I P I I P chance: 25%
    29. Object Responsive areas intact readout performance (% correct) scrambled voxels scrambled labels chance level 4AFC
    30. Object Responsive areas 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    31. Object Responsive areas 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    32. Object Responsive areas 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    33. Object Responsive areas 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    34. Object Responsive areas 75 o intact readout performance (% correct) scrambled voxels scrambled labels 50 * * * chance 25 level 4AFC 0 P-P I-I P-I I-P
    35. Retinotopic areas (V1+V2) intact readout performance (% correct) scrambled voxels scrambled labels chance level 4AFC
    36. Retinotopic areas (V1+V2) 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    37. Retinotopic areas (V1+V2) 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    38. Retinotopic areas (V1+V2) 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    39. Retinotopic areas (V1+V2) 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    40. Retinotopic areas (V1+V2) 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 chance 25 level 4AFC 0 P-P I-I P-I I-P
    41. Retinotopic areas (V1+V2) 75 intact readout performance (% correct) scrambled voxels scrambled labels 50 * chance 25 level 4AFC 0 P-P I-I P-I I-P
    42. Similarity between P/I classifiers OR P classifiers 0.2 0.1 0.0 -0.1 I classifiers
    43. Similarity between P/I classifiers OR Ret P classifiers 0.2 0.1 0.0 -0.1 I classifiers
    44. Similarity between P/I classifiers OR subject AH017 P classifiers subject I classifiers JL003
    45. Similarity between P/I classifiers VVIQ score W −B P-I sim = In answering items 1 to 4, think of some relative or friend whom you frequently see W +B (but who is not with you at present) and consider carefully the picture that comes before your mind’s eye. 1 The exact contour of face, head, shoulders and body. 2 Characteristic poses of head, attitudes of body etc. 3 The precise carriage, length of step, etc. in walking. 4 The different colors worn in some familiar clothes. Visualize the rising sun. Consider carefully the picture that comes before your mind’s eye. 5 The sun is rising above the horizon into a hazy sky. 6 The sky clears and surrounds the sun with blueness. 7 Clouds. A storm blows up, with flashes of lightening. 8 A rainbow appears. Think of the front of a shop, which you often go to. Consider the picture that comes before your mind’s eye. 9 The overall appearance of the shop from the opposite side of the road. 10 A window display including colors, shape and details of individual items for sale. 11 You are near the entrance. The color, shape and details of the door. 12 You enter the shop and go to the counter. The counter assistant serves you. Money changes hands. Finally, think of a country scene, which involves trees, mountains and a lake. Consider the picture that comes before your mind’s eye. 13 The contours of the landscape. 14 The color and shape of the trees. 15 The color and shape of the lake. 16 A strong wind blows on the tree and on the lake causing waves. Marks, 1973; Amedi et al ’05; Cui et al ’07
    46. Predicting “vividness of imagery” from fMRI signal 5.0 4.0 R² = 0.7045 3.0 VVIQ score 2.0 1.0 0 1.85 1.90 1.95 2.00 2.05 P-I similarity
    47. Summary Robust readout of mental imagery Robust generalization between I and P suggesting that I and P elicit very similar patterns of neural activity Preliminary data suggest that this similarity predicts the subjective vividness of imagery in individual subjects
    48. perception-fixation imagery-fixation perception-imagery (p<0.01) (p<0.01) (p<0.01)
    49. Object responsive (OR) Separate localizer runs for faces, scenes, objects vs. scrambled images (p < 10-4, uncorrected) Distributed voxels in ventral temporal cortex (see Haxby et al ’01) Includes LOC, FFA and PPA
    50. d at anel the line as a Feedback more diffuse than tion feedforward connections no- Shmuel et al. • Functional Organization of Feedback Projections V1 rre- nta- The was sot- rox- that tion anel as a defi- the reen V2 V1 re- mea- rom mal Schmuel et al ’05
    51. Significant main effects: Category Classification condition Interaction effect Performance for faces and buildings > than for food and tools Very similar results after excluding FFA and PPA
    52. Importance maps
    53. Importance maps perception imagery
    54. for OR and Figure 6B for the retinotopic ROI. A 2-way repeated measures ANOVA weights (within category/across category) x ROI revealed a significant main effec Importance maps weights (F(1,9) = 54.22; p < 0.0001), a significant main effect of ROI (F(1,9) = 52.67; 0.0001) and a significant interaction effect (F(1,9) = 48.94; p < 0.0005). Post- Bonferonni corrected comparisons revealed that the weight map overlap within cate was higher than across category, and higher in OR than in retinotopic voxels. perception imagery
    55. Subjective measure of vividness Sample question from the VVIQ: “Visualize the sun rising above a rocky mountain range into a bright sky. How vivid is your mental picture on a scale from 1 to 5, where 1 is akin to a photograph, and 5 is a pictureless concept?” Marks, 1973; McKelvie & Demers ‘79; McKelvie ’94; Hatakeyama ’97; Amedi et al ’05; Cui et al ’07
    56. Vividness of imagery Different people report experiencing different levels of vividness of imagery Could the similarity between the P-I conditions explain the subjective vividness of mental imagery across participants?

    + tserretserre, 6 months ago

    custom

    573 views, 0 favs, 0 embeds more stats

    This is a presentation done at the Vision Science S more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 573
      • 573 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 0
    • Downloads 15
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    -->
    Search
    RSS Feed
    What's new?

    © 2009 SlideShare Inc. All Rights Reserved