Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex

Talk I have delivered at SfN 2018 about our work at Institute of Computer Science of University of Tartu in collaboration Lyon Neuroscience Research Center. We use deep intracranial recordings from humans to compare visual representations of human visual system with the visual representations learned by deep convolutional neural networks.

Published paper: https://www.nature.com/articles/s42003-018-0110-y
Code and data: https://github.com/kuz/Human-Intracranial-Recordings-and-DCNN-to-Compare-Biological-and-Artificial-Mechanisms-of-Vision

  • Login to see the comments

  • Be the first to like this

Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex

  1. 1. Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex Ilya Kuzovkin, Raul Vicente, Mathilde Petton, Jean-Philippe Lachaux, Monica Baciu, Philippe Kahane, Sylvain Rheims, Juan R. Vidal and Jaan Aru
  2. 2. Biological system of vision
  3. 3. Biological system of vision
  4. 4. Biological system of vision
  5. 5. Artificial system of vision
  6. 6. Artificial system of vision
  7. 7. Artificial system of vision
  8. 8. Artificial system of vision
  9. 9. Artificial system of vision
  10. 10. Artificial system of vision
  11. 11. Is there a correspondence between the hierarchies of features of artificial and biological systems of vision?
  12. 12. Güçlü, U. & van Gerven, M. A. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. J. Neurosci. 35, 10005–10014 (2015). Yamins, D. L. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19, 356–365 (2016). Eickenberg, M., Gramfort, A., Varoquaux, G. & Thirion, B. Seeing it all: convolutional network layers map the function of the human visual system. Neuroimage 152, 184–194 (2016) Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A. & Oliva, A. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Sci. Rep. 6, (2016).
  13. 13. x100
  14. 14. x100
  15. 15. x100 ~100 probes per subject 9073 probes total
  16. 16. Quantify the diagonal trend ρalign = Spearman( Assignment to DNN layers, Assignment to Brodmann areas)
  17. 17. Quantify the diagonal trend ρalign = Spearman( Assignment to DNN layers, Assignment to Brodmann areas)
  18. 18. - simple features (mapped to lower layers of DCNN) - complex features (higher layers)
  19. 19. Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex https://github.com/kuz/Human-Intracranial-Recordings-and-DCNN-to-Compare- Biological-and-Artificial-Mechanisms-of-Vision https://www.nature.com/articles/s42003-018-0110-y

×