My talk slide at Dresden (Brain Dynamics on Multiple Scales - Paradigms, their Relations, and Integrated Approaches)

1. 17 June 20 @
Brain Dynamics on Multiple Scales - Paradigms, their Relations, and Integrated Approaches
Nao Tsuchiya 土谷 尚嗣
School of Psychological Sciences &
Monash Institute of Cognitive & Clinical Neuroscience (MICCN)
Monash University, Australia
"What is it like to be a bat?"
- a pathway to the answer
from integrated information theory of consciousness

2. My lab’s goal: to reveal the physical basis
of consciousness
4 empirical approaches:
1. Boundary of consciousness
- vs non-conscious processing
- vs unconscious states
2. Relationship between consciousness and associated processes
- attention, working memory, metacognition, expectation, report
3.Analysis of multi-electrode neural recordings
- fly LFP to human ECoG
4. Empirical testing of the theory of consciousness
- integrated information theory of consciousness
- predictive coding / free energy

3. Talk overview
• “What it is like to be a bat?”
• definitions, motivations
• Overview of IIT
• Empirical testing of IIT
• How to approach the bat problem?

4. What it is like to be a bat?
• Thomas Nagel 1974
• Clarify the difficulty of mind-body problem
• Bat - echolocation system

5. • Why does vision feels like vision?
• Why any visual experience is more
similar to other visual experience than
auditory experience?
• Is bat’s echolocation similar to vision,
audition, non-conscious or anything else?

6. bat’s experience
• Impossible stupid question?
• No, it can drive science of consciousness
• What’s the relevance to society?
• Any studeis of mental disease in model
animals: depression in mice (?)
• Theories that explain all available evidence
and make correct predictions can be
trusted when making extraordinary
predictions

7. A new framework for studying the relationship?
Relationship between qualia and mathematical structures
Tsuchiya, Taguchi, Saigo 2016 Neuroscience Research
mathematical
structures

8. Integrated information theory
of consciousness
• Starts from phenomenology, identifies five
axioms (1. existence, 2. composition, 3.
information, 4. integration, 5. exclusion)
• Tries to seek for potential physical
mechanisms that can support the axioms
Tononi 2004, 2008, Oizumi et al 2014 PLoS Comp Bio

9. Consciousness is highly informative
• Each experience is highly unique and excludes all other possible
experiences

10. Consciousness is composed of various aspects
Consciousness is integrated
Consciousness is excluded outside of the certain boundary

11. • Based on the 5 phenomenological axioms,
IIT derives 5 mathematical postulates.
• Using the postulates, IIT predicts a
particular level and contents of
consciousness of a system, based on how
the network is connected and which
state the system is in.

12. Integrated information theory in a nut shell
Copy
Copy
Tsuchiya 2017 Philosophy Compass

13. H=log2(4)=2 I=H-H*=2 Φ=I-I*=2
ΦAB, ΦAC,
ΦBC, ΦABC
Integrated information theory in a nut shell
Tsuchiya 2017 Philosophy Compass

14. Entropy H of X (assuming Gaussian
distribution of X):
−500 0 500 1000
100
0
100
0
100
0
100
0
-500 0 500 1000
time from stimulus onset (msec)
XD
XC
XB
XA
Oizumi et al 2016 PLoS Comp
Continuous variables with Gaussian assumption
https://figshare.com/articles/phi_toolbox_zip/3203326
Measures variability of responses

15. Entropy H of X (assuming Gaussian
distribution of X):
−500 0 500 1000
100
0
100
0
100
0
100
0
-500 0 500 1000
time from stimulus onset (msec)
XD
XC
XB
XA
Oizumi et al 2016 PLoS Comp
Φ*
= I − I*
Continuous variables with Gaussian assumption
https://figshare.com/articles/phi_toolbox_zip/3203326

16. Integrated information
=
loss of predictability of past states
based on current states,
when system is cut
(Oizumi et al 2016 PLoS Comp)
Intuitive explanation 1

17. (Oizumi et al 2016 PLoS Comp)

18. Integrated information
=
a distance between the actual and
disconnected model
Oizumi,Tsuchiya,Amari 2016 PNAS
Intuitive explanation 2

19. x1
x2
y1
y2
Difference
x1
x2
y1
y2
!me !me
Constraints
Integrated informa!on
Generaliza!on of the original measure proposed by Tononi in his
Integrated Informa!on Theory (Tononi, 2008).
Distance
Full model Disconnected model

20. Stochas(c interac(on
nforma(on
ed informa(on
Transfer entropy
y1
y2
x1
x2
y1
y2
y1
y2
x1
x2
y1
y2
Stochas(c inte
Mutual informa(on
Integrated informa(on
Transfer entro
x1
x2
y1
y2
x1
x2
x1
x2
y1
y2
x1
x2

21. A unified framework for causal interactions
based on information geometry
Oizumi,Tsuchiya,Amari 2016 PNAS

22. Isomorphism between structure of conscious experience
and structure of integrated information?

23. Mutual information between present and past (tau=6-10 ms)
−400 −200 0 200 400 600 800
5
5.2
5.4
5.6
H/chan
−400 −200 0 200 400 600 800
4.2
4.4
4.6
4.8
H
cond
/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
MI/chan
1
H ≈ log|Σ|
I = H - HCOND
HCOND
I*
−400 −200 0 200 400 600 800
4.2
4.4
4.6
4.8
Hcond
/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
MI/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
I*/chan
−400 −200 0 200 400 600 800
0
0.2
0.4
phi*
time from I onset
I = H - HCOND
φ* = I - I*
HCOND
I*
Haun et al 2016 bioRxiv
11 possible subsets
for 4 channels:
6 x 2 channels (AB,AC,..)
4 x 3 channels (ABC, …)
1 x 4 channels (ABCD)

24. Mutual information between present and past (tau=6-10 ms)
−400 −200 0 200 400 600 800
5
5.2
5.4
5.6
H/chan
−400 −200 0 200 400 600 800
4.2
4.4
4.6
4.8
H
cond
/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
MI/chan
1
H ≈ log|Σ|
I = H - HCOND
HCOND
I*
−400 −200 0 200 400 600 800
4.2
4.4
4.6
4.8
Hcond
/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
MI/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
I*/chan
−400 −200 0 200 400 600 800
0
0.2
0.4
phi*
time from I onset
I = H - HCOND
φ* = I - I*
HCOND
I*
Gaussian assumption
Randomness of the system
- entropy: H(Xt)
Haun et al 2016 bioRxiv

25. −400 −200 0 200 400 600 800
4.2
4.4
4.6
4.8
Hcond
/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
MI/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
I*/chan
−400 −200 0 200 400 600 800
0
0.2
0.4
phi*
time from I onset
I = H - HCOND
φ* = I - I*
HCOND
I*
−400 −200 0 200 400 600 800
5
5.2
5.4
5.6
H/chan
−400 −200 0 200 400 600 800
4.2
4.4
4.6
4.8
H
cond
/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
MI/chan
1
H ≈ log|Σ|
I = H - HCOND
HCOND
I*
−400 −200 0 200 400 600 800
4.2
4.4
4.6
4.8
Hcond
/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
MI/chan
−400 −200 0 200 400 600 800
0.6
0.8
1
I*/chan
−400 −200 0 200 400 600 800
0
0.2
0.4
phi*
time from I onset
I = H - HCOND
φ* = I - I*
HCOND
I*
Mutual information between
present and past (tau=3 ms)
Haun et al 2016 bioRxiv

29. Comparing contents of consciousness across three tasks

30. Unmasked conditions - 3 of 5 categories shown
Haun et al 2016 bioRxiv

31. Backward masking - 17 ms face in one of 4 quadrants
Haun et al 2016 bioRxiv

32. Visibility=4 Visibility=3 Visibility=2 Visibility=1
High
contrast
face
Middle
contrast
face
Low
contrast
face

33. 0.1
1
Φ
CFSBM UNM
Faces(CFS,BM,UNM) Masked Faces & Mondrians (CFS,BM,UNM)Houses (UNM) Tools (UNM)
0.1
1
I
0.1
1
H
Haun et al 2016 bioRxiv
Phi, but not I and H, are correlated with contents of consciousness
on a trial-by-trial basis

34. Quantifying isomorphism across different tasks and
subjects as classification accuracy
Haun et al 2016 bioRxiv

36. Summary of this talk
• Overview of IIT
• Integrated information as:
• loss of decoding accuracy (Φ*)
• distance between full and disconnected
models (ΦG)
• Empirical testing of IIT
• Phi patterns: hierarchy of causal interactions
better correlates with conscious perception
than entropy or mutual information.

37. A path to bats’ experience…
• Can be compared between sensory
modalities
• Across individuals
• Across species
• Common topological properties of phi*
pattern for vision vs audition?
• What is it like to be a bat?
• Dissolution of the Hard Problem?

38. Acknowledgment
• Thanks for your attention & consciousness!
• Support
• ARC Future Fellowship
• ARC Discovery Project
• JST PRESTO grant