Consciousness Quantify qualia, similarity rating, massive report paradigm, attention, yoneda lemma, category theory, functor

1. Quantifying qualia
from 11am-: IIT symposium
2020 Aug 1 @ 43rd Annual Meeting of the Japan Neuroscience Society
Nao Tsuchiya 土谷 尚嗣 Twitter: @conscious_tlab
School of Psychological Sciences and Turner Institute for Brain and Mental Health
Monash University, Australia

2. Hypothesis
Structure of consciousness
Structure of brain
Structure of information
Substrate-independent
Filter out non-conscious

3. [Goal] Charactrerize
1. Structure of information via IIT
2. Structure of phenomenology via
similarity rating
attentional manipulation
massive report paradigm
3. Relationship between two types of structures
via category theory
[Aim] Search for an isomorphism between
phenomenal structure and informational structure

4. 1. How can we investigate
the relationship between
the phenomenological and
the informational structures?

5. Why category theory?
• Can make a bridge between structures of IIT and
phenomenology
• Can offer a powerful conceptual framework for
• Making progress in the isomorphism search
• Inspiring novel methods to reveal an aspect of
the phenomenal structure

6. Category Theory
as a framework to characterze
relations among distinct domains
• Geometry and algebra (Eilenberg & MacLane 1945)
• Quantum mechanics, topology, logic, and
computation (Baez & Stay 2009)
• Phenomenology and information
• Information & IIT (Manin 2020 Arxiv, Vigneaux 2019 ArXiv, Baoudot
2015 Entropy, Northoff,Tsuchiya, Saigo 2019 Entropy, Kleiner & Tull
2020 Arxiv)

7. Finding functors as a step
before finding isomorphisms
• A mapping from category X toY that preserve
the structure of X
• e.g., Phe->IIT functor would preserve
“similarity” of percept in Phe to “similarity”
in IIT
• IIT->Phe functor should do the same
• A and B are isomorphic if
• F:A->B, G:B->A, and 1A=F;G, 1B=G;F
• Then, F and G are called isomorphism

8. Tsuchiya,Taguchi, Saigo 2016 Neuroscience Research,
Tsuchiya & Saigo 2020 PsyARxiv (under review)

9. Functor
• Advantage:
• Functor is much looser than “isomorphism”
• Existence of a functor is empirically testable
from both directions

10. Category theory inspires
phenomenology research
TheYoneda Lemma:

11. A web of relationships to translate concepts and
percepts between subjects
• Shepard 1970, Edelmann 1999, Kriegeskorte 2013
ABSURDIST by Goldstone et al 2004

12. 2.What is the structure of
consciousness?

13. • Two candidate paradigms to examine the
structure of consciousness
• Similarity structure of qualia (qualia in a
narrow sense)
• Conceptual structure of a moment of
experience (qualia in a broad sense)
• Perturb the structure through manipulation
of attention

14. Similarity ratings
• Can potentially serve like a “distance” to
characterize structure of information (e.g.,
IIT’s MICS) and structure of conscious
phenomenology

15. Any moment of conscious experience
(=qualia) is structured

16. Consciousness is defined by the relationship with other
possible consciousness, not perceived at the moment

17. Various Structure of Contents of Consciousness
(=Qualia)
Color SoundFace

18. Empirical estimation of Qualia structure
Rowe, D’Souza, van Boxtel, Tsuchiya (in preparation)
Poster: https://figshare.com/articles/Joanita_poster_share_pdf/10258580

19. Single task
Similarity Matrix

20. Single task
Similarity Matrix
Single task
Similarity Matrix
Single task
Multi Dimensional Scaling

21. Single task
Similarity Matrix
Single task
Multi Dimensional Scaling
Dual task
Similarity Matrix

22. Single task
Similarity Matrix
Single task
Multi Dimensional Scaling
Dual task
Similarity Matrix
Dual task
Multi Dimensionla Scaling

23. Interim summary:
• Attention as a way to perturb
phenomenology.
• Some phenomenal strcutre can be distrupted
when top-down spatial attention is withdran,
whole others can remain intact.
• A potential tool to investigate isomorphism
between the phenomenal and the
informaitonal structures

24. Any moment of conscious experience
seems highly informative & structured
Koh, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

25. Q. Is a moment of conscious experience
highly informative?
-> Can we distinguish many different experiences
among all possible experiences?
Q. Is conscious experience meaningfully structured?
-> Do people describe a moment of experience
in a coherent structure?

26. Lab (N=10) & Online (N=670) testing over 400 images
-> ~1300 trials, 4000 unique words
Koh, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

27. Koh, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

28. An exemplar response 67 ms
133 ms 267 ms
Koh, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

29. For none of other >400 images, no people
reported seeing “eiffel-tower”
13 / 27 people reported seeing “eiffel-tower”
on this image.
“Leave-one-out analysis”
Highly selective & specific -> AUC=1.0
Koh, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

30. Exemplar top intersubjectively
informative words
AUC
Worst examples:
AUC
Koh, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

31. • Across 400 images and 670 participants,
mean AUC per word was 0.64 (for
SOA=67ms) and 0.69 (for SOA=133 or
266ms) when 5 words were reported on a
given image.
• Empirically, neally all pairs of words had no
correlations in reported patterns.

32. Freely described words are not correlated
(4185 words among 570 images ; 8755020 correlation values)
0
.25
.50
.75
1
0.75 0.80 0.85 0.90 0.95 1.00
Cumulativeproportion
Koh, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

33. • Across 400 images and 670 participants,
mean AUC per word was 0.64 (for
SOA=67ms) and 0.69 (for SOA=133 or
266ms) when 5 words were reported on a
given image.
• Empirically, neally all pairs of words had no
correlations in reported patterns.
• Participants generated highly informative
(specific and selective) reports

34. How well can we distinguish distinct concepts that
accompany conscious experiences?
How much do we see?
What we see > We can report
50ms presentation
(Sperling, 1960)
Traditional estimate ~ 40 bits/s; Norretranders 1991

35. blue-
hat boy coat country horse jarchee old stand sunlight
zoomed
-in
build
building
city
daytime
people
sea
sky
skyline
tree
water
Present
words
Absent words
Present > Absent
Present = Absent
Present < Absent
N=15, 133ms, 20 quesitons
AUC=0.74
100*2*(0.74-0.5)/0.13
=369 bits/s
blue-
hat boy coat country horse jarchee old stand sunlight
zoomed
-in
build
building
city
daytime
people
sea
sky
skyline
tree
water
ent
ds
Present > Absent
Present = Absent
Present < Absent
Gallagher, Koh, Walther, Loeffler, Nishimoto, van Boxtel, Tsuchiya
(2018 SOBR poster; in preparation)

36. Liang, Gallagher, Nishimoto, Tsuchiya (2019 SOBR poster; in preparation)

37. Part 2 summary
• Structure of a moment of conscious experience
• Much richer and conceptually richly reportable
and recognizable than previously thought
• Consistent with “informativeness” &
“composition” axioms (providing more numerical
supports)
• Possible to model phenomenology
mathematically?
• Tsuchiya & Saigo 2020 PsyArxiv, Prentner 2019 Conscious
& Cognition, Haun & Tononi 2019 Entrpopy

38. Takehome messages and
future directions
• To crack the fundamenal question on the physical basis of
consciousness, it is necessary:
1.To characterize the structure of qualia
2.To derive the structure of information from neural data
3.To link between the phenomenal and the informational
domain

39. Acknowledgment
• Thanks for your attention & consciousness!
• Support
• ARC Discovery Project
• NHMRC Ideas grants
• Templeton World Charity Foundation
Monash Neuroscience of Consciousness (MoNoC)
Melbourne-Monash Consciousness Research (MMCR) network