• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Talk esscs2010
 

Talk esscs2010

on

  • 346 views

Greco-Carrea, Grounding compositional symbols (talk at ESSCS meeting in London, 6-7 July 2010)

Greco-Carrea, Grounding compositional symbols (talk at ESSCS meeting in London, 6-7 July 2010)

Statistics

Views

Total Views
346
Views on SlideShare
346
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Talk esscs2010 Talk esscs2010 Presentation Transcript

    • Grounding compositional symbols: no composition without discrimination? Alberto Greco, Elena Carrea University of Genoa, Italy greco@unige.it elena.carrea@unige.it ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Overview  What is Symbol Grounding?  Symbol Grounding and Compositionality  Why is Compositionality relevant to Cognitive Sciences?  Purposes of the Study  Experiment  Provisional conclusions ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem Searle (1980) argued against the symbolic model of mind with the mental experiment of the Chinese Room. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem Can we say that this man understands Chinese? ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem Harnad’s Merry-go-round: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem Harnad’s Merry-go-round: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem Harnad’s Merry-go-round: This problem is similar to having to learn Chinese with only a Chinese/Chinese dictionary. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem How can symbolic systems be linked to the world? Can a purely symbolic model be grounded only in other meaningless symbols? This is the Symbol Grounding problem (Harnad 1990). ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem Harnad’s solution: grounding experience must be sensorimotor… ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • The Symbol Grounding problem Symbols (words) acquire their meaning only when they are associated with sensorimotor grounding representations. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Overview  What is Symbol Grounding?  Symbol Grounding and Compositionality  Why is Compositionality relevant to Cognitive Sciences?  Purposes of the Study  Experiment  Provisional conclusions ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Compositionality In Harnad's view, grounding has been conceived as compositional: from sensorimotor toil to symbolic theft, thanks to propositions. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Compositionality An example: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Compositionality An example: “a zebra is like a horse with stripes”… ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Compositionality What is compositionality? A definition of compositionality from a conference on this topic*: “Compositionality is a key feature of structured representational systems, be they linguistic, mental or neuronal. A system of representations is compositional just in case the semantic values of complex representations are determined by the semantic values of their parts.” * Compositionality, Concepts, and Cognition: An Interdisciplinary Conference in Cognitive Science, Düsseldorf, Germany, February 28 to March 3, 2004 ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Compositionality  Systematic combination of meaningful components,  According to syntactical rules.  Meaning of complex expressions,  Productivity (linguistic and conceptual). ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Overview  What is Symbol Grounding?  Symbol Grounding and Compositionality  Why is Compositionality relevant to Cognitive Sciences?  Purposes of the Study  Experiment  Provisional conclusions ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Compositionality Why is it relevant to Cognitive Sciences?  Formal languages of mathematics, logic, and computer science are (usually) considered as compositional.  Linguistics: are natural languages compositional?  Philosophy: is whole meaning really coming out from part meaning? What about context?  Psychology: is representation compositional? Is language of thought compositional? (from cognitivism to ecological, embodiment, re-enacting theories).  Cognitive and neural modelling (AI, connectionism, robotics): no easy compositionality in neural networks; binding problem; Can analog features be represented compositionally? ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Overview  What is Symbol Grounding?  Symbol Grounding and Compositionality  Why is Compositionality relevant to Cognitive Sciences?  Purposes of the Study  Experiment  Provisional conclusions ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study Key-question: What is complex symbolic composition based on? …a corresponding composition of grounding sensorimotor representations? ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study Key-question (in other words): Are structured grounding representations needed, to express meanings that have a complex structure? ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study Key-question (in other words): Are structured grounding representations needed, to express meanings that have a complex structure? How can we study this empirically? ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study How could we empirically test whether grounding representations have been established? Starting from scratch… from nonsense, to meaning… ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study Our solution: association task between nonsense symbols (words) and target perceptual or motor stimulus patterns. nonsense words perceptual or motor patterns (arbitrary, not yet grounded symbols) ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study nonsense words perceptual or motor patterns Our assumption: successful learning of this association shows that symbols have been grounded, and that a corresponding grounding representation has been established. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study The experiment should allow us to distinguish between two possibilities: blue circle - compositional Grounding REPresentations: separate GREPs for each part (when a concept GREPs comes from the combination of the parts ); - holistic GREPs: these combined blue-circle concepts are best learned by bringing them back to a single unifying GREP. GREP ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study Previous research on compositional grounding with motor patterns Greco & Caneva (2005) associated 3 pseudo-words to 3 features (e.g. GAB DIN FIT or TANEC for “push right hand”). compositional holistic Greco & Caneva (2009) associated 2 pseudo-words to 2 features (a particular nonsense motor pattern & hands up, down, fist) (e.g. BASPI NOLE or TERPESOVA). compositional holistic And in other conditions (e.g. incremental learning) holistic again …no consistent results ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study Inconsistency in these results may be explained by several factors: • difficulty of learning tasks, • different learning methods, • words not balanced across conditions… ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study We presupposed that: • a single symbol is best supported by a single Grounding REPresentation (GREP) when a pattern tends to be represented holistically • separate symbols (to be combined) are best supported by separate GREPs when pattern features tend to have separate representations ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study We presupposed that: • a single symbol is best supported by a single Grounding REPresentation (GREP) when a pattern tends to be represented holistically • separate symbols (to be combined) are best supported by separate GREPs when pattern features tend to have separate representations But we really don’t know much about the nature of motor representations (do they tend to be represented analytically or holistically?). We need stimuli that have a known representation… ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Purposes of the Study We presupposed that: • a single symbol is best supported by a single Grounding REPresentation (GREP) when a pattern tends to be represented holistically • separate symbols (to be combined) are best supported by separate GREPs when pattern features tend to have separate representations But we really don’t know much about the nature of motor representations (do they tend to be represented analytically or holistically?). We need stimuli that have a known representation… ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Overview  What is Symbol Grounding?  Symbol Grounding and Compositionality  Why is Compositionality relevant to Cognitive Sciences?  Purposes of the Study  Experiment  Provisional conclusions ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment From literature… Handel & Imai (1972) showed different kinds of properties of visual objects: Integral Separable ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment From literature… Handel & Imai (1972) showed different kinds of properties of visual objects: Integral Separable When attributes are fused together, and are perceived as one. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment From literature… Handel & Imai (1972) showed different kinds of properties of visual objects: Integral Separable When attributes are When attributes can be fused together, and are easily perceived perceived as one. separately. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment We repeated some classical experiments of Handel & Imai to select good (integral and separable) stimuli for our study. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment  It is not an absolute separation;  integral and separable stimuli, not features. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment INTEGRAL STIMULI: We put the figures in jars to make the task more engaging. As integral stimuli, we used triangles with the same blue which varied according to a matrix with 4 degrees of brightness and saturation. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment INTEGRAL STIMULI: We put the figures in jars to make the task more engaging. As integral stimuli, we used triangles with the same blue which varied according to a matrix with 4 degrees of brightness and saturation. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment INTEGRAL STIMULI: We put the figures in jars to make the task more engaging. As integral stimuli, we used triangles with the same blue which varied according to a matrix with 4 degrees of brightness and saturation. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment INTEGRAL STIMULI: We put the figures in jars to make the task more engaging. As integral stimuli, we used triangles with the same blue which varied according to a matrix with 4 degrees of brightness and saturation. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment INTEGRAL STIMULI: We put the figures in jars to make the task more engaging. As integral stimuli, we used triangles with the same blue which varied according to a matrix with 4 degrees of brightness and saturation. An example: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment SEPARABLE STIMULI: We used colored polygons as separable stimuli. We put into a matrix 4 shapes and 4 colors as separable stimuli. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment SEPARABLE STIMULI: We used colored polygons as separable stimuli. We put into a matrix 4 shapes and 4 colors as separable stimuli. An example: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment SEPARABLE STIMULI: We used colored polygons as separable stimuli. We put into a matrix 4 shapes and 4 colors as separable stimuli. An example: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment SEPARABLE STIMULI: We used colored polygons as separable stimuli. We put into a matrix 4 shapes and 4 colors as separable stimuli. An example: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment SEPARABLE STIMULI: We used colored polygons as separable stimuli. We put into a matrix 4 shapes and 4 colors as separable stimuli. An example: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment In our task meaningless labels were associated with integral and separable stimuli. We set also two conditions for the labels:  holistic (a single word for the stimulus as a whole);  compositional (two words that describe two features of the stimulus). ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment Four conditions: compositional holistic label label (1 word) (2 words) integral stimuli P Q separable stimuli R S ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment Conceptual universe: nole bote sove pofe baspi terpesova pincelura volsicoda tispi mutiralda feltorana patrasina respi rispaguna balartoca luticanza cuspi sertamina mertogala dortamana ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment • Participants (28 undergraduate students) saw the instructions on a PC screen. • The instructions explained that the task was to learn and remember the names of some jars. STAGES: 1.familiarization with the stimuli (judging the similarity between pairs of jars); 2.associative Learning (Ss. saw a jar and its name); 3.test (Ss. should click the correct name after seeing the jar). ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment Results: After familiarization (similarity judgments), we found the same results reported in the literature. (distance best measured by euclidean metric for integral pairs, by city-block metric for separable pairs). ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment Expectations ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment Results: ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Experiment Results (percentage of correct answers): ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Overview  What is Symbol Grounding?  Symbol Grounding and Compositionality  Why is Compositionality relevant to Cognitive Sciences?  Purposes of the Study  Experiment  Provisional conclusions ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Integral stimuli: the association with 2 words is more difficult because of what we called “competitive grounding”. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Integral stimuli: the association with 2 words is more difficult because of what we called “competitive grounding”. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Integral stimuli: the association with 2 words is more difficult because of what we called “competitive grounding”. First the subject thinks that baspi- BOTE is a certain blue. In a second moment another blue is called tispi-BOTE (because of the same lighting). This creates a competition between the 2 representation. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Integral stimuli: the association with 2 words is more difficult because of what we called “competitive grounding”. First the subject thinks that baspi- BOTE is a certain blue. In a second moment another blue is called tispi-BOTE (because of the same lighting). This creates a competition between the 2 representation. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Separable stimuli: learning with 2 words did not show any extra-benefit because of the limited number of words and stimuli to recall. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Separable stimuli: learning with 2 words did not show any extra-benefit because of the limited number of words and stimuli to recall. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Separable stimuli: learning with 2 words did not show any extra-benefit because of the limited number of words and stimuli to recall. Compositionality probably shows its convenience with a higher number of words and stimuli. With only 8 words it’s easier to learn a single label. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Separable stimuli: learning with 2 words did not show any extra-benefit because of the limited number of words and stimuli to recall. Compositionality probably shows its convenience with a higher number of words and stimuli. With only 8 words it’s easier to learn a single label. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions How can we explain these results? • Separable stimuli: learning with 2 words did not show any extra-benefit because of the limited number of words and stimuli to recall. Compositionality probably shows its convenience with a higher number of words and stimuli. With only 8 words it’s easier to learn a single label. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions For this reason we are doing a new experiment with 12 words and stimuli to learn. We are interested in studying if compositionality becomes convenient (with separable stimuli) beyond a critical number of stimuli. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions Limitations: Using meaningless words is not “ecological”; It’s difficult to create words free from participant’s personal association with common words;  It’s difficult to neutralized the prior knowledge (e.g. about geometric shapes). ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions Benefits:  We set a paradigm for testing grounding representation with a learning task;  We can test whether 2 labels of the compositional condition really have 2 different grounding representations. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions In the final step of the experiment we asked participants to select among 4 jars the one described by a particular sentence. In this set 2 jars shared 1 attribute (color) but not the other (shape). In other words: 2 jars were BASPI but only 1 was BASPI-NOLE. Then we asked the opposite task (two NOLE but only one BASPI). ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010
    • Provisional conclusions By using this setting we could exclude that the participants memorized the compositional sentence as a single word. ESSCS 2010 London, 7^ July 2010 lunedì 26 luglio 2010