PowerPoint Presentation


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Adding a new descriptor is not a matter of adding a new proposition to the list: each previously inserted descriptor constrains and modifies the integration of following descriptors.
  • Embodied Meaning
    instead of meaningless symbols, representations could have a structure that is lawfully related to the objects being represented. Different approach: Possibilities of analogical representation underlying language comprehension.
    Embodied approach to Meaning
    Perceptual systems facilitate and enable our interaction with the real world
    To do this world is conceptualized as patterns of possible bodily interactions
    The meaning of a particular object, sentence is what the person can do with that object event or sentence
    Embodied Meaning / Embodied Representations: embodied meaning captures spatial-functional properties of the object or situation. EMBODIED: because the world is perceived in terms of its potential for interaction with an individual’s body.The perception of projectable properties is in terms of patterns of possible action: how we can grasp, move, leap over a certain object. This action based coding depends on the capabilities of our bodies
    Symbol Grounding Problem and Embodied Representations
    How does this answer objections opposed to traditional approaches?
    Embodied representations do not need to be mapped onto the world to become meaningful because they arise from the world. Embodied representations are directly grounded by virtue of being lawfully and analogically related to properties of the world and how those properties are transduced py perceptual action systems. (Harnad 1990; 1993)
    Meaningful action oriented component of conceptualization is not abstract and amodal. It reflects how our bodies can interact with objects. Embodied representations do not need to be mapped on the world to be grounded.
    Terms definition:
    Projectablerefers to properties of the environment that can be specified by
    Information available
    Nonprojectable properties that must be provided by other sources
    e.g. spatial layout of a place is a projectable property whereas “ownership” is a nonprojectable property that comes from experience.
    Notions of mesh: patterns of actions underlying one idea can be integrated with patterns of action underlying another. Patterns mutually modify and constrain each other because the conjoint actions must be possible given the constraints our body and the environment impose upon us. Mutual modification of patterns of action underlies construction of meaning from words whose senses are jointly modified by the context in which they occur. When patterns mash or combine they modify each other because the must be combined in a way that satisfies the constraints imposed upon bodily interaction.
    Example of cognitive meshing: Basalou: Imagine a ball, imagine it has stripes, imagine it is defloated.pg.6 right side
  • Scanner data, white matter, blow up to see structures, colour regions
  • PowerPoint Presentation

    1. 1. Situation Models and Embodied Language Processes Franz Schmalhofer University of Osnabrück / Germany 1) Memory and Situation Models 2) Computational Modeling of Inferences 3) What Memory and Language are for 4) Neural Correlates 5) Integration of Behavioral Experiments and Neural Correlates (ERP; fMRI) by Formal Models
    2. 2. Cognition and Knowledge • Traditional: – Cognition = Computation – Representation by propositions – Propositions are abstract relations • Embodiment of Meaning – Cognition is serving perception and actions – Representation = Patterns of possible bodily interactions with the world (lawfully related to the world) – What an object, event, sentence means for you, is what you can do with the object, event, sentence.
    3. 3. Embodiment of Memory (Glenberg, 1997) • Projectable properties: information available through the senses • Non-projectable properties: information available through other sources (e.g. memory) • Conceptualization: Combination (mesh) of projectable and non-projectable properties  Primary function of memory is to mesh the embodied conceptualizations of projectable properties of the environment with embodied experiences that provide non-projectable properties
    4. 4. Embodiment of Memory • Evidence for embodiment... • ...and affect: – Forced frowning or smiling influences affective judgments (Berkowitz & Trocolli 1990, Berkowitz et. al 1993) • ...and imagery: – Actually rotating facilitates orientation opposed to imagining rotating(Montello & Presson 1993, Rieser etl al. 1994). • ...and memory: – Retrieval of memorized spatial layouts depends on position on body axis (Bryant 1992).
    5. 5. Embodied Memory • Combination of patterns of possible action = meshing. • Meshing of patterns of action derived from: – projectable properties of the environment – non-projectable properties • as (spatial-functional) constraint satisfaction.
    6. 6. Function of Memory • Meshing projectable & non-projectable properties: • Meshing is important for: – perception – imagination – Comprehension • Projectable properties as well as non-projectable properties can be meshed with each-other.
    7. 7. Interaction of perceptions and memory • Environment has to be primary • Clamping projectable properties keeps the system reality-oriented • Experiences stay individuated
    8. 8. Updating memory • Experiences are shifts between conceptualizations • Trajectories from one pattern of action to another • Trajectories more often used become reinforced
    9. 9. Prediction and Planning • Hypothetical conceptualizations by using trajectories from memory • But: – Simulating action does not change the environment – Clamped projectable properties provide wrong constraints for prediction – Environment has to be suppressed (effortfull process) • Suppression loosens tie to reality
    10. 10. Two modes of memory (like the distinction between implicit/explicit) • Automatic – Meshing of projectable and nonprojectable properties – Causes conceptual priming: based on non-projectable properties, (therefore semantic) • Effortful – Suppression of projectable properties, conceptualization by trajectories from memory
    11. 11. Cognitive Meshing • Imagine a ball • Now Imagine that it has yellow and white stripes • Now Imagine that it is deflated • Mutual modification of mashed pattern: not only the ball but also the stripes become deformed when the ball is deflated. • Patterns of actions to the same spatio-functional constraints.
    12. 12. What memory and language are for • The primary function of memory “is to mesh the embodied conceptualizations of projectable properties of the environment with embodied experiences that provide nonprojectable properties…This meshed conceptualization, the meaning, is in the service of control of action in a three-dimensional environment“ (Glenberg 1997) • “Language is a surrogate for experience” (Taylor and Tversky, 1992):
    13. 13. Summary: Embodied Representations & Symbol Grounding • Embodied Meaning – action based coding of objects and situations • Embodied representations are lawfully and analogically related to properties of the world (Harnard, 1990, 1993) • Notion of “mesh”: mutual modification of patterns of action (Glenberg, 1997) • Meaning of a situation is a meshed pattern of possible actions = embodied conceptualization.
    14. 14. Overview of the methods of cognitive neuropsychology/science (I) • Advances in science by technology: – Invention of the telescope in 1608 changed astronomers‘ observational methods – If well-formulated questions are not asked, even the most powerful tools will not provide sensible answers • Cognitive Psychology / Computer Modeling • Neuroanatomy – Gross Neuroanatomy (general structures and connections) – Fine Neuroanatomy (components of individual neurons)
    15. 15. Overview of the methods of cognitive neuropsychology/science (II) • Neurophysiology (experimental methods used with animals) – Electrical stimulation – Single-cell recording – Lesions – Genetic manipulations • Neurology – Structural imaging and neurological damage – Causes of neurological disorders (vascular disorders,tumors, degenerous and infectious disorders, traumata, epilepsies) – Functional neuro-surgery
    16. 16. Overview of the methods of cognitive neuropsychology/science (III) • Converging methods – Cognitive deficits following brain damage – Virtual lesions: Transcranial magnetic stimulation (TMS) – Functional imaging • Electrical and magnetic signals in the brain (EEG, MEG) • Metabolic signals – Positron emission tomography (PET): regional cerebral blood flow; – fMRI: blood oxygenation level dependent effect or BOLD effect
    17. 17. Brain functions (1810-1819) • Do parts of the brain working independently enable the mind? (componential hypothesis) • Franz Joseph Gall and J. G. Spurzheim – 35 specific brain functions – Language, color perception, hope, self-esteem – With practice, areas grow, causing a bump in the overlying skull – Anatomical personology – phrenology • Does the whole brain work in concert? (wholistic hypothesis) • Pierre Flourens (1794-1867) – All sensations, all perceptions and all volitions occupy the same seat in these cerebral organs. – The faculty of sensation, percept and volition is then esssentially one faculty. – Empirical evidence: no matter where he leasoned a bird brain, the bird recovered
    18. 18. Language Areas • Broca • Wernicke
    19. 19. Neuroanatomist Korbinian Brodman (1909) • Analyzed cellular organization of the cortex • Tissue stains to visualize different brain regions • To a large extent cytoarchitecturally described brain areas do indeed represent functionally distinct brain regions
    20. 20. Componential or wholistic? Again the question • Camillo Golgi – Developed stain that impregnated individual neurons – Believed the whole brain to be a continuous mass of tissue that shares a common cytoplasm • Cajal – Used Golgi stains – Identified the unitary nature of neurons – Transmittion of information by electricity
    21. 21. How does the nervous system work (20-th century) • Understand how single neurons behave and interact • Knowing all the elements, can we figure out the system? • Billions of neurons • Brain-damaged humans show lack of typical symptoms • Impossible to localize „higher cognitive functions“ Jackson: Lesion might well affect other structures in the brain because the lesion might have damaged neurons connected to other regions; diaschisis: damage of one part can create problems for another. Gestaltist view: The whole is different from the sum of its parts
    22. 22. Summary • Localists – Wrong, in that they tried to map behaviors and perceptions into single locations in the cortex – Any behavior is produced by many areas – Complex functions • Globalists – A function can be achieved in numerous different ways (in this sense the globalists were right) • But – simple processes that are recruited to exercise an ability are localized
    23. 23. Event-Related Brain Potentials
    24. 24. What are event-related brain potentials (ERPs) • Like EEG, but related to an event (a task, e.g. making a decision, reading a word, etc.) • The ERP (a few µV is small in relation to the EEG (about 50µV) • The international 10-20 system (Jasper 1958) allows for between-laboratory and between-experiment comparisons
    25. 25. EEG profiles obtained during various states of consciousness After Penfield and Jasper (1954)
    26. 26. How ERPs are obtained from EEG-data
    27. 27. Schematic representation of ERP-Procedure
    28. 28. ERP-components
    29. 29. ERP-Components • Usually labeled by polarity and latency, P300, P3 (ordinal latency of the component) • Scalp locations, e.g. frontal P300 • Psychological or experimental conditions – Novelty P3 – Readiness potential – Mismatch negativity, MMN • Sensory or exogeneous • Interaction subject – response (task requirements) endogenous
    30. 30. From the brain to the scalp • Distant manifestations of activations of populations of neurons (recorded on surface of skull) • Requirements – Neurons must act synchronously – Electric fields must be oriented so that they cumulate – Therefore only a subset of neural activity is visible – Open field organizations (dentritic trees are ordered), neurons are organized in layers, most of cortex, parts of thalamus, cerebellum and others • Presynaptic potentials (spikes) high frequency • Postsynaptic potentials (slower), summation thereof
    31. 31. Forward and backward solutions – inverse dipole modeling
    32. 32. Conclusions from ERPs • Just a sample of neuronal activity – If you find the same effect in different experimental conditions – If you do not find an effect
    33. 33. From the scalp to the brain: Inferring the sources of ERPs • Observations are: voltages differences between scalp electrodes and a reference electrode. • Identify neural generators of ERPs: – indefinite number of unknown parameters – No unique solution – Head is not a homogeneous medium – Difficult to compute • Non-invasive and invasive techniques – Dense electrode arrays and source monitoring – Neurophysiological knowledge, other imaging techniques • Invasive techniques – Implanting electrodes, lesion studies with animals
    34. 34. The concept of components • Voltage x time x location function. • Segments of the ERP waveform to covary in response to a specific experimental condition – Positive, negative – Aspects of the ERP waveform – In terms of neural structures that generate them • But a peak may be the sum of several functionally and structurally distinct components
    35. 35. Quantification of ERP components • Artifacts: eyeballs, eyelids, muscles of the head. • Signal to noise ratio – ERPs are constant over trials – Noise is random – ERPs are independent of the background noise • Peak measurements • Covariation measures (e.g. covary with condition) • Source-activity measures (algorithms for dipole, Loreta, Baillet & Garnero, 1997) spatio-temporal dipole model; distributed source models
    36. 36. Problems in Component Measurements • Are components identical. • Revisions of component classification • Components overlap • Principle component analysis (application statistics, linear algebra) • Subtraction procedure, only amplitude, not latency vary across conditions
    37. 37. Experimental logic • Discovery: establishing functional significance • Components antecedents (I.e. Experimental manipulations) • Consequences of variation • Speculations about the psychological or neuropsychological function it manifests
    38. 38. Psycho physiological inference • Conditions are different • Conditions differ at a particular time • Conditions differ with respect to the latency of some process • Conditions differ with respect to the degree to which some process occurs
    39. 39. Some ERP-findings • Movement-related potentials – Lateralized readiness potential (LRP) – Contingent negative variation (CNV) – Error-related negativity (ERN) • Sensory components – The early negatives (ERPs and locus of selective attention) – The middle latency cognitive components (mismatch negativity of MMN) – N200s (or N2) • The late cognitive ERPs: – P300, elicited by deviant stimuli – The “frontal” P3, elicited by novel stimuli, novelty P3 (no memory template is available
    40. 40. Some ERP-findings (continued) • ERP effects associated with subsequent memory – Distinctive word (van Restoff, character change, large P300s; recalled ones show larger P300s as compared to not recalled ones. – Same-different task (Sanquist et al., 1980) larger amplitude P300s were better recognized in subsequent recognition test. – Two-process model of recognition (large P300 when explicit recollection as opposed to “just know” • N 400 (language-related) – More prolonged over the right rather than the left hemisphere – N400 may be generated by the parahippocampal anterior fusiform gyrus – A distinctively semantic process – Inversely related to the subject’s expectancy (cloze probability) – Semantically related to sentence completion produce smaller N 400. “The pizza was too hot to drink / cry”.
    41. 41. Brain Imaging
    42. 42. Brain Imaging • Magnetic resonance imaging (MRI) • Angelo Mosso (1846-1910): correlation blood flow – neuronal activity • Seymour Kety (1915-2000), Lou Sokoloff quantified relation (middle of the 20-th century) • Position Emission tomography (PET) • 1980s: Michael Posner, Steve Peterson: Study human cognition by PET; Marcus Raichle • Donders (1868) method of subtraction
    43. 43. Subtraction Method
    44. 44. Brain imaging (fMRI) • Behavior of hydrogen atoms or protons in a magnetic field • Paul Lauterbur: MRI (Nobel-prize, 2003) • Seiji Ogawa: functional states of the brain (fMRI); – Amount of oxygen carried by hemoglobin chances the degree to which hemoglobin disturbs a magnetic field – Tracking blood flow – BOLD-signal: blood oxygen level dependent
    45. 45. Steps in analysis: cortex-segmentation
    46. 46. Computational Modeling and fMRI
    47. 47. A prominent cognitive architecture ACT-R (Anderson & Lebiere, 1998)
    48. 48. How to map time predictions to the BOLD-signal
    49. 49. Experimental Predictions by the ACT-R model
    50. 50. Comparison of model predictions and observations by a measure of proportionality
    51. 51. Summary • Embodied memory representations • New methods of cognitive science • ERP • fMRI • Modeling