Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

PowerPoint Presentation






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Adding a new descriptor is not a matter of adding a new proposition to the list: each previously inserted descriptor constrains and modifies the integration of following descriptors.
  • Embodied Meaning instead of meaningless symbols, representations could have a structure that is lawfully related to the objects being represented. Different approach: Possibilities of analogical representation underlying language comprehension. Embodied approach to Meaning Perceptual systems facilitate and enable our interaction with the real world To do this world is conceptualized as patterns of possible bodily interactions The meaning of a particular object, sentence is what the person can do with that object event or sentence Embodied Meaning / Embodied Representations: embodied meaning captures spatial-functional properties of the object or situation. EMBODIED: because the world is perceived in terms of its potential for interaction with an individual’s body.The perception of projectable properties is in terms of patterns of possible action: how we can grasp, move, leap over a certain object. This action based coding depends on the capabilities of our bodies Symbol Grounding Problem and Embodied Representations How does this answer objections opposed to traditional approaches? Embodied representations do not need to be mapped onto the world to become meaningful because they arise from the world. Embodied representations are directly grounded by virtue of being lawfully and analogically related to properties of the world and how those properties are transduced py perceptual action systems. (Harnad 1990; 1993) Meaningful action oriented component of conceptualization is not abstract and amodal. It reflects how our bodies can interact with objects. Embodied representations do not need to be mapped on the world to be grounded. Terms definition: Projectable refers to properties of the environment that can be specified by Information available Nonprojectable properties that must be provided by other sources e.g. spatial layout of a place is a projectable property whereas “ownership” is a nonprojectable property that comes from experience. 2)3) Notions of mesh: patterns of actions underlying one idea can be integrated with patterns of action underlying another. Patterns mutually modify and constrain each other because the conjoint actions must be possible given the constraints our body and the environment impose upon us. Mutual modification of patterns of action underlies construction of meaning from words whose senses are jointly modified by the context in which they occur. When patterns mash or combine they modify each other because the must be combined in a way that satisfies the constraints imposed upon bodily interaction. Example of cognitive meshing: Basalou: Imagine a ball, imagine it has stripes, imagine it is defloated.pg.6 right side
  • Scanner data, white matter, blow up to see structures, colour regions

PowerPoint Presentation PowerPoint Presentation Presentation Transcript

  • Situation Models and Embodied Language Processes
    • Franz Schmalhofer
      • University of Osnabrück / Germany
    • Memory and Situation Models
    • Computational Modeling of Inferences
    • What Memory and Language are for
    • Neural Correlates
    • Integration of Behavioral Experiments and Neural Correlates (ERP; fMRI) by Formal Models
  • Cognition and Knowledge
    • Traditional:
      • Cognition = Computation
      • Representation by propositions
      • Propositions are abstract relations
    • Embodiment of Meaning
      • Cognition is serving perception and actions
      • Representation = Patterns of possible bodily interactions with the world (lawfully related to the world)
      • What an object, event, sentence means for you, is what you can do with the object, event, sentence.
  • Embodiment of Memory (Glenberg, 1997)
    • Projectable properties: information available through the senses
    • Non-projectable properties: information available through other sources (e.g. memory)
    • Conceptualization: Combination (mesh) of projectable and non-projectable properties
    •  Primary function of memory is to mesh the embodied conceptualizations of projectable properties of the environment with embodied experiences that provide non-projectable properties
  • Embodiment of Memory
    • Evidence for embodiment...
    • ...and affect:
      • Forced frowning or smiling influences affective judgments (Berkowitz & Trocolli 1990, Berkowitz et. al 1993)
    • ...and imagery:
      • Actually rotating facilitates orientation opposed to imagining rotating(Montello & Presson 1993, Rieser etl al. 1994).
    • ...and memory:
      • Retrieval of memorized spatial layouts depends on position on body axis (Bryant 1992).
  • Embodied Memory
    • Combination of patterns of possible action
    • = meshing.
    • Meshing of patterns of action derived from:
      • projectable properties of the environment
      • non-projectable properties
    • as (spatial-functional) constraint satisfaction.
  • Function of Memory
    • Meshing projectable & non-projectable properties:
    • Meshing is important for:
      • perception
      • imagination
      • Comprehension
    • Projectable properties as well as non-projectable properties can be meshed with each-other.
  • Interaction of perceptions and memory
    • Environment has to be primary
    • Clamping projectable properties keeps the system reality-oriented
    • Experiences stay individuated
  • Updating memory
    • Experiences are shifts between conceptualizations
    • Trajectories from one pattern of action to another
    • Trajectories more often used become reinforced
  • Prediction and Planning
    • Hypothetical conceptualizations by using trajectories from memory
    • But:
      • Simulating action does not change the environment
      • Clamped projectable properties provide wrong constraints for prediction
      • Environment has to be suppressed (effortfull process)
    • Suppression loosens tie to reality
  • Two modes of memory (like the distinction between implicit/explicit)
    • Automatic
      • Meshing of projectable and nonprojectable properties
      • Causes conceptual priming: based on non-projectable properties, (therefore semantic)
    • Effortful
      • Suppression of projectable properties, conceptualization by trajectories from memory
  • Cognitive Meshing
    • Imagine a ball
    • Now Imagine that it has yellow and white stripes
    • Now Imagine that it is deflated
    • Mutual modification of mashed pattern: not only the ball but also the stripes become deformed when the ball is deflated.
    • Patterns of actions to the same spatio-functional constraints.
  • What memory and language are for
    • The primary function of memory “is to mesh
    • the embodied conceptualizations of projectable
    • properties of the environment with embodied
    • experiences that provide nonprojectable
    • properties…This meshed conceptualization,
    • the meaning, is in the service of control of
    • action in a three-dimensional environment“
    • (Glenberg 1997)
    • “ Language is a surrogate for experience” (Taylor and Tversky, 1992):
  • Summary: Embodied Representations & Symbol Grounding
    • Embodied Meaning – action based coding of objects and situations
    • Embodied representations are lawfully and analogically related to properties of the world (Harnard, 1990, 1993)
    • Notion of “mesh”: mutual modification of patterns of action (Glenberg, 1997)
    • Meaning of a situation is a meshed pattern of possible actions = embodied conceptualization.
  • Overview of the methods of cognitive neuropsychology/science (I)
    • Advances in science by technology:
      • Invention of the telescope in 1608 changed astronomers‘ observational methods
      • If well-formulated questions are not asked, even the most powerful tools will not provide sensible answers
    • Cognitive Psychology / Computer Modeling
    • Neuroanatomy
      • Gross Neuroanatomy (general structures and connections)
      • Fine Neuroanatomy (components of individual neurons)
  • Overview of the methods of cognitive neuropsychology/science (II)
    • Neurophysiology (experimental methods used with animals)
      • Electrical stimulation
      • Single-cell recording
      • Lesions
      • Genetic manipulations
    • Neurology
      • Structural imaging and neurological damage
      • Causes of neurological disorders (vascular disorders,tumors, degenerous and infectious disorders, traumata, epilepsies)
      • Functional neuro-surgery
  • Overview of the methods of cognitive neuropsychology/science (III)
    • Converging methods
      • Cognitive deficits following brain damage
      • Virtual lesions: Transcranial magnetic stimulation (TMS)
      • Functional imaging
        • Electrical and magnetic signals in the brain (EEG, MEG)
        • Metabolic signals
          • Positron emission tomography (PET): regional cerebral blood flow;
          • fMRI: blood oxygenation level dependent effect or BOLD effect
  • Brain functions (1810-1819)
    • Do parts of the brain working independently enable the mind? ( componential hypothesis )
    • Franz Joseph Gall and J. G. Spurzheim
      • 35 specific brain functions
      • Language, color perception, hope, self-esteem
      • With practice, areas grow, causing a bump in the overlying skull
      • Anatomical personology
      • phrenology
    • Does the whole brain work in concert? ( wholistic hypothesis )
    • Pierre Flourens (1794-1867)
      • All sensations, all perceptions and all volitions occupy the same seat in these cerebral organs.
      • The faculty of sensation, percept and volition is then esssentially one faculty.
      • Empirical evidence: no matter where he leasoned a bird brain, the bird recovered
  • Language Areas
    • Broca
    • Wernicke
  • Neuroanatomist Korbinian Brodman (1909)
    • Analyzed cellular organization of the cortex
    • Tissue stains to visualize different brain regions
    • To a large extent cytoarchitecturally described brain areas do indeed represent functionally distinct brain regions
  • Componential or wholistic? Again the question
    • Camillo Golgi
      • Developed stain that impregnated individual neurons
      • Believed the whole brain to be a continuous mass of tissue that shares a common cytoplasm
    • Cajal
      • Used Golgi stains
      • Identified the unitary nature of neurons
      • Transmittion of information by electricity
  • How does the nervous system work (20-th century)
    • Understand how single neurons behave and interact
    • Knowing all the elements, can we figure out the system?
    • Billions of neurons
    • Brain-damaged humans show lack of typical symptoms
    • Impossible to localize „higher cognitive functions“
    Jackson: Lesion might well affect other structures in the brain because the lesion might have damaged neurons connected to other regions; diaschisis: damage of one part can create problems for another. Gestaltist view: The whole is different from the sum of its parts
  • Summary
    • Localists
      • Wrong, in that they tried to map behaviors and perceptions into single locations in the cortex
      • Any behavior is produced by many areas
      • Complex functions
    • Globalists
      • A function can be achieved in numerous different ways (in this sense the globalists were right)
    • But
      • simple processes that are recruited to exercise an ability are localized
  • Event-Related Brain Potentials
  • What are event-related brain potentials (ERPs)
    • Like EEG, but related to an event (a task, e.g. making a decision, reading a word, etc.)
    • The ERP (a few µV is small in relation to the EEG (about 50µV)
    • The international 10-20 system (Jasper 1958) allows for between-laboratory and between-experiment comparisons
  • EEG profiles obtained during various states of consciousness After Penfield and Jasper (1954)
  • How ERPs are obtained from EEG-data
  • Schematic representation of ERP-Procedure
  • ERP-components
  • ERP-Components
    • Usually labeled by polarity and latency, P300, P3 (ordinal latency of the component)
    • Scalp locations, e.g. frontal P300
    • Psychological or experimental conditions
      • Novelty P3
      • Readiness potential
      • Mismatch negativity, MMN
    • Sensory or exogeneous
    • Interaction subject – response (task requirements) endogenous
  • From the brain to the scalp
    • Distant manifestations of activations of populations of neurons (recorded on surface of skull)
    • Requirements
      • Neurons must act synchronously
      • Electric fields must be oriented so that they cumulate
      • Therefore only a subset of neural activity is visible
      • Open field organizations (dentritic trees are ordered), neurons are organized in layers, most of cortex, parts of thalamus, cerebellum and others
    • Presynaptic potentials (spikes) high frequency
    • Postsynaptic potentials (slower), summation thereof
  • Forward and backward solutions – inverse dipole modeling
  • Conclusions from ERPs
    • Just a sample of neuronal activity
      • If you find the same effect in different experimental conditions
      • If you do not find an effect
  • From the scalp to the brain: Inferring the sources of ERPs
    • Observations are: voltages differences between scalp electrodes and a reference electrode.
    • Identify neural generators of ERPs:
      • indefinite number of unknown parameters
      • No unique solution
      • Head is not a homogeneous medium
      • Difficult to compute
    • Non-invasive and invasive techniques
      • Dense electrode arrays and source monitoring
      • Neurophysiological knowledge, other imaging techniques
    • Invasive techniques
      • Implanting electrodes, lesion studies with animals
  • The concept of components
    • Voltage x time x location function.
    • Segments of the ERP waveform to covary in response to a specific experimental condition
      • Positive, negative
      • Aspects of the ERP waveform
      • In terms of neural structures that generate them
    • But a peak may be the sum of several functionally and structurally distinct components
  • Quantification of ERP components
    • Artifacts: eyeballs, eyelids, muscles of the head.
    • Signal to noise ratio
      • ERPs are constant over trials
      • Noise is random
      • ERPs are independent of the background noise
    • Peak measurements
    • Covariation measures (e.g. covary with condition)
    • Source-activity measures (algorithms for dipole, Loreta, Baillet & Garnero, 1997) spatio-temporal dipole model; distributed source models
  • Problems in Component Measurements
    • Are components identical.
    • Revisions of component classification
    • Components overlap
    • Principle component analysis (application statistics, linear algebra)
    • Subtraction procedure, only amplitude, not latency vary across conditions
  • Experimental logic
    • Discovery: establishing functional significance
    • Components antecedents (I.e. Experimental manipulations)
    • Consequences of variation
    • Speculations about the psychological or neuropsychological function it manifests
  • Psycho physiological inference
    • Conditions are different
    • Conditions differ at a particular time
    • Conditions differ with respect to the latency of some process
    • Conditions differ with respect to the degree to which some process occurs
  • Some ERP-findings
    • Movement-related potentials
      • Lateralized readiness potential (LRP)
      • Contingent negative variation (CNV)
      • Error-related negativity (ERN)
    • Sensory components
      • The early negatives (ERPs and locus of selective attention)
      • The middle latency cognitive components (mismatch negativity of MMN)
      • N200s (or N2)
    • The late cognitive ERPs:
      • P300, elicited by deviant stimuli
      • The “frontal” P3, elicited by novel stimuli, novelty P3 (no memory template is available
  • Some ERP-findings (continued)
    • ERP effects associated with subsequent memory
      • Distinctive word (van Restoff, character change, large P300s; recalled ones show larger P300s as compared to not recalled ones.
      • Same-different task (Sanquist et al., 1980) larger amplitude P300s were better recognized in subsequent recognition test.
      • Two-process model of recognition (large P300 when explicit recollection as opposed to “just know”
    • N 400 (language-related)
      • More prolonged over the right rather than the left hemisphere
      • N400 may be generated by the parahippocampal anterior fusiform gyrus
      • A distinctively semantic process
      • Inversely related to the subject’s expectancy (cloze probability)
      • Semantically related to sentence completion produce smaller N 400. “The pizza was too hot to drink / cry”.
  • Brain Imaging
  • Brain Imaging
    • Magnetic resonance imaging (MRI)
    • Angelo Mosso (1846-1910): correlation blood flow – neuronal activity
    • Seymour Kety (1915-2000), Lou Sokoloff quantified relation (middle of the 20-th century)
    • Position Emission tomography (PET)
    • 1980s: Michael Posner, Steve Peterson: Study human cognition by PET; Marcus Raichle
    • Donders (1868) method of subtraction
  • Subtraction Method
  • Brain imaging (fMRI)
    • Behavior of hydrogen atoms or protons in a magnetic field
    • Paul Lauterbur: MRI (Nobel-prize, 2003)
    • Seiji Ogawa: functional states of the brain (fMRI);
      • Amount of oxygen carried by hemoglobin chances the degree to which hemoglobin disturbs a magnetic field
      • Tracking blood flow
      • BOLD-signal: blood oxygen level dependent
  • Steps in analysis: cortex-segmentation
  • Computational Modeling and fMRI
  • A prominent cognitive architecture ACT-R (Anderson & Lebiere, 1998)
  • How to map time predictions to the BOLD-signal
  • Experimental Predictions by the ACT-R model
  • Comparison of model predictions and observations by a measure of proportionality
  • Summary
    • Embodied memory representations
    • New methods of cognitive science
    • ERP
    • fMRI
    • Modeling