the human
perception and cognition
computational analogy

  input           processing          output




perception        cognition           action
             (thinking and memory) (motor system)
a cycle of interaction

             cognition



perception               action


             the world
perception
‘the’ five senses

                      sight
                              distant
                     sound
   complex           smell    closer
pain, temperature,
 pressure, texture
                     touch    contact

     largely
                      taste   inside!
      smell!
‘the’ five senses

           sight
          sound
not the   whole
          smell     story
          touch
           taste
outside and within

     exteroception        interoception


sight, sound,                 headaches,
smell, touch,                   hunger,
    taste                    proprioception

  sensing the                sensing the
    outside                     inside
outside and within

      exteroception                    interoception


sight,
sight, sound,                              headaches,
smell, most used sense in interfaces
        touch,                               hunger,
    taste (e.g. retina displays!)         proprioception

   sensing the half the brain
         around
                                          sensing the
          dedicated to vision
     outside                                 inside
sight and sound



   in short
the eye(s) and vision
on the front
  iris & pupil (lens) – adjust light level and focus
  two of them – 3D vision – but not the only way
at the back
  fovea – detailed and colour ... but very small,
  peripheral vision – b&w, movement, grabs attention
inside
  two pathways – fast, old, unconscious
                                  + slower, newer,
    conscious
the ears and sound
inherently temporal
  no ‘still’ sound => issues for media
two of them & pinna
  directional ... stereo & full 3D
little physical focus, but ...
  attention – cocktail part effect
  background – context and grab attention
eye & ear ... common

attention    – focus limited
                               construction
                                              filling ga
                                              meaning
            grabbing attention


periphery – context, broader, but unconscious
cognition
memory and thinking
memory
 in short
types of memory
        timescale
  seconds       sensory memory
                short-term memory (STM)
  minutes
               mezzanine memory
hours/days


  lifetime
               long-term memory (LTM)
comparison

              short term        long term mezzanine
 capacity     7+/-2 chunks         big!            ?

  decay           ~30s           lifetime ?   mins–hours
              needs rehearsal

laying down      implicit       repetition      implicit

  neural   electrical      physical           chemical?
mechanism neuron firing synapse growth           LTP
types of LTM
episodic
   serial memory of events
               may be hard to recall out of order
   => story and narrative
semantic / declarative
   facts, concepts, skills
               links and associations
   => recognition vs recall

                         http://en.wikipedia.org/wiki/File:The_Tyger_BM_a_1794.j
semantic not photographic

  perception

         encode            reconstruct

                                   memories

all memories are constructed
                            confabulation,
  false memories
mezzanine memory
      what have I been doing recently?
                                 short-term episodic memory

      what am I doing now?
                                 context – activates LTM
                   cockpits and control rooms
                          situation awareness
                   reading
                          long-term working memory

http://en.wikipedia.org/wiki/File:Swiss_Saab_2000_Cockpit.jpg
implicit / tacit as well as explicit
automatic reactions
motor memories
expert knowledge
                proceduralisation


               activation & context
    implicit                                   explicit
                 introspection
                & externalisation

                          http://www.flickr.com/photos/protographer23/358987856/
thinking
logical?
Aristotle & formal logic   all men are mortal
                           Socrates is a man
                           => Socrates is mortal
Wason card test
 possible rule: vowel one side ⇒even on other
 is it right?


            E      7       K     2
            3
            4      U
                   P             A
                                 Z
reasoning – the real world

confirmationnot falsification

abductionnot deduction

satisficingnot optimality
special purpose intelligence

Wason
   rule: vowel one side ⇒even on other

            E       7        K      2
  social
social
      rule: take newspaper ⇒pay for it
   rule: take newspaper ⇒pay for it
                             no
           paper   no   £   paper   £
below the surface
              linear – single ‘stream of
 conscious    consciousness’
              interruptions = disruptions
              multitasking – really rapid switching


              priming, insight


unconscious     non-linear, fuzzy, intuitive, associative
                share with animals
into action
      and
into the world
we live to act

             cognition



perception               action


             the world
low level
automatic reactions
  innate and learnt
           e.g. heat reflex


hand–eye coordination
  Fitts’ Law:                                s
            time ~ log (distance/size)
                                         d
perception and action

             single stream of consciousness
attention    single stream of activity


            grabbing attention
                     often physical movement

periphery   feeds unconsciousness
            associations
higher level
longer–term conscious planning and activity

moment-to-moment
 Norman’s execution evaluation cycle
   user establishes the goal
   formulates intention
   specifies actions at interface                goal
   executes action
                                execution               evaluation
   perceives system state
   interprets system state                     system
   evaluates system state with respect to goal
Human error - slips and
             mistakes
slip
       understand system and goal
       correct formulation of action
       incorrect action

mistake
       may not even have right goal!

Fixing things?
  slip – better interface design
  mistake – better understanding of system
all in the mind?

reminders, lists,
     notes, workings

we think using the world
  distributed cognition
  situated action
  extended mind

The Human perception & Overview

  • 1.
  • 2.
    computational analogy input processing output perception cognition action (thinking and memory) (motor system)
  • 3.
    a cycle ofinteraction cognition perception action the world
  • 4.
  • 5.
    ‘the’ five senses sight distant sound complex smell closer pain, temperature, pressure, texture touch contact largely taste inside! smell!
  • 6.
    ‘the’ five senses sight sound not the whole smell story touch taste
  • 7.
    outside and within exteroception interoception sight, sound, headaches, smell, touch, hunger, taste proprioception sensing the sensing the outside inside
  • 8.
    outside and within exteroception interoception sight, sight, sound, headaches, smell, most used sense in interfaces touch, hunger, taste (e.g. retina displays!) proprioception sensing the half the brain around sensing the dedicated to vision outside inside
  • 9.
  • 10.
    the eye(s) andvision on the front iris & pupil (lens) – adjust light level and focus two of them – 3D vision – but not the only way at the back fovea – detailed and colour ... but very small, peripheral vision – b&w, movement, grabs attention inside two pathways – fast, old, unconscious + slower, newer, conscious
  • 11.
    the ears andsound inherently temporal no ‘still’ sound => issues for media two of them & pinna directional ... stereo & full 3D little physical focus, but ... attention – cocktail part effect background – context and grab attention
  • 12.
    eye & ear... common attention – focus limited construction filling ga meaning grabbing attention periphery – context, broader, but unconscious
  • 13.
  • 14.
  • 15.
    types of memory timescale seconds sensory memory short-term memory (STM) minutes mezzanine memory hours/days lifetime long-term memory (LTM)
  • 16.
    comparison short term long term mezzanine capacity 7+/-2 chunks big! ? decay ~30s lifetime ? mins–hours needs rehearsal laying down implicit repetition implicit neural electrical physical chemical? mechanism neuron firing synapse growth LTP
  • 17.
    types of LTM episodic serial memory of events may be hard to recall out of order => story and narrative semantic / declarative facts, concepts, skills links and associations => recognition vs recall http://en.wikipedia.org/wiki/File:The_Tyger_BM_a_1794.j
  • 18.
    semantic not photographic perception encode reconstruct memories all memories are constructed  confabulation, false memories
  • 19.
    mezzanine memory what have I been doing recently? short-term episodic memory what am I doing now? context – activates LTM cockpits and control rooms situation awareness reading long-term working memory http://en.wikipedia.org/wiki/File:Swiss_Saab_2000_Cockpit.jpg
  • 20.
    implicit / tacitas well as explicit automatic reactions motor memories expert knowledge proceduralisation activation & context implicit explicit introspection & externalisation http://www.flickr.com/photos/protographer23/358987856/
  • 21.
  • 22.
    logical? Aristotle & formallogic all men are mortal Socrates is a man => Socrates is mortal Wason card test possible rule: vowel one side ⇒even on other is it right? E 7 K 2 3 4 U P A Z
  • 23.
    reasoning – thereal world confirmationnot falsification abductionnot deduction satisficingnot optimality
  • 24.
    special purpose intelligence Wason rule: vowel one side ⇒even on other E 7 K 2 social social rule: take newspaper ⇒pay for it rule: take newspaper ⇒pay for it no paper no £ paper £
  • 25.
    below the surface linear – single ‘stream of conscious consciousness’ interruptions = disruptions multitasking – really rapid switching priming, insight unconscious non-linear, fuzzy, intuitive, associative share with animals
  • 26.
    into action and into the world
  • 27.
    we live toact cognition perception action the world
  • 28.
    low level automatic reactions innate and learnt e.g. heat reflex hand–eye coordination Fitts’ Law: s time ~ log (distance/size) d
  • 29.
    perception and action single stream of consciousness attention single stream of activity grabbing attention often physical movement periphery feeds unconsciousness associations
  • 30.
    higher level longer–term consciousplanning and activity moment-to-moment Norman’s execution evaluation cycle user establishes the goal formulates intention specifies actions at interface goal executes action execution evaluation perceives system state interprets system state system evaluates system state with respect to goal
  • 31.
    Human error -slips and mistakes slip understand system and goal correct formulation of action incorrect action mistake may not even have right goal! Fixing things? slip – better interface design mistake – better understanding of system
  • 32.
    all in themind? reminders, lists, notes, workings we think using the world distributed cognition situated action extended mind

Editor's Notes

  • #3 brain image: CC. Patrick J. Lynch, http://en.wikipedia.org/w/index.php?title=File:Skull_and_brain_normal_human.svg&page=1
  • #29 http://en.wikipedia.org/wiki/File:Descartes-reflex.JPG