Perception and
cognition
in design
PC3001/PC4001/PC5001/CS2847 User Centred Design - Lecture 2
Szonya Durant
Drop-in sessions: (in person + online) Tuesday 3-4pm (online) Fri 2.30-3.30 pm
Remember to log your attendance! 1
Aims
To discuss which psychological considerations may be important and and why in designing
displays and interfaces.
To consider how different aspects of Psychology play a role and the interplay between them in this
applied context.
To be able to apply theory in a practical context of display design.
2
Outline Low level perception – what visual
characteristics effect how well we
can perceive the contents of a
display?
Attention – how can displays
help/hinder our ability to be aware
of relevant information?
Memory – how challenging are the
requirements of the interface?
3
Wickens et al.
Pp. 4
Visual Acuity
Acuity
Visual resolution
The smallest object you can see
Vernier acuity
Foveal vs peripheral location
4
http://www.michaelbach.de/ot/lum-hyperacuity/index.html
Landolt C
Snellen chart
5
fovea more peripheral
fovea periphery
eccentricity (mm)
cones/mm2
(x 1000)
Visual Acuity
Ageing
6
Age range 43-54 55-64 65-74 75+
Num. of
letters read 57 55.1 52 44.8
best-corrected visual acuity
Klein et al. 1995
Brightness and
contrast
Brightness threshold
More important is contrast – relative brightness.
Visual impairment – raises the contrast threshold as well as
reducing visual acuity E.g. Ross, Bron & Clarke (1984).
Some evidence of higher sensitivity in dyslexics (Bednerak & Grabowska, 2002).
7
High Low
Colour
Colour discrimination
We can distinguish hue, saturation, luminance.
Colour blindness
The most common form occurs in men and leads to difficulty in
seeing differences in red and green.
8% of men and 0.5% of women have the common form of red-green
colour blindness.
CVD – colour vision deficiency.
8
2 7 8 9 1 2
https://nei.nih.gov/health/color_blindness/facts_about
https://www.color-blindness.com/ishihara-38-plates-cvd-test/
Colour
Colour contrast
The brain detects differences between
these channels - What we see depends on context
(Johnson, 2020)
Colour similarity
The colour wheel captures similarity,
you would want to use colours far from each
other to distinguish different objects
(Zanker, 2010, colour plate 3, Fig.3)
10
Movement Motion threshold – Humans are very sensitive to
motion, very small displacements can be detected.
Flicker and motion onset/offset can be used to attract
attention.
Apparent motion - Dmin is defined as the minimum
distance over which spatial change is seen as motion
and Dmax is the maximum spatial gap over which
motion is seen (Baker & Braddick, 1985). Dmin can be
as low as 0.01deg, Dmax can be many degrees for
moving objects.
11
Attention in
displays
Attention is a limited resource
Attention can be thought of as a ‘bottle-neck’
of processing.
We do not take in all that is presented in the
scene at any given time.
The visual attributes above need to be
considered along with what the other objects
in the display are and what the task is.
Visual attention is usually closely linked with
where the eyes are looking – can use eye
tracking as measure.
12
Selective
Visual
Attention
Useful task types to think about:
1. General orientation and scene
scanning
2. Supervisory control
3. Noticing
4. Searching
5. Reading
6. Confirming
13
Wickens (2013) pg 50
Salience Refers to how much an object stands out
against a background.
The most famous model of salience is that
of Itti & Koch – given an image it tries to
predict which areas will draw the eyes (Itti &
Koch, 2001).
Displays increase salience when trying to
attract attention – studies have looked are
real world effectiveness such as flashing
light on an altometer (Wickens (2013) pg
51).
14
Shen & Zhao (2014)
Effort – the
cost of shifting
attention
Spatial proximity
Cost of moving eyes
Cost of moving head
15
Wickens (2013)
page 51
Rotation
0 4 20-30 90-120
**
*
* *
*
IAE
Info
Access
effort
Spatial separation between two items (degrees of visual angle)
Foveal Eye field Head field (neck rotation)
Full body
Task relevance Can be defined positively (the value or
relevance of information for a task).
Or negatively (e.g. the cost of missing a
warning).
At any point in time we may have many
tasks, the more highly weighted ones my
override the others.
16
Expectancy
(expectation)
Attention can be guided to areas where
information is rapidly changing (e.g. curvy
road edges attract more attention than
straight Wickens (2013) page 52).
Our prior experience may guide our
attention to where we expect information to
be.
Contextual cues (see later) such as arrows
lead to expectation and thus guide
attention.
17
Noticing
Change blindness
Although the human visual systems is
usually sensitive to change, sometimes we
fail to see changes.
Inside the lab this is called change
blindness (Rensink 2002).
Can be caused by some sort of visual
interruption, such as an eye movement at
the the wrong time.
18
Inattentional blindness - even when something is very salient it can go unnoticed (Simons and
Chabris 1999).
Difficulty of primary task with degree of visual similarity between unexpected task and primary
event.
Look, but fail to see reported in driving (e.g. Crundell et al. 2012).
Needs to be considered if you want people for example to pay attention to a warning on an
interface.
19
Visual search
A useful paradigm for examining the
mechanisms of attention.
Some feature ‘pop out’, some need ‘serial
search’.
Also a real-life task when looking at a
display.
Examples of real life search?
20
L
L
L
L
T
T
L
L L
T
T
T
T L
L
L
L
L
L
L
L
L
L L
L
L
L
L L
L
Serial search –
Target
absent/present
21
Search set (N)
Time to
target
Target present
Target absent
4 8 12
XY
KF
R P J F
K M S T
L M K Z
T D X U
R U L Q
Search for K
ap
aa ST = ap + bN/2
ST = aa + bN
Sternberg 1966, Wickens (2013) page 57
Clutter
This is a term closely related to search given a
certain target describes how the rest of the
contents of the display impede visual search.
Numerosity clutter - too many other objects.
Proximity or readout clutter - too many
nearby objects, hence local and global
density clutter is also referred to.
Disorganizational clutter - lack of spatial
structure in the background objects can
interfere.
Heterogeneous clutter – many different
distractors make search difficult.
22
Wickens (2013) page 61
Clutter
23
Disorganizational clutter
Wickens (2013) page 61
24
Gestalt psychology described the principles
that makes us visually group objects together
Zanker (2010) pg 165
Pragnanz
‘simplest
explanation’
Good
continuation Closure
Proximity Similarity Common fate
Directing and
guiding
attention
Central cue
Arrows have been shown to automatically
guide attention (Tipples, 2002).
Peripheral cue
By making things become salient they can
catch attention.
An example of this is highlighting in a
display – but this has to not obscure the
area of interest.
25
Wickens (2013) page 60
Tipples, 2002
Working
memory
Temporary attention demanding store, used
to store information before we use it or
combine different bits of information.
Closely related to attention – e.g. in visual
search we are looking for a target that we
are holding in working memory.
Failures occur in a work context, for
example carrying out a list of tasks,
checking a list, remembering a procedure.
26
Working
memory
Lab measures:
digit span, word span, reading span, operation
span, counting span.
Correlate with real world human performance
in tasks such as
reading and listening comprehension,
academic performance, multi-tasking,
language comprehension, ability to follow
directions, vocabulary learning, note taking,
writing, reasoning, learning to programme,
making complex aviation decisions.
Wickens (2013) page199
27
Working memory
Limited capacity and duration
28
Retention interval (s)
Recall
accuracy
Chance
100%
1 item
3 items
5 items
7 items
0 30
Similar decay curves found for
navigational information and
information used by radar
controllers.
Magic number of 7 +/- 2
Wickens (2013) page 203
Working
memory
Interference and confusion
Having to carry out another task whilst
holding items in memory will interfere with
the ability to recall the original items
(retroactive interference) but can also
affect memory for new items (proactive
interference).
e.g, 10s delay needed before material
presented in a verbal exchange stopped
disrupting the memory for the next
exchange
29
Wickens (2013) page 206
Working memory Individual differences (e.g. age)
30
Gajewski et al. 2018
20-40 41-60 61-80
Conclusion We have seen the importance of considering the
first steps of perception and cognition in
designing visual displays and interfaces.
We have seen how psychological theory and lab
studies can be applied to user centred design.
In the workshop we will test out some of the
limitations on our own perception and cognition
in perceiving displays.
31
Recommended
reading
Wickens, C. D. (2013) Engineering Psychology and Human Performance (4th Ed) Routledge.
Chapter 3 & Chapter 7
Johnson, J. (2020). Designing with the mind in mind: simple guide to understanding user interface
design guidelines. Morgan Kaufmann. Chapter 1-3, 5-7
Zanker, J. (2010). Sensation, perception and action: an evolutionary perspective. Macmillan
International Higher Education. Chapter 4
32
33
Anstis, S. M. (1974). A chart demonstrating variations in acuity with retinal position. Vision research, 14(7), 589.
Bednarek, D. B., & Grabowska, A. (2002). Luminance and chromatic contrast sensitivity in dyslexia: the magnocellular
deficit hypothesis revisited. Neuroreport, 13(18), 2521-2525.
Crundall, D., Crundall, E., Clarke, D., & Shahar, A. (2012). Why do car drivers fail to give way to motorcycles at t-
junctions?. Accident Analysis & Prevention, 44(1), 88-96.
Gajewski, P. D., Hanisch, E., Falkenstein, M., Thönes, S., & Wascher, E. (2018). What does the n-back task measure as
we get older? Relations between working-memory measures and other cognitive functions across the lifespan. Frontiers
in psychology, 9, 2208.
Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature reviews neuroscience, 2(3), 194.
Klein, R., Wang, Q., Klein, B. E., Moss, S. E., & Meuer, S. M. (1995). The relationship of age-related maculopathy,
cataract, and glaucoma to visual acuity. Investigative ophthalmology & visual science, 36(1), 182-191.
Full reference list
Ross, J. E., Bron, A. J., & Clarke, D. D. (1984). Contrast sensitivity and visual disability in chronic simple
glaucoma. British journal of ophthalmology, 68(11), 821-827.
Shen, C., & Zhao, Q. (2014, September). Webpage saliency. In European conference on computer vision (pp. 33-
46). Springer, Cham.
Tipples, J. (2002). Eye gaze is not unique: Automatic orienting in response to uninformative arrows.
Psychonomic bulletin & review, 9(2), 314-318.
Universale, C. O. (1984). Visual acuity measurement standard. Visual Functions Committee.
Westheimer, G., & McKee, S. P. (1977) Spatial configurations for visual hyperacuity. Vision research, 17(8), 941-
947

Perception and Cognition

  • 1.
    Perception and cognition in design PC3001/PC4001/PC5001/CS2847User Centred Design - Lecture 2 Szonya Durant Drop-in sessions: (in person + online) Tuesday 3-4pm (online) Fri 2.30-3.30 pm Remember to log your attendance! 1
  • 2.
    Aims To discuss whichpsychological considerations may be important and and why in designing displays and interfaces. To consider how different aspects of Psychology play a role and the interplay between them in this applied context. To be able to apply theory in a practical context of display design. 2
  • 3.
    Outline Low levelperception – what visual characteristics effect how well we can perceive the contents of a display? Attention – how can displays help/hinder our ability to be aware of relevant information? Memory – how challenging are the requirements of the interface? 3 Wickens et al. Pp. 4
  • 4.
    Visual Acuity Acuity Visual resolution Thesmallest object you can see Vernier acuity Foveal vs peripheral location 4 http://www.michaelbach.de/ot/lum-hyperacuity/index.html Landolt C Snellen chart
  • 5.
    5 fovea more peripheral foveaperiphery eccentricity (mm) cones/mm2 (x 1000)
  • 6.
    Visual Acuity Ageing 6 Age range43-54 55-64 65-74 75+ Num. of letters read 57 55.1 52 44.8 best-corrected visual acuity Klein et al. 1995
  • 7.
    Brightness and contrast Brightness threshold Moreimportant is contrast – relative brightness. Visual impairment – raises the contrast threshold as well as reducing visual acuity E.g. Ross, Bron & Clarke (1984). Some evidence of higher sensitivity in dyslexics (Bednerak & Grabowska, 2002). 7 High Low
  • 8.
    Colour Colour discrimination We candistinguish hue, saturation, luminance. Colour blindness The most common form occurs in men and leads to difficulty in seeing differences in red and green. 8% of men and 0.5% of women have the common form of red-green colour blindness. CVD – colour vision deficiency. 8 2 7 8 9 1 2 https://nei.nih.gov/health/color_blindness/facts_about https://www.color-blindness.com/ishihara-38-plates-cvd-test/
  • 10.
    Colour Colour contrast The braindetects differences between these channels - What we see depends on context (Johnson, 2020) Colour similarity The colour wheel captures similarity, you would want to use colours far from each other to distinguish different objects (Zanker, 2010, colour plate 3, Fig.3) 10
  • 11.
    Movement Motion threshold– Humans are very sensitive to motion, very small displacements can be detected. Flicker and motion onset/offset can be used to attract attention. Apparent motion - Dmin is defined as the minimum distance over which spatial change is seen as motion and Dmax is the maximum spatial gap over which motion is seen (Baker & Braddick, 1985). Dmin can be as low as 0.01deg, Dmax can be many degrees for moving objects. 11
  • 12.
    Attention in displays Attention isa limited resource Attention can be thought of as a ‘bottle-neck’ of processing. We do not take in all that is presented in the scene at any given time. The visual attributes above need to be considered along with what the other objects in the display are and what the task is. Visual attention is usually closely linked with where the eyes are looking – can use eye tracking as measure. 12
  • 13.
    Selective Visual Attention Useful task typesto think about: 1. General orientation and scene scanning 2. Supervisory control 3. Noticing 4. Searching 5. Reading 6. Confirming 13 Wickens (2013) pg 50
  • 14.
    Salience Refers tohow much an object stands out against a background. The most famous model of salience is that of Itti & Koch – given an image it tries to predict which areas will draw the eyes (Itti & Koch, 2001). Displays increase salience when trying to attract attention – studies have looked are real world effectiveness such as flashing light on an altometer (Wickens (2013) pg 51). 14 Shen & Zhao (2014)
  • 15.
    Effort – the costof shifting attention Spatial proximity Cost of moving eyes Cost of moving head 15 Wickens (2013) page 51 Rotation 0 4 20-30 90-120 ** * * * * IAE Info Access effort Spatial separation between two items (degrees of visual angle) Foveal Eye field Head field (neck rotation) Full body
  • 16.
    Task relevance Canbe defined positively (the value or relevance of information for a task). Or negatively (e.g. the cost of missing a warning). At any point in time we may have many tasks, the more highly weighted ones my override the others. 16
  • 17.
    Expectancy (expectation) Attention can beguided to areas where information is rapidly changing (e.g. curvy road edges attract more attention than straight Wickens (2013) page 52). Our prior experience may guide our attention to where we expect information to be. Contextual cues (see later) such as arrows lead to expectation and thus guide attention. 17
  • 18.
    Noticing Change blindness Although thehuman visual systems is usually sensitive to change, sometimes we fail to see changes. Inside the lab this is called change blindness (Rensink 2002). Can be caused by some sort of visual interruption, such as an eye movement at the the wrong time. 18
  • 19.
    Inattentional blindness -even when something is very salient it can go unnoticed (Simons and Chabris 1999). Difficulty of primary task with degree of visual similarity between unexpected task and primary event. Look, but fail to see reported in driving (e.g. Crundell et al. 2012). Needs to be considered if you want people for example to pay attention to a warning on an interface. 19
  • 20.
    Visual search A usefulparadigm for examining the mechanisms of attention. Some feature ‘pop out’, some need ‘serial search’. Also a real-life task when looking at a display. Examples of real life search? 20 L L L L T T L L L T T T T L L L L L L L L L L L L L L L L L
  • 21.
    Serial search – Target absent/present 21 Searchset (N) Time to target Target present Target absent 4 8 12 XY KF R P J F K M S T L M K Z T D X U R U L Q Search for K ap aa ST = ap + bN/2 ST = aa + bN Sternberg 1966, Wickens (2013) page 57
  • 22.
    Clutter This is aterm closely related to search given a certain target describes how the rest of the contents of the display impede visual search. Numerosity clutter - too many other objects. Proximity or readout clutter - too many nearby objects, hence local and global density clutter is also referred to. Disorganizational clutter - lack of spatial structure in the background objects can interfere. Heterogeneous clutter – many different distractors make search difficult. 22 Wickens (2013) page 61
  • 23.
  • 24.
    24 Gestalt psychology describedthe principles that makes us visually group objects together Zanker (2010) pg 165 Pragnanz ‘simplest explanation’ Good continuation Closure Proximity Similarity Common fate
  • 25.
    Directing and guiding attention Central cue Arrowshave been shown to automatically guide attention (Tipples, 2002). Peripheral cue By making things become salient they can catch attention. An example of this is highlighting in a display – but this has to not obscure the area of interest. 25 Wickens (2013) page 60 Tipples, 2002
  • 26.
    Working memory Temporary attention demandingstore, used to store information before we use it or combine different bits of information. Closely related to attention – e.g. in visual search we are looking for a target that we are holding in working memory. Failures occur in a work context, for example carrying out a list of tasks, checking a list, remembering a procedure. 26
  • 27.
    Working memory Lab measures: digit span,word span, reading span, operation span, counting span. Correlate with real world human performance in tasks such as reading and listening comprehension, academic performance, multi-tasking, language comprehension, ability to follow directions, vocabulary learning, note taking, writing, reasoning, learning to programme, making complex aviation decisions. Wickens (2013) page199 27
  • 28.
    Working memory Limited capacityand duration 28 Retention interval (s) Recall accuracy Chance 100% 1 item 3 items 5 items 7 items 0 30 Similar decay curves found for navigational information and information used by radar controllers. Magic number of 7 +/- 2 Wickens (2013) page 203
  • 29.
    Working memory Interference and confusion Havingto carry out another task whilst holding items in memory will interfere with the ability to recall the original items (retroactive interference) but can also affect memory for new items (proactive interference). e.g, 10s delay needed before material presented in a verbal exchange stopped disrupting the memory for the next exchange 29 Wickens (2013) page 206
  • 30.
    Working memory Individualdifferences (e.g. age) 30 Gajewski et al. 2018 20-40 41-60 61-80
  • 31.
    Conclusion We haveseen the importance of considering the first steps of perception and cognition in designing visual displays and interfaces. We have seen how psychological theory and lab studies can be applied to user centred design. In the workshop we will test out some of the limitations on our own perception and cognition in perceiving displays. 31
  • 32.
    Recommended reading Wickens, C. D.(2013) Engineering Psychology and Human Performance (4th Ed) Routledge. Chapter 3 & Chapter 7 Johnson, J. (2020). Designing with the mind in mind: simple guide to understanding user interface design guidelines. Morgan Kaufmann. Chapter 1-3, 5-7 Zanker, J. (2010). Sensation, perception and action: an evolutionary perspective. Macmillan International Higher Education. Chapter 4 32
  • 33.
    33 Anstis, S. M.(1974). A chart demonstrating variations in acuity with retinal position. Vision research, 14(7), 589. Bednarek, D. B., & Grabowska, A. (2002). Luminance and chromatic contrast sensitivity in dyslexia: the magnocellular deficit hypothesis revisited. Neuroreport, 13(18), 2521-2525. Crundall, D., Crundall, E., Clarke, D., & Shahar, A. (2012). Why do car drivers fail to give way to motorcycles at t- junctions?. Accident Analysis & Prevention, 44(1), 88-96. Gajewski, P. D., Hanisch, E., Falkenstein, M., Thönes, S., & Wascher, E. (2018). What does the n-back task measure as we get older? Relations between working-memory measures and other cognitive functions across the lifespan. Frontiers in psychology, 9, 2208. Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature reviews neuroscience, 2(3), 194. Klein, R., Wang, Q., Klein, B. E., Moss, S. E., & Meuer, S. M. (1995). The relationship of age-related maculopathy, cataract, and glaucoma to visual acuity. Investigative ophthalmology & visual science, 36(1), 182-191. Full reference list Ross, J. E., Bron, A. J., & Clarke, D. D. (1984). Contrast sensitivity and visual disability in chronic simple glaucoma. British journal of ophthalmology, 68(11), 821-827. Shen, C., & Zhao, Q. (2014, September). Webpage saliency. In European conference on computer vision (pp. 33- 46). Springer, Cham. Tipples, J. (2002). Eye gaze is not unique: Automatic orienting in response to uninformative arrows. Psychonomic bulletin & review, 9(2), 314-318. Universale, C. O. (1984). Visual acuity measurement standard. Visual Functions Committee. Westheimer, G., & McKee, S. P. (1977) Spatial configurations for visual hyperacuity. Vision research, 17(8), 941- 947