This document discusses visual thinking tools and space-time cognitive models to support design. It presents two examples: (1) tools for finding patterns in whale tracking data that turn time into spatial patterns, improving efficiency over playback tools by a factor of 100; and (2) comparing zooming versus multiple windows for comparing visual scenes, finding that multiple windows are better for higher memory loads. The key is understanding perception and cognitive processes to design effective visual thinking tools.

1. Visual Thinking and Visual
Thinking Tools:
Space, Time and Simple Cognitive Models to Support Design
Colin Ware
Data Visualization Research Lab, CCOM,
University of New Hampshire

2. Architecture for visual thinking

3. Central Problem: How do we perceive
the world in all its rich detail?

4. The Nature of Visual Space

6. Capacity of visual working
memory (Vogal, Woodman, Luck,
2001)



Task – change detection
Can see 3.3 objects
Each object can be complex
1 second

7. Sequential comparison task

12. Solution



“The world is its own memory” O’Regan
Task-related active vision
“What you see is what you need”
Treish et al. (2003)

Seeing is a process that helps us solve problems

13. Task-related eye movements 2005
Hayhoe and Ballard,

14. Example 2: How to get focus and
context?



Zooming (Bedersen)
Linked windows
(Fowler & Ware)
Fisheye (Furnas,
Carpendale)

15. Zooming Vs Multiple Windows
(Matt Plumlee)

Problem: When do we need extra
windows?
Comparing parts of a visual scene.

2 solutions: Zooming, multiple windows


16. Task: searching for target patterns that match
Conditons:
Zooming vs
Windows + eye movements
1,3,5,7 items per cluster

17. Cognitive Model (grossly
simplified)

Time = setup cost +
number of “visits” x time per visit
Number of visits is a function of number of
objects to be compared and visual
working memory capacity.
Visits = n/M


18. Prediction
As targets (and visual working
memory load) increases, multiple
Windows become more attractive.
Results

19. Design heuristic

When we need to compare more two or
three simple pattern components add
windows.

20. Example 2

Tools for finding new underwater
behaviors from humpback whale tag data
(Why turning time into space is a good idea)

21. The gear
Antenna
Big
Eyes
DTAG Mark Johnson
Dave Wiley

24. Task: find new behaviors
= stereotyped patterns
Cognitive Algorithm
 repeat




Review behavior sequence looking for
patterns. Remember patterns.
Look for more instances.
until no new patterns

25. The old
way

26. Solution 1. GeoZui 4D

27. Cognitive process for finding new
behaviors
stereotyped patterns
Cognitive Algorithm
 repeat





Review behavior sequence looking for patterns by
playback. Remember patterns using space-time
notes.
Look for more instances. May involve reviewing all
other whale tracks.
Until no new patterns
Cost k*playback time.

28. Solution 2: trackplot

29. Foraging patterns

30. Traversing

31. 2006
Mostly
04
06
07
06

32. Process for finding new behaviors
stereotyped patterns
Cognitive algorithm
 Get to a good viewpoint
 repeat





Review behavior sequence looking for patterns eye
movements. Remember patterns using visual working
memory.
Look for more instances. May involve reviewing all
other whale tracks. Can be posted on the wall
until no new patterns
Cost Nav + Eye Movement time *pattern
matching.

33. Gain in efficiency – from
playback tool to pattern finding
tool

Many hours (with playback)
A few minutes (with patterns)

Approximately a factor of 100


34. Design heuristic

Whenever possible: Turn time into a
spatial pattern – one that converts critical
events into shapes or patterns

Try to make natural mappings – proper
use of texture color, etc.

35. How to get perceptual and
cognitive principles into the
Anderson
designer’s head?
ACT-R
Shneiderman
Overview
Zoom&filter
Details on Demand

38. ME Graph
Constellation

39. Activities at all levels

41. Example 1: Network diagram

43. Where are we going?

Simple cognitive process models involving


Perceptual and cognitive operations
Interaction methods
For Design

44. Acknowlegements
NSF ARDA NOAA ONR
David Wiley, Rusty Bobrow, Matt Plumlee, Anne Gilman

46. Where are we going?
DESIGN of visual thinking tools
We need to understand perception and the cognitive process

47. The machinery
Display
Features
A
B
Proto-objects and
C
Patterns
D
Verbal
Working
Memory
OBJECT
FILES
“Nexus”
Visual
Working
Memory
GIST
Visual
Query
Egocentric object and
Pattern map

48. Change Blindness
Simons and Levin

49. Follow up study with eye tracker
Multi-window condition only.
One eye movement
(from cluster to
cluster) per visual
object
Similar result in Gajewski and Henderson, 2005

53. Sketching ideas:
Many people have noted that we
are cognitive cyborgs:
Edwin Hutchins, Herbert Simon,
Don Norman.. Etc.

54. The query process

55. Tuning the visual system