Some Theory and Practice on
Patterns – In Theory
Yann-Gaël Guéhéneuc
NII, Tokyo, Japan
12/02/14

This work is licensed und...
2/216
Did you know that…

3/216
Vision is our dominant sense?

4/216
5/216
6/216
How does vision work?

7/216
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Visual System
 Vision

Science: Photons to Phenomenology
by Stephen E. Palmer

10/216
Visual System

Retinal
Image

Image-based
Stage

Surface-based
Stage

Item-based
Stage

Category-based
Stage

Working
Memo...
Retinal
Image

Visual System
Region
Analysis

Figure/Group
Distinction

Visual
Interpolation

Items
Grouping

Image-based
...
Retinal
Image

Visual System
Region
Analysis

Figure/Group
Distinction

Visual
Interpolation

Items
Grouping

Image-based
...
Retinal
Image

Visual System
Region
Analysis

Figure/Group
Distinction

Visual
Interpolation

Items
Grouping

Image-based
...
Retinal
Image

Visual System
Region
Analysis

Figure/Group
Distinction

Visual
Interpolation

Items
Grouping

Image-based
...
Retinal
Image

Visual System
Region
Analysis

Figure/Group
Distinction

Visual
Interpolation

Items
Grouping

Image-based
...
Visual System
 Retinal

image

17/216
Visual System
 Retinal

image

18/216
Visual System
 Region

analysis

– Distinguish items based on their regions

19/216
Visual System
 Region

analysis

– Distinguish items based on their regions

Zhuolin Jiang and Larry S. Davis ; Submodula...
Visual System
 Items

grouping

– Gestalt theory
• Gestalten: to put in form, to give structure

Fritz Perls
*Berlin, Jul...
Visual System

22/216
Visual System

23/216
Visual System

24/216
Visual System

25/216
Visual System


Items grouping
– Laws of the Gestalt

26/216
Visual System


Items grouping
– Laws of the Gestalt
• Good form

27/216
Visual System


Items grouping
– Laws of the Gestalt
• Good form

28/216
Visual System


Items grouping
– Laws of the Gestalt
• Good form

• Continuity

29/216
Visual System


Items grouping
– Laws of the Gestalt
• Good form

• Continuity

30/216
Visual System


Items grouping
– Laws of the Gestalt
• Good form

• Proximity

• Continuity

31/216
Visual System


Items grouping
– Laws of the Gestalt
• Good form

• Proximity

• Continuity

32/216
Visual System


Items grouping
– Laws of the Gestalt
• Good form

• Proximity

• Continuity
• Similitude

33/216
Visual System


Items grouping
– Laws of the Gestalt

34/216
Visual System


Items grouping
– Laws of the Gestalt

• Common destiny

35/216
Visual System


Items grouping
– Laws of the Gestalt

• Common destiny

36/216
Visual System


Items grouping
– Laws of the Gestalt
• Closure
• Common destiny

37/216
Visual System


Items grouping
– Laws of the Gestalt
• Closure
• Common destiny

Completion

38/216
Visual System
 Items

grouping

– Involvement of the memory

39/216
Visual System
 Items

grouping

– Involvement of the memory

40/216
Visual System
 Comparison

– Involvement of the memory

41/216
Visual System
 Comparison

– Involvement of the memory
• Spatial

42/216
Visual System
 Comparison

– Involvement of the memory
• Spatial
• Temporal

43/216
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Memory Models
 One

model

Items
(Visuo-spatial scratchpad)

Articulatory
Loop

Central
Executive
Long-term
Memory

45/21...
Memory Models
 One

model

Items
(Visuo-spatial scratchpad)

Articulatory
Loop

Central
Executive
Long-term
Memory

Episo...
Memory Models
 One

model

Items
(Visuo-spatial scratchpad)

Articulatory
Loop

Central
Executive
Long-term
Memory

Episo...
Memory Models
 One

model

Episodic
Memory

Procedural
Memory

Semantic
Memory

Objects
Representations

Categories
Repre...
Memory Models
 Another

model

49/216
Memory Models
 Yet

another model

50/216
Memory Models
 Yet

another model

51/216
Memory Models
 Yet,

yet another model

52/216
Memory Models
 Miller,

1956

53/216
Memory Models
 Miller,

1956

MRBXQVLWQK

54/216
Memory Models
 Miller,

1956

?

55/216
Memory Models
 Miller,

1956

?

56/216
Memory Models
 Miller,

1956

?
The Magical Number Seven, Plus or Minus Two:
Some Limits on Our Capacity for Processing I...
Memory Models
 Brooks,

1978

58/216
Memory Models
 Brooks,

1978

– Problem domain

59/216
Memory Models
 Brooks,

1978

– Problem domain
– Executing program

60/216
Memory Models
 Brooks,

1978

– Problem domain
– Executing program
– Knowledge domains
• Hypothesis-and-verify process
• ...
Memory Models
 Brooks,

1978

– Problem domain
– Executing program
– Knowledge domains
• Hypothesis-and-verify process
• ...
Memory Models
 Soloway,

1983

– Experts’ knowledge that novices do not have
• Programming plans
– RUNNING TOTAL LOOP PLA...
Memory Models
 Soloway,

1983

64/216
Memory Models
 Soloway,

1983

An IF should be used when a statement body is guaranteed
to be executed only once and a WH...
Memory Models
 Soloway,

1983

An IF should be used when a statement body is guaranteed
to be executed only once and a WH...
Memory Models
 Soloway,

1983

An IF should be used when a statement body is guaranteed
to be executed only once and a WH...
Memory Models
 Soloway,

1983

An IF should be used when a statement body is guaranteed
to be executed only once and a WH...
Memory Models
 Pennington,

1987

– Multiple abstractions for computer programs
•
•
•
•

(COBOL but may be true for other...
Memory Models
 Pennington,

1987

70/216


Abstraction of function
71/216


Abstraction of data-flow
72/216


Abstraction of control-flow
73/216


Abstraction of conditionalised actions
74/216


75/216
Memory Models
 Pennington,

1987

– Programming knowledge
• Text structure knowledge
– Sequences
– Iterations
– Condition...
Memory Models
 Pennington,

1987

– Which knowledge and what abstraction?

77/216
Memory Models
 Pennington,

1987

– Which knowledge and what abstraction?

Text structure

78/216


Abstraction of control-flow
79/216
Memory Models
 Van

Mayrhauser, 1995

– From previous works by
•
•
•
•
•
•
•

Pennington
Soloway et al.
Rist
Van Mayhause...
Memory Models
 Van

Mayrhauser, 1995

81/216
Memory Models
 Van

Mayrhauser, 1995

Integrated model

82/216


83/216


84/216


85/216


86/216
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Mental Models
“The image of the world around us, which we
carry in our head, is just a model. Nobody in
his head imagines ...
Mental Models






89/216
Mental Models
 Mental

models






90/216
Mental Models
 Mental

models
 Formal rules of inference




91/216
Mental Models
 Mental

models
 Formal rules of inference
 Domain-specific rules of inference



92/216
Mental Models
 Mental

models
 Formal rules of inference
 Domain-specific rules of inference
 Probabilities


93/216
Mental Models
 Mental

models
 Formal rules of inference
 Domain-specific rules of inference
 Probabilities
…

94/216
Mental Models

95/216
Memory Models

96/216
Mental Models

97/216
Mental Models
 Context

matters!

98/216
Mental Models
 Context

matters!

99/216
Mental Models
 Context

matters!

100/216
Mental Models
 Context

matters!

101/216
Mental Models
 Context

matters!

102/216
Mental Models
 Minsky,

1974

– Frames
• “data-structure for representing a stereotyped situation”
• “network of nodes an...
Mental Models
 Minsky,

1974

– Vision
– Speech
– Knowledge
– Control

104/216
Mental Models
 Minsky,

1974

– Vision
– Speech
– Knowledge
– Control

105/216
Mental Models
 Minsky,

1974

– Vision
• A frame, once evoked on the basis of partial evidence
or expectation, would firs...
Mental Models
 Minsky,

1974

– Vision
– Speech
– Knowledge
– Control

107/216
Mental Models
 Minsky,

1974

– Knowledge
• Similarity networks

108/216
Mental Models
 Minsky,

1974

– Knowledge
• Similarity networks

109/216
Mental Models
 Minsky,

1974

– Vision
– Speech
– Knowledge
– Control

110/216
Mental Models
 Minsky,

1974

– Control
• TOP-DOWN or LATERAL: Should one make a pass
over all the terminals first or sho...
Mental Models
 Soloway,

1986

– In the context of analyzing problems and
constructing programs
– Goals
• Requirements
• ...
Mental Models
 Soloway,

1986

“Master and novice chess players were shown a
meaningful chess board, and asked to recall ...
Mental Models
 Soloway,

1986

– Micro-strategies
• Ask questions
• Seek answers
• Repeat
Difficult with delocalised plan...
Mental Models
 Rich

and Waters, 1990

(IBM’s Harlan Mills idea of
“chief programmer teams”)
“[T]o provide software engin...
Mental Models
 Rich

and Waters, 1990

“Between the engineer and the Apprentice […],
the key to effective cooperation is ...
Mental Models
 Rich

and Waters, 1990

“[An algorithmic] cliché contains both fixed parts
and parts that vary from one oc...
Mental Models
 Rich

and Waters, 1990

– Key properties of algorithmic clichés
•
•
•
•

Canonical form
Language independe...
Mental Models
 Rich

and Waters, 1990

“The Plan Calculus is intended to be a blueprint
language for software.”
(p. 29)

...
Mental Models
 Rich

and Waters, 1990
if:negative
T

F

then:negate

T

F
end:join

120/216
Mental Models
 Rich

and Waters, 1990
if:negative
T

F

absolute-value
then:negate

T

F
end:join

121/216
Mental Models
 Rich

and Waters, 1990
input:

next:apply-function

input:
generate

continue:
generation
next:

*

output...
(On Gaming…)




Visual system
Memory models
Mental models




http://arstechnica.com/science/2014/02/how-action-gam...
(On Gaming…)




Visual system
Memory models
Mental models




http://arstechnica.com/science/2014/02/how-action-gam...
(On Gaming…)




Visual system
Memory models
Mental models




http://arstechnica.com/science/2014/02/how-action-gam...
(On Gaming…)




Visual system
Memory models
Mental models
“[G]amers [were] better at picking out details in a
cluttere...
(On Gaming…)




Visual system
Memory models
Mental models
“[G]amers [were] better at picking out details in a
cluttere...
(On Gaming…)




Visual system
Memory models
Mental models
“[G]amers [were] better at picking out details in a
cluttere...
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Schema Theory

“The captain asked the passengers to fasten
the seat belts. They were ready to take off.”
—Kohls and Scheit...
Schema Theory

131/216
Schema Theory
“This comprehensive situation is an
interrelation of events and entities, and is
stored in an internal data ...
Schema Theory
“This comprehensive situation is an
interrelation of events and entities, and is
stored in an internal data ...
Schema Theory
“Human brains operate fundamentally in
terms of pattern recognition rather than of
logic. They are highly co...
Schema Theory
 Piaget,

1969

– Child development theory
– Schema and associated processes

135/216
Schema Theory
 Piaget,

1969

– Child development theory
– Schema and associated processes

136/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• From Plato and Aristotle: essential commonalities,
basi...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Abstract representation of multiple instances of the
sa...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Rang...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Rang...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Rang...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Rang...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Prototypical object

• Object space and (mental) object...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Prototypical object

• Object space and (mental) object...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Prototypical object

• Object space and (mental) object...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Prototypical object

• Object space and (mental) object...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Prototypical object

• Object space and (mental) object...
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Part-of

• Coupling

• Is-a
148/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Part-of

• Coupling

• Is-a
149/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Part-of

• Coupling

• Is-a
150/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Part-of

• Coupling

• Is-a
151/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Hierarchy
– Contexts
– Forces
– Probabilities

152/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Hierarchy
– Contexts
– Forces
– Probabilities

153/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Hierarchy
– Contexts
– Forces
– Probabilities

154/216
Schema Theory
 Piaget,

1969

– Schema and associated processes
• Use

155/216
156/216
(Schema Theory  Programming)
 Papert

worked with Piaget from 1958 to
1963 and afterwards
– Constructionism
• Learning a...
(Schema Theory  Programming)



Papert developed the Logo programming language
within the constructionism theory circa. ...
(Schema Theory  Programming)
 Explanations

of Logo programming using
a simplified cognitive development model
Corporal ...
(Schema Theory  Programming)
 Explanations

of object-oriented programming using the same simplified model
Corporal  Vi...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Comprehension: pattern matching of memorised
schema th...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Accretion: specific situation or experience stored in
...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Accretion: specific situation or experience stored in
...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Accretion: specific situation or experience stored in
...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Tuning: modifications to existing schemas
– Variables
...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Tuning: modifications to existing schemas
– Variables
...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Restructuring: ex nihilo / copy of existing schemas
– ...
Schema Theory
 Rumelhart

and Norman, 1978

– Models of learning
• Restructuring: ex nihilo / copy of existing schemas
– ...
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Software Patterns


Schemas
– Variables
– Ranges of values
• Forces

– Constraints among
variables and their
values

177/...
Software Patterns


Schemas
– Variables
– Ranges of values
• Forces

– Constraints among
variables and their
values
What ...
Software Patterns


Schemas
– Variables
– Ranges of values
• Forces

– Constraints among
variables and their
values
Possi...
Software Patterns


Schemas
– Variables

Expected relations, interactions…

– Ranges of values
• Forces

– Constraints am...
Software Patterns
 Schema

theory

“The pattern is an attempt to discover some
invariant features, which distinguishes go...
Software Patterns
 Knowledge

and–or procedure

“The solution part of a good pattern describes
both a process and a thing...
Software Patterns
 Space

and generation

“A pattern describes a coherent yet infinite
design space, not a finite set of ...
Software Patterns


Use design patterns
–
–
–
–

Composite
Decorator
Observer
…

to generate the
architecture of the
prog...
Software Patterns
 Wholeness

– Quality Without a Name
• Forces

“Having an ideal concept of a thing lets one
judge the b...
Software Patterns

186/216
Software Patterns
“Important assumptions
– That patterns can be codified in such a way that
they can be shared between dif...
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Empirical Studies
“Eye movements are uniquely poised between
perception and cognition. They are central to the
function of...
Empirical Studies
“Eye movements are uniquely poised between
perception and cognition. They are central to the
function of...
Empirical Studies
“Eye movements are uniquely poised between
perception and cognition. They are central to the
function of...
Empirical Studies
“Eye movements are uniquely poised between
perception and cognition. They are central to the
function of...
Empirical Studies
“Eye movements are uniquely poised between
perception and cognition. They are central to the
function of...
194/216
http://web-tan.forum.impressrd.jp/e/2009/04/02/5280

195/216
http://another-voice.jp/jp/katsu/professional_view/01/index.shtml

196/216
http://another-voice.jp/jp/katsu/professional_view/01/index.shtml

197/216
Empirical Studies
 Data

– Fixations
– Saccades
– Scan path
 Analyses

– Centres of interests
– Effort
– Duration
–…
198...
Empirical Studies

199/216
200/216
201/216
Empirical Studies
 SR

Research Eye-link II

– www.sr-research.com
– Intrusive
– Two infra-red cameras and one sensor
– E...
203/216
204/216
Empirical Studies
 Tobii

1750 eye-tracker

– www.tobii.se
– Non-intrusive (no head mount)
– Two cameras built into a 17 ...
206/216
207/216
Empirical Studies
 Seeing

Machines FaceLab 5

– www.seeingmachines.com
– Non-intrusive
– Two infra-red cameras
– Error r...
Outline
 Visual

system
 Memory models
 Mental models
 Schema theory
 Software patterns
 Empirical studies
 Conclus...
Conclusion
“That practices can be codified in such a
way that they can be shared between
different designers”
—Zhang and B...
Conclusion
“That practices can be codified in such a
way that they can be shared between
different designers”
—Zhang and B...
Conclusion
“That practices can be codified in such a
way that they can be shared between
different designers”
—Zhang and B...
Conclusion
“That practices can be codified in such a
way that they can be shared between
different designers”
—Zhang and B...
Conclusion
 Software

patterns

– Schema theory
– Knowledge and–or procedure
– Space and generation
– Wholeness

214/216
Conclusion
 Empirical

studies

– Patterns use and usefulness
•
•
•
•

Faults
Changes
Comprehension
…

– Many opportuniti...
Conclusion
 Empirical

studies

– Patterns use and usefulness
•
•
•
•

Faults
Changes
Comprehension
…

– Many opportuniti...
Upcoming SlideShare
Loading in...5
×

Software Design Patterns in Theory

474

Published on

Following the previous lectures that focus on the practical uses of good and bad practices, we frame this practices in the context of the developers' cognitive characteristics. We first recall that developers' use sight first and foremost to acquire and reason about their systems. Then, we cast the use of patterns by developers and then discuss the visual system, memory models, and mental models.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
474
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Software Design Patterns in Theory

  1. 1. Some Theory and Practice on Patterns – In Theory Yann-Gaël Guéhéneuc NII, Tokyo, Japan 12/02/14 This work is licensed under a Creative Commons Attribution-NonCommercialShareAlike 3.0 Unported License
  2. 2. 2/216
  3. 3. Did you know that… 3/216
  4. 4. Vision is our dominant sense? 4/216
  5. 5. 5/216
  6. 6. 6/216
  7. 7. How does vision work? 7/216
  8. 8. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 8/216
  9. 9. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 9/216
  10. 10. Visual System  Vision Science: Photons to Phenomenology by Stephen E. Palmer 10/216
  11. 11. Visual System Retinal Image Image-based Stage Surface-based Stage Item-based Stage Category-based Stage Working Memory 11/216
  12. 12. Retinal Image Visual System Region Analysis Figure/Group Distinction Visual Interpolation Items Grouping Image-based Stage Surface-based Stage Item-based Stage Category-based Stage Working Memory 12/216
  13. 13. Retinal Image Visual System Region Analysis Figure/Group Distinction Visual Interpolation Items Grouping Image-based Stage Surface-based Stage Comparison Decision Item-based Stage Category-based Stage Working Memory 13/216
  14. 14. Retinal Image Visual System Region Analysis Figure/Group Distinction Visual Interpolation Items Grouping Image-based Stage Surface-based Stage Comparison Decision Item-based Stage Items (Visuo-spatial scratchpad) Articulatory Loop Category-based Stage Central Executive Long-term Memory Working Memory 14/216
  15. 15. Retinal Image Visual System Region Analysis Figure/Group Distinction Visual Interpolation Items Grouping Image-based Stage Surface-based Stage Comparison Decision Item-based Stage Items (Visuo-spatial scratchpad) Articulatory Loop Category-based Stage Central Executive Long-term Memory Working Memory 15/216
  16. 16. Retinal Image Visual System Region Analysis Figure/Group Distinction Visual Interpolation Items Grouping Image-based Stage Surface-based Stage Comparison Decision Item-based Stage Items (Visuo-spatial scratchpad) Articulatory Loop Category-based Stage Central Executive Long-term Memory Working Memory 16/216
  17. 17. Visual System  Retinal image 17/216
  18. 18. Visual System  Retinal image 18/216
  19. 19. Visual System  Region analysis – Distinguish items based on their regions 19/216
  20. 20. Visual System  Region analysis – Distinguish items based on their regions Zhuolin Jiang and Larry S. Davis ; Submodular Salient Region Detection ; Conference on Computer Vision and Pattern Recognition, IEEE, 2013 20/216
  21. 21. Visual System  Items grouping – Gestalt theory • Gestalten: to put in form, to give structure Fritz Perls *Berlin, July 8, 1893 †Chicago, March 14, 1970 21/216
  22. 22. Visual System 22/216
  23. 23. Visual System 23/216
  24. 24. Visual System 24/216
  25. 25. Visual System 25/216
  26. 26. Visual System  Items grouping – Laws of the Gestalt 26/216
  27. 27. Visual System  Items grouping – Laws of the Gestalt • Good form 27/216
  28. 28. Visual System  Items grouping – Laws of the Gestalt • Good form 28/216
  29. 29. Visual System  Items grouping – Laws of the Gestalt • Good form • Continuity 29/216
  30. 30. Visual System  Items grouping – Laws of the Gestalt • Good form • Continuity 30/216
  31. 31. Visual System  Items grouping – Laws of the Gestalt • Good form • Proximity • Continuity 31/216
  32. 32. Visual System  Items grouping – Laws of the Gestalt • Good form • Proximity • Continuity 32/216
  33. 33. Visual System  Items grouping – Laws of the Gestalt • Good form • Proximity • Continuity • Similitude 33/216
  34. 34. Visual System  Items grouping – Laws of the Gestalt 34/216
  35. 35. Visual System  Items grouping – Laws of the Gestalt • Common destiny 35/216
  36. 36. Visual System  Items grouping – Laws of the Gestalt • Common destiny 36/216
  37. 37. Visual System  Items grouping – Laws of the Gestalt • Closure • Common destiny 37/216
  38. 38. Visual System  Items grouping – Laws of the Gestalt • Closure • Common destiny Completion 38/216
  39. 39. Visual System  Items grouping – Involvement of the memory 39/216
  40. 40. Visual System  Items grouping – Involvement of the memory 40/216
  41. 41. Visual System  Comparison – Involvement of the memory 41/216
  42. 42. Visual System  Comparison – Involvement of the memory • Spatial 42/216
  43. 43. Visual System  Comparison – Involvement of the memory • Spatial • Temporal 43/216
  44. 44. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 44/216
  45. 45. Memory Models  One model Items (Visuo-spatial scratchpad) Articulatory Loop Central Executive Long-term Memory 45/216
  46. 46. Memory Models  One model Items (Visuo-spatial scratchpad) Articulatory Loop Central Executive Long-term Memory Episodic Memory Procedural Memory Semantic Memory 46/216
  47. 47. Memory Models  One model Items (Visuo-spatial scratchpad) Articulatory Loop Central Executive Long-term Memory Episodic Memory Procedural Memory Semantic Memory Objects Representations Categories Representations 47/216
  48. 48. Memory Models  One model Episodic Memory Procedural Memory Semantic Memory Objects Representations Categories Representations 48/216
  49. 49. Memory Models  Another model 49/216
  50. 50. Memory Models  Yet another model 50/216
  51. 51. Memory Models  Yet another model 51/216
  52. 52. Memory Models  Yet, yet another model 52/216
  53. 53. Memory Models  Miller, 1956 53/216
  54. 54. Memory Models  Miller, 1956 MRBXQVLWQK 54/216
  55. 55. Memory Models  Miller, 1956 ? 55/216
  56. 56. Memory Models  Miller, 1956 ? 56/216
  57. 57. Memory Models  Miller, 1956 ? The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information 57/216
  58. 58. Memory Models  Brooks, 1978 58/216
  59. 59. Memory Models  Brooks, 1978 – Problem domain 59/216
  60. 60. Memory Models  Brooks, 1978 – Problem domain – Executing program 60/216
  61. 61. Memory Models  Brooks, 1978 – Problem domain – Executing program – Knowledge domains • Hypothesis-and-verify process • Importance of documentation 61/216
  62. 62. Memory Models  Brooks, 1978 – Problem domain – Executing program – Knowledge domains • Hypothesis-and-verify process • Importance of documentation 62/216
  63. 63. Memory Models  Soloway, 1983 – Experts’ knowledge that novices do not have • Programming plans – RUNNING TOTAL LOOP PLAN – ITEM SEARCH LOOP PLAN –… • Rules of programming discourse – An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed –… 63/216
  64. 64. Memory Models  Soloway, 1983 64/216
  65. 65. Memory Models  Soloway, 1983 An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed 65/216
  66. 66. Memory Models  Soloway, 1983 An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed 66/216
  67. 67. Memory Models  Soloway, 1983 An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed 67/216
  68. 68. Memory Models  Soloway, 1983 An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed 68/216
  69. 69. Memory Models  Pennington, 1987 – Multiple abstractions for computer programs • • • • (COBOL but may be true for other languages) Abstraction of function Abstraction of data-flow Abstraction of control-flow Abstraction of conditionalised actions 69/216
  70. 70. Memory Models  Pennington, 1987 70/216
  71. 71.  Abstraction of function 71/216
  72. 72.  Abstraction of data-flow 72/216
  73. 73.  Abstraction of control-flow 73/216
  74. 74.  Abstraction of conditionalised actions 74/216
  75. 75.  75/216
  76. 76. Memory Models  Pennington, 1987 – Programming knowledge • Text structure knowledge – Sequences – Iterations – Conditionals • Plan knowledge – – – – Plans are intermediate-level programming concepts Summing Hashing Counting … 76/216
  77. 77. Memory Models  Pennington, 1987 – Which knowledge and what abstraction? 77/216
  78. 78. Memory Models  Pennington, 1987 – Which knowledge and what abstraction? Text structure 78/216
  79. 79.  Abstraction of control-flow 79/216
  80. 80. Memory Models  Van Mayrhauser, 1995 – From previous works by • • • • • • • Pennington Soloway et al. Rist Van Mayhauser and Vans Letovsky Brooks Schneiderman and Mayer 80/216
  81. 81. Memory Models  Van Mayrhauser, 1995 81/216
  82. 82. Memory Models  Van Mayrhauser, 1995 Integrated model 82/216
  83. 83.  83/216
  84. 84.  84/216
  85. 85.  85/216
  86. 86.  86/216
  87. 87. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 87/216
  88. 88. Mental Models “The image of the world around us, which we carry in our head, is just a model. Nobody in his head imagines all the world, government or country. He has only selected concepts, and relationships between them, and uses those to represent the real system.” —Jay Wright Forrester, 1971 88/216
  89. 89. Mental Models      89/216
  90. 90. Mental Models  Mental models     90/216
  91. 91. Mental Models  Mental models  Formal rules of inference    91/216
  92. 92. Mental Models  Mental models  Formal rules of inference  Domain-specific rules of inference   92/216
  93. 93. Mental Models  Mental models  Formal rules of inference  Domain-specific rules of inference  Probabilities  93/216
  94. 94. Mental Models  Mental models  Formal rules of inference  Domain-specific rules of inference  Probabilities … 94/216
  95. 95. Mental Models 95/216
  96. 96. Memory Models 96/216
  97. 97. Mental Models 97/216
  98. 98. Mental Models  Context matters! 98/216
  99. 99. Mental Models  Context matters! 99/216
  100. 100. Mental Models  Context matters! 100/216
  101. 101. Mental Models  Context matters! 101/216
  102. 102. Mental Models  Context matters! 102/216
  103. 103. Mental Models  Minsky, 1974 – Frames • “data-structure for representing a stereotyped situation” • “network of nodes and relations” and “slots” – Frame-systems • Important actions are transformations between the frames of a system • Different frames of a system share the same terminals (and slots) – Matching process 103/216
  104. 104. Mental Models  Minsky, 1974 – Vision – Speech – Knowledge – Control 104/216
  105. 105. Mental Models  Minsky, 1974 – Vision – Speech – Knowledge – Control 105/216
  106. 106. Mental Models  Minsky, 1974 – Vision • A frame, once evoked on the basis of partial evidence or expectation, would first direct a test to confirm its own appropriateness • It would request information needed to assign values to those terminals that cannot retain their default assignments • If informed about a transformation (e.g., an impending motion), it would transfer control to the appropriate other frame of that system 106/216
  107. 107. Mental Models  Minsky, 1974 – Vision – Speech – Knowledge – Control 107/216
  108. 108. Mental Models  Minsky, 1974 – Knowledge • Similarity networks 108/216
  109. 109. Mental Models  Minsky, 1974 – Knowledge • Similarity networks 109/216
  110. 110. Mental Models  Minsky, 1974 – Vision – Speech – Knowledge – Control 110/216
  111. 111. Mental Models  Minsky, 1974 – Control • TOP-DOWN or LATERAL: Should one make a pass over all the terminals first or should one attempt a complete, detailed instantiation of some supposedly most critical one? • CENTRAL CONTROL: Should a frame, once activated, "take over" and control its instantiation, or should a central process organize the operation 111/216
  112. 112. Mental Models  Soloway, 1986 – In the context of analyzing problems and constructing programs – Goals • Requirements • Bugs – Plans • “Canned” solutions • With “meaning” 112/216
  113. 113. Mental Models  Soloway, 1986 “Master and novice chess players were shown a meaningful chess board, and asked to recall the pieces […]. The masters recalled more of the pieces than did the novices. Next, the masters and novices were shown a board in which the chess pieces were placed randomly on the board. […] The performance of the masters was the same as that of the novices.” —Chase and Simon, 1973 113/216
  114. 114. Mental Models  Soloway, 1986 – Micro-strategies • Ask questions • Seek answers • Repeat Difficult with delocalised plans – Macro-strategies • Systematic (scalability?) • As-needed (correctness?) 114/216
  115. 115. Mental Models  Rich and Waters, 1990 (IBM’s Harlan Mills idea of “chief programmer teams”) “[T]o provide software engineers with intelligent assistance.” (p. 2) “The intended interaction between a software engineer […] is modeled after interaction with a human assistant.” (p. 2) 115/216
  116. 116. Mental Models  Rich and Waters, 1990 “Between the engineer and the Apprentice […], the key to effective cooperation is shared knowledge.” (p. 4) – Shared knowledge • About the system being worked on • About concepts of software engineering  Clichés 116/216
  117. 117. Mental Models  Rich and Waters, 1990 “[An algorithmic] cliché contains both fixed parts and parts that vary from one occurrence of the cliché to the next […], [it] may also include constraints […].” (p. 12) “Non-algorithmic clichés are represented […] using standard frame-based knowledge representation techniques […].” (p. 21) 117/216
  118. 118. Mental Models  Rich and Waters, 1990 – Key properties of algorithmic clichés • • • • Canonical form Language independence Convenient manipulation Expressiveness – A representation of clichés  The Plan Calculus 118/216
  119. 119. Mental Models  Rich and Waters, 1990 “The Plan Calculus is intended to be a blueprint language for software.” (p. 29) • Formal representation for – Programs – Algorithmic clichés Subroutines Flowcharts Program schemas Program generators Macros Flowchart schemas Formal grammars Program transformation Plan calculus Logic Data abstraction 119/216
  120. 120. Mental Models  Rich and Waters, 1990 if:negative T F then:negate T F end:join 120/216
  121. 121. Mental Models  Rich and Waters, 1990 if:negative T F absolute-value then:negate T F end:join 121/216
  122. 122. Mental Models  Rich and Waters, 1990 input: next:apply-function input: generate continue: generation next: * output: generation 122/216
  123. 123. (On Gaming…)    Visual system Memory models Mental models    http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/ 123/216
  124. 124. (On Gaming…)    Visual system Memory models Mental models    http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/ 124/216
  125. 125. (On Gaming…)    Visual system Memory models Mental models    http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/ 125/216
  126. 126. (On Gaming…)    Visual system Memory models Mental models “[G]amers [were] better at picking out details in a cluttered scene” http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/ 126/216
  127. 127. (On Gaming…)    Visual system Memory models Mental models “[G]amers [were] better at picking out details in a cluttered scene” “[G]amers were able to produce accurate estimates [of relative counts]” http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/ 127/216
  128. 128. (On Gaming…)    Visual system Memory models Mental models “[G]amers [were] better at picking out details in a cluttered scene” “[G]amers were able to produce accurate estimates [of relative counts]” “Gamers found it easier to perform mental rotations of geometric shapes” http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/ 128/216
  129. 129. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 129/216
  130. 130. Schema Theory “The captain asked the passengers to fasten the seat belts. They were ready to take off.” —Kohls and Scheiter, 2008 130/216
  131. 131. Schema Theory 131/216
  132. 132. Schema Theory “This comprehensive situation is an interrelation of events and entities, and is stored in an internal data structure that can be activated by recognizing its typical features. Such data structures, or schemas, are mental representations in an individual’s mind.” —Kohls and Scheiter, 2008 132/216
  133. 133. Schema Theory “This comprehensive situation is an interrelation of events and entities, and is stored in an internal data structure that can be activated by recognizing its typical features. Such data structures, or schemas, are mental representations in an individual’s mind.” —Kohls and Scheiter, 2008 133/216
  134. 134. Schema Theory “Human brains operate fundamentally in terms of pattern recognition rather than of logic. They are highly constructive in settling on given schema and at the same time are constantly open to error.” —Edelman, 2006 (With minor adaptation) 134/216
  135. 135. Schema Theory  Piaget, 1969 – Child development theory – Schema and associated processes 135/216
  136. 136. Schema Theory  Piaget, 1969 – Child development theory – Schema and associated processes 136/216
  137. 137. Schema Theory  Piaget, 1969 – Schema and associated processes • From Plato and Aristotle: essential commonalities, basic properties that are distinctive of an object To Kant: link from stimuli to “pure” concepts – Perceived chair vs. “ideal” chair – No a-priori knowledge • Gick and Holyoak: learning by examples and problem solving (like in mathematics) 137/216
  138. 138. Schema Theory  Piaget, 1969 – Schema and associated processes • Abstract representation of multiple instances of the same kinds of – – – – – Concept Situation Plan Behaviour … to recognise similar / discriminate dissimilar experiences, access common concepts, draw inferences, create goals, develop plans, use skills138/216
  139. 139. Schema Theory  Piaget, 1969 – Schema and associated processes • Variables (or slots or attributes) • (Constrained) Ranges of values – People’s experience – Constants – Optional or mandatory • Constraints among variables and their values 139/216
  140. 140. Schema Theory  Piaget, 1969 – Schema and associated processes • Variables (or slots or attributes) • (Constrained) Ranges of values – People’s experience – Constants – Optional or mandatory • Constraints among variables and their values 140/216
  141. 141. Schema Theory  Piaget, 1969 – Schema and associated processes • Variables (or slots or attributes) • (Constrained) Ranges of values – People’s experience – Constants – Optional or mandatory • Constraints among variables and their values 141/216
  142. 142. Schema Theory  Piaget, 1969 – Schema and associated processes • Variables (or slots or attributes) • (Constrained) Ranges of values – People’s experience – Constants – Optional or mandatory • Constraints among variables and their values 142/216
  143. 143. Schema Theory  Piaget, 1969 – Schema and associated processes • Prototypical object • Object space and (mental) object generator 143/216
  144. 144. Schema Theory  Piaget, 1969 – Schema and associated processes • Prototypical object • Object space and (mental) object generator 144/216
  145. 145. Schema Theory  Piaget, 1969 – Schema and associated processes • Prototypical object • Object space and (mental) object generator 145/216
  146. 146. Schema Theory  Piaget, 1969 – Schema and associated processes • Prototypical object • Object space and (mental) object generator 146/216
  147. 147. Schema Theory  Piaget, 1969 – Schema and associated processes • Prototypical object • Object space and (mental) object generator 147/216
  148. 148. Schema Theory  Piaget, 1969 – Schema and associated processes • Part-of • Coupling • Is-a 148/216
  149. 149. Schema Theory  Piaget, 1969 – Schema and associated processes • Part-of • Coupling • Is-a 149/216
  150. 150. Schema Theory  Piaget, 1969 – Schema and associated processes • Part-of • Coupling • Is-a 150/216
  151. 151. Schema Theory  Piaget, 1969 – Schema and associated processes • Part-of • Coupling • Is-a 151/216
  152. 152. Schema Theory  Piaget, 1969 – Schema and associated processes • Hierarchy – Contexts – Forces – Probabilities 152/216
  153. 153. Schema Theory  Piaget, 1969 – Schema and associated processes • Hierarchy – Contexts – Forces – Probabilities 153/216
  154. 154. Schema Theory  Piaget, 1969 – Schema and associated processes • Hierarchy – Contexts – Forces – Probabilities 154/216
  155. 155. Schema Theory  Piaget, 1969 – Schema and associated processes • Use 155/216
  156. 156. 156/216
  157. 157. (Schema Theory  Programming)  Papert worked with Piaget from 1958 to 1963 and afterwards – Constructionism • Learning as a reconstruction rather than as a transmission of knowledge • Learning is most effective when part of learner’s activity to build a meaningful product http://www.lispcast.com/programming-language-stages-of-development 157/216
  158. 158. (Schema Theory  Programming)  Papert developed the Logo programming language within the constructionism theory circa. 1967 158/216
  159. 159. (Schema Theory  Programming)  Explanations of Logo programming using a simplified cognitive development model Corporal  Visual  Symbolic – The child learns to see her own perspective from the outside in the turtle – The child sees the results of her commands immediately (cf. chick sexing) – The child could think of the result of her commands before their execution 159/216
  160. 160. (Schema Theory  Programming)  Explanations of object-oriented programming using the same simplified model Corporal  Visual  Symbolic – The ability to program from the perspective of a single object at a time – The ability to think in terms of visual, archetypal interactions (client-server, MVC, etc.) – The ability to build abstractions with simple yet powerful rules and meta-programming 160/216
  161. 161. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes 161/216
  162. 162. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes 162/216
  163. 163. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes • Animal face • human face • Bob’s face 163/216
  164. 164. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes • Animal face • human face • Bob’s face 164/216
  165. 165. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes • Animal face • human face • Bob’s face 165/216
  166. 166. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes • Animal face • human face • Bob’s face 166/216
  167. 167. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes • Animal face • human face • Bob’s face 167/216
  168. 168. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Comprehension: pattern matching of memorised schema that cooperates and competes • Animal face • human face • Bob’s face 168/216
  169. 169. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Accretion: specific situation or experience stored in schemas used to reconstruct original experience – Episodic memory – Specific details 169/216
  170. 170. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Accretion: specific situation or experience stored in schemas used to reconstruct original experience – Episodic memory – Specific details 곰 세 마리가 한 집에 있어 아빠 곰 엄마 곰 애기 곰 아빠 곰은 뚱뚱해 엄마 곰은 날씬해 애기 곰은 너무 귀여워 170/216
  171. 171. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Accretion: specific situation or experience stored in schemas used to reconstruct original experience – Episodic memory – Specific details 곰 세 마리가 한 집에 있어 아빠 곰 엄마 곰 애기 곰 아빠 곰은 뚱뚱해 엄마 곰은 날씬해 애기 곰은 너무 귀여워 Why/How is specific information forgotten/replaced by generalised knowledge? 171/216
  172. 172. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Tuning: modifications to existing schemas – Variables – Ranges – Probabilities 172/216
  173. 173. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Tuning: modifications to existing schemas – Variables – Ranges – Probabilities 173/216
  174. 174. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Restructuring: ex nihilo / copy of existing schemas – Schema generation by analogies with existing schema, is-a – Schema induction from existing schema, has-a 174/216
  175. 175. Schema Theory  Rumelhart and Norman, 1978 – Models of learning • Restructuring: ex nihilo / copy of existing schemas – Schema generation by analogies with existing schema, is-a – Schema induction from existing schema, has-a 175/216
  176. 176. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 176/216
  177. 177. Software Patterns  Schemas – Variables – Ranges of values • Forces – Constraints among variables and their values 177/216
  178. 178. Software Patterns  Schemas – Variables – Ranges of values • Forces – Constraints among variables and their values What class plays this role? 178/216
  179. 179. Software Patterns  Schemas – Variables – Ranges of values • Forces – Constraints among variables and their values Possible classes playing this role plus their relations, implementation… 179/216
  180. 180. Software Patterns  Schemas – Variables Expected relations, interactions… – Ranges of values • Forces – Constraints among variables and their values 180/216
  181. 181. Software Patterns  Schema theory “The pattern is an attempt to discover some invariant features, which distinguishes good places from bad places with respect to some particular system of forces.” —Christopher Alexander, 1979 181/216
  182. 182. Software Patterns  Knowledge and–or procedure “The solution part of a good pattern describes both a process and a thing: the ‘thing’ is created by the ‘process’.” —Christopher Alexander, 1979 “Furthermore, a pattern tells about a form not only what it is but also what it does.” —Christopher Alexander, 1964 182/216
  183. 183. Software Patterns  Space and generation “A pattern describes a coherent yet infinite design space, not a finite set of implementations in that space.” —Frank Buschmann, Kevlin Henney, and Douglas C. Schmidt, 2007 183/216
  184. 184. Software Patterns  Use design patterns – – – – Composite Decorator Observer … to generate the architecture of the program 184/216
  185. 185. Software Patterns  Wholeness – Quality Without a Name • Forces “Having an ideal concept of a thing lets one judge the beauty of it” —Kohls and Scheiter, 2008 185/216
  186. 186. Software Patterns 186/216
  187. 187. Software Patterns “Important assumptions – That patterns can be codified in such a way that they can be shared between different designers – That reuse will lead to “better” designs. There is an obvious question here of what constitutes “better”, but a key measure is maintainability” —Zhang and Budgen, 2012 (With minor adaptations) 187/216
  188. 188. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 188/216
  189. 189. Empirical Studies “Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a random sample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequently, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.” —Richardson, Dale, and Spivey, 2013 189/216
  190. 190. Empirical Studies “Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a random sample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequently, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.” —Richardson, Dale, and Spivey, 2013 190/216
  191. 191. Empirical Studies “Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a random sample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequently, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.” —Richardson, Dale, and Spivey, 2013 191/216
  192. 192. Empirical Studies “Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a random sample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequent, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.” —Richardson, Dale, and Spivey, 2013 192/216
  193. 193. Empirical Studies “Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a random sample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequent, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.” —Richardson, Dale, and Spivey, 2013 193/216
  194. 194. 194/216
  195. 195. http://web-tan.forum.impressrd.jp/e/2009/04/02/5280 195/216
  196. 196. http://another-voice.jp/jp/katsu/professional_view/01/index.shtml 196/216
  197. 197. http://another-voice.jp/jp/katsu/professional_view/01/index.shtml 197/216
  198. 198. Empirical Studies  Data – Fixations – Saccades – Scan path  Analyses – Centres of interests – Effort – Duration –… 198/216
  199. 199. Empirical Studies 199/216
  200. 200. 200/216
  201. 201. 201/216
  202. 202. Empirical Studies  SR Research Eye-link II – www.sr-research.com – Intrusive – Two infra-red cameras and one sensor – Error rate < 0.5º – Sampling rate of 500Hz – Access eye position < 3.0 ms. – 0.01º RMS resolution (tilt)  EyeLink Data Viewer 202/216
  203. 203. 203/216
  204. 204. 204/216
  205. 205. Empirical Studies  Tobii 1750 eye-tracker – www.tobii.se – Non-intrusive (no head mount) – Two cameras built into a 17 in flat screen – Records audio and video of a session – Error rate < 0.5º – Sampling rate of 50Hz  ClearView software 205/216
  206. 206. 206/216
  207. 207. 207/216
  208. 208. Empirical Studies  Seeing Machines FaceLab 5 – www.seeingmachines.com – Non-intrusive – Two infra-red cameras – Error rate < 1º – Sampling rate of 60Hz – Complete head-tracking ±90º (y), ±45º (x) – Complete eye-tracking ±45º (y), ±22º (x)  GazeTracker 208/216
  209. 209. Outline  Visual system  Memory models  Mental models  Schema theory  Software patterns  Empirical studies  Conclusion 209/216
  210. 210. Conclusion “That practices can be codified in such a way that they can be shared between different designers” —Zhang and Budgen, 2012  What format?  How memorised?  How acquired? 210/216
  211. 211. Conclusion “That practices can be codified in such a way that they can be shared between different designers” —Zhang and Budgen, 2012  What format?  How memorised?  How acquired? Schema theory 211/216
  212. 212. Conclusion “That practices can be codified in such a way that they can be shared between different designers” —Zhang and Budgen, 2012  What format?  How memorised?  How acquired? Schema theory Long-term memory 212/216
  213. 213. Conclusion “That practices can be codified in such a way that they can be shared between different designers” —Zhang and Budgen, 2012  What format?  How memorised?  How acquired? Schema theory Long-term memory Gestalt theory 213/216
  214. 214. Conclusion  Software patterns – Schema theory – Knowledge and–or procedure – Space and generation – Wholeness 214/216
  215. 215. Conclusion  Empirical studies – Patterns use and usefulness • • • • Faults Changes Comprehension … – Many opportunities to understand program comprehension better! 215/216
  216. 216. Conclusion  Empirical studies – Patterns use and usefulness • • • • Faults Changes Comprehension … – Many opportunities to understand program comprehension better! 216/216
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×