Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Graph comprehension model talk, Birkbeck and Toulouse Le Mirail, February 2012

344 views

Published on

Published in: Science
  • Be the first to comment

  • Be the first to like this

Graph comprehension model talk, Birkbeck and Toulouse Le Mirail, February 2012

  1. 1. A computational model of graph comprehension David Peebles February 15, 2012
  2. 2. Outline of the talk 1. Introduction • Empirical studies • Previous models of graph use 2. The ACT-R cognitive architecture 3. An ACT-R model of graph comprehension
  3. 3. Graphical displays of information • Increasingly used in an ever-widening range of activities • A current data explosion with new representations and software tools being developed all the time • “Open data” and greater public access to information visualisations: • http://www.informationisbeautiful.net • http://www.guardian.co.uk/news/datablog • http://infosthetics.com/ • http://flowingdata.com/ • The emergence of visual analytics (Thomas and Cook, 2005) • A growing need to teach graphicacy and understand the cognitive and visual processes that underlie it
  4. 4. Costs and benefits of using graphical representations Benefits • Information can be grouped by location, reducing search • Allows perceptual reasoning and rapid identification of salient aspects of data (Larkin and Simon, 1987) Costs • People interpret graphical elements differently (Zacks and Tversky, 1999) • Interpretation can be distorted by visual features (Peebles, 2008)
  5. 5. Bar and line graphs Bar graphs • Interpreted as representing categories • People typically encode height and separate values • Better for comparing single quantities Line graphs • Interpreted as representing ordered/scaled data • People typically encode slope and continuous change • Better for identifying trends
  6. 6. Line graphs 0 1 2 3 4 5 6 7 8 9 10 JanFebMar AprMayJun Jul AugSepOctNovDec AmountProduced Month Amount of Silver and Gold Produced (in tonnes) by ‘Precious Metals Plc.’ in 1996 Silver Gold Similar, informationally equivalent, graphs not computationally equivalent (Larkin and Simon, 1987) 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 Gold Silver Amount of Silver and Gold Produced (in tonnes) by ‘Precious Metals Plc.’ in 1996 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec “How much silver was produced when 4 tonnes of gold were produced? (Peebles and Cheng, 2003)
  7. 7. Police performance monitors (Peebles, 2008)
  8. 8. The case for simulation modelling • There has been a great deal of graph comprehension research (e.g., Carswell and Wickens, 1990; Kosslyn, 1994; Pinker, 1981; Shah and Carpenter, 1995) • Still impossible to produce precise advice about optimal display for particular communicative goal (Meyer, 2000; Kosslyn, 2006) • Too many variables interacting in complex ways • Definitive experiments difficult as too many variations in stimulus conditions possible. • Computer simulation models of graph reading and comprehension • allow testing of graphical designs • allow exploration of ideas and hypotheses about cognitive and visual mechanisms (McClelland, 2009)
  9. 9. Previous process models of graph use • GOMS-based (task-analytic) models of graph question answering • UCIE (Lohse, 1993) MA-P (Gillan, 1994) 1. Identify perceptual and cognitive operators from task or verbal protocol analyses 2. Construct sequences of operators (with execution times) 3. Predict scan paths and task completion times • Computational models • BOZ graph generation tool (Casner, 1991) • CaMerA model of economics expert with multiple representations (Tabachneck-Schijf et al., 1997) • ACT-R graph reading model (Peebles and Cheng, 2003) • None of these models are of graph comprehension
  10. 10. Cognitive architectures • “Unified theories of cognition” (Newell, 1990) • Theories of core cognitive components, representational formats, and processes (e.g., memory, cognitive control) • Main examples of symbolic cognitive architectures: • Soar (Newell, 1990; Laird, 2012) • Epic (Kieras and Meyer, 1997) • ACT-R (Anderson, 2007) • All three based on production system architecture • All can interact with simulated task environments • This allows models of interaction with external representations to be developed
  11. 11. The ACT-R cognitive architecture • Hybrid architecture with symbolic and subsymbolic components • Production system model of procedural memory & cognitive control • Semantic network model of declarative memory • Activation-based learning, memory retrieval & forgetting mechanisms • Simulated eyes & hands for interacting with computer-based tasks Visual Module ACT−R Buffers Environment Pattern Matching Execution Production Module Motor Control State Problem State Memory Procedural Memory Declarative
  12. 12. Interaction graphs Low High 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night • Display data from 2 × 2 factorial research designs • Novices often find interpretation difficult as they differ from more familiar bar and line graphs Low High 10 20 30 40 50 60 70 80 90 100 Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night • Come in bar and line forms (both equally likely) • Primary task is to comprehend relationships between variables, not identify specific values
  13. 13. How people use interaction graphs Low High 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night • Novices’ interpretations of line graphs significantly worse than bar graphs (Peebles and Ali, 2009) • Drawn by salience of lines to identify legend variable & ignore x axis variable Low High 10 20 30 40 50 60 70 80 90 100 Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night • Expert users’ interpretations also affected by graph type • New line graph design improved novice performance to bar graph level (Ali and Peebles, submitted)
  14. 14. Alternative designs Low High 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night • “Combined” graph – no significant improvement 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of Day PercentError Experience Low High Time of Day Day Night • “Colour match” graph – significant improvement to bar graph level
  15. 15. An ACT-R model of interaction graph comprehension Aims • Simulate verbal protocols and eye movement scan paths of experts and novices interpreting interaction graphs • Provide a detailed characterisation of: • the prior knowledge required to interpret interaction graphs • the information extracted from the diagram • the knowledge generated during the comprehension process • the cognitive and perceptual operations involved in interpreting interaction graphs • the strategic processes that control comprehension • what underlies the differences between expert and novice performance
  16. 16. Stages of comprehension • Verbal protocols reveal that experts produce qualitative descriptions of the differences between conditions, not individual values. • Comprehension is typically carried out in two main phases: 1. Variable identification. Labels categorised as dependent or independent according to location, and associated with levels and identifiers: left or right (x axis), or colour (legend) 2. Pattern recognition and description. Distances between plot points compared and used to probe long-term declarative memory for interpretive knowledge • Success – retrieved knowledge is used to provide an interpretation • Failure – simply describe the identification or comparison process being carried out
  17. 17. Prior declarative graph knowledge (1) • Three types of knowledge required to interpret interaction graphs: 1. The typical allocation of DV and IVs to graph axes and legend Low High 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night
  18. 18. Prior declarative graph knowledge (2) 2. The principle that distance between graphical elements indicates magnitude of relationship between conceptual entities the elements represent Low High 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of DayPercentError Experience Time of Day Day Night
  19. 19. Prior declarative graph knowledge (3) 3. Graphical/spatial indicators of three interpretive facts: • Simple effects. Distance between two plot points • Main effects. Differences in the y-axis location of the midpoints between two pairs of plot points. • Interactions. Differences in the inter-point distances between levels, combined with information about their point ordering. Low High 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night
  20. 20. Representing and encoding information in the graph • Model encodes four x-y coordinate points and spatial distances between them • Encoded numerically then translated to symbolic descriptions (e.g., “small”, “very large”). “Elementary perceptual tasks” (Cleveland and McGill, 1984) • Encode spatial distance between plot points • Compare magnitude of two distances and produce a symbolic description of difference. Low High 10 20 30 40 50 60 70 80 90 100 ● ● Percent Error as a function of Experience and Time of Day PercentError Experience Time of Day Day Night
  21. 21. Knowledge generated during comprehension • Knowledge chunks representing the variables • Three knowledge structures that accumulate and associate items of graph and interpretive information when identifying: • Simple effects • Main effects • Interactions
  22. 22. Stages of comprehension • Comprehension proceeds in the following order: 1. Read title. Identify variable names and create memory chunks. 2. Seek variable labels, identify what they are by their location and if required, associate with label levels 3. Associate variable levels with indicators (position or colour) 4. Look at plot region and attempt to interpret distances. If a highly salient pattern exists (e.g., cross, large gap) process that first 5. Continue until no more patterns are recognised • Individual production rule for each pattern • No production rule then pattern not processed
  23. 23. Example model output 1 seek text at top of display. . . 2 [m-yield] = [var] 3 [as] [a] [function] [of] [p-density] = [var] 4 [and] [n-level] = [var] 5 seek text at far right of display. . . 6 [n-level] at [far right] = [independent] var 7 look to nearest text. . . 8 [low] = level of [n-level] 9 [high] = level of [n-level] 10 seek objects in plot region. . . 11 [blue] [line] 12 no memory for [blue] look to legend 13 [blue] [rect]. seek near text. . . [blue] = [low] 14 [green] [rect]. seek near text. . . [green] = [high] 15 seek text at far left of display. . . 16 [m-yield] at [far left] = [dependent] var 17 seek text at bottom of display. . . 18 [p-density] at [bottom] = [independent] var 19 look to nearest text. . . 20 [compact] = level of [p-density]. [compact] = [right] 21 [sparse] = level of [p-density]. [sparse] = [left]
  24. 24. Example model output 22 identify legend levels. . . 23 [0.0] diff [blue] = [no] [simple] effect [low] [n-level] 24 [0.5] diff [green] [modest] [simple] effect [high] [n-level] 25 compare [blue] & [green] levels. . . 26 [small] diff. [high] [n-level] > [low] [n-level] 27 [small] [main] effect [n-level] 28 identify x axis levels. . . 29 [0.0] diff [left] = [no] [simple] effect [sparse] [p-density] 30 [0.5] diff [right] = [modest] [simple] effect [compact] [p-density] 31 compare [left] & [right] levels. . . 32 [small] diff. [compact] [p-density] > [sparse] [p-density] 33 [small] [main] effect [p-density] 34 compare left and right patterns. . . 35 [0.5] diff in distance between points. [right] bigger 36 [modest] diff & [same] point order = [modest] [interaction] 37 for [sparse] [p-density] [low] [n-level] = [high] [n-level] 38 for [compact] [p-density] [high] [n-level] > [low] [n-level]
  25. 25. Limitations of the model • Model currently focuses on graph knowledge and not the effects of domain knowledge. Experts do display such domain-general graph knowledge. • Early visual processes are not specified. ACT-R models of visual and spatial processing are being developed. • Although based on human data, the model has not been rigorously compared to eye movement and verbal protocol data yet. • Model of novice users has not been developed yet • Selectively remove production rules and interpretive declarative knowledge.
  26. 26. Conclusions • Currently the only computational model of graph comprehension • Given a data set from a 2 × 2 factorial research design it will produce an expert level description of the effects • The model also describes the patterns used to produce the interpretation • The model associates variables and their levels, and levels with their colour identifiers using mechanisms that are functionally equivalent to the Gestalt laws of proximity and similarity respectively. • The current model can be considered a first approximation to a more detailed model that incorporates additional factors to broaden the scope of behaviour accounted for. • The model will form the basis of a more general graph comprehension model that can interpret bar interaction graphs and more general bar and line graphs.
  27. 27. References (1) • Ali, N. and Peebles, D. (submitted). The effect of gestalt laws of perceptual organisation on the comprehension of three-variable bar and line graphs • Anderson, J. R. (2007). How can the human mind occur in the physical universe? Oxford University Press, New York, NY • Carswell, C. M. and Wickens, C. D. (1990). The perceptual interaction of graphical attributes: Configurality, stimulus homogeneity, and object interaction. Perception & Psychophysics, 47:157–168 • Casner, S. M. (1991). A task-analytic approach to the automated design of graphic presentations. ACM Transactions on Graphics, 10:111–151 • Gillan, D. J. (1994). A componential model of human interaction with graphs: 1. linear regression modelling. Human Factors, 36(3):419–440
  28. 28. References (2) • Kieras, D. E. and Meyer, D. E. (1997). An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction, 12:391–438 • Kosslyn, S. M. (1994). Elements of graph design. W. H. Freeman & Co., New York • Kosslyn, S. M. (2006). Graph design for the eye and mind. Oxford University Press, New York • Laird, J. E. (2012). The Soar cognitive architecture. MIT Press, Cambridge, Mass • Lohse, G. L. (1993). A cognitive model for understanding graphical perception. Human-Computer Interaction, 8:353–388 • Meyer, J. (2000). Performance with tables and graphs: effects of training and a visual search model. Ergonomics, 43(11):1840–1865
  29. 29. References (3) • McClelland, J. L. (2009). The place of modeling in cognitive science. Topics in Cognitive Science, 1:11–38 • Newell, A. (1990). Unified theories of cognition. The William James lectures. Harvard University Press • Peebles, D. (2008). The effect of emergent features on judgments of quantity in configural and seperable displays. Journal of Experimental Psychology: Applied, 14(2):85–100 • Peebles, D. and Ali, N. (2009). Differences in comprehensibility between three-variable bar and line graphs. In Proceedings of the Thirty-first Annual Conference of the Cognitive Science Society, pages 2938–2943, Mahwah, NJ. Lawrence Erlbaum Associates • Peebles, D. and Cheng, P. C.-H. (2003). Modeling the effect of task and graphical representation on response latency in a graph reading task. Human Factors, 45:28–45
  30. 30. References (4) • Pinker, S. (1981). A theory of graph comprehension. Occasional Paper 10, Center for Cognitive Science, Massachusetts Institute of Technology • Shah, P. and Carpenter, P. A. (1995). Conceptual limitations in comprehending line graphs. Journal of Experimental Psychology: General, 124:43–62 • Tabachneck-Schijf, H. J. M., Leonardo, A. M., and Simon, H. A. (1997). CaMeRa: A computational model of multiple representations. Cognitive Science, 21:305–350 • Thomas, J. J. and Cook, K. A. (2005). Illuminating the path: The research and development agenda for visual analytics. IEEE Press • Zacks, J. and Tversky, B. (1999). Bars and lines: A study of graphic communication. Memory and Cognition, 27(6):1073–1079

×