Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

EWIC talk - 07 June, 2018


Published on

Talk at the 16th European Workshop on Imagery and Cognition, Padua, Italy. 7-9 June 2018

Published in: Science
  • Be the first to comment

  • Be the first to like this

EWIC talk - 07 June, 2018

  1. 1. Multiple representations and visual mental imagery in cognitive architectures David Peebles Reader in Cognitive Science Department of Psychology June 7, 2018
  2. 2. Outline of the talk Cognitive architectures & the Common Model of Cognition Attempts to model visual mental imagery An update on the very long history of representation as it now applies to current work in cognitive architectures Multiple representation cognition (Peebles & Cheng, 2017) Apologies if some of this is familiar to you
  3. 3. Cognitive architectures Theories of the core, immutable structures and processes of the human cognitive system. Computer architecture Perception Action Task Environment Learning Procedural Perceptual Learning Learning Declarative Selection Action Short−term memory Long−term memory Procedural Long−term memory Declarative Human cognitive architecture Combine AI methods and cognitive psychology theory. How cognition, perception, and motor processes interact and integrate to produce complex, real-world behaviour. Running software systems.
  4. 4. An emerging standard model 49 architectures currently under active development (Kotseruba & Tsotsos, 2018). Two dominant: ACT-R and Soar (both approx. 30 years old). Much consolidation and convergence over past decade. ‘Common Model of Cognition’ (Laird, Lebiere, & Rosenbloom, 2017)
  5. 5. Representations in cognitive architectures Cognitive architectures come from symbolic, propositional AI tradition (cf. Pylyshyn). colour(square, dark-cyan) colour(circle, violet-red) left-of(square, circle) Descriptive representations. Discrete, amodal, symbolic or numerical entities. Human visual mental imagery uses depictive array-like representations and visual areas in occipital lobe (Kosslyn, Thompson, & Ganis, 2006). Impossible to determine which representations are being used (Anderson, 1978)
  6. 6. Symbolic approaches to spatial reasoning Most use only symbolic descriptive representations CogSketch (Forbus, Usher, Lovett, Lockwood, & Wetzel, 2011) Diagram Representation System (DRS) (Chandrasekaran, Kurup, Banerjee, Josephson, & Winkler, 2004) ACT-R (Peebles, 2013; Peebles & Cheng, 2003) Much spatial reasoning is qualitative though Often ad hoc mechanisms not intrinsic to the architecture Quebec Mississippi 0 10 20 30 40 50 60 70 80 90 100 q q q q Plant CO2 Uptake as a function of Plant Type and Treatment PlantCO2Uptake Plant Type Treatment Chilled Non−chilled
  7. 7. Early attempts at array-based representations Computational Imagery (Glasgow & Papadias, 1992) Computation with Multiple Representations (CaMeRa) model (Tabachneck-Schijf, Leonardo, & Simon, 1997)
  8. 8. Retinotopic Reasoning (R2) architecture Aims to model the computational properties of mental imagery (Kunda, McGreggor, & Goel, 2013). Based, in large part, on array based (non-symbolic) representations and operators. Successfully applied to: (a) Raven’s Progressive Matrices test, (b) Embedded Figures test, (c) Block Design test, and (d) Paper Folding test.
  9. 9. Retinotopic Reasoning (R2) architecture Similar to CaMeRa (Tabachneck-Schijf et al., 1997)
  10. 10. Soar/SVS Soar/SVS (Spatial/Visual System) (Lathrop, Wintermute, & Laird, 2011; Wintermute, 2012) “[M]ental imagery if functionally essential to spatial and visual cognition”.
  11. 11. Soar/SVS Soar/SVS (Spatial/Visual System) (Lathrop et al., 2011; Wintermute, 2012)
  12. 12. Similarities in these non-symbolic approaches Employ representations consisting of two-dimensional arrays of pixels. Operators to manipulate array objects (affine transformations, composition etc.). Forward and backward connections to higher-level (numerical, symbolic) representations (CaMeRa, Soar/SVS) Important in that they allow cognitive modelling of processes akin to those used in visual mental imagery. None explicitly address the issue of multiple representation cognition
  13. 13. Multi-representation cognition An example Imagine a square with sides of one unit. At opposite corners of the square add circles with radii 2 3 of a unit centred on the corners. Do the two circles: 1. overlap? 2. just touch? 3. not touch?
  14. 14. Mathematics-based solution 1. “According to Pythagoras’ theorem, the length of the diagonal between the two opposite corners is the square root of 2”. 2. “That’s about 1.4 units so if we divide that by 2, the centre of the square is about 0.7 units from each corner”. 3. “The radius of each circle is about 0.66 units, so neither circle’s perimeter will reach the centre of the square”. 4. “Therefore the circles do not touch”.
  15. 15. Imagery-based solution 1. “I’m imagining a square and I can see that circles with unit radius will definitely overlap; in fact they intersect each other at the other corners of the square”. 2. “Now I’m imagining circles of 1 2 unit and I can see that they clearly don’t meet. In fact the circles cross the mid-point of each side of the square and curve away from the centre”. 3. “Now I’m thinking of circles with radii of 2 3 . It’s hard to be certain how big they should be, but they seem to just touch each other”.
  16. 16. Multi-representation cognition Two solutions rely upon different mental representations Declarative and mathematical Visuo-spatial and exploiting imagery in the mind’s eye. Understanding computational costs of each representation Potential for cognitive off-loading User’s familiarity with each representation Compatibility of different representations Metacognitive knowledge and processes Monitoring and control processes to handle the selection and monitoring of, transitions between, and integration of different representations. Meta-level information about the characteristics of different representational formats (e.g., level of precision afforded, ease of computation, suitability for a given problem etc.). Initial selection of representations
  17. 17. Conclusions 1. The age-old debate about representations underlying imagery are still going—and has a new impetus. 2. Most cognitive architectures remain with just symbolic representations (probably for pragmatic reasons rather than deep theoretical commitments). 3. New attempts to integrate non-symbolic, ‘depictive’, array-based representations into architectures. 4. The use (or not) of mental imagery requires meta-cognitive knowledge and processes which need to be incorporated into cognitive architectures (Peebles & Cheng, 2017).
  18. 18. References I Anderson, J. R. (1978). Arguments concerning representations for mental imagery.. Psychological Review, 85(4), 249. Chandrasekaran, B., Kurup, U., Banerjee, B., Josephson, J. R., & Winkler, R. (2004). An architecture for problem solving with diagrams [Lecture notes in artificial intelligence 2980]. In A. Blackwell, K. Marriott, & A. Shimojima (Eds.), Diagrammatic representation and inference (pp. 235–256). Berlin: Springer-Verlag. Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2011). CogSketch: Sketch understanding for cognitive science research and for education. Topics in Cognitive Science, 3(4), 648–666. Glasgow, J. & Papadias, D. (1992). Computational imagery. Cognitive Science, 16(3), 355–394. Kosslyn, S. M., Thompson, W. L., & Ganis, G. (2006). The case for mental imagery. Oxford University Press.
  19. 19. References II Kotseruba, I. & Tsotsos, J. K. (2018). A review of 40 years of cognitive architecture research: Core cognitive abilities and practical applications. arXiv preprint, abs/1610.08602. Retrieved from Kunda, M., McGreggor, K., & Goel, A. K. (2013). A computational model for solving problems from the raven’s progressive matrices intelligence test using iconic visual representations. Cognitive Systems Research, 22, 47–66. Laird, J. E., Lebiere, C., & Rosenbloom, P. S. (2017). A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics. AI Magazine. 38(4). Lathrop, S. D., Wintermute, S., & Laird, J. E. (2011). Exploring the functional advantages of spatial and visual cognition from an architectural perspective. Topics in Cognitive Science, 3(4), 796–818.
  20. 20. References III Peebles, D. (2013). Strategy and pattern recognition in expert comprehension of 2×2 interaction graphs. Cognitive Systems Research, 24, 43–51. Peebles, D. & Cheng, P. C.-H. (2003). Modeling the effect of task and graphical representation on response latency in a graph reading task. Human Factors, 45, 28–45. Peebles, D. & Cheng, P. C.-H. (2017, September 11). Multiple representations in cognitive architectures. In AAAI fall symposium 2017: ‘‘a standard model of the mind”. FS-17-01–FS-17-05. American Association for the Advancement of Artificial Intelligence. Washington, VA. Tabachneck-Schijf, H. J. M., Leonardo, A. M., & Simon, H. A. (1997). CaMeRa: A computational model of multiple representations. Cognitive Science, 21, 305–350. Wintermute, S. (2012). Imagery in cognitive architecture: Representation and control at multiple levels of abstraction. Cognitive Systems Research, 19, 1–29.