Ken Forbus presented “Software Social Organisms: Implications for measuring AI progress.”


Published on

Kenneth D. Forbus, Walter P. Murphy Professor of Computer Science and Professor of Education at Northwestern University, gave a wonderful presentation “Software Social Organisms: Implications for measuring AI progress” as part of our Cognitive Systems Institute Speaker Series.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ken Forbus presented “Software Social Organisms: Implications for measuring AI progress.”

  1. 1. Software Social Organisms: Implications for Measuring AI Progress Kenneth D. Forbus Northwestern University
  2. 2. Big Picture • Achieving human-level AI = learning how to create sufficiently smart software social organisms • No single test will be sufficient to measure progress • Evaluation should be ability to participate in our culture – Apprentices – Provides multiple dimensions within which progress can be measured • Modalities, range of domains, levels of knowledge…
  3. 3. Today’s AI systems can be fast and effective. But they requires careful design for narrow circumstances, maintenance by highly trained personnel What if AI systems were as robust, trainable, and taskable as mammals?
  4. 4. Companion Cognitive Architecture • Approach: Understand how to build Software social organisms that – Work with people using natural modalities – Learn and adapt over extended periods of time – Maintain themselves – Capable of relating to people as apprentices, not tools • Companion architecture incorporates other hypotheses as well – Centrality of analogy, qualitative representations • But the idea of software social organisms is more general, and is a path to human-level AI
  5. 5. Why Organisms? • Autonomy – We do not have direct access to the internal representations of children or our collaborators – We communicate via language, sketching, … • Adaptation – Incremental learning in everyday life – Building up knowledge over weeks, months, years • New interest in interactive task learning Kirk & Laird, 2013 Hinrichs & Forbus 2013
  6. 6. Why Social? • Being social is a driver of intelligence – e.g. Tomasello’s research • Cultural knowledge is central in human life – We can all think about things we never experienced • Plate tectonics, the Revolutionary War, … – Stories are an important component of human learning • e.g. Winston’s research • Smart organisms want to cooperate and collaborate as well as compete – AI has started to model this, e.g. Grosz’s research
  7. 7. Human-level AI • Will end up being social organisms • Sufficiently smart software social organisms will be human-level AIs – Sufficiently smart = capable of learning to perform a sufficiently broad range of tasks that people do, with the same or better levels of performance
  8. 8. Apprenticeship • Natural trajectory for bringing people into a role – Also provides one trajectory for evaluations • Start as a student – Lots of “book learning”, interaction – Tests to gauge learning – Legitimate peripheral participation • Expand range of responsibilities – Joint work – Negotiated roles • Autonomous operation with a community – Performs well on its own, or with others at their level – Can ask for help, and help others
  9. 9. Substrate Capacities Needed for Human-like Social Organism • Autonomy – Its own needs, drives, and capabilities for acting and learning • Operates in environments that support shared focus – Each participant has some information about what others can sense – Participants can make focus of attention known to each other easily • Language sufficient to express beliefs, hypotheticals • Ability to build models of the intent of others – Implies learning the types of goals they can have, and how available actions feed into those goals – Models of needs, drives as wellsprings of particular goals • Strong interest in interactions with conspecifics – Helping and teaching
  10. 10. Implications for Evaluation • No single test will work – Won’t measure adaptability and breadth – Can always be hacked • Need to evaluate via ability to participate in our culture – Do real work – Interact with people appropriately – Learn and adapt about its work, others, and itself – Without AI experts constantly fiddling with its internals
  11. 11. Scalable Evaluation Dimensions • Modalities – Text, sketching, speech, vision, mobility, … • Starting point – Nothing versus some prior knowledge versus lots • Level of domain knowledge – Elementary school  Professional life • Range of tasks responsible for – Single task versus multiple – Interactive task learning • Learning ability – About its job, its people, and itself – Amount and quantity of data required
  12. 12. Summary • Human level AI = sufficiently smart software social organisms • Apprenticeship trajectory provides framework for many kinds of evaluation – Initially a student, learning from instructors through carefully designed exercises – Working as an assistant to a mentor, with increasing responsibility as it learns – Working autonomously, reporting back periodically for debriefing, review, and retasking Check out QRGLab channel on YouTube