Susan epstein at ibm csig speaker series


Published on

IBM Cognitive Systems Institute Group Speaker Series Call - Susan Epstein presenting.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Susan epstein at ibm csig speaker series

  1. 1. FORR A Cognitive Architecture for Expertise Susan L. Epstein The Graduate Center and Hunter College of The City University of New York
  2. 2. Executive summary •  FORR (FOr the Right Reasons) is an architecture •  FORR-based systems develop expertise •  FORR-based systems learn quickly from problem solving experience •  FORR-based systems are built from §  World knowledge (descriptives) §  Good reasons for making decisions (Advisors) •  FORR-based systems can restructure their decision process •  FORR confirmed cognitively plausible on human subjects 2Background • FORR • Applications
  3. 3. People, agents and expertise •  People are our best model of intelligent agents §  Some human approaches work well on really hard problems §  Their methods are robust to imperfect data §  They pursue multiple goals •  If an agent is to collaborate with people, it is necessary to understand human decision processes •  A cognitively plausible agent simulates significant human characteristics •  Expert does things faster and better than the rest of us [D’Andrade 1990] 3Background • FORR • Applications
  4. 4. Characteristics of human experts •  They work in a domain (set of related problem classes) •  They satisfice = make good enough decisions •  They entertain multiple decision-making heuristics [Ratterman & Epstein 1995] •  They access multiple representations •  They do situation-based reasoning [Klein & Calderwood 1991] •  Human experts are made, not born 4 Learning is the hallmark of human intelligence Background • FORR • Applications
  5. 5. Agent architecture •  Postulates general principles •  System shell for diverse domains •  Requirements for cognitive plausibility §  Display reasonable behavior •  Make obvious decisions •  Avoid obvious errors •  Solve easy problems quickly §  Balance accuracy and speed §  Be robust to error §  Tolerate and reason with inconsistent, incomplete, noisy data §  Learn 5Background • FORR • Applications Do forever Sense the world Select an action Execute that action
  6. 6. Fundamental issues for a learning architecture •  What is there to learn? •  From whom to learn? •  When to learn? •  How to learn? •  How to use learned knowledge to make decisions? •  How to manage reality and noise? 6Background • FORR • Applications
  7. 7. Cornerstones of FORR’s pragmatic approach •  Expertise is learned, that is, it develops with experience •  Easy questions should have fast (reactive) answers •  Satisfice = make good enough decisions in a simplified model of a complex world (and recover if need be) •  Exploit synergy inherent in multiplicity §  Multiple domain-dependent representations §  Multiple domain-dependent heuristics for decision making §  Multiple learning methods •  Maintain flexibility §  Decouple data, learning methods, and decision methods §  Restructure its own decision-making process •  Transparency: explain decisions FORR's building blocks are descriptives and Advisors 7Background • FORR • Applications
  8. 8. Multiple representations •  Descriptive = a shared data object §  Value provided on demand §  Defined with functions that determine how and when to update it §  Value may be learned •  Although a descriptive has a single representation, many descriptives can represent the same world state •  Examples: X-O-blank empty/occupied lines on the board 8Background • FORR • Applications
  9. 9. Multiple ways to use knowledge •  Operationalization = how to use a data object •  Although a descriptive has a single representation, it can be operationalized in many ways Ways to reason about the empty/occupied squares Calculate possible actions Predict opponent's move Ways to reason about the lines Report a result Finish a winning line Block your opponent’s winning line Create a fork Plan a win on a specific line 9Background • FORR • Applications
  10. 10. An Advisor operationalizes descriptives •  Implements a class-independent, action-selection rationale •  Limitedly-rational (resource-limited) procedure •  Input: state of the world + descriptives + possible actions •  Output: comments whose strengths express intensity of support or opposition to individual actions (or sets of actions) •  Domain-specific < Advisor, action, strength> Advisor current state possible actions relevant descriptives 10Background • FORR • Applications
  11. 11. Often, Advisors disagree 11 O X X O Panic (prevent immediate loss) Worried (prevent long- range loss) Victory (win!) And rely on learned descriptives •  Good openings •  Endgame play •  Strategies … Background • FORR • Applications
  12. 12. More about Advisors •  Advisors have different properties §  Some are always right §  Some need more time to decide §  Some would like to make a sequence of decisions, not just one •  Comments are opinions from the perspective of the Advisor's rationale §  On a single action do x x is better than y don’t do z do x or y x is a 10, y is an 8, but z is a –3 §  On an (unordered or fully or partially ordered) set of actions do x and y do p and then q do p and then do q and r 12Background • FORR • Applications
  13. 13. FORR (FOr the Right Reasons) •  Premise: synergy among domain-specific rationales solves problems •  Descriptives isolate representation from reasoning •  Advisor hierarchy §  Tier 1: correct, quick, pre-sequenced §  Tier 2: reactive plan rationales §  Tier 3: voting among heuristics based on their comment strengths and learned weights 13 <AdvisorA, action2, 10> <AdvisorA, action4, 8> <AdvisorA, action7, 6> <AdvisorB, action2, 7> <AdvisorB, action3, 9> <AdvisorC, action1, 9> <AdvisorC, action2, 7> <AdvisorC, action3, 9> <AdvisorC, action7, 9> … Voting For Advisor i and action j argmax j diwicij∑ Background • FORR • Applications
  14. 14. The FORR decision cycle 14 take actionyes Tier 1: Reaction from perfect knowledge Victory T-11 T-1n… Decision? no Background • FORR • Applications state actions descriptives
  15. 15. The FORR decision cycle 15 take actionyes Tier 1: Reaction from perfect knowledge Victory T-11 T-1n… Decision? begin planyes Tier 2: Plans triggered by situation recognition no T-21 T-22 T-2m… Decision? Background • FORR • Applications state actions descriptives
  16. 16. The FORR decision cycle 16 take actionyes Tier 1: Reaction from perfect knowledge Victory T-11 T-1n… Decision? begin planyes no T-32T-31 T-3k… …Tier 3: Heuristic reactions Voting take action Tier 2: Plans triggered by situation recognition no T-21 T-22 T-2m… Decision? Background • FORR • Applications state actions descriptives
  17. 17. How to develop a problem solver 17 •  Specialize FORR with domain knowledge §  Problem classes §  Advisors §  Descriptives with learning methods •  To solve a class of problems robustly, FORR learns §  Descriptives’ values §  Rationales’ relative utility §  New Advisors §  How to reorganize tier 3 Domain knowledge FORR FORR-based problem solver Learned problem solver Problem class Experience WARNING: problem solving often provides noisy data Background • FORR • Applications
  18. 18. FORR-based single agents 18 •  Hoyle learned to play 19 two-person, perfect-information, finite-board games as well or better than human / machine expert [Epstein, 2001] •  Ariadne learned to navigate efficiently in grid worlds, despite perceptual limitations and no map [Epstein, 1995] •  ACE learned to solve constraint satisfaction problems and rediscovered the Brélaz heuristic [ Epstein & Freuder, 2005] •  SemaFORR: controls an autonomous search-and-rescue robot [Epstein, Schneider, Ozgelen, Munoz, Costantino, Sklar & Parsons, 2012] Background • FORR • Applications
  19. 19. Lessons learned 19 •  Reactive plans work well •  Elimination of inaccurate heuristics produces substantial speedup •  Lazy descriptive computation also provides speedup •  Self-awareness supports transparency •  Advisor weights may have problem-stage context •  Weight learning has subtle pitfalls (example extraction) •  Autonomous restructuring must balance accuracy against risk •  Sometimes it is more efficient not to reason at all Background • FORR • Applications
  20. 20. FORR-based collaborating agents 20 •  Co-FORR: 5 collaborating agents for 2D park design [Epstein, 1998] •  FORRSooth: learned to conduct a spoken dialogue with a library patron who orders books [Epstein, Passonneau, Gordon, & Ligorio, 2012] •  SemaFORR: controls autonomous search-and-rescue robot team [Epstein, Aroor, Evanusa, Sklar & Parsons, 2015] Each new domain poses new challenges whose solution strengthens FORR Background • FORR • Applications
  21. 21. FORR-based results 21 •  PhD theses §  Shih on learning multiple behavior sequences, 2000 §  Lock on learning multiple plans from behavior sequences, 2003 §  Petrovic on weight learning for multiple Advisors, 2008 §  Ligorio on learning to select attributes, 2011 §  Li on representation and exploitation of multiple complex relationships, 2011 §  Yun on parallelization of multiple solvers, 2013 §  Osisek on application of multiple relationships in recommendation (in progress) §  Aroor on reactive planning for multiple robots (in progress) •  Applications to bioinformatics (with Dr. Lei Xie) §  Protein-protein interaction networks §  Virtual drug screening Background • FORR • Applications
  22. 22. Take home message To develop expertise FORR learns to harness the synergy of multiplicity in representation and reasoning 22Background • FORR • Applications
  23. 23. Acknowledgements We gratefully acknowledge the support of The National Science Foundation CUNY’s High Performance Computing Center Continued thanks to my collaborators Gene Freuder Rebecca Passonneau Rick Wallace Lei Xie Elizabeth Sklar Simon Parsons and a host of undergraduate and graduate students with whom I continue to learn 23
  24. 24. Selected references •  Epstein, S. L. 2001. Learning to Play Expertly: A Tutorial on Hoyle. In Machine Learning in Game Playing •  Epstein, S. L. 1998. Pragmatic Navigation: Reactivity, Heuristics, and Search. Artificial Intelligence, 100 (1-2): 275-322. •  Epstein, S. L., E. C. Freuder and M. Wallace 2005. Learning to Support Constraint Programmers. Computational Intelligence 21(4): 337-371. •  Epstein, S. L., R. J. Passonneau, T. Ligorio and J. Gordon. 2012. Data Mining to Support Human-Machine Dialogue for Autonomous Agents. In Proceedings of Agents and Data Mining Interaction (ADMI2011). •  Epstein, S.L., Aroor, A., Evanusa, M., Sklar, E.I., Simon, S. 2015. Navigation with Learned Spatial Affordances. In Proceedings of CogSci 2015. 24