Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanfo...
Definition of a Machine Learning System that improves task performance by acquiring knowledge based on partial task experi...
Elements of a Learning System experience/ environment knowledge learning method performance element
Elements of Classification Learning observed examples category descriptions learning mechanism classification mechanism
Five Paradigms for Classification Learning Rule Induction Decision-Tree Induction Case-Based Learning Neural Networks Prob...
 
 
 
 
 
 
 
 
Category Learning in Humans <ul><li>graded and imprecise nature of concepts </li></ul><ul><li>categories stored at differe...
 
Learning for Problem Solving problem-solving experience problem-solving knowledge learning mechanism problem solver
Operators for the Blocks World (pickup (?x) (on ?x ?t) (table ?t) (clear ?x) (arm-empty)  =>  (<add>  (holding ?x)) (<dele...
State Space for the Blocks World
Inducing Search-Control Knowledge <ul><li>One approach to learning for problem solving induces search-control rules from s...
Induction from Solution Paths <ul><li>For each operator O, </li></ul><ul><li>For each solution path P, </li></ul><ul><ul><...
Labeled Operators on a Solution Path
Search-Control Rules for the Blocks World ((holding ?x) (table ?t)  (goal (on ?x ?y)) (<not> (clear ?y))  => (putdown ?x))...
Learning for Means-Ends Analysis <ul><li>Some problem-solving systems use means-ends analysis, which:  </li></ul><ul><li>s...
A Means-Ends Problem-Solving Trace
Forming Macro-Operators <ul><li>An alternative approach to learning for problem solving constructs macro-operators from so...
Partitioning a Solution into Macro-Operators
Human Learning and Problem Solving <ul><li>reduced search with increased experience </li></ul><ul><li>reduced access to in...
Cognitive Architectures and Learning <ul><li>specifies the infrastructure that holds constant over domains, as opposed to ...
Selected References Billman, D., Fisher, D., Gluck, M., Langley, P., & Pazzani, M. (1990). Computational models of categor...
End of Presentation
Upcoming SlideShare
Loading in …5
×

mlrev.ppt

373 views

Published on

  • Be the first to comment

  • Be the first to like this

mlrev.ppt

  1. 1. Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, CA http://cll.stanford.edu/~langley/ May 28, 2004 Computational Learning for Classification and Problem Solving
  2. 2. Definition of a Machine Learning System that improves task performance by acquiring knowledge based on partial task experience a software artifact
  3. 3. Elements of a Learning System experience/ environment knowledge learning method performance element
  4. 4. Elements of Classification Learning observed examples category descriptions learning mechanism classification mechanism
  5. 5. Five Paradigms for Classification Learning Rule Induction Decision-Tree Induction Case-Based Learning Neural Networks Probabilistic Learning
  6. 14. Category Learning in Humans <ul><li>graded and imprecise nature of concepts </li></ul><ul><li>categories stored at different levels of generality </li></ul><ul><li>incremental processing of experience </li></ul><ul><li>effects of training order on learned knowledge </li></ul><ul><li>ability to learn from unlabeled cases </li></ul><ul><li>particular rates of category acquisition </li></ul>Human categorization exhibits clear characteristics: Many approaches to computational category learning ignore these phenomena.
  7. 16. Learning for Problem Solving problem-solving experience problem-solving knowledge learning mechanism problem solver
  8. 17. Operators for the Blocks World (pickup (?x) (on ?x ?t) (table ?t) (clear ?x) (arm-empty) => (<add> (holding ?x)) (<delete> (on ?x ?t) (clear ?x) (arm-empty))) (unstack (?x ?y) (on ?x ?y) (block ?y) (clear ?x) (arm-empty) => (<add> (holding ?x) (clear ?y)) (<delete> (on ?x ?y) (clear ?x) (arm-empty))) (putdown (?x) (holding ?x) (table ?t) => (<add> (on ?x ?t) (arm-empty)) (<delete> (holding ?x))) (stack (?x ?y) (holding ?x) (block ?y) (clear ?y) (  ?x ?y) => (<add> (on ?x ?y) (arm-empty)) (<delete> (holding ?x) (clear ?y))) Most formulations of the blocks world assume four operators: Each operator has a name, arguments, preconditions, an add list, and a delete list.
  9. 18. State Space for the Blocks World
  10. 19. Inducing Search-Control Knowledge <ul><li>One approach to learning for problem solving induces search-control rules from solution paths. </li></ul><ul><li>Given: A set of (possibly opaque) operators </li></ul><ul><li>Given: A test for achievement of goal states </li></ul><ul><li>Given: Solution paths that lead to goal states </li></ul><ul><li>Acquire: Control knowledge that improves problem solving </li></ul><ul><li>This scheme involves recasting the task in terms of supervised concept learning. </li></ul><ul><li>The resulting control rules reduce the effective branching factor during search. </li></ul>
  11. 20. Induction from Solution Paths <ul><li>For each operator O, </li></ul><ul><li>For each solution path P, </li></ul><ul><ul><li>Mark all cases of operator O on P as positive instances </li></ul></ul><ul><ul><li>Mark all cases of operator O one step off P as negative instances </li></ul></ul><ul><li>Induce rules that cover positive but not negative instances of O. </li></ul><ul><li>One can use any supervised concept learning method to this end, including ones that rely on non-logical formalisms. </li></ul><ul><li>However, many problem-solving domains require induction over relational descriptions. </li></ul>
  12. 21. Labeled Operators on a Solution Path
  13. 22. Search-Control Rules for the Blocks World ((holding ?x) (table ?t) (goal (on ?x ?y)) (<not> (clear ?y)) => (putdown ?x)) ((holding ?x) (table ?t) (goal (on ?y ?x)) (goal (on ?z ?y)) => (putdown ?x)) ((holding ?x) (block ?y) (clear ?y) (  ?x ?y) (goal (on ?x ?y)) (on ?y ?z) (goal (on ?y ?z)) => (stack ?x ?y)) ((on ?x ?y) (block ?y) (clear ?x) (arm-empty) (on ?y ?z) (<not> (goal (on ?y ?z)) => (unstack ?x ?y)) ((on ?x ?y) (block ?y) (clear ?x) (arm-empty) (<not> (goal (on ?x ?y)) => (unstack ?x ?y)) Quinlan’s FOIL system induces a number of selection rules: Note that these rules are sensitive to the description of the goal.
  14. 23. Learning for Means-Ends Analysis <ul><li>Some problem-solving systems use means-ends analysis, which: </li></ul><ul><li>selects operators whose preconditions are not yet satisfied; </li></ul><ul><li>divides problems recursively into simpler subproblems. </li></ul><ul><li>Similar techniques can learn control knowledge for this paradigm. </li></ul><ul><li>Means-ends analysis has advantages over state-space search, in that it can transfer knowledge about solved subproblems. </li></ul><ul><li>Much of the work on learning in planning systems has relied on this approach. </li></ul>
  15. 24. A Means-Ends Problem-Solving Trace
  16. 25. Forming Macro-Operators <ul><li>An alternative approach to learning for problem solving constructs macro-operators from solution paths. </li></ul><ul><li>Given: A set of transparent operators for a domain </li></ul><ul><li>Given: Solution paths that lead to goal states </li></ul><ul><li>Acquire: Sequences of operators that improve problem solving </li></ul><ul><li>This scheme involves logically composing the conditions and effects of operators. </li></ul><ul><li>The resulting macro-operators reduce the effective depth of search required to find a solution. </li></ul>
  17. 26. Partitioning a Solution into Macro-Operators
  18. 27. Human Learning and Problem Solving <ul><li>reduced search with increased experience </li></ul><ul><li>reduced access to intermediate results </li></ul><ul><li>automatization and Einstellung effects </li></ul><ul><li>asymmetric transfer of expertise </li></ul><ul><li>incremental learning at particular rates </li></ul>Human learning in problem-solving domains exhibits: Computational methods for learning in problem solving address some but not all of these phenomena.
  19. 28. Cognitive Architectures and Learning <ul><li>specifies the infrastructure that holds constant over domains, as opposed to knowledge, which can vary. </li></ul><ul><li>commits to representations and organizations of knowledge in long-term and short-term memory; </li></ul><ul><li>commits to performance processes that operate on these mental structures and learning mechanisms that generate them; </li></ul><ul><li>comes with a programming language for encoding knowledge and constructing intelligent systems. </li></ul>Many computational psychological models are cast within some theory of the human cognitive architecture that: Most architectures (e.g., ACT, Soar, I CARUS ) use rules or similar formalisms and focus on multi-step reasoning or problem solving.
  20. 29. Selected References Billman, D., Fisher, D., Gluck, M., Langley, P., & Pazzani, M. (1990). Computational models of category learning. Proceedings of the Thirteenth Annual Conference of the Cognitive Science Society (pp. 989-996). Cambridge, MA: Lawrence Erlbaum. Langley, P. (1995). Elements of machine learning . San Francisco: Morgan Kaufmann. Shavlik, J. W., & Dietterich, T. G. (Eds.). (1990). Readings in machine learning . San Francisco: Morgan Kaufmann. VanLehn, K. (1989). Problem solving and cognitive skill acquisition. In M. I. Posner (Ed.), Foundations of cognitive science . Cambridge, MA: MIT Press.
  21. 30. End of Presentation

×