This document discusses machine learning techniques for planning, specifically macro learning. It defines macro learning as a process where a planner solves a problem and induces macros (compiled sequences of operators) from the solution. The planner can regress through states backwards to determine the weakest precondition of the operator sequence for achieving the goal. These macros can then be stored and reused to help solve future planning problems more efficiently by skipping search if the preconditions are met. However, learning too many macros can also increase the search space and time spent looking for applicable macros.

1. AI – Week 21 Machine Learning: Macro Learning Lee McCluskey, room 2/09 Email [email_address] http://scom.hud.ac.uk/scomtlm/cha2555/

2. Term 2: Summary 13 - Introduction to planning and learning 14 - Introduction to formulations/jargon of Planning 15 - Operator Schemas and state space search 16 - Planner Implementation: BreadthFS in Prolog 17 - Planner Implementation: BreadthFS, BestFS, Heuristics 18 - Graphplan Planning Alg 19 - Graphplan Planning Alg 20 – reading week 21 – Knowledge Acquisition + GIPO 22 - Learning example 1: Macro Learning  ****** we are here 23 - Learning example 2: GIPO's Opmaker 24 - Learning example 3: Information Extraction

3. Types of Learning Learning by ROTE (remember facts) - this is purely storing and remembering facts without integrating or recognising the meaning of the facts Learning by BEING TOLD (programmed) - this is storing and remembering facts / procedures, but implies some kind of understanding / integration of what is being told, with previous knowledge. Learning by EXAMPLE/ANALOGY (trained/taught) this invovles a benevolent teacher who gives classified examples to the leaner. The learner performs some generalisation the examples to infer new knowledge. Previous knowledge maybe used to steer the generalisations. In analogy the learner performs the generalisation based on some previously learnt situation.

4. Types of Learning Learning by OBSERVATION (self-taught) this is similar to the category above but without classification by teacher - the learner uses pre-learned information to help classify observation (eg conceptual clustering) Learning by DISCOVERY this is the highest level of learning covering invention etc and is composed of some of the other types below TWO ASPECTS OF LEARNING: KNOWLEDGE/SKILL ACQUISITION Inputting NEW knowledge KNOWLEDGE/SKILL REFINEMENT Changing/integrating old knowledge to create better (operational) knowledge (Inputs no or little new knowledge)

5. Recap: Knowledge Acquisition Knowledge Acquisition is the process of encoding knowledge in a way that intelligent processes can use effectively. ? Do we always need an AI expert to encode knowledge? ? Can we get programs to learn – or acquire knowledge for themselves ? Next week we will see how GIPO can be used to learn new planning operators

6. KNOWLEDGE/SKILL REFINEMENT Changing/integrating old knowledge to create better (operational) knowledge (Inputs no or little new knowledge) Examples: Learning heuristics (improve search) Re-representing knowledge (improve search space) Learning procedures (remove search altogether!) Automatically removing bugs in representations

7. KNOWLEDGE/SKILL REFINEMENT: MACRO ACQUISITION AND USE FOR AI PLANNING Roughly: a planner solves a problem and induces one or more macros from the solution sequence by “compiling” the operator sequence into one macro. Definition: WP (weakest precondition) of operator O to achieve goal(s) G WP(O,G) = Elements of G that O does not achieve UNION O’s preconditions Learning task: Find a solution T = (o(1),..,o(N)) to goal G from initial state s(0) Form a Macro- Operator (macro) based on: Pre-condition: WP (T, G) Post-condition: G

8. Macro acquisition: algorithm Starting with goal G, we regress through the states backwards: Assume we have the last operator o(N) applied to s(N-1) to form the final state s(N), with add list o(N).add and precondition o(N).pre Then regressing G through o(N) gives NewG = WP(o(N),T) = G – o(N).add UNION with o(N).pre NewG can now be regressed further until the initial state is reached. The full regressed goal is the weakest precondition that T achieves the goal G.

9. Macro Use The triple (G, WP(T,G), T) can be stored and retrieved: Given a planning problem (S, G1), IF S => WP(T,G) and G => G1 then achieve G1 by applying sequence T to S. The stored‘ macro (G, WP(T,G), T) can be further generalised by changing the constants to variables ranging through all objects of the same sort as the original constants. In OCL for example, as all objects of the same sort share the same behaviour, this generalisation has some justification The macro could increase future performance as it may cut out the need to search for a solution to G.

10. Macro (+Learning) Utility There are other kinds of macro creation: for a solution of size N, N -1 macros can be created as each of the regressed goals can form a macro. This may cause utility problems , however. Too many or too general macros may Increase the search space Increase the time of searching as the planner spends time looking for macros

11. Conclusion There are various kinds of Learning manifest in nature and in AI Two important roles for Learning are in Knowledge Acquisition and Knowledge Refinement Macro acquisition is a form of KR where procedures are learned to make plan generation more efficient Sometimes, Learned information can degrade performance