Your SlideShare is downloading. ×
0
A Brief Survey of Machine Learning <ul><li>Used materials from: </li></ul><ul><li>William H. Hsu   </li></ul><ul><li>Linda...
ML Lectures Outline: what we will discuss? <ul><li>Why machine learning? </li></ul><ul><li>Brief Tour of Machine Learning ...
What is Learning? <ul><li>“ Learning denotes changes in a system that ... enable a system to do the same task more efficie...
Why Machine Learning? What ML can do? <ul><li>Discovers  new things  or structures that are  unknown to humans </li></ul><...
Why Machine Learning? <ul><li>New Computational Capability </li></ul><ul><ul><li>Database mining:   </li></ul></ul><ul><ul...
Why Machine Learning? <ul><li>Better Understanding of Human Learning and Teaching </li></ul><ul><ul><li>Understand and imp...
A General Model of Learning Agents
Three Aspects of Learning Systems <ul><ul><li>1 . Models:   </li></ul></ul><ul><ul><ul><li>decision trees,  </li></ul></ul...
What are the aspects of research on Learning? <ul><li>1.  Theory of Learning </li></ul><ul><ul><li>Computational learning ...
Some Issues in Machine Learning <ul><li>What  Algorithms Can Approximate Functions Well? When ? </li></ul><ul><li>How  Do ...
Major Paradigms of Machine Learning <ul><li>Rote Learning   </li></ul><ul><ul><li>One-to-one mapping from inputs to stored...
Major Paradigms of Machine Learning <ul><li>Neural Networks </li></ul><ul><li>Reinforcement  </li></ul><ul><ul><li>Feedbac...
The Inductive Learning Problem <ul><li>Induce rules  that  extrapolate   from a given set of examples </li></ul><ul><ul><l...
The Inductive Learning Problem <ul><li>Concept learning   </li></ul><ul><ul><li>Called also  Classification </li></ul></ul...
Supervised  Concept Learning <ul><li>Given a  training set  of  positive and negative examples  of a concept </li></ul><ul...
Inductive Learning  Framework <ul><li>Raw input data from sensors are preprocessed to obtain a  feature vector , X, that a...
Inductive Learning by  Nearest-Neighbor  Classification <ul><li>One simple approach to inductive learning is to  save each...
Learning  Decision Trees <ul><li>Goal:  Build a  decision tree  for classifying examples as positive or negative instances...
Preference Bias:  Ockham's Razor <ul><li>Aka  Occam’s Razor , Law of Economy, or  Law of Parsimony </li></ul><ul><li>Princ...
Inductive Learning and Bias <ul><li>Suppose that we want to learn a function f(x) = y and we are given  some sample (x,y) ...
Example of using probabilities to create trees:  Huffman code <ul><li>In 1952 MIT student David Huffman devised, in the co...
Huffman code example as a  prototypical idea  from  other area <ul><li>Message  Probability. </li></ul><ul><li>A .125 </li...
<ul><li>If a  set T of records  is partitioned into disjoint exhaustive classes (C1,C2,..,Ck) on the basis of the value of...
Gain <ul><li>Consider the quantity Gain(X,T) defined as </li></ul><ul><li>Gain(X,T) = Info(T) - Info(X,T) </li></ul><ul><l...
Rule and Decision Tree Learning <ul><li>Example: Rule Acquisition from Historical Data </li></ul><ul><li>Data </li></ul><u...
Neural Network Learning <ul><li>A utonomous  L earning  V ehicle  I n a  N eural  N et (ALVINN):  Pomerleau  et al </li></...
Specifying A Learning Problem <ul><li>Learning = Improving with Experience at Some Task </li></ul><ul><ul><li>Improve over...
Example : Learning to Play Checkers <ul><li>Type of Training Experience </li></ul><ul><ul><li>Direct or indirect? </li></u...
A Target Function for Learning to Play Checkers <ul><li>Possible Definition </li></ul><ul><ul><li>If  b  is a final board ...
A Training Procedure for  Learning to Play Checkers <ul><li>Obtaining Training Examples </li></ul><ul><ul><li>the target f...
Design Choices for Learning to Play Checkers Completed Design Determine Type of Training Experience Games against experts ...
Example of Interesting Application: Data Mining Database Mining NCSA  D2K  - http://www.ncsa.uiuc.edu/STI/ALG
Example: Reasoning (Inference, Decision Support) Cartia  ThemeScapes -  http://www.cartia.com 6500 news stories from the W...
Example: Planning and Control Normal Ignited Engulfed Destroyed Extinguished Fire Alarm Flooding DC-ARM  - http://www-kbs....
Relevant Disciplines <ul><li>Artificial Intelligence </li></ul><ul><li>Bayesian Methods </li></ul><ul><li>Cognitive Scienc...
Upcoming SlideShare
Loading in...5
×

Introduction to Machine Learning.

574

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
574
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Introduction to Machine Learning."

  1. 1. A Brief Survey of Machine Learning <ul><li>Used materials from: </li></ul><ul><li>William H. Hsu </li></ul><ul><li>Linda Jackson </li></ul><ul><li>Lex Lane </li></ul><ul><li>Tom Mitchell </li></ul><ul><ul><ul><ul><li>Machine Learning, Mc Graw Hill 1997 </li></ul></ul></ul></ul><ul><ul><ul><li>Allan Moser </li></ul></ul></ul><ul><ul><ul><li>Tim Finin, </li></ul></ul></ul><ul><ul><ul><li>Marie desJardins </li></ul></ul></ul><ul><ul><ul><li>Chuck Dyer </li></ul></ul></ul>
  2. 2. ML Lectures Outline: what we will discuss? <ul><li>Why machine learning? </li></ul><ul><li>Brief Tour of Machine Learning </li></ul><ul><ul><li>A case study </li></ul></ul><ul><ul><li>A taxonomy of learning </li></ul></ul><ul><ul><li>Intelligent systems engineering: specification of learning problems </li></ul></ul><ul><li>Issues in Machine Learning </li></ul><ul><ul><li>Design choices </li></ul></ul><ul><ul><li>The performance element: intelligent systems </li></ul></ul><ul><li>Some Applications of Learning </li></ul><ul><ul><li>Database mining , reasoning (inference/decision support), acting </li></ul></ul><ul><ul><li>Industrial usage of intelligent systems </li></ul></ul><ul><ul><li>Robotics </li></ul></ul>
  3. 3. What is Learning? <ul><li>“ Learning denotes changes in a system that ... enable a system to do the same task more efficiently the next time.” -- Herbert Simon </li></ul><ul><li>“ Learning is constructing or modifying representations of what is being experienced.” -- Ryszard Michalski </li></ul><ul><li>“ Learning is making useful changes in our minds.” -- Marvin Minsky </li></ul>definitions
  4. 4. Why Machine Learning? What ML can do? <ul><li>Discovers new things or structures that are unknown to humans </li></ul><ul><ul><li>Examples: </li></ul></ul><ul><ul><ul><li>Data mining, </li></ul></ul></ul><ul><ul><ul><li>Knowledge Discovery in Databases </li></ul></ul></ul><ul><li>Fills in skeletal or incomplete specifications about a domain </li></ul><ul><ul><li>Large, complex AI systems cannot be completely derived by hand </li></ul></ul><ul><ul><li>They require dynamic updating to incorporate new information. </li></ul></ul><ul><ul><li>Learning new characteristics : </li></ul></ul><ul><ul><ul><li>1. expands the domain or expertise </li></ul></ul></ul><ul><ul><ul><li>2. lessens the &quot;brittleness&quot; of the system </li></ul></ul></ul><ul><li>Using learning, the software agents can adapt to : </li></ul><ul><ul><li>to their users, </li></ul></ul><ul><ul><li>to other software agents, </li></ul></ul><ul><ul><li>to the changing environment. </li></ul></ul>
  5. 5. Why Machine Learning? <ul><li>New Computational Capability </li></ul><ul><ul><li>Database mining: </li></ul></ul><ul><ul><ul><li>converting (technical) records into knowledge </li></ul></ul></ul><ul><ul><li>Self-customizing programs: </li></ul></ul><ul><ul><ul><li>learning news filters, </li></ul></ul></ul><ul><ul><ul><li>adaptive monitors </li></ul></ul></ul><ul><ul><li>Learning to act: </li></ul></ul><ul><ul><ul><li>robot planning, </li></ul></ul></ul><ul><ul><ul><li>control optimization, </li></ul></ul></ul><ul><ul><ul><li>decision support </li></ul></ul></ul><ul><ul><li>Applications that are hard to program: </li></ul></ul><ul><ul><ul><li>automated driving, </li></ul></ul></ul><ul><ul><ul><li>speech recognition </li></ul></ul></ul>
  6. 6. Why Machine Learning? <ul><li>Better Understanding of Human Learning and Teaching </li></ul><ul><ul><li>Understand and improve efficiency of human learning </li></ul></ul><ul><ul><li>Use to improve methods for teaching and tutoring people </li></ul></ul><ul><ul><ul><li>e.g., better computer-aided instruction. ( can our robot-head teach English?) </li></ul></ul></ul><ul><ul><li>Cognitive science: theories of knowledge acquisition (e.g., through practice) </li></ul></ul><ul><ul><li>Performance elements : reasoning (inference) and recommender systems </li></ul></ul><ul><li>Time is Right </li></ul><ul><ul><li>Recent progress in algorithms and theory </li></ul></ul><ul><ul><li>Rapidly growing volume of online data from various sources </li></ul></ul><ul><ul><li>Available computational power </li></ul></ul><ul><ul><li>Growth and interest of learning-based industries (e.g., data mining/KDD) </li></ul></ul>
  7. 7. A General Model of Learning Agents
  8. 8. Three Aspects of Learning Systems <ul><ul><li>1 . Models: </li></ul></ul><ul><ul><ul><li>decision trees, </li></ul></ul></ul><ul><ul><ul><li>linear threshold units (winnow, weighted majority), </li></ul></ul></ul><ul><ul><ul><li>neural networks, </li></ul></ul></ul><ul><ul><ul><li>Bayesian networks (polytrees, belief networks, influence diagrams, HMMs), </li></ul></ul></ul><ul><ul><ul><li>genetic algorithms, </li></ul></ul></ul><ul><ul><ul><li>instance-based (nearest-neighbor) </li></ul></ul></ul><ul><ul><li>2. Algorithms (e.g., for decision trees): </li></ul></ul><ul><ul><ul><li>ID3, </li></ul></ul></ul><ul><ul><ul><li>C4.5, </li></ul></ul></ul><ul><ul><ul><li>CART, </li></ul></ul></ul><ul><ul><ul><li>OC1 </li></ul></ul></ul><ul><ul><li>3. Methodologies: </li></ul></ul><ul><ul><ul><li>supervised, </li></ul></ul></ul><ul><ul><ul><li>unsupervised, </li></ul></ul></ul><ul><ul><ul><li>reinforcement; </li></ul></ul></ul><ul><ul><ul><li>knowledge-guided </li></ul></ul></ul>
  9. 9. What are the aspects of research on Learning? <ul><li>1. Theory of Learning </li></ul><ul><ul><li>Computational learning theory (COLT): complexity, limitations of learning </li></ul></ul><ul><ul><li>Probably Approximately Correct (PAC) learning </li></ul></ul><ul><ul><li>Probabilistic, statistical, information theoretic results </li></ul></ul><ul><li>2. Multistrategy Learning: </li></ul><ul><ul><li>Combining Techniques, </li></ul></ul><ul><ul><li>Knowledge Sources </li></ul></ul><ul><li>3. Create and collect Data: </li></ul><ul><ul><li>Time Series, </li></ul></ul><ul><ul><li>Very Large Databases (VLDB), </li></ul></ul><ul><ul><li>Text Corpora </li></ul></ul><ul><li>4. Select good applications </li></ul><ul><ul><li>Performance element: </li></ul></ul><ul><ul><ul><li>classification, </li></ul></ul></ul><ul><ul><ul><li>decision support, </li></ul></ul></ul><ul><ul><ul><li>planning, </li></ul></ul></ul><ul><ul><ul><li>control </li></ul></ul></ul><ul><ul><li>Database mining and knowledge discovery in databases (KDD) </li></ul></ul><ul><ul><li>Computer inference: learning to reason </li></ul></ul>
  10. 10. Some Issues in Machine Learning <ul><li>What Algorithms Can Approximate Functions Well? When ? </li></ul><ul><li>How Do Learning System Design Factors Influence Accuracy? </li></ul><ul><ul><li>Number of training examples </li></ul></ul><ul><ul><li>Complexity of hypothesis representation </li></ul></ul><ul><li>How Do Learning Problem Characteristics Influence Accuracy? </li></ul><ul><ul><li>Noisy data </li></ul></ul><ul><ul><li>Multiple data sources </li></ul></ul><ul><li>What Are The Theoretical Limits of Learnability? </li></ul><ul><li>How Can Prior Knowledge of Learner Help? </li></ul><ul><li>What Clues Can We Get From Biological Learning Systems? </li></ul><ul><li>How Can Systems Alter Their Own Representation? </li></ul>
  11. 11. Major Paradigms of Machine Learning <ul><li>Rote Learning </li></ul><ul><ul><li>One-to-one mapping from inputs to stored representation. </li></ul></ul><ul><ul><li>&quot;Learning by memorization.” </li></ul></ul><ul><ul><li>Association-based storage and retrieval. </li></ul></ul><ul><li>Clustering </li></ul><ul><li>Analogue </li></ul><ul><ul><li>Determine correspondence between two different representations </li></ul></ul><ul><li>Induction </li></ul><ul><ul><li>Use specific examples to reach general conclusions </li></ul></ul><ul><li>Discovery </li></ul><ul><ul><li>Unsupervised, specific goal not given </li></ul></ul><ul><li>Genetic Algorithms </li></ul>
  12. 12. Major Paradigms of Machine Learning <ul><li>Neural Networks </li></ul><ul><li>Reinforcement </li></ul><ul><ul><li>Feedback given at end of a sequence of steps. </li></ul></ul><ul><ul><li>Feedback can be positive or negative reward </li></ul></ul><ul><ul><ul><li>Assign reward to steps by solving the credit assignment problem — </li></ul></ul></ul><ul><ul><ul><ul><li>which steps should receive credit or blame for a final result? </li></ul></ul></ul></ul>
  13. 13. The Inductive Learning Problem <ul><li>Induce rules that extrapolate from a given set of examples </li></ul><ul><ul><li>These rules should make “accurate” predictions about future examples. </li></ul></ul><ul><li>Supervised versus Unsupervised learning </li></ul><ul><ul><li>Learn an unknown function f(X) = Y, where: </li></ul></ul><ul><ul><ul><li>X is an input example and </li></ul></ul></ul><ul><ul><ul><li>Y is the desired output . </li></ul></ul></ul><ul><ul><li>Supervised learning implies we are given a training set of (X, Y) pairs by a &quot;teacher.“ </li></ul></ul><ul><ul><li>Unsupervised learning means we are only given the Xs and some (ultimate) feedback function on our performance. </li></ul></ul>
  14. 14. The Inductive Learning Problem <ul><li>Concept learning </li></ul><ul><ul><li>Called also Classification </li></ul></ul><ul><ul><li>Given a set of examples of some concept/class/category, determine if a given example is an instance of the concept or not. </li></ul></ul><ul><ul><li>If it is an instance, we call it a positive example . </li></ul></ul><ul><ul><li>If it is not, it is called a negative example . </li></ul></ul>
  15. 15. Supervised Concept Learning <ul><li>Given a training set of positive and negative examples of a concept </li></ul><ul><ul><li>Usually each example has a set of features/attributes </li></ul></ul><ul><li>Construct a description that will accurately classify whether future examples are positive or negative. </li></ul><ul><li>That is, </li></ul><ul><ul><li>learn some good estimate of function f </li></ul></ul><ul><ul><li>given a training set {(x1, y1), (x2, y2), ..., (xn, yn)} </li></ul></ul><ul><ul><li>where each yi is either + (positive) or - (negative). </li></ul></ul><ul><ul><li>f is a function of the features/attributes </li></ul></ul>
  16. 16. Inductive Learning Framework <ul><li>Raw input data from sensors are preprocessed to obtain a feature vector , X, that adequately describes all of the relevant features for classifying examples. </li></ul><ul><li>Each x is a list of (attribute, value) pairs. For example, </li></ul><ul><ul><li>X = [Person:Sue, EyeColor:Brown, Age:Young, Sex:Female] </li></ul></ul><ul><li>The number and names of attributes (aka features ) is fixed (positive, finite). </li></ul><ul><li>Each attribute has a fixed, finite number of possible values. </li></ul><ul><li>Each example can be interpreted as a point in an n-dimensional feature space , where n is the number of attributes. </li></ul>
  17. 17. Inductive Learning by Nearest-Neighbor Classification <ul><li>One simple approach to inductive learning is to save each training example as a point in feature space </li></ul><ul><li>Classify a new example by giving it the same classification (+ or -) as its nearest neighbor in Feature Space. </li></ul><ul><ul><li>1. A variation involves computing a weighted sum of class of a set of neighbors </li></ul></ul><ul><ul><ul><li>where the weights correspond to distances </li></ul></ul></ul><ul><ul><li>2. Another variation uses the center of class </li></ul></ul><ul><li>The problem with this approach is that it doesn't necessarily generalize well if the examples are not well &quot;clustered.&quot; </li></ul>
  18. 18. Learning Decision Trees <ul><li>Goal: Build a decision tree for classifying examples as positive or negative instances of a concept using supervised learning from a training set. </li></ul><ul><li>A decision tree is a tree where </li></ul><ul><ul><li>each non-leaf node is associated with an attribute (feature) </li></ul></ul><ul><ul><li>each leaf node is associated with a classification (+ or -) </li></ul></ul><ul><ul><li>each arc is associated with one of the possible values of the attribute at the node where the arc is directed from. </li></ul></ul><ul><li>Generalization: allow for >2 classes </li></ul><ul><ul><li>e.g., {sell, hold, buy} </li></ul></ul>Color Shape Size + + - Size + - + big big small small round square red green blue
  19. 19. Preference Bias: Ockham's Razor <ul><li>Aka Occam’s Razor , Law of Economy, or Law of Parsimony </li></ul><ul><li>Principle stated by William of Ockham (1285-1347/49), a scholastic, that </li></ul><ul><ul><li>“ non sunt multiplicanda entia praeter necessitatem” </li></ul></ul><ul><ul><li>or, entities are not to be multiplied beyond necessity. </li></ul></ul><ul><li>The simplest explanation that is consistent with all observations is the best. </li></ul><ul><li>Therefore, the smallest decision tree that correctly classifies all of the training examples is best. </li></ul><ul><li>Finding the provably smallest decision tree is NP-Hard </li></ul><ul><li>Therefore we do not construct the absolute smallest tree consistent with the training examples. </li></ul><ul><li>We construct a tree that is pretty small . </li></ul>
  20. 20. Inductive Learning and Bias <ul><li>Suppose that we want to learn a function f(x) = y and we are given some sample (x,y) pairs , as in figure (a). </li></ul><ul><li>There are several hypotheses we could make about this function, e.g.: (b), (c) and (d). </li></ul><ul><li>A preference for one over the others reveals the bias of our learning technique, e.g.: </li></ul><ul><ul><li>prefer piece-wise functions </li></ul></ul><ul><ul><li>prefer a smooth function </li></ul></ul><ul><ul><li>prefer a simple function and treat outliers as noise </li></ul></ul>
  21. 21. Example of using probabilities to create trees: Huffman code <ul><li>In 1952 MIT student David Huffman devised, in the course of doing a homework assignment , an elegant coding scheme </li></ul><ul><li>This scheme is optimal in the case where all symbols’ probabilities are integral powers of 1/2. </li></ul><ul><li>A Huffman code can be built in the following manner: </li></ul><ul><ul><li>1. Rank all symbols in order of probability of occurrence. </li></ul></ul><ul><ul><li>2. Successively combine the two symbols of the lowest probability to form a new composite symbol; </li></ul></ul><ul><ul><ul><li>eventually we will build a binary tree where each node is the probability of all nodes beneath it. </li></ul></ul></ul><ul><ul><li>3. Trace a path to each leaf, noticing the direction at each node. </li></ul></ul>
  22. 22. Huffman code example as a prototypical idea from other area <ul><li>Message Probability. </li></ul><ul><li>A .125 </li></ul><ul><li>B .125 </li></ul><ul><li>C .25 </li></ul><ul><li>D .5 </li></ul>If we need to send many messages (A,B,C or D) and they have this probability distribution and we use this code , then over time, the average bits/message should approach 1.75 (= 0.125*3+0.125*3+0.25*2*0.5*1) .5 .5 1 .125 .125 .25 A C B D .25 0 1 0 0 1 1
  23. 23. <ul><li>If a set T of records is partitioned into disjoint exhaustive classes (C1,C2,..,Ck) on the basis of the value of the categorical attribute, then the information needed to identify the class of an element of T is </li></ul><ul><ul><li>Info(T) = I(P) </li></ul></ul><ul><li> where P is probability distribution of partition (C1,C2,..,Ck): </li></ul><ul><ul><li>P = (|C1|/|T|, |C2|/|T|, ..., |Ck|/|T|) </li></ul></ul><ul><li>If we partition T w.r.t attribute X into sets {T1,T2, ..,Tn} then the information needed to identify the class of an element of T becomes the weighted average of the information needed to identify the class of an element of Ti, </li></ul><ul><ul><li>i.e. the weighted average of Info(Ti): </li></ul></ul><ul><ul><li>Info(X,T) =  |Ti|/|T| * Info(Ti) =  |Ti|/|T| * log |Ti|/|T| </li></ul></ul>
  24. 24. Gain <ul><li>Consider the quantity Gain(X,T) defined as </li></ul><ul><li>Gain(X,T) = Info(T) - Info(X,T) </li></ul><ul><li>This represents the difference between </li></ul><ul><ul><li>information needed to identify an element of T and </li></ul></ul><ul><ul><li>information needed to identify an element of T after the value of attribute X has been obtained, </li></ul></ul><ul><li> that is, this is the gain in information due to attribute X. </li></ul><ul><li>We can use this to rank attributes and to build decision trees where at each node is located the attribute with greatest gain among the attributes not yet considered in the path from the root. </li></ul><ul><li>The intents of this ordering are twofold: </li></ul><ul><ul><li>1. To create small decision trees so that records can be identified after only a few questions. </li></ul></ul><ul><ul><li>2. To match a hoped for minimality of the process represented by the records being considered (Occam's Razor). </li></ul></ul>We will use this idea to build decision trees, ID3
  25. 25. Rule and Decision Tree Learning <ul><li>Example: Rule Acquisition from Historical Data </li></ul><ul><li>Data </li></ul><ul><ul><li>Patient 103 (time = 1): Age 23, First-Pregnancy: no, Anemia: no, Diabetes: no, Previous-Premature-Birth: no, Ultrasound: unknown , Elective C-Section: unknown , Emergency-C-Section: unknown </li></ul></ul><ul><ul><li>Patient 103 (time = 2): Age 23, First-Pregnancy: no, Anemia: no, Diabetes: yes , Previous-Premature-Birth: no, Ultrasound: abnormal , Elective C-Section: no, Emergency-C-Section: unknown </li></ul></ul><ul><ul><li>Patient 103 (time = n): Age 23, First-Pregnancy: no, Anemia: no, Diabetes: no, Previous-Premature-Birth: no, Ultrasound: unknown , Elective C-Section: no, Emergency-C-Section: YES </li></ul></ul><ul><li>Learned Rule </li></ul><ul><ul><li>IF no previous vaginal delivery , AND abnormal 2nd trimester ultrasound , AND malpresentation at admission , AND no elective C-Section THEN probability of emergency C-Section is 0.6 </li></ul></ul><ul><ul><li>Training set: 26/41 = 0.634 </li></ul></ul><ul><ul><li>Test set: 12/20 = 0.600 </li></ul></ul>
  26. 26. Neural Network Learning <ul><li>A utonomous L earning V ehicle I n a N eural N et (ALVINN): Pomerleau et al </li></ul><ul><ul><li>http://www.cs.cmu.edu/afs/cs/project/alv/member/www/projects/ALVINN.html </li></ul></ul><ul><ul><li>Drives 70mph on highways </li></ul></ul>
  27. 27. Specifying A Learning Problem <ul><li>Learning = Improving with Experience at Some Task </li></ul><ul><ul><li>Improve over task T , </li></ul></ul><ul><ul><li>with respect to performance measure P , </li></ul></ul><ul><ul><li>based on experience E . </li></ul></ul><ul><li>Example: Learning to Play Checkers </li></ul><ul><ul><li>T : play games of checkers </li></ul></ul><ul><ul><li>P : percent of games won in world tournament </li></ul></ul><ul><ul><li>E : opportunity to play against self </li></ul></ul><ul><li>Refining the Problem Specification: Issues </li></ul><ul><ul><li>What experience? </li></ul></ul><ul><ul><li>What exactly should be learned? </li></ul></ul><ul><ul><li>How shall it be represented ? </li></ul></ul><ul><ul><li>What specific algorithm to learn it? </li></ul></ul><ul><li>Defining the Problem Milieu </li></ul><ul><ul><li>Performance element: </li></ul></ul><ul><ul><ul><li>How shall the results of learning be applied? </li></ul></ul></ul><ul><ul><li>How shall the performance element be evaluated? The learning system? </li></ul></ul>
  28. 28. Example : Learning to Play Checkers <ul><li>Type of Training Experience </li></ul><ul><ul><li>Direct or indirect? </li></ul></ul><ul><ul><li>Teacher or not? </li></ul></ul><ul><ul><li>Knowledge about the game (e.g., openings/endgames)? </li></ul></ul><ul><li>Problem: Is Training Experience Representative (of Performance Goal)? </li></ul><ul><li>Software Design </li></ul><ul><ul><li>Assumptions of the learning system: legal move generator exists </li></ul></ul><ul><ul><li>Software requirements: </li></ul></ul><ul><ul><ul><li>generator, </li></ul></ul></ul><ul><ul><ul><li>evaluator(s), </li></ul></ul></ul><ul><ul><ul><li>parametric target function </li></ul></ul></ul><ul><li>Choosing a Target Function </li></ul><ul><ul><li>ChooseMove: Board  Move (action selection function, or policy ) </li></ul></ul><ul><ul><li>V: Board  R (board evaluation function) </li></ul></ul><ul><ul><li>Ideal target V ; approximated target </li></ul></ul><ul><ul><li>Goal of learning process: operational description (approximation) of V </li></ul></ul>
  29. 29. A Target Function for Learning to Play Checkers <ul><li>Possible Definition </li></ul><ul><ul><li>If b is a final board state that is won, then V(b) = 100 </li></ul></ul><ul><ul><li>If b is a final board state that is lost, then V(b) = -100 </li></ul></ul><ul><ul><li>If b is a final board state that is drawn, then V(b) = 0 </li></ul></ul><ul><ul><li>If b is not a final board state in the game, then V(b) = V(b’) where b’ is the best final board state that can be achieved starting from b and playing optimally until the end of the game </li></ul></ul><ul><ul><li>Correct values, but not operational </li></ul></ul><ul><li>Choosing a Representation for the Target Function </li></ul><ul><ul><li>Collection of rules? </li></ul></ul><ul><ul><li>Neural network? </li></ul></ul><ul><ul><li>Polynomial function (e.g., linear, quadratic combination) of board features? </li></ul></ul><ul><ul><li>Other? </li></ul></ul><ul><li>A Representation for Learned Function </li></ul><ul><ul><li>bp/rp = number of black/red pieces; bk/rk = number of black/red kings; bt/rt = number of black/red pieces threatened (can be taken on next turn) </li></ul></ul>
  30. 30. A Training Procedure for Learning to Play Checkers <ul><li>Obtaining Training Examples </li></ul><ul><ul><li>the target function </li></ul></ul><ul><ul><li>the learned function </li></ul></ul><ul><ul><li>the training value </li></ul></ul><ul><li>One Rule For Estimating Training Values: </li></ul><ul><li>Choose Weight Tuning Rule </li></ul><ul><ul><li>Least Mean Square (LMS) weight update rule: REPEAT </li></ul></ul><ul><ul><ul><ul><li>Select a training example b at random </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Compute the error(b) for this training example </li></ul></ul></ul></ul><ul><ul><ul><ul><li>For each board feature f i , update weight w i as follows: where c is a small, constant factor to adjust the learning rate </li></ul></ul></ul></ul>
  31. 31. Design Choices for Learning to Play Checkers Completed Design Determine Type of Training Experience Games against experts Games against self Table of correct moves Determine Target Function Board  value Board  move Determine Representation of Learned Function Polynomial Linear function of six features Artificial neural network Determine Learning Algorithm Gradient descent Linear programming
  32. 32. Example of Interesting Application: Data Mining Database Mining NCSA D2K - http://www.ncsa.uiuc.edu/STI/ALG
  33. 33. Example: Reasoning (Inference, Decision Support) Cartia ThemeScapes - http://www.cartia.com 6500 news stories from the WWW in 1997
  34. 34. Example: Planning and Control Normal Ignited Engulfed Destroyed Extinguished Fire Alarm Flooding DC-ARM - http://www-kbs.ai.uiuc.edu
  35. 35. Relevant Disciplines <ul><li>Artificial Intelligence </li></ul><ul><li>Bayesian Methods </li></ul><ul><li>Cognitive Science </li></ul><ul><li>Computational Complexity Theory </li></ul><ul><li>Control Theory </li></ul><ul><li>Information Theory </li></ul><ul><li>Neuroscience </li></ul><ul><li>Philosophy </li></ul><ul><li>Psychology </li></ul><ul><li>Statistics </li></ul>Machine Learning Symbolic Representation Planning/Problem Solving Knowledge-Guided Learning Bayes’s Theorem Missing Data Estimators PAC Formalism Mistake Bounds Language Learning Learning to Reason Optimization Learning Predictors Meta-Learning Entropy Measures MDL Approaches Optimal Codes ANN Models Modular Learning Occam’s Razor Inductive Generalization Power Law of Practice Heuristic Learning Bias/Variance Formalism Confidence Intervals Hypothesis Testing
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×