Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

  • Be the first to comment

  • Be the first to like this


  1. 1. Machine learning Overview PD. Dr. Gabriella Kókai [email_address] Friedrich-Alexander-Universität Lehrstuhl für Informatik 2 Raum 04.131 Tel: 8528996
  2. 2. Machine Learning: Content <ul><li>Why Machine Learning? </li></ul><ul><li>How can a learning problem be defined </li></ul><ul><li>Designing a learning system: learning to play checker </li></ul><ul><li>Perspectives and questions in ML </li></ul><ul><li>Summary </li></ul>
  3. 3. Why Machine Learning? (1/10) <ul><li>Webster 's definition of 'learn' </li></ul><ul><ul><li>'To gain knowledge, or understanding of, or skill in by study instruction or experience‘ </li></ul></ul><ul><li>Simons' definition (Machine Learning I, 1993, Chapter 2.) </li></ul><ul><ul><li>'Learning denotes changes in the system that are adaptive in the sense that they enable the system to do the same task or tasks drawn from the same population more effectively the next time‘ </li></ul></ul><ul><li>Donald Michie's Definition (Computer Journal 1991) </li></ul><ul><ul><li>'A learning system uses sample data to generate an update basis for improved (performance) on subsequent data from the same source and express the new basis in intelligible symbolic form' </li></ul></ul>
  4. 4. Why Machine Learning? (2/10) <ul><li>Machine learning is typically thought of as a sup-topic of artificial intelligence. </li></ul><ul><li>It is inspired by several disciplines </li></ul>Machine Learning Cognitive Science Statistic Pattern Recognition Computer Science
  5. 5. Why Machine Learning? (3/10) <ul><li>Relevant topics: </li></ul><ul><ul><li>Artificial Intelligence:Learning: Learning symbolic representation of concepts, ML as search problem , Prior knowledge + training examples guide the learning-process </li></ul></ul><ul><ul><li>Bayesian Methods:Calculating probabilities of the hypotheses, Bayesian-classifier </li></ul></ul><ul><ul><li>Theory of the computational complexity: Theoretical bounds of the complexity for different learning task measured in the terms of the computational effort, number of different training examples, the number of mistakes required in order to learn </li></ul></ul><ul><ul><li>Information theory:Measurement of the entropy, minimal description length, optimal codes and their relationship to optimal training sequences for encoding a hypothesis </li></ul></ul><ul><ul><li>Philosophy: Occam's razor suggesting the simpliest hypothesis is the best </li></ul></ul><ul><ul><li>Psychology and Neurobiology: Motivation of NN the power law of the practice </li></ul></ul><ul><ul><li>Statistics : Characterisation of the errors (e.g. bias,variance), that occur when estimating the accuracy of hypothesis based, confidence interval, statistical tests </li></ul></ul><ul><li>Goal: Description of the different learning paradigms, the algorithms, the theoretical results and applications </li></ul>
  6. 6. Why Machine Learning?(4/10) <ul><li>Dimension: Constraints </li></ul><ul><ul><li>Task/objective </li></ul></ul><ul><ul><ul><li>Learning task </li></ul></ul></ul><ul><ul><ul><li>Performance task </li></ul></ul></ul><ul><ul><li>Availability of the background knowledge </li></ul></ul><ul><ul><ul><li>Encoded </li></ul></ul></ul><ul><ul><ul><li>Interactive </li></ul></ul></ul><ul><ul><li>Availability of data </li></ul></ul><ul><ul><ul><li>Incremental vs. batch </li></ul></ul></ul><ul><ul><ul><li>Passive vs. active </li></ul></ul></ul><ul><ul><li>Characteristics of the data </li></ul></ul><ul><ul><ul><li>Static vs. drifting </li></ul></ul></ul><ul><ul><ul><li>Propositional or first-order </li></ul></ul></ul>
  7. 7. Why Machine Learning?(5/10) <ul><li>Dimension: Approach </li></ul><ul><ul><li>Search mechanism </li></ul></ul><ul><ul><ul><li>Top-Down (model driven) </li></ul></ul></ul><ul><ul><ul><li>Bottom-up (data driven) </li></ul></ul></ul><ul><ul><ul><li>Many others </li></ul></ul></ul><ul><ul><li>Reasoning methods </li></ul></ul><ul><ul><ul><li>Induction, abduction, deduction </li></ul></ul></ul>
  8. 8. Why Machine Learning? (6/10) <ul><li>Deductive Reasoning: </li></ul><ul><li>Inductive Reasoning: </li></ul><ul><li>Abductive Reasoning: </li></ul>
  9. 9. Why Machine Learning? (7/10) <ul><ul><li>Evaluation Methodologies </li></ul></ul><ul><ul><ul><li>Mathematical </li></ul></ul></ul><ul><ul><ul><ul><li>Previously: Learning in the limit </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Now: PAC (Probably Approximately Correct) </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>More tolerant </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Addresses efficiency constraints </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Recent: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Best cases analysis (Helpful Teacher Model) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Average case analysis (constraining assumption) </li></ul></ul></ul></ul></ul><ul><ul><ul><li>Empirical:When mathematical analysis isn't obvious </li></ul></ul></ul><ul><ul><ul><ul><li>Popular </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Data intensive </li></ul></ul></ul></ul><ul><ul><ul><li>Psychological </li></ul></ul></ul><ul><ul><ul><ul><li>Goal: Model human learning behaviour </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Method: Comparison with subject data </li></ul></ul></ul></ul>
  10. 10. Why Machine Learning? (8/10) <ul><li>Knowledge-Poor Supervised Learning </li></ul><ul><ul><li>Given: A training set of annotated instances </li></ul></ul><ul><ul><li>To Induce : A hypothesis (concept description) </li></ul></ul><ul><li>Knowledge-Intensive Supervised Learning </li></ul><ul><ul><li>Given : A set of training instances + a hypothesis of the target concept + background knowledge </li></ul></ul><ul><ul><li>To Induce: A modified hypothesis (concept description) that is consistent with the domain theory & the training instances </li></ul></ul><ul><li>Unsupervised learning: clustering </li></ul><ul><ul><li>Given: A set of unclassified instances I Have not any special target attribute </li></ul></ul><ul><ul><li>To Do: Create a set of clusters for I according to their presumed classes Clusters need not to be disjoint Clusters can be hierarchically related </li></ul></ul>
  11. 11. Why Machine Learning? (9/10) <ul><li>Paradigms knowledge-poor supervised learning: </li></ul><ul><ul><li>Concept learning </li></ul></ul><ul><ul><li>Decision tree (ID3, TIDT) </li></ul></ul><ul><ul><li>Rule based </li></ul></ul><ul><ul><li>Lazy learning </li></ul></ul><ul><ul><li>Genetic algorithms </li></ul></ul><ul><ul><li>Neural networks </li></ul></ul><ul><ul><li>Bayesian networks </li></ul></ul><ul><li>Paradigms knowledge-intensive supervised learning: </li></ul><ul><ul><li>Explanation based learning </li></ul></ul><ul><ul><li>Inductive Logic Programming </li></ul></ul><ul><li>Unsupervised learning </li></ul><ul><ul><li>Bayesian learning </li></ul></ul><ul><ul><li>Clustering </li></ul></ul>
  12. 12. Why Machine Learning? (10/10) <ul><li>Importance: How can computers be programmed that they 'learn' </li></ul><ul><li>Machine learning natural learning </li></ul><ul><li>Application areas </li></ul><ul><ul><li>Data mining: automatic detection of regularity in big amounts of data </li></ul></ul><ul><ul><li>Implementation of software, which cannot be easily programmed by hand </li></ul></ul><ul><ul><li>Self adaptive programs: programs for playing </li></ul></ul><ul><li>Theoretical results: Connection among the number of training examples, the hypothesis and the expected error </li></ul><ul><li>Biological studies </li></ul>
  13. 13. How can the learning problem be defined <ul><li>Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P , if its performance at tasks in T , as measured by P improves with experience E </li></ul><ul><li>Example: Learning to play checker </li></ul><ul><ul><li>Task T : design a program to learn to play checker </li></ul></ul><ul><ul><li>Performance measure P : The percentage of the games won </li></ul></ul><ul><ul><li>Experience E : Playing against itself </li></ul></ul>
  14. 14. Content <ul><li>Why Machine Learning? </li></ul><ul><li>How can the learning problem be defined </li></ul><ul><ul><li>Choosing the training experience </li></ul></ul><ul><ul><li>Choosing the target function </li></ul></ul><ul><ul><li>Choosing the representation of the target function </li></ul></ul><ul><ul><li>Choosing a function approximation algorithm </li></ul></ul><ul><li>Designing a learning system: learning to play checker </li></ul><ul><li>Perspectives and questions in ML </li></ul><ul><li>Summary </li></ul>
  15. 15. Choosing the Training Experience (1/2) <ul><li>What experience is provided </li></ul><ul><ul><li>Direct or indirect feedback regarding the choices executed by the system </li></ul></ul><ul><ul><ul><li>Direct: Individual checker board states and the correct move for each </li></ul></ul></ul><ul><ul><ul><li>Indirect: move sequences and final outcomes Problem: determining the degree to which each move in the sequence deserves credit or blame for the final outcome ( credit assignment ) </li></ul></ul></ul><ul><ul><li>The rate of the controls of the sequence of the training examples by the learning system </li></ul></ul><ul><ul><ul><li>The teacher selects informative board states and provides the correct move for each </li></ul></ul></ul><ul><ul><ul><li>The learner might itself propose board states that it finds particularly confusing and ask the teacher for the correct move </li></ul></ul></ul><ul><ul><ul><li>The learner may have complete control over both the board states and the (indirect) training classification, as it does when it learns playing against itself with no teacher </li></ul></ul></ul>
  16. 16. Choosing the Training Experience (2/2) <ul><ul><li>How well does it represent the distribution of examples over which the final system performance P must be measured Problem: The distribution of the training examples is identical to the distribution of the test examples </li></ul></ul><ul><ul><li>A checkers learning problem: </li></ul></ul><ul><ul><ul><li>Task T : playing checker </li></ul></ul></ul><ul><ul><ul><li>Performance measure P : percentage of games won in the world tournament </li></ul></ul></ul><ul><ul><ul><li>Training experience E : games played against itself </li></ul></ul></ul>
  17. 17. Choosing the Target Function (1/2) <ul><li>What type of knowledge will be learned and how will this be used by the performaning program </li></ul><ul><ul><li>Example: The program needs to learn how to choose the best move from any board state </li></ul></ul><ul><ul><li>ChooseMove: B: the set of legal board state M: the set of legal moves </li></ul></ul><ul><ul><li>Problem: difficult to learn if only the kind of indirect training experience is available to our system => B : the set of legal board states : some real value </li></ul></ul>
  18. 18. Choosing the Target Function (2/2) <ul><li>Question: Definition of the target function V : </li></ul><ul><ul><li>If b is a final board state that is won, then </li></ul></ul><ul><ul><li>If b is a final board state that is lost, then </li></ul></ul><ul><ul><li>If b is a final board state that is drawn, then </li></ul></ul><ul><ul><li>If b is not a final state in the game, then where b ' is the best final board state that can be achieved starting from b and playing optimally until the end of the game (assuming the opponent plays optimally as well). </li></ul></ul><ul><li>Problem: While this definition specifies a value of V(b) for every board state b recursively , this definition is not usable by our checker's player because it is not efficiently computable </li></ul><ul><li>Solution: Discovering an operational description of the ideal target function V , Difficult => learning some approximation </li></ul>
  19. 19. Choosing a Function Approximation Algorithm (1/2) <ul><li>How can be represented? </li></ul><ul><ul><li>For any given board state, the function will be calculated as a linear combination of weights </li></ul></ul><ul><ul><ul><li>bp(p): the number of black pieces on the board </li></ul></ul></ul><ul><ul><ul><li>rp(b): the number of red pieces on the board </li></ul></ul></ul><ul><ul><ul><li>bk(b): the number of black kings on the board </li></ul></ul></ul><ul><ul><ul><li>rk(b): the number of red kings on the board </li></ul></ul></ul><ul><ul><ul><li>bt(b): the number of black pieces threatened by red (i.e., which can be captured on red's next turn) </li></ul></ul></ul><ul><ul><ul><li>rt(b): the number of red pieces threatened by black </li></ul></ul></ul>
  20. 20. Choosing a Function Approximation Algorithm (2/2) <ul><li>Partial design of a checker learning program: </li></ul><ul><ul><li>Task T: playing checker </li></ul></ul><ul><ul><li>Performance measure P: percentage of games won in the world tournament </li></ul></ul><ul><ul><li>Training experience E: games played against itself </li></ul></ul><ul><ul><li>Target function </li></ul></ul><ul><ul><li>Target function representation : </li></ul></ul>
  21. 21. Choosing a Function Approximation Algorithm: Estimating Training Values <ul><li>How to assign training values to the more numerous intermediate board states? </li></ul><ul><ul><li>Approach: assign the training value of for any intermediate board state b to be , where is the learner's current approximation to V and where Successor(b) denotes the next board state following b for which it is again the program's turn to move. </li></ul></ul><ul><ul><li>Rule for estimating the training values: </li></ul></ul>
  22. 22. Choosing a Function Approximation Algorithm: Adjusting the Weights <ul><li>LMS Weight update rule (choosing the weights to best fit the set of training examples) </li></ul><ul><ul><li>Best fit:minimise the squared error E between the training values and the values predicted by the hypothesis: </li></ul></ul><ul><ul><li>For each training example </li></ul></ul><ul><ul><ul><li>Use the current weights to calculate: </li></ul></ul></ul><ul><ul><ul><li>For each update c is a small constant that moderates the size weight update. </li></ul></ul></ul>
  23. 23. Some Issues in Machine Learning <ul><li>What algorithms can approximate functions well (and when?) </li></ul><ul><li>How does the number of training examples influence the accuracy? </li></ul><ul><li>How does the complexity of the hypothesis representation impact it? </li></ul><ul><li>How does noisy data influence the accuracy? </li></ul><ul><li>What are the theoretical limits of learnability? </li></ul><ul><li>How can prior knowledge of the learner help? </li></ul><ul><li>What clues can we get from a biological learning system? </li></ul><ul><li>How can systems alter their own representation? </li></ul>
  24. 24. Summary <ul><li>Goal: Building computer programs that improve their performance at some task through experience </li></ul><ul><li>Application domain: </li></ul><ul><ul><li>Data Mining: discover automatically implicit regularities in large data sets </li></ul></ul><ul><ul><li>Poorly understood domains where humans might not have the knowledge needed to develop effective algorithms </li></ul></ul><ul><ul><li>Domains where the program must dynamically adapt to changing conditions </li></ul></ul><ul><li>ML draws on ideas from several sets of disciplines, including artificial intelligence, probability and statistics, computational complexity information theory, psychology and neurobiology, control theory and philosophy </li></ul><ul><li>Well defined learning problem = well specified task + performance metric + source of training examples </li></ul>