Artificial Intelligence Lecture 15 Introduction to Machine Learning
Overview <ul><li>Machine Learning  </li></ul><ul><li>ID3 Decision Tree Algorithm </li></ul><ul><li>Discussions </li></ul>
Machine Learning <ul><li>Supervised Learning </li></ul><ul><ul><li>Training examples consist of pairs of input vectors, an...
Inductive Learning Basics <ul><li>Inferring a boolean/real-valued function from training examples </li></ul><ul><li>A trai...
Hypothesis <ul><li>Any function that approximates the given set of examples </li></ul>Bias : preference for one hypothesis...
Hypothesis Space <ul><li>A set of all hypotheses consistent with data denoted by H:{H 1 , H 2 , …, H n } </li></ul><ul><li...
ID3 Decision Tree Algorithm
Training Examples
Decision Tree for  PlayTennis ? ? PlayTennis Wind Humidity Temperature Outlook Weak High Hot Overcast Weak High Hot Sunny
Decision Tree Representation <ul><li>Decision Tree </li></ul><ul><ul><li>Each internal node tests an attribute </li></ul><...
Entropy
Information Gain
ID3 Algorithm
Training Examples
Selecting the Best Attribute
 
Hypothesis Space Search by ID3
Discussions
Hypothesis Space Search by ID3 <ul><li>Hypothesis space is complete </li></ul><ul><ul><li>Target function surely in there ...
Inductive Bias in ID3 <ul><li>Note the hypothesis space H is the power set of instances X. Unbiased? </li></ul><ul><li>Not...
Summary <ul><li>Machine Learning </li></ul><ul><li>ID3 Decision Tree Algorithm </li></ul><ul><li>Discussions </li></ul>
Upcoming SlideShare
Loading in …5
×

Introduction to Machine Learning

772 views

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
772
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
37
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Introduction to Machine Learning

  1. 1. Artificial Intelligence Lecture 15 Introduction to Machine Learning
  2. 2. Overview <ul><li>Machine Learning </li></ul><ul><li>ID3 Decision Tree Algorithm </li></ul><ul><li>Discussions </li></ul>
  3. 3. Machine Learning <ul><li>Supervised Learning </li></ul><ul><ul><li>Training examples consist of pairs of input vectors, and desired outputs </li></ul></ul><ul><li>Unsupervised Learning </li></ul><ul><ul><li>Training examples do not contain hints about correct outputs </li></ul></ul><ul><ul><li>Usually used to identify unusual structures in data </li></ul></ul>
  4. 4. Inductive Learning Basics <ul><li>Inferring a boolean/real-valued function from training examples </li></ul><ul><li>A training example is a pair of (x, f(x)) </li></ul><ul><ul><li>x is the input </li></ul></ul><ul><ul><li>f(x) is the output of the function applied to x </li></ul></ul>
  5. 5. Hypothesis <ul><li>Any function that approximates the given set of examples </li></ul>Bias : preference for one hypothesis beyond mere consistency (a) (b) (c) (d)
  6. 6. Hypothesis Space <ul><li>A set of all hypotheses consistent with data denoted by H:{H 1 , H 2 , …, H n } </li></ul><ul><li>Inductive learning is searching for a good hypothesis in the hypothesis space </li></ul><ul><li>Occam’s razor: prefer the simplest hypothesis consistent with data </li></ul>Inductive Learning Assumption : Any hypothesis found to approximate the target function well over a sufficiently large set of training examples will also approximate the target function well over other unobserved examples .
  7. 7. ID3 Decision Tree Algorithm
  8. 8. Training Examples
  9. 9. Decision Tree for PlayTennis ? ? PlayTennis Wind Humidity Temperature Outlook Weak High Hot Overcast Weak High Hot Sunny
  10. 10. Decision Tree Representation <ul><li>Decision Tree </li></ul><ul><ul><li>Each internal node tests an attribute </li></ul></ul><ul><ul><li>Each branch takes an attribute value </li></ul></ul><ul><ul><li>Each leaf node predict a class label </li></ul></ul><ul><li>Disjunction of conjunctions of a set of attribute values </li></ul>
  11. 11. Entropy
  12. 12. Information Gain
  13. 13. ID3 Algorithm
  14. 14. Training Examples
  15. 15. Selecting the Best Attribute
  16. 17. Hypothesis Space Search by ID3
  17. 18. Discussions
  18. 19. Hypothesis Space Search by ID3 <ul><li>Hypothesis space is complete </li></ul><ul><ul><li>Target function surely in there … </li></ul></ul><ul><li>Outputs a single hypothesis (Which one?) </li></ul><ul><ul><li>Cannot determine how many alternatives </li></ul></ul><ul><li>No back tracking </li></ul><ul><ul><li>Local minima … </li></ul></ul><ul><li>Use statistical properties of all training data at each step in search </li></ul><ul><ul><li>Robust to noisy data </li></ul></ul>
  19. 20. Inductive Bias in ID3 <ul><li>Note the hypothesis space H is the power set of instances X. Unbiased? </li></ul><ul><li>Not really. </li></ul><ul><ul><li>Preference for short trees, and for those with high information gain attributes near the root </li></ul></ul><ul><ul><li>Bias is a preference for some hypotheses, rather than a restriction of hypothesis space H, e.g. target concept is not in H. </li></ul></ul>
  20. 21. Summary <ul><li>Machine Learning </li></ul><ul><li>ID3 Decision Tree Algorithm </li></ul><ul><li>Discussions </li></ul>

×