machine learning

1. 1-1
Introduction

2. Concept Learning 1-2
Lectures 5,6
Concept Learning
EE 439 Introduction to
Machine Learning
Chapter 4 of Peter Flach, “Machine Learning: The Art
And Science Of Algorithms That Make Sense Of Data”,
Cambridge university press, 2012

3. 1-3
Logical models
❑ The hallmark of logical models is that they use
logical expressions or concepts to divide the instance
space into segments.
❑ The goal is to find a segmentation such that the data
in each segment is more homogeneous
(predominantly of one class), with respect to the task
to be solved.
❑ There are essentially two kinds of logical models: tree
models and rule models.
Concept Learning

4. 1-4
Logical models
❑ Rule models consist of a collection of implications or
if–then rules,
❖ if-part defines a segment, and
❖ the then-part defines the behaviour of the model in
this segment.
❑ Tree models are a restricted kind of rule model where
the if-parts of the rules are organized in a tree
structure.
Concept Learning

5. 1-5
Logical models
Concept Learning

6. 1-6
Logical models
❑ In this lecture, we consider methods for learning
logical expressions or concepts from examples,
which lies at the basis of both tree models and rule
models.
❑ In concept learning, we only learn a description for
the positive class, and label everything that doesn’t
satisfy that description as negative.
Concept Learning

7. 1-7
The hypothesis space
❑ The simplest concept learning setting is where we
restrict the logical expressions describing concepts to
conjunctions of literals.
Concept Learning

8. 1-8
The hypothesis space
Concept Learning

9. 1-9
The hypothesis space
Concept Learning

10. 1-10
The hypothesis space
❑ Despite the simplicity of this example, the space of
possible concepts – usually called the hypothesis
space – is already fairly large.
❑ Let’s assume we have three different lengths: 3, 4
and 5 metres, while the other three features have two
values each.
❑ Possible instances = ?
❑ Possible instances = 3·2·2·2 = 24
Concept Learning

11. 1-11
The hypothesis space
❑ How many conjunctive concepts are there using
these same features?
❑ Treat the absence of a feature as an additional
‘value’.
❑ This gives a total of 4 · 3 · 3 · 3 = 108 different
concepts.
Concept Learning

12. 1-12
The hypothesis space
❑ Sets of instances – is much larger:
❖ 2^24, which is more than 16 million!
❑ If you pick a random set of instances, the probability
that you can’t find a conjunctive concept that exactly
describes those instances is
❖ 0.9999
❑ This is actually a good thing, as it forces the learner
to generalise beyond the training data and cover
instances that it hasn’t seen before.
Concept Learning

13. 1-13
The hypothesis space
Concept Learning

14. 1-14
The hypothesis space
❑ If we rule out all concepts that don’t cover at least
one of the instances in Example 4.1, the hypothesis
space is reduced to 32 conjunctive concepts.
Concept Learning

15. 1-15
The hypothesis space
Concept Learning

16. 1-16
The hypothesis space
❑ Insisting that any hypothesis cover all three instances
reduces this further to only four concepts:
1) Gills = no,
2) Beak = yes ,
3) empty concept (postulates that everything is a
dolphin)
4) Gills = no ^ Beak = yes
❑ The least general one – is called the least general
generalisation (LGG).
Concept Learning

17. 1-17
The hypothesis space
❑ If we want to be a bit more adventurous, we could
choose one of the more general hypotheses such as
Gills = no or Beak = yes
❑ We don’t want to choose the most general
hypothesis, which is simply that every animal is a
dolphin, as this would clearly be an over-
generalisation.
❑ Negative examples are very useful to prevent over-
generalistion.
Concept Learning

18. 1-18
The hypothesis space
Concept Learning

19. 1-19
The hypothesis space
Concept Learning

20. 1-20
Version space
❑ A concept is complete if it covers all positive
examples.
❑ A concept is consistent if it covers none of the
negative examples.
❑ The version space is the set of all complete and
consistent concepts.
Concept Learning

21. Concept Learning 1-21
Chapter 2 of E. Alpaydin, “Introduction to Machine
Learning”, 4th Edition The MIT press, 2020

22. 1-22
Learning a Class from Examples
❑ Let us say we want to learn the class, C, of a “family
car”.
❑ We have a set of examples of cars, and we have a
group of experts that we survey to whom we show
these cars.
❑ The people look at the cars and label them; the cars
that they believe are family cars are positive
examples, and the other cars are negative examples.
Concept Learning

23. 1-23
Learning a Class from Examples
❑ Class learning is finding a description that is shared
by all the positive examples and none of the negative
examples.
❑ We can make a prediction:
❖ Given a car that we have not seen before, by checking
with the description learned, we will be able to say
whether it is a family car or not.
❑ We can do knowledge extraction:
❖ Sponsored by a car company, and the aim is to
understand what people expect from a family car.
Concept Learning

24. 1-24
Learning a Class from Examples
❑ Among all features a of car, we consider the price
and engine power to be the features that separate a
family car from other type of cars.
❑ These two attributes are the inputs to the class
recognizer.
❑ Other attributes such as seating capacity and color
are considered to be irrelevant.
Concept Learning

25. 1-25
Learning a Class from Examples
Concept Learning

26. 1-26
Learning a Class from Examples
Concept Learning

27. 1-27
Learning a Class from Examples
Concept Learning

28. 1-28
Learning a Class from Examples
Concept Learning

29. 1-29
Learning a Class from Examples
Concept Learning

30. 1-30
Learning a Class from Examples
Concept Learning

31. 1-31
Learning a Class from Examples
Concept Learning

32. 1-32
Learning a Class from Examples
Concept Learning

33. 1-33
Learning a Class from Examples
Concept Learning

34. 1-34
Learning a Class from Examples
Concept Learning

35. 1-35
Learning a Class from Examples
Concept Learning

36. 1-36
Model Selection and
Generalization
Concept Learning

37. 1-37
Model Selection and
Generalization
Concept Learning

38. 1-38
Model Selection and
Generalization
Concept Learning

39. 1-39
Model Selection and
Generalization
Concept Learning

40. 1-40
Model Selection and
Generalization
Concept Learning

41. 1-41
Model Selection and
Generalization
Concept Learning

42. 1-42
Model Selection and
Generalization
Concept Learning

43. 1-43
Model Selection and
Generalization
Concept Learning

44. 1-44
Dimensions of a Supervised
Machine Learning Algorithm
Concept Learning

45. 1-45
Dimensions of a Supervised
Machine Learning Algorithm
Concept Learning

46. 1-46
Dimensions of a Supervised
Machine Learning Algorithm
Concept Learning