Lecture3 - Machine Learning

Introduction to Machine
Learning
Lecture 3

Albert Orriols i Puig
aorriols@salle.url.edu
i l @ ll ld

Artificial Intelligence – Machine Learning
Enginyeria i Arquitectura La Salle
gy q
Universitat Ramon Llull

Recap of Lecture 2
Machine learning
Learning = Improving with experience at some task
Improve over task T
With respect to a performance measure P
Based on experience E

Three especial niches
Data mining: extract information from historical data to help
g p
decision making
Software applications that are too complex to build a hard-
pp p
wired solution for
Self customizing p g
g programs

Slide 2
Artificial Intelligence Machine Learning

Today’s Agenda

Characteristics Desired for ML Methods
General issues
Concepts that will be used through lectures
Summary of the Paradigms that We Won’t
y g
Study
Summary of the P bl
S f th Problems th t W Will Study
that We St d

Slide 3

Characteristics Desired ML
We would like our ML techniques to have the following
q g
properties
Be able to generalize but not too much
generalize,
Be robust
Be li bl
B reliable
Learn models of high quality
Be scalable and efficient
Be explicative
Be determinist

Slide 4

Be able to generalize, but not too much
g ,
We learn from a set of examples
Imagine that we are doing d t regression
I i th t d i data i

Examples (observations)

-- Real domain

Learned function

We only know the examples {e1, e2, e3, e4, e5, e6, e7, e8, e9}
We do not know the real distribution
So, does the learning function fits t e real d st but o
t e ea g u ct o ts the ea distribution?

Slide 5

Be able to generalize, but not too much
g ,
Examples (observations)

-- Real domain

Learned function

What could have happened?
at cou d a e appe ed
I may not be a good representation of the original distribution
The ML method may not work well (overfitting)

So, what should we do?
Assume that I is a good representative of the original distribution
g p g
Go for the simplest solution

Slide 6

Be robust
Real-world is imperfect and our measurements of real world
may be e e more imperfect
ay even o e pe ec
Therefore, we will deal with domains with
Noise
Uncertainty
Vagueness
We have to keep this in mind when designing our algorithms

Slide 7

Learn models of high quality
gq y Test set

How do we evaluate learning quality? New instance

Information based Knowledge
on experience extraction
Learner Model
Dataset

Predicted Output

Training set
g

More advanced validation methods:
k-fold cross-validation
Holdout

Slide 8

Be reliable
What do you prefer?
Do not predict something that you doubt about?
Or just bet for an option?
Classes are cost sensitive?
Cl t iti ?
What happens if I say that a patient, who has actually cancer,
is healthy?
What happens if I say that a patient, who is actually healthy,
has cancer?
Do I prefer to model one class as opposed to the other?
Fraud detection (0.1% of fraudulent transactions)
(% )
Geez, I modeled perfectly the non-fraudulent transactions!
Am I successful?

Slide 9

Be scalable and efficient
Huge amount of data
Information hidden i th
If ti hidd in these d t
data
I need to process them quickly!

Two types o costs
o of costs:
Cost to build the model
Cost to classify new test examples
y p

Slide 10


Be explicative
Should
Sh ld I care about giving an explanation?
b t ii l ti ?
Text/speech recognition
fast. huge,
Things happen too fast If errors are not too huge I do not care if
I read “a” instead of “e”
Medical diagnosis
g
I really care about obtaining an accurate explanation, since the
diagnosis may involve applying surgery to a patient or not

Slide 11

Be determinist

If my data does not change
The learned model should be always the same
The answer for a given test instance should be always the
same

If my data changes
I should adapt to the changes

Slide 12

Paradigms in ML

Typically, techniques in ML have been divided in
different paradigms
Inductive learning
Explanation-based learning
p g
Analogy-based learning
Evolutionary learning
Connectionist Learning

Slide 13

Inductive Learning
Induce rules, trees or, in general, patterns from a set of
, ,g ,p
examples
Start from a specific experience
Draw inferences or generalizations from it

That is
Initial state: Original data
State: Symbolic description of the data with a certain degree of
generalization/specialization
Final state: Model with maximum generalization that implies the
input data

Slide 14

Explanation-Based Learning
Deduce information from a set of observations
Humans learn a lot from few examples
Machine: use results f
M hi lt from one example t solve th next
l to l the t
problem

Domain theory for the problem

EBL New domain theory
Goal concept

Training example

Slide 15

Explanation-Based Learning
Domain:
D i
R1: striped(x) ^ feline(x) tiger(x)
tiger (Flare)
R2: runs(x) feline(x)
R3: ca
3 carnivorous(x) ^ has Tail(x)
o ous( ) as_ a ( ) feline(x)
e e( )
R4: eats_meat(x) carnivorous(x)
R5: teeth(x) ^mammal(x) carnivorous(x) feline (Flare) striped (Flare)
R6: hairy(x) mammal(x)
R7: feeds milk(x)
feeds_milk(x) mammal(x)
R8: warm_blood(x) mammal(x)
carnivorous (Flare)
runs (Flare) has_tail (Flare)
Goal: TIGER
Example:
feeds_milk( Flare )
has_tail ( Flare )
eats_meat
eats meat (Flare) mammal (Flare) teeth (Flare)
striped ( Flare )
teeth ( Flare)

hairy (Flare) feeds_milk (Flare) warm_blood (Flare)

Slide 16

Explanation-based Learning
Example
Goal: Get to Brecon
Training data
Near (Cardiff, Brecon)
Airport (Cardiff)
Domain Knowledge
Near(x,y) ^ holds( loc(x), s ) holds( loc(y), result(drive(x,y),s) )
Airport(z) loc(z),
loc(z) result( fly(z), s )
fly(z)

Operational criterion: We must express concept definition in pure description
language syntax
Our goal can be expressed as
Holds ( loc(Brecon), s)

Slide 17

Learning Based on Analogy
A is similar to A’ according to α
α
A A’
If I have B, can I get B’?
Learn the causality relationship β β
β'
β
Transform α to α’
α'
B B’
Get B according to B and α’
B’ α

Where is the trick?
In learning α’ and β

Partial mapping

Previously
New Problem
solved problem
Derivation
Transformation

Solution to the Solution of this
problem known problem

Slide 18

Evolutionary Learning
Nature as problem solver
p
Nature evolved adapted solutions to life
Let’s
L t’ use thi concepts t learn f
this t to l from experience
i

Slide 19

Connectionist Learning
Mimic brain structure to build machines that are able to
learn
A brain consists of
Connected neurons that behave in a specific way
Let’s assume that this behavior can be coded functionally

Slide 20

Problems That We’ll Study
Typical ML courses go through the different families
yp g g
Structured courses
Big i t
Bi picture of th diff
f the different l
t learning paradigms
i di
However
Emergence of hybrid intelligent systems
Concepts come all mixed together
g
We are engineers. We need to solve problems
So,
So we propose to go problem-oriented
problem oriented
Techniques of different paradigms will come on our way

Slide 21

Problems That We’ll Study

Data classification: C4.5, kNN, Naïve Bayes …
1.

Statistical learning: SVM
2.
2

Association analysis: A-priori
3.

Link mining: Page Rank
4.

Clustering: k-means
g
5.

Reinforcement learning: Q-learning, XCS
6.

Regression
7.
7

Genetic Fuzzy Systems
8.

Slide 22

Next Class

How I Would Like my Problem to Look Like?
Summary of the Paradigms that we Won’t Study

Slide 23

Lecture3 - Machine Learning

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Lecture3 - Machine Learning

Similar to Lecture3 - Machine Learning (20)

More from Albert Orriols-Puig

More from Albert Orriols-Puig (20)

Recently uploaded

Recently uploaded (20)

Lecture3 - Machine Learning