Lecture3 - Machine Learning


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Lecture3 - Machine Learning

  1. 1. Introduction to Machine Learning Lecture 3 Albert Orriols i Puig aorriols@salle.url.edu i l @ ll ld Artificial Intelligence – Machine Learning Enginyeria i Arquitectura La Salle gy q Universitat Ramon Llull
  2. 2. Recap of Lecture 2 Machine learning Learning = Improving with experience at some task Improve over task T With respect to a performance measure P Based on experience E Three especial niches Data mining: extract information from historical data to help g p decision making Software applications that are too complex to build a hard- pp p wired solution for Self customizing p g g programs Slide 2 Artificial Intelligence Machine Learning
  3. 3. Today’s Agenda Characteristics Desired for ML Methods General issues Concepts that will be used through lectures Summary of the Paradigms that We Won’t y g Study Summary of the P bl S f th Problems th t W Will Study that We St d Slide 3 Artificial Intelligence Machine Learning
  4. 4. Characteristics Desired ML We would like our ML techniques to have the following q g properties Be able to generalize but not too much generalize, Be robust Be li bl B reliable Learn models of high quality Be scalable and efficient Be explicative Be determinist Slide 4 Artificial Intelligence Machine Learning
  5. 5. Characteristics Desired ML Be able to generalize, but not too much g , We learn from a set of examples Imagine that we are doing d t regression I i th t d i data i Examples (observations) -- Real domain Learned function We only know the examples {e1, e2, e3, e4, e5, e6, e7, e8, e9} We do not know the real distribution So, does the learning function fits t e real d st but o t e ea g u ct o ts the ea distribution? Slide 5 Artificial Intelligence Machine Learning
  6. 6. Characteristics Desired ML Be able to generalize, but not too much g , Examples (observations) -- Real domain Learned function What could have happened? at cou d a e appe ed I may not be a good representation of the original distribution The ML method may not work well (overfitting) So, what should we do? Assume that I is a good representative of the original distribution g p g Go for the simplest solution Slide 6 Artificial Intelligence Machine Learning
  7. 7. Characteristics Desired ML Be robust Real-world is imperfect and our measurements of real world may be e e more imperfect ay even o e pe ec Therefore, we will deal with domains with Noise Uncertainty Vagueness We have to keep this in mind when designing our algorithms Slide 7 Artificial Intelligence Machine Learning
  8. 8. Characteristics Desired ML Learn models of high quality gq y Test set How do we evaluate learning quality? New instance Information based Knowledge on experience extraction Learner Model Dataset Predicted Output Training set g More advanced validation methods: k-fold cross-validation Holdout Slide 8 Artificial Intelligence Machine Learning
  9. 9. Characteristics Desired ML Be reliable What do you prefer? Do not predict something that you doubt about? Or just bet for an option? Classes are cost sensitive? Cl t iti ? What happens if I say that a patient, who has actually cancer, is healthy? What happens if I say that a patient, who is actually healthy, has cancer? Do I prefer to model one class as opposed to the other? Fraud detection (0.1% of fraudulent transactions) (% ) Geez, I modeled perfectly the non-fraudulent transactions! Am I successful? Slide 9 Artificial Intelligence Machine Learning
  10. 10. Characteristics Desired ML Be scalable and efficient Huge amount of data Information hidden i th If ti hidd in these d t data I need to process them quickly! Two types o costs o of costs: Cost to build the model Cost to classify new test examples y p Slide 10 Artificial Intelligence Machine Learning
  11. 11. Characteristics Desired ML Be explicative Should Sh ld I care about giving an explanation? b t ii l ti ? Text/speech recognition fast. huge, Things happen too fast If errors are not too huge I do not care if I read “a” instead of “e” Medical diagnosis g I really care about obtaining an accurate explanation, since the diagnosis may involve applying surgery to a patient or not Slide 11 Artificial Intelligence Machine Learning
  12. 12. Characteristics Desired ML Be determinist If my data does not change The learned model should be always the same The answer for a given test instance should be always the same If my data changes I should adapt to the changes Slide 12 Artificial Intelligence Machine Learning
  13. 13. Paradigms in ML Typically, techniques in ML have been divided in different paradigms Inductive learning Explanation-based learning p g Analogy-based learning Evolutionary learning Connectionist Learning Slide 13 Artificial Intelligence Machine Learning
  14. 14. Inductive Learning Induce rules, trees or, in general, patterns from a set of , ,g ,p examples Start from a specific experience Draw inferences or generalizations from it That is Initial state: Original data State: Symbolic description of the data with a certain degree of generalization/specialization Final state: Model with maximum generalization that implies the input data Slide 14 Artificial Intelligence Machine Learning
  15. 15. Explanation-Based Learning Deduce information from a set of observations Humans learn a lot from few examples Machine: use results f M hi lt from one example t solve th next l to l the t problem Domain theory for the problem EBL New domain theory Goal concept Training example Slide 15 Artificial Intelligence Machine Learning
  16. 16. Explanation-Based Learning Domain: D i R1: striped(x) ^ feline(x) tiger(x) tiger (Flare) R2: runs(x) feline(x) R3: ca 3 carnivorous(x) ^ has Tail(x) o ous( ) as_ a ( ) feline(x) e e( ) R4: eats_meat(x) carnivorous(x) R5: teeth(x) ^mammal(x) carnivorous(x) feline (Flare) striped (Flare) R6: hairy(x) mammal(x) R7: feeds milk(x) feeds_milk(x) mammal(x) R8: warm_blood(x) mammal(x) carnivorous (Flare) runs (Flare) has_tail (Flare) Goal: TIGER Example: feeds_milk( Flare ) has_tail ( Flare ) eats_meat eats meat (Flare) mammal (Flare) teeth (Flare) striped ( Flare ) teeth ( Flare) hairy (Flare) feeds_milk (Flare) warm_blood (Flare) Slide 16 Artificial Intelligence Machine Learning
  17. 17. Explanation-based Learning Example Goal: Get to Brecon Training data Near (Cardiff, Brecon) Airport (Cardiff) Domain Knowledge Near(x,y) ^ holds( loc(x), s ) holds( loc(y), result(drive(x,y),s) ) Airport(z) loc(z), loc(z) result( fly(z), s ) fly(z) Operational criterion: We must express concept definition in pure description language syntax Our goal can be expressed as Holds ( loc(Brecon), s) Slide 17 Artificial Intelligence Machine Learning
  18. 18. Learning Based on Analogy A is similar to A’ according to α α A A’ If I have B, can I get B’? Learn the causality relationship β β β' β Transform α to α’ α' B B’ Get B according to B and α’ B’ α Where is the trick? In learning α’ and β Partial mapping Previously New Problem solved problem Derivation Transformation Solution to the Solution of this problem known problem Slide 18 Artificial Intelligence Machine Learning
  19. 19. Evolutionary Learning Nature as problem solver p Nature evolved adapted solutions to life Let’s L t’ use thi concepts t learn f this t to l from experience i Slide 19 Artificial Intelligence Machine Learning
  20. 20. Connectionist Learning Mimic brain structure to build machines that are able to learn A brain consists of Connected neurons that behave in a specific way Let’s assume that this behavior can be coded functionally Slide 20 Artificial Intelligence Machine Learning
  21. 21. Problems That We’ll Study Typical ML courses go through the different families yp g g Structured courses Big i t Bi picture of th diff f the different l t learning paradigms i di However Emergence of hybrid intelligent systems Concepts come all mixed together g We are engineers. We need to solve problems So, So we propose to go problem-oriented problem oriented Techniques of different paradigms will come on our way Slide 21 Artificial Intelligence Machine Learning
  22. 22. Problems That We’ll Study Data classification: C4.5, kNN, Naïve Bayes … 1. Statistical learning: SVM 2. 2 Association analysis: A-priori 3. Link mining: Page Rank 4. Clustering: k-means g 5. Reinforcement learning: Q-learning, XCS 6. Regression 7. 7 Genetic Fuzzy Systems 8. Slide 22 Artificial Intelligence Machine Learning
  23. 23. Next Class How I Would Like my Problem to Look Like? Summary of the Paradigms that we Won’t Study Slide 23 Artificial Intelligence Machine Learning
  24. 24. Introduction to Machine Learning Lecture 3 Albert Orriols i Puig aorriols@salle.url.edu i l @ ll ld Artificial Intelligence – Machine Learning Enginyeria i Arquitectura La Salle gy q Universitat Ramon Llull