Lecture24

1,389
-1

Published on

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,389
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
74
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Lecture24

  1. 1. Introduction to Machine Learning Lecture 24 Learning Classifier Systems Albert Orriols i Puig http://www.albertorriols.net htt // lb t i l t aorriols@salle.url.edu Artificial Intelligence – Machine Learning g g Enginyeria i Arquitectura La Salle Universitat Ramon Llull
  2. 2. Recap of Lecture 23 Michigan-style LCS g y Environment Sensorial Action Reward state Online rule evaluator: • XCS: Q-Learning (Sutton & Barto, 1998) Classifier 1 Learning Any Representation: y p Uses Widrow-Hoff delta rule Classifier 2 Classifier production rules, genetic programs, System Classifier n perceptrons, SVMs Rule evolution: Genetic Typically, a GA (Holland, 75; Algorithm Goldberg, 89) applied on the population. Slide 2 Artificial Intelligence Machine Learning
  3. 3. Recap of Lecture 23 Main characteristics of XCS Population-based method Independent classifiers Id d tl ifi Works under a reinforcement learning paradigm, but can be also applied t supervised l l li d to i d learning ( d t f i (and to function ti approximation) Classifiers evolved b a genetic algorithm Cl ifi l d by ti l ith Slide 3 Artificial Intelligence Machine Learning
  4. 4. Today’s Agenda Examples of domains Another step toward cognitive systems Anticipatory Classifier System Slide 4 Artificial Intelligence Machine Learning
  5. 5. Applications of LCS Examples of domains p Reinforcement learning Supervised l S i d learning i [Function approximation – not seen herein] Real life Real-life applications Data Mining Modeling market traders Autonomous robotics Modeling artificial ecosystems … Slide 5 Artificial Intelligence Machine Learning
  6. 6. Example in Reinf. Learning Example: simple maze problem p p p XCS solves more complex reinforcement learning prob : prob.: Complex mazes Mountain car Slide 6 Artificial Intelligence Machine Learning
  7. 7. Example in Reinf. Learning Performance in the Maze6 problem (Butz et al.) p ( ) Slide 7 Artificial Intelligence Machine Learning
  8. 8. Example in Sup. Learning Solving large, non-linear, complex boolean functions 000 0#######:0 000 1#######:1 001 #0######:0 001 #1######:1 010 ##0#####:0 010 ##1#####:1 011 ###0####:0 011 ###1####:1 100 ####0###:0 100 ####1###:1 101 #####0##:0 101 #####1##:1 110 ######0#:0 110 ######1#:1 111 #######0:0 111 #######1:1 Solving multiplexer problems up to 135 input variables Slide 8 Artificial Intelligence Machine Learning
  9. 9. Current Real-Life Applications Data mining Most important application domain of LCSs John H. Holmes Epidemiologic study by means of LCSs Hidden relationships among variables p g discovered by LCSs Xavier Llorà et al. Better than Human Capability in Diagnosing Prostate Cancer Using Infrared Spectroscopic imaging gg Many other applications and miners: GALE, GAX GAssist, UCS, GALE GAX, GAssist UCS MIPS … See: Bull, Bernadó-Mansilla & Holmes (eds) Learning Classifier Systems in Data mining. Springer (2008) Artificial Intelligence Machine Learning Slide 9
  10. 10. Current Real-Life Applications Modeling market traders g LETS project: Evolving artificial traders for successful market trading (Sonia Sc u e bu g et a , 2007) ad g (So a Schulenburg al, 00 ) Evolutionary economics: Create trend followers and value investors Let them interact Evolve a population of strategies Slide 10 Artificial Intelligence Machine Learning
  11. 11. Current Real-Life Applications Autonomous Robotics Robot shaping: Early efforts of Marco Dorigo and Marco Colombetti (1997) Small mobile robots equipped with sensors and motors Robots connected in real time by various sorts of modem cable Robots controlled by LCS, ICS, running on desktop computers Constant stream of positive/negative rewards (bucket brigade) Tasks solved: Following lights Gather food and run home Hunt around for a light hidden behind and obstacle Impressive results, high performance Recent applications to model robotic problems performed in the University of West England Slide 11 Artificial Intelligence Machine Learning
  12. 12. Current Real-Life Applications Modeling Artificial Ecosystems g y Eden: Artificial Life environment (Jon McCromak, 2004) Model of an environment where evolvable rule based rule-based classifier systems drive agent behavior. Autonomous LCSs or agents compete for limited resources. Agents can: g Make and listen to sounds Forage for food g Encounter predators Mate with each other Goal: Maintain the audience in tension without fitness needing the audience explicitly perform fitness selection Slide 12 Artificial Intelligence Machine Learning
  13. 13. Toward Cognitive Systems Cognitive systems g y Cognitive systems are natural or artificial information p ocess g systems, c ud g ose espo s b e o perception, processing sys e s, including those responsible for pe cep o , learning, reasoning and decision-making and for communication and action LCS originally devised as cognitive systems A step further Anticipatory LCS Slide 13 Artificial Intelligence Machine Learning
  14. 14. Anticipatory LCS Anticipations influence cognitive systems p g y LCS learned: Conditions x actions x prediction Anticipatory learning processes learn Condition x action x effect relations Let’s see the Anticipatory LCS (ACS2) Slide 14 Artificial Intelligence Machine Learning
  15. 15. ACS ENVIRONMENT A1 σt σt+1 Population [P] Match Set [M] C1 – A1 – E1 Action Set [A] C2 – A2 – E2 C1 – A1 – E1 C3 – A3 – E3 C3 – A3 – E3 C4 – A1 – E4 C1 – A1 – E1 C4 – A1 – E4 C5 – A5 – E5 C4 – A1 – E4 C6 – A6 – E6 C6 – A6 – E6 C7 – A1 – E7 C7 – A1 – E7 C7 – A1 – E7 … C9 – A9 – E9 C8 – A8 – E8 … C9 – A9 – E9 … Anticipatory Learning Process compare Slide 15 Artificial Intelligence Machine Learning
  16. 16. Next Class Big i t Bi picture of what we have seen so far f ht h f New challenges in machine learning Slide 16 Artificial Intelligence Machine Learning
  17. 17. Introduction to Machine Learning Lecture 24 Learning Classifier Systems Albert Orriols i Puig http://www.albertorriols.net htt // lb t i l t aorriols@salle.url.edu Artificial Intelligence – Machine Learning g g Enginyeria i Arquitectura La Salle Universitat Ramon Llull
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×