Your SlideShare is downloading. ×
XCS: Current capabilities and future  challenges
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

XCS: Current capabilities and future challenges

790
views

Published on

Martin Butz presents the current state-of-the-union of XCS

Martin Butz presents the current state-of-the-union of XCS

Published in: Technology, Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
790
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. XCS: Current capabilities and future challenges Martin V. Butz Department of Cognitive Psychology (III) University of Würzburg, Germany http://www-illigal.ge.uiuc.edu/~butz mbutz@psychologie.uni-wuerburg.de 02/23/2006
  • 2. Overview 1. The XCS Classifier System 2. XCS - Capabilities 3. XCS - Future Challenges 4. Summary & Conclusions 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 2
  • 3. 1. The XCS Classifier System 1. The XCS Classifier System 1. Framework 2. Evolutionary Pressures 3. Computational Complexity 4. General Learning Intuition 2. XCS – Capabilities 3. XCS – Future Challenges 4. Summary & Conclusions 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 3
  • 4. 1 The XCS Classifier System The XCS Classifier System • Learning classifier system – Rule-based representation of condition-action-predictions – Steady-state GA for evolution of conditions – Gradient-based techniques for estimation of predictions • Major characteristics: – Q-learning based reinforcement learning – Relative accuracy-based fitness – Action-set restricted selection, that is, niche selection – Panmictic (population-wide) deletion Goal: Learn a complete maximally accurate, maximally general predictive model. 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 4
  • 5. 1 The XCS Classifier System Rule Evaluation • Gradient-based techniques for derivation of prediction and error estimates • Q-learning derived update • Propagation of reward possible • Rule quality depends on inverse of error estimate • Accuracy of rule prediction determines fitness 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 5
  • 6. 1 The XCS Classifier System Evolutionary Algorithm • Fixed population size • Steady-state genetic algorithm • Two reproductions and deletions per iteration – Reproduction in action set based on fitness – Deletion from whole population based on coverage • Genetic operators: – Mutation – Recombination 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 6
  • 7. 1 The XCS Classifier System Learning Suitability • XCS represents its solution by a collection of sub-solutions (that is, the population of classifiers). • XCS learns an effective problem space clustering (subspaces) in its conditions. • Clusters (subspaces) evolve to enable maximally accurate predictions. – Accuracy can be bounded (error threshold ε0 and population size relation). – Basically any form of prediction is possible (e.g. reward, next sensory input, function value). 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 7
  • 8. 1 The XCS Classifier System XCS Power • Combination of – Gradient-based techniques (to generate predictions) – Evolutionary techniques (to generate features / clusters) • Advantage: – Usage of gradient-based error-feedback learning where possible – Usage of evolutionary techniques • Where error-feedback is hard or impossible to propagate into • Where error-feedback learning gets easily stuck in local optima • Thus: – Combine best approximation technique (error-feedback learning) with best evolutionary technique (representation and operators) 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 8
  • 9. 2. XCS – Current Capabilities 1. The XCS Classifier System 2. XCS – Current Capabilities 1. Binary and Real-world Classification Problems 2. Function Approximation Problems 3. (Multistep) Reinforcement Learning Problems 3. XCS – Future Challenges 4. Summary & Conclusions 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 9
  • 10. 2 XCS – Current Capabilities The Multiplexer Problem Optimal solution representation C A R εF 000### 0 1000 0 1 Problem instance Class 000### 1 0 01 000000 0 001### 0 0 01 001000 1 001### 1 1000 0 1 000111 0 01#0## 0 1000 0 1 011011 0 01#0## 1 0 01 101101 0 01#1## 0 0 01 01#1## 1 1000 0 1 100010 1 10##0# 0 1000 0 1 100101 0 … … … …… 110000 0 … … 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 10
  • 11. 2 XCS – Current Capabilities XCS Performance in MP 70 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 11
  • 12. 2 XCS – Current Capabilities Hierarchical Classification Problem • Hierarchical problems with low order dependencies (“building blocks”) and further high-order dependencies • BB structures are re-used on the higher level to derive problem class. • Example: Hierarchical 3-parity, 6-multiplexer problem: 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 12
  • 13. 2 XCS – Capabilities XCS/BOA Performance 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 13
  • 14. 2 XCS – Capabilities Classification of Real-World Datasets • Conditions are coded with attributes dependent on type of attribute in dataset (interval coding or Binary coding). • Experiments in 42 datasets (from UCI and other sources) • Comparisons with ten other ML systems (pairwise t-test) • XCS learns competitively but it is a much more general learning system. XCS Maj. 1-R C4.5 Naïve PART IB1 IB3 SMO SMO SMO Bayes (poly) (pol.3) (rad.) 99% 38/0 29/1 5/8 19/12 5/6 13/7 9/11 9/17 8/13 23/8 95% 38/0 30/1 5/9 19/12 7/6 14/7 9/15 9/18 9/14 24/9 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 14
  • 15. 2 XCS – Capabilities Piecewise Linear Function Approximation f(x,y) 1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 0.8 1 0 0.2 0.4 0.4 0.6 0.6 0.8 0 0.2 1 y x 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 15
  • 16. 2 XCS – Capabilities Performance in 3D Sinusoidal Function 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 16
  • 17. 2 XCS – Capabilities Performance Evaluation in Maze6 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 17
  • 18. 2 XCS – Capabilities Performance Maze6 plus Irrelevant Bits 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 18
  • 19. 3. XCS – Future Challenges 1. The XCS Classifier System 2. XCS – Performance Demonstration 3. XCS – Future Challenges 1. Representation & Operators 2. Niching 3. Dynamic Problems 4. Compactness of solution / population 5. Hierarchical classifier systems 4. Summary & Conclusions 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 19
  • 20. 4.1 XCS –Future Challenges Representation & Operators • Different representations of conditions – Binary, real-valued, mixed – Kernels, bases – Combinations, Hybrids • Different representations of predictions – Constant, Linear, Polynomial – State prediction, property prediction – Control variable prediction • Most suitable operators for representations – Approximation operators (use best gradient method) – Genetic operators (mind XCS problem bounds) 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 20
  • 21. 4.1 XCS –Future Challenges XCS and Niching • Currently: – Niching is done occurrence-based – Number of classifiers in large problem niches unnecessary large – Number of classifiers in small but hard to approximate problem spaces potentially too small -> niche loss • Additional balancing mechanisms might be necessary! 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 21
  • 22. 4.1 XCS –Future Challenges Dynamic Problems • Currently: XCS was applied mainly to static problems – Makes iterative, adaptive approach not really necessary • Dynamic problems – Concept class changes – Reward distribution changes – Problem sampling changes • Question: – How quickly can XCS adapt to these changes? – Can we improve adaptation? 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 22
  • 23. Compactness of Solution / Population • Problem in XCS: – Population sizes get rather big. – Final solution is not very compact – Solutions indicate overfitting problems in dataminig problems • Main generalization mechanism purely based on occurrence frequency • Borders between different classes are ill-defined. • How can we efficiently compact the population online or using post-processing (some approaches available)? 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 23
  • 24. Hierarchical Classifier System • Local structures are often used by many higher-order structures (decomposability of environment, problems, etc.) • Can we build higher-level classifier structures that build on evolving lower-level structures…. • The hierarchical boolean function problems as a start? 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 24
  • 25. 4. Summary and Conclusions 1. The XCS Classifier System 2. XCS – Performance Demonstration 3. XCS – Future Challenges 4. Summary & Conclusions 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 25
  • 26. 4 Summary and Conclusions Conclusions • XCS is designed to – Cluster the problem space to enable – Maximally accurate predictions • XCS is a highly flexible learning system – Conditions of various types possible – Predictions of various types possible • Major XCS challenges lie in the further development of – Representation & Operators – Niching – Dynamic Problems – Compactness of solution / population – Hierarchical classifier systems • XCS has a big potential due to the combination of – Gradient-based update mechanisms – Evolutionary-based feature extraction mechanisms 05/16/2006 XCS: Current Capabilities & Future Challenges Martin V. Butz 26
  • 27. Thank You for Your Attention