0
Fuzzy-UCS: Preliminary Results


                         Albert Orriols-Puig1,2
                                Orriols P...
Motivation


       Michigan-style LCSs for supervised learning. Eg. XCS and UCS
       – Evolve online highly accurate mo...
Framework


       Genetic Fuzzy Systems
       – Change the rule representation to fuzzy rules
       – Provide a robust,...
Aim


       Propose Fuzzy-UCS
       – Accuracy based Michigan-style LFCS
         Accuracy-based Michigan style
       –...
Outline



        1. Description of Fuzzy-UCS
        1D      ii      fF      UCS

        2.
        2 Experimental Meth...
1. Description of Fuzzy-UCS
                                                                                     2. Experi...
1. Description of Fuzzy-UCS
                                                                                              ...
Description of Fuzzy-UCS
             p            y



         Describe the different components
         1. Rule repres...
1. Description of Fuzzy-UCS
                                                                             2. Experimental M...
1. Description of Fuzzy-UCS
                                                                                              ...
1. Description of Fuzzy-UCS
                                                                                    2. Experim...
1. Description of Fuzzy-UCS
                                                                         2. Experimental Metho...
1. Description of Fuzzy-UCS
                                                                          2. Experimental Meth...
Description of Fuzzy-UCS
             p            y



         Describe the different components
         1. Rule repres...
1. Description of Fuzzy-UCS
                                                                                    2. Experim...
1. Description of Fuzzy-UCS
                                                                      2. Experimental Methodol...
1. Description of Fuzzy-UCS
                                                                                              ...
Description of Fuzzy-UCS
             p            y



         Describe the different components
         1. Rule repres...
1. Description of Fuzzy-UCS
                                                                                          2. E...
1. Description of Fuzzy-UCS
                                                                                     2. Experi...
Description of Fuzzy-UCS
             p            y



         Describe the different components
         1. Rule repres...
1. Description of Fuzzy-UCS
                                                                        2. Experimental Method...
1. Description of Fuzzy-UCS
                                                                                             2...
1. Description of Fuzzy-UCS
                                                                                     2. Experi...
1. Description of Fuzzy-UCS
                                                                         2. Experimental Metho...
1. Description of Fuzzy-UCS
                                                                                    2. Experim...
1. Description of Fuzzy-UCS
                                                                      2. Experimental Methodol...
Fuzzy-UCS: Preliminary Results


                         Albert Orriols-Puig1,2
                                Orriols P...
Upcoming SlideShare
Loading in...5
×

IWLCS'2007: Fuzzy-UCS: Preliminary Results

479

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
479
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "IWLCS'2007: Fuzzy-UCS: Preliminary Results"

  1. 1. Fuzzy-UCS: Preliminary Results Albert Orriols-Puig1,2 Orriols Puig Jorge Casillas2 Ester Bernadó-Mansilla1 1Research Group in Intelligent Systems Enginyeria i Arquitectura La Salle, Ramon Llull University 2Dept. Computer Science and Artificial Intelligence University of Granada
  2. 2. Motivation Michigan-style LCSs for supervised learning. Eg. XCS and UCS – Evolve online highly accurate models – Competitive to the most-used machine learning techniques • (Bernadó et al, 02; Wilson, 02; Bacardit & Butz, 04; Butz, 06; Orriols & Bernadó, 07) Main drawback: Intepretability of the rule sets – Number of rules or classifiers • Reduction schemes (Wilson, 02; Fu & Davis, 02; Dixon et al., 03) – Intervalar representation of continuous attributes: [li, ui] . Semantic-free variables Enginyeria i Arquitectura la Salle Slide 2 GRSI
  3. 3. Framework Genetic Fuzzy Systems – Change the rule representation to fuzzy rules – Provide a robust, flexible, and powerful methodology to deal with noisy imprecise and incomplete data data. noisy, imprecise, Michigan-style Learning Fuzzy-Classifier Systems (LFCS) – (Valenzuela-Radón, 91 & 98) – (Parodi & Bonelli, 93) – (Furuhashi, Nakaoka & Uchikawa, 94) – (Velasco, 98) – (Ishibuchi, Nakashima & Murata, 99 & 05): First LFCS for pattern classification – (Casillas, Carse & Bull, 07) Fuzzy-XCS Enginyeria i Arquitectura la Salle Slide 3 GRSI
  4. 4. Aim Propose Fuzzy-UCS – Accuracy based Michigan-style LFCS Accuracy-based Michigan style – Supervised learning scheme – Derived from UCS (Bernado & Garrell, 2003) • Introduction of a linguistic fuzzy representation • Modification of all operators that deal with rules – We expect: • Achieve similar performance than UCS • Higher interpretability since we would deal with linguistic rules • Lower number of fuzzy rules in the final population Enginyeria i Arquitectura la Salle Slide 4 GRSI
  5. 5. Outline 1. Description of Fuzzy-UCS 1D ii fF UCS 2. 2 Experimental Methodology 3. Results 4. Conclusions Enginyeria i Arquitectura la Salle Slide 5 GRSI
  6. 6. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of UCS p 3. Results 4. Conclusions 4C li Rule representation: – Binary variables: {0 1 #} {0, 1, – Continuous variables: [li, ui] –E Eg: IF x1 Є [l1, u1] ^ x2 Є [l2, u2] … ^ xn Є[ln, nn] THEN class1 – Matching instance e: for all ei: li ≤ ei ≤ ui – Set of parameters: • Accuracy • Fitness • Numerosity • Experience • Correct set size Enginyeria i Arquitectura la Salle Slide 6 GRSI
  7. 7. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of UCS p 3. Results 4. Conclusions 4C li Stream of Environment examples Match Set M t h S t [M] Problem instance P bl it + output class acc F num cs ts exp 1C A acc F num cs ts exp 3C A Population [P] acc F num cs ts exp 5C A acc F num cs ts exp 6C A … acc F num cs ts exp 1C A acc F num cs ts exp 2C A acc F num cs ts exp 3C A acc F num cs ts exp correct set 4C A Classifier acc F num cs ts exp generation 5C A Parameters acc F num cs ts exp Match set 6C A Update generation … Correct Set [C] 3 C A acc F num cs ts exp # Correct Deletion Selection, Reproduction, acc = 6 C A acc F num cs ts exp mutation Experience p … If there are no classfiers in Fitness = accν Genetic Algorithm [C], covering is triggered Enginyeria i Arquitectura la Salle Slide 7 GRSI
  8. 8. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 8 GRSI
  9. 9. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions Rule representation – Linguistic fuzzy rules – E.g.: IF x1 is A1 and x2 is A2 … and xn is An THEN class1 Disjunction of linguistic fuzzy terms – All variables share th same semantics i bl h the ti – Example: Ai = {small, medium, large} IF x1 is small and x2 is medium or large THEN class1 – Codification: IF [100 | 011] THEN class1 Enginyeria i Arquitectura la Salle Slide 9 GRSI
  10. 10. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions How do we know if a given input is small, medium or large? g p , g – Each linguistic term defined by a membership function Belongs to medium with a degree of 0 8 0.8 Belongs to small with a degree of 0 2 0.2 ei Attribute value Triangular-shaped membership functions Enginyeria i Arquitectura la Salle Slide 10 GRSI
  11. 11. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions Matching degree uAk(e) gg () [,] [0,1] k: IF x1 is small and x2 is medium or large THEN class1 Example: (e1, e2) 0.8 08 0.2 0.2 e1 e2 T-conorm: bounded sum max ( 1, 0.8 + 0.2) = 1 T-norm: product uAk(e) = 1 * 0.2 = 0.2 Enginyeria i Arquitectura la Salle Slide 11 GRSI
  12. 12. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li The role of matching changes: • UCS: A rule matches or not an example (binary function) • Fuzzy-UCS: A rule matches an example with a certain degree Enginyeria i Arquitectura la Salle Slide 12 GRSI
  13. 13. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Each classifier has the following parameters: 1. 1 Weight per class wj • Soundness with which the rule predicts the class j. • The class value is dynamic and corresponds to the class j with higher wj 2. Fitness: • Quality of the rule 3. Other parameters directly inherited from UCS: • numerosity • correct set size • experience Enginyeria i Arquitectura la Salle Slide 13 GRSI
  14. 14. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 14 GRSI
  15. 15. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4C li 4. Conclusions Learning interaction: – The environment provides an example e and its class c – Match set creation: all classifiers that match with uAk(x) > 0 – Correct set creation: all classifiers that advocate c – Covering: if there is not a classifier that maximally matches e • Create the classifier that match the input example with maximum degree. • Generalize the condition with probability P# For each variable: A1 A2 A3 Enginyeria i Arquitectura la Salle Slide 15 GRSI
  16. 16. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Parameters’ Update – Experience: – Sum of correct matching per class j cmj: Enginyeria i Arquitectura la Salle Slide 16 GRSI
  17. 17. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Parameters’ Update – Use cm to update of the weights per each class: • Rule that only matches instances of class c: • wc = 1 • For all the other classes j: wj = 0 – Calculate the fitness Pressuring toward rules that correctly match instances of only one class Enginyeria i Arquitectura la Salle Slide 17 GRSI
  18. 18. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 18 GRSI
  19. 19. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Discovery component – Steady state niched GA Steady-state – Roulette wheel selection Instances that have a higher g matching degree have more opportunities of being selected Enginyeria i Arquitectura la Salle Slide 19 GRSI
  20. 20. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Discovery component – Crossover and mutation applied on the antecedent • 2 point crossover IF [100 | 011] THEN class1 IF [101 | 100] THEN class1 • Mutation: – Expansion p IF [101 | 011] THEN class1 [ ] IF [100 | 011] THEN class1 [ ] – Contraction IF [100 | 001] THEN class1 IF [100 | 011] THEN class1 – Shift IF [010 | 011] THEN class1 IF [100 | 011] THEN class1 Enginyeria i Arquitectura la Salle Slide 20 GRSI
  21. 21. Description of Fuzzy-UCS p y Describe the different components 1. Rule representation and matching 2. Learning interaction 3. Discovery component 3 Di t 4. Fuzzy-UCS in test mode Enginyeria i Arquitectura la Salle Slide 21 GRSI
  22. 22. 1. Description of Fuzzy-UCS 2. Experimental Methodology Description of Fuzzy-UCS p y 3. Results 4. Conclusions 4C li Class inference of a test example e – Winner rule • Inference: Select the rule that maximizes uAk(e) · Fk • Reduction: Only keep in the final population that rules that maximize uAk(e) · Fk at least for one training example – Average vote • Inference: All experienced rules vote for the class they predict. The most voted class is returned. Reduction: O l k Only keep experienced rules with positive fit i dl ith iti fitness i th • R d ti in the final population Enginyeria i Arquitectura la Salle Slide 22 GRSI
  23. 23. 1. Description of Fuzzy-UCS 2. Experimental Methodology Experimental Methodology p gy 3. Results 4. Conclusions 4C li Methodology – Compare Fuzzy-UCS to UCS, C4.5, and SMO. – 10-fold cross-validation – Averages over 10 runs – 5 linguistic labels #Inst #Fea #Re #In #No #Cl %Min %Max %MisAtt 625 4 4 0 0 3 7,8 46,1 0 bal 345 6 6 0 0 2 42 58 0 bpa p 303 13 6 0 7 2 45,5 54,5 15,4 h-c 150 4 4 0 0 3 33,3 33,3 0 irs 768 8 8 0 0 2 34,9 65,1 0 pim 215 5 5 0 0 3 14 60 0 thy 699 9 0 9 0 2 34,5 , 65,5 , 11,1 , wbcd 178 13 13 0 0 3 27 39,9 0 wne Enginyeria i Arquitectura la Salle Slide 23 GRSI
  24. 24. 1. Description of Fuzzy-UCS 2. Experimental Methodology Results 3. Results 4. Conclusions 4C li • 1st objective: Competitive in terms of performance Significant difference of Fuzzy-UCS with winner rule: Significant difference of Fuzzy-UCS with average vote: Parameters: iter = 100,000, N = 6,400, F0 = 0.99, v=10, {θGA, θdel, θsub} = 50, x =0.8, u=0.04, P#=0.6, 5 linguistic labels Enginyeria i Arquitectura la Salle Slide 24 GRSI
  25. 25. 1. Description of Fuzzy-UCS 2. Experimental Methodology Results 3. Results 4. Conclusions 4C li • 2nd objective: Improve the interpretability Example of rules evolved by UCS for iris Example of rules evolved by Fuzzy-UCS for iris – Linguistic terms: {XS, S, M, L, XL} Enginyeria i Arquitectura la Salle Slide 25 GRSI
  26. 26. 1. Description of Fuzzy-UCS 2. Experimental Methodology Results 3. Results 4. Conclusions 4C li Number of rules WRule Avote UCS 80 1293 1392 bal bl 60 1849 2075 bpa 145 4441 2265 hc h-c 18 477 624 irs 166 3344 2908 pim 32 1142 856 thy 136 3018 1111 wbcd 101 3984 3618 wne Enginyeria i Arquitectura la Salle Slide 26 GRSI
  27. 27. 1. Description of Fuzzy-UCS 2. Experimental Methodology Conclusions and Further Work 3. Results 4. Conclusions 4C li Conclusions – We proposed a Michigan-style LFCS for supervised learning – Competitive with respect to UCS, SMO, and C4.5 –I Improvement in terms of interpretability with respect t UCS ti t fi t t bilit ith t to Further work – Evolve more reduced populations pp – Enhance the comparison with new real-world problems – Compare to other LFCS – Exploit the incremental learning approach to dig large datasets Enginyeria i Arquitectura la Salle Slide 27 GRSI
  28. 28. Fuzzy-UCS: Preliminary Results Albert Orriols-Puig1,2 Orriols Puig Jorge Casillas2 Ester Bernadó-Mansilla1 1Research Group in Intelligent Systems Enginyeria i Arquitectura La Salle, Ramon Llull University 2Dept. Computer Science and Artificial Intelligence University of Granada
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×