Pattern Recognition 41 (2008) 1824-1833

   Presenter: Chia-Ming Wang
Goal and Contribution
Goal and Contribution
• Construct a fuzzy classifier (Goal)
Goal and Contribution
• Construct a fuzzy classifier (Goal)
 • map the attrs. to predefined fuzzy sets
Goal and Contribution
• Construct a fuzzy classifier (Goal)
 • map the attrs. to predefined fuzzy sets
 • rules with conf. a...
Goal and Contribution
• Construct a fuzzy classifier (Goal)
 • map the attrs. to predefined fuzzy sets
 • rules with conf. a...
Goal and Contribution
• Construct a fuzzy classifier (Goal)
 • map the attrs. to predefined fuzzy sets
 • rules with conf. a...
Goal and Contribution
• Construct a fuzzy classifier (Goal)
 • map the attrs. to predefined fuzzy sets
 • rules with conf. a...
Goal and Contribution
• Construct a fuzzy classifier (Goal)
 • map the attrs. to predefined fuzzy sets
 • rules with conf. a...
The used antecedent
                        fuzzy sets
             1.0
Membership




                                   ...
Determination of Cj and CFj
1. calculate the compatibility of each training pattern xp with the rule Rj
                  ...
Structure of the goal classifier
Structure of the goal classifier
                Classifier #1


                Set of rules
                for class #1

...
Procedure of SAFCS
T = Tmax
Scurrent = Sinit
Sbest = Scurrent
EFcurrent = NNCP(Scurrent)
EFbest = NNCP(Sbest)
While (T ≥ T...
Procedure of SAFCS
                                                      Mb
                                              ...
Metropolis Procedure
Snew = Perturb (Scurrent)              # generate new S
EFnew = NNCP(Snew)
ΔEF = EFnew - EFcurrent
IF...
Perturbation(3 func.)
1. Modify
  •  select a rule from S randomly
  •  modified one or more antecedent of it
  •  if the c...
Experiments
Parameters
 Parameters                            Values
 Initial set of rule size (Ninit)       50
 Initial temperature (...
Competes
• C4.5:
• IBk: nearest neighbor, k = 3
• Naive Bayes
• LIBSVM
• XCS: Michigan approach
• GAssist: Pittsburgh appr...
Dataset (UCI)
                                                                                                            ...
Progress 1/2
Progress 2/2
Accuracies
                                                                                                               ...
Upcoming SlideShare
Loading in …5
×

Data Mining With A Simulated Annealing Based Fuzzy Classification System

811 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
811
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide





















  • Data Mining With A Simulated Annealing Based Fuzzy Classification System

    1. 1. Pattern Recognition 41 (2008) 1824-1833 Presenter: Chia-Ming Wang
    2. 2. Goal and Contribution
    3. 3. Goal and Contribution • Construct a fuzzy classifier (Goal)
    4. 4. Goal and Contribution • Construct a fuzzy classifier (Goal) • map the attrs. to predefined fuzzy sets
    5. 5. Goal and Contribution • Construct a fuzzy classifier (Goal) • map the attrs. to predefined fuzzy sets • rules with conf. and target label (How?)
    6. 6. Goal and Contribution • Construct a fuzzy classifier (Goal) • map the attrs. to predefined fuzzy sets • rules with conf. and target label (How?) • combination optimization problem (6 ) n
    7. 7. Goal and Contribution • Construct a fuzzy classifier (Goal) • map the attrs. to predefined fuzzy sets • rules with conf. and target label (How?) • combination optimization problem (6 ) n • Use SA to find a set of fuzzy rules (Contribution)
    8. 8. Goal and Contribution • Construct a fuzzy classifier (Goal) • map the attrs. to predefined fuzzy sets • rules with conf. and target label (How?) • combination optimization problem (6 ) n • Use SA to find a set of fuzzy rules (Contribution) • authors said
    9. 9. Goal and Contribution • Construct a fuzzy classifier (Goal) • map the attrs. to predefined fuzzy sets • rules with conf. and target label (How?) • combination optimization problem (6 ) n • Use SA to find a set of fuzzy rules (Contribution) • authors said
    10. 10. The used antecedent fuzzy sets 1.0 Membership 1.small 2.medium small S MS M ML L 3.medium 4.medium large Attribute Value 0.0 1.0 5.large 6.don’t care Membership 1.0 if x1 is small and x2 is medium and DC x3 is don’t care Encode: 136 Attribute Value 0.0 1.0
    11. 11. Determination of Cj and CFj 1. calculate the compatibility of each training pattern xp with the rule Rj µ j (x p ) = µ j1 (x p1 ) ×L × µ jn (x pn ), p = 1,K , m 2. for each class, calculate the relative sum of compatibility grades of the training patterns in class h with the rule Rj ∑ βClass h (R j ) = µ j (x p ) / N Class h x p ∈Class h 3. Find class hj hat { } if 0 or conflict, set Cj be φ βClass h j (R j ) = max βClass1 (R j ),L , βClass c (R j ) ) 4. if Cj = φ, set CFj of rule Rj to 0. Otherwise c ( ) where β = ∑ βClass h (R j ) / (c − 1) CFj = βClass h (R j ) − β / ∑ βClass h (R j ) ) ) j h=1 h≠ h j 5. classify the sample xp with rule set S { } reject if µ j (x p ) = 0 ∀R j ∈S µ j* (x p ) ⋅ CFj* = max µ j (x p ) ⋅ CFj R j ∈S
    12. 12. Structure of the goal classifier
    13. 13. Structure of the goal classifier Classifier #1 Set of rules for class #1 Classifier #2 Set of rules Decision for class #2 Detected Test . Fusion Class Dataset . . Classifier #c Set of rules for class #c
    14. 14. Procedure of SAFCS T = Tmax Scurrent = Sinit Sbest = Scurrent EFcurrent = NNCP(Scurrent) EFbest = NNCP(Sbest) While (T ≥ Tmin) For i = 1 to k Call Metropolis(Scurrent, EFcurrent, Sbest, EFbest,T) Time = Time + k k=β×k T = α ×T Return(Sbest)
    15. 15. Procedure of SAFCS Mb 1 −ΔEFb T = Tmax # M = Mg + Mb ∑ ΔEFb Tmax ΔEFb = = Mb ln(Pinit ) Scurrent = Sinit # Ninit i=1 Sbest = Scurrent N EFcurrent = NNCP(Scurrent) #NNCP(S)= m− ∑ NCP(Rj ) EFbest = NNCP(Sbest) j=1 While (T ≥ Tmin) # Tmin = 0.01 For i = 1 to k # k is num of calls of metropolis Call Metropolis(Scurrent, EFcurrent, Sbest, EFbest,T) Time = Time + k # Time is the spend time so far k=β×k # β is a constant (set to 1) T = α ×T # α is the cooling rate (set to 0.9) Return(Sbest)
    16. 16. Metropolis Procedure Snew = Perturb (Scurrent) # generate new S EFnew = NNCP(Snew) ΔEF = EFnew - EFcurrent IF (ΔEF < 0), Then # better rule set Scurrent = Snew IF EFnew < EFbest , Then # better than best Sbest = Snew ELSEIF (rand(0,1) < exp(-ΔEF/T)), Then # accept, too Scurrent = Snew
    17. 17. Perturbation(3 func.) 1. Modify • select a rule from S randomly • modified one or more antecedent of it • if the consequent is equal, then replace; otherwise, repeated 2. Delete f itnessmax (SClass h ) − f itness(R) P (R) = select with, f itnessmax (SClass h ) − f itnessmin (SClass h ) 3. Create the same as modify, but add (NB:change more linguistic values than“Modify”, they said for jump)
    18. 18. Experiments
    19. 19. Parameters Parameters Values Initial set of rule size (Ninit) 50 Initial temperature (Tmax) 100 Final temperature (Tmin) 0.01 Cooling rate (α) 0.90 # Iteration at each temperature (k) 40 Iteration increment rate (β) 1 Estimate: 88 × 40 = 3520 iterations (keep in mind)
    20. 20. Competes • C4.5: • IBk: nearest neighbor, k = 3 • Naive Bayes • LIBSVM • XCS: Michigan approach • GAssist: Pittsburgh approach
    21. 21. Dataset (UCI) 1829 H. Mohamadi et al. / Pattern Recognition 41 (2008) 1824 – 1833 Table 1 Features of the data sets used in computational experiments Name #Instance #Attribute #Real. #Nominal #Class Dev. cla. (%) Mag. cla. (%) Min. cla. (%) bswd 625 4 4 – 3 18.03 46.08 7.84 cra 690 15 6 9 2 5.51 55.51 44.49 ion 351 34 34 – 2 14.10 64.10 35.90 iris 150 4 4 – 3 – 33.33 33.33 lab 57 16 8 8 2 14.91 64.91 35.09 pima 768 8 8 – 2 15.10 65.10 34.90 wave 5000 40 40 – 3 0.36 33.84 33.06 wine 178 13 13 – 3 5.28 39.89 26.97 Dev.cla., deviation of class distribution; Mag. Cla, percentage of majority class instances; Min. Cla, percentage of minority class instances. IBk [31] is the nearest neighbor classifier technique. It uses Table 2 Parameters specification in computer simulations for the SAFCS the whole training set as the core of the classifier and Euclidean distance to select the k nearest instances. The class prediction Parameter Value 10-fold cross validation provided by the system is the majority class in these k examples. Initial set of rules size (Ninit ) 50 Here, k is set equal to 3. Initial temperature (Tmax ) 100 Naïve Bayes [32] is a very simple Bayesian network approach Final temperature (Tmin ) 0.01 that assumes that the predictive attributes are conditionally Cooling rate ( ) 0.90 independent given the class and also that no hidden or latent # Iteration at each temperature (k) 40 Iteration increment rate ( ) 1 attributes influence the prediction process. These assumptions
    22. 22. Progress 1/2
    23. 23. Progress 2/2
    24. 24. Accuracies 1831 H. Mohamadi et al. / Pattern Recognition 41 (2008) 1824 – 1833 Table 3 Train set and test set accuracies of different algorithms on eight UCI data sets (mean ± standard deviation) Data set Algorithm C4.5 IBk Naïve Bayes SVM GAssist XCS SAFCS 95.19 ± 1.28 89.93 ± 0.68 90.53 ± 0.54 91.92 ± 0.25 91.01 ± 0.19 92.14 ± 0.28 94.63 ± 0.46 bswd Train set accuracy % 91.43 ± 1.25 77.66 ± 2.91 86.09 ± 2.72 90.90 ± 1.43 89.62 ± 2.22 81.10 ± 3.80 90.47 ± 1.36 Test set accuracy % 98.90 ± 0.73 90.31 ± 0.86 91.05 ± 0.52 82.58 ± 0.82 55.51 ± 0.08 91.07 ± 0.73 94.25 ± 0.54 cra Train set accuracy % 85.77 ± 3.27 85.55 ± 3.45 84.73 ± 4.04 81.07 ± 5.32 55.51 ± 0.70 85.62 ± 4.00 85.60 ± 3.5 Test set accuracy % 99.86 ± 0.24 98.68 ± 0.54 90.94 ± 0.59 93.00 ± 0.42 94.19 ± 0.64 96.90 ± 0.74 99.66 ± 0.34 ion Train set accuracy % 92.71 ± 5.01 88.97 ± 5.91 85.66 ± 4.66 91.50 ± 4.70 92.14 ± 4.62 90.10 ± 4.70 91.89 ± 4.65 Test set accuracy % 99.85 ± 0.19 98.00 ± 0.61 96.59 ± 0.49 96.67 ± 0.53 97.11 ± 0.64 98.33 ± 0.79 99.10 ± 1.19 iris Train set accuracy % 96.66 ± 3.09 94.22 ± 5.37 94.89 ± 6.37 96.22 ± 5.36 96.22 ± 4.77 95.20 ± 5.87 94.70 ± 5.10 Test set accuracy % 100 ± 0.00 91.58 ± 4.00 98.77 ± 1.55 95.92 ± 1.60 96.04 ± 0.93 99.92 ± 0.24 99.96 ± 0.08 lab Train set accuracy % 97.83 ± 5.33 80.31 ± 17.44 95.38 ± 7.75 93.76 ± 10.50 93.35 ± 8.32 97.77 ± 5.98 83.50 ± 14.80 Test set accuracy % 98.90 ± 0.67 84.43 ± 2.41 85.67 ± 0.65 77.07 ± 0.61 78.27 ± 0.53 83.11 ± 0.82 87.55 ± 0.59 pima Train set accuracy % 77.32 ± 4.70 75.44 ± 4.79 74.52 ± 3.91 75.30 ± 4.45 74.46 ± 5.19 72.40 ± 5.30 75.71 ± 4.41 Test set accuracy % 97.29 ± 0.61 81.59 ± 0.21 78.28 ± 0.60 85.02 ± 0.18 wave Train set accuracy % N/A N/A N/A 80.00 ± 1.16 75.93 ± 2.10 79.89 ± 1.40 76.01 ± 1.97 Test set accuracy % N/A N/A N/A 100 ± 0.00 100 ± 0.00 98.86 ± 0.54 97.27 ± 0.53 98.67 ± 0.45 99.33 ± 0.32 99.98 ± 0.04 wine Train set accuracy % 98.10 ± 3.40 94.24 ± 6.44 96.61 ± 4.02 97.20 ± 3.43 96.33 ± 4.13 95.60 ± 4.90 97.63 ± 3.02 Test set accuracy % The best values are in bold. C4.5 IBk NB LIBSVM Gassist XCS SAFCS

    ×