Nuria Macia, Jaume Bacardit, Ester Bernadó-Mansilla "Testing learning classifier systems", IWLCS, 2011

1. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla
nmacia@salle.url.edu
jaume.bacardit@nottingham.ac.uk
esterb@salle.url.edu

2. Methodology for learners’ assessment

4. The UCI repository. 134 classification problems
Source: GECCO’11 Proceedings

5. Algorithm refinement Knowledge extraction
Behind the experiments

6. Standard comparison
Source: Jaume Bacardit, Edmund K. Burke, and Natalio Krasnogor. Improving the scalability of rule-based evolutionary learning. (2009)

7. Taxonomy of problems

8. Complexity coverage

9. • How many data sets should we use in the experiments?
• Which ones? Synthetic data sets? Real-world problems?
Both?
• Is the UCI our best sample?
• How should we select our referenced learners?
• Should we keep performing comparisons over an arbitrary
set of problems?
• How can we characterise problems?
• More coverage? Benchmarks?
• Should we tackle one problem at a time?
• …
Questions. Answers?

10. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla
nmacia@salle.url.edu
jaume.bacardit@nottingham.ac.uk
esterb@salle.url.edu