The document discusses a methodology for assessing learners using the UCI repository, highlighting 134 classification problems. It raises several important questions regarding the selection of data sets, problem characterization, and evaluation benchmarks. The authors aim to improve the scalability of rule-based evolutionary learning through refined algorithms and knowledge extraction.

1. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla
nmacia@salle.url.edu
jaume.bacardit@nottingham.ac.uk
esterb@salle.url.edu

2. Methodology for learners’ assessment

4. The UCI repository. 134 classification problems
Source: GECCO’11 Proceedings

5. Algorithm refinement Knowledge extraction
Behind the experiments

6. Standard comparison
Source: Jaume Bacardit, Edmund K. Burke, and Natalio Krasnogor. Improving the scalability of rule-based evolutionary learning. (2009)

7. Taxonomy of problems

8. Complexity coverage

9. • How many data sets should we use in the experiments?
• Which ones? Synthetic data sets? Real-world problems?
Both?
• Is the UCI our best sample?
• How should we select our referenced learners?
• Should we keep performing comparisons over an arbitrary
set of problems?
• How can we characterise problems?
• More coverage? Benchmarks?
• Should we tackle one problem at a time?
• …
Questions. Answers?

10. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla
nmacia@salle.url.edu
jaume.bacardit@nottingham.ac.uk
esterb@salle.url.edu