SlideShare a Scribd company logo
1 of 10
Download to read offline
Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla

                                 nmacia@salle.url.edu
                     jaume.bacardit@nottingham.ac.uk
                                  esterb@salle.url.edu
Methodology for learners’ assessment
The UCI repository. 134 classification problems



Source: GECCO’11 Proceedings
Algorithm refinement        Knowledge extraction




              Behind the experiments
Standard comparison



Source: Jaume Bacardit, Edmund K. Burke, and Natalio Krasnogor. Improving the scalability of rule-based evolutionary learning. (2009)
Taxonomy of problems
Complexity coverage
• How many data sets should we use in the experiments?
• Which ones? Synthetic data sets? Real-world problems?
  Both?
• Is the UCI our best sample?
• How should we select our referenced learners?
• Should we keep performing comparisons over an arbitrary
  set of problems?
• How can we characterise problems?
• More coverage? Benchmarks?
• Should we tackle one problem at a time?
• …
                      Questions. Answers?
Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla

                                 nmacia@salle.url.edu
                     jaume.bacardit@nottingham.ac.uk
                                  esterb@salle.url.edu

More Related Content

More from Daniele Loiacono

GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013Daniele Loiacono
 
EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013Daniele Loiacono
 
2013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-20132013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-2013Daniele Loiacono
 
2012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-20122012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-2012Daniele Loiacono
 
2012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-20122012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-2012Daniele Loiacono
 
2012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-20122012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-2012Daniele Loiacono
 
Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012Daniele Loiacono
 
XCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental ForgettingXCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental ForgettingDaniele Loiacono
 
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...Daniele Loiacono
 
Introducing LCS to Digital Design Verification
Introducing LCS to Digital Design VerificationIntroducing LCS to Digital Design Verification
Introducing LCS to Digital Design VerificationDaniele Loiacono
 
A temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networksA temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networksDaniele Loiacono
 
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...Daniele Loiacono
 
Automatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier SystemsAutomatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier SystemsDaniele Loiacono
 
Voting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label ClassificationVoting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label ClassificationDaniele Loiacono
 
2011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-20112011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-2011Daniele Loiacono
 
2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-20102010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-2010Daniele Loiacono
 
2010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-20102010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-2010Daniele Loiacono
 
2010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-20102010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-2010Daniele Loiacono
 
Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010Daniele Loiacono
 

More from Daniele Loiacono (20)

GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013GPUs for GEC Competition @ GECCO-2013
GPUs for GEC Competition @ GECCO-2013
 
EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013EvoRobocode Competition @ GECCO-2013
EvoRobocode Competition @ GECCO-2013
 
2013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-20132013 Simulated Car Racing @ GECCO-2013
2013 Simulated Car Racing @ GECCO-2013
 
2012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-20122012 Simulated Car Racing Championship @ CIG-2012
2012 Simulated Car Racing Championship @ CIG-2012
 
2012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-20122012 Simulated Car Racing Championship @ GECCO-2012
2012 Simulated Car Racing Championship @ GECCO-2012
 
2012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-20122012 Simulated Car Racing Championship @ Evo*-2012
2012 Simulated Car Racing Championship @ Evo*-2012
 
Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012Computational Intelligence in Games Tutorial @GECCO2012
Computational Intelligence in Games Tutorial @GECCO2012
 
XCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental ForgettingXCSF with Local Deletion: Preventing Detrimental Forgetting
XCSF with Local Deletion: Preventing Detrimental Forgetting
 
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
Random Artificial Incorporation of Noise in a Learning Classifier System Envi...
 
One Step Fits All
One Step Fits AllOne Step Fits All
One Step Fits All
 
Introducing LCS to Digital Design Verification
Introducing LCS to Digital Design VerificationIntroducing LCS to Digital Design Verification
Introducing LCS to Digital Design Verification
 
A temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networksA temporal classifier system using spiking neural networks
A temporal classifier system using spiking neural networks
 
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
Confusion Matrices for Improving Performance of Feature Pattern Classifier Sy...
 
Automatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier SystemsAutomatically Defined Functions for Learning Classifier Systems
Automatically Defined Functions for Learning Classifier Systems
 
Voting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label ClassificationVoting Based Learning Classifier System for Multi-Label Classification
Voting Based Learning Classifier System for Multi-Label Classification
 
2011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-20112011 Simulated Car Racing Championship @ GECCO-2011
2011 Simulated Car Racing Championship @ GECCO-2011
 
2010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-20102010 Simulated Car Racing Championship @ CIG-2010
2010 Simulated Car Racing Championship @ CIG-2010
 
2010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-20102010 Simulated Car Racing Championship @ GECCO-2010
2010 Simulated Car Racing Championship @ GECCO-2010
 
2010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-20102010 Simulated Car Racing Championship @ WCCI-2010
2010 Simulated Car Racing Championship @ WCCI-2010
 
Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010Car Setup Optimization Competition @ EvoStar 2010
Car Setup Optimization Competition @ EvoStar 2010
 

Recently uploaded

“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...Marc Dusseiller Dusjagr
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13Steve Thomason
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docxPoojaSen20
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdfQucHHunhnh
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdfSoniaTolstoy
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Sapana Sha
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphThiyagu K
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxpboyjonauth
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingTechSoup
 
CARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxCARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxGaneshChakor2
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityGeoBlogs
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Educationpboyjonauth
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)eniolaolutunde
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxSayali Powar
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformChameera Dedduwage
 

Recently uploaded (20)

“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
 
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptxINDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13
 
mini mental status format.docx
mini    mental       status     format.docxmini    mental       status     format.docx
mini mental status format.docx
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptx
 
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdfTataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy Consulting
 
CARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxCARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptx
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activity
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Education
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy Reform
 

Testing learning classifier systems

  • 1. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla nmacia@salle.url.edu jaume.bacardit@nottingham.ac.uk esterb@salle.url.edu
  • 3.
  • 4. The UCI repository. 134 classification problems Source: GECCO’11 Proceedings
  • 5. Algorithm refinement Knowledge extraction Behind the experiments
  • 6. Standard comparison Source: Jaume Bacardit, Edmund K. Burke, and Natalio Krasnogor. Improving the scalability of rule-based evolutionary learning. (2009)
  • 9. • How many data sets should we use in the experiments? • Which ones? Synthetic data sets? Real-world problems? Both? • Is the UCI our best sample? • How should we select our referenced learners? • Should we keep performing comparisons over an arbitrary set of problems? • How can we characterise problems? • More coverage? Benchmarks? • Should we tackle one problem at a time? • … Questions. Answers?
  • 10. Núria Macià, Jaume Bacardit, and Ester Bernadó-Mansilla nmacia@salle.url.edu jaume.bacardit@nottingham.ac.uk esterb@salle.url.edu