SlideShare a Scribd company logo
1 of 26
Fighting Knowledge Acquisition Bottleneck  with   Argument Based Machine Learning Martin Mozina, Matej Guid, Jana Krivec, Aleksander Sadikov  and  Ivan Bratko ECAI 2008 Faculty of Computer and Information Science University of Ljubljana, Slovenia
Motivation for Knowledge Acquisition with Argument Based Machine Learning ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Combining Machine Learning and Expert Knowledge ,[object Object],[object Object],[object Object],IF ... THEN ... IF ... THEN ... ...
Combining Machine Learning and Expert Knowledge ,[object Object],[object Object],[object Object],IF ... THEN ... IF ... THEN ... ...
Combining Machine Learning and Expert Knowledge ,[object Object],[object Object],[object Object],IF ... THEN ... IF ... THEN ... ... ABML
Definition of  Argument Based Machine Learning ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],BK,T e i T a i e i
Argument Based Rule Learning ,[object Object],[object Object],[object Object],Name RegularJob Rich AccountStatus HairColor CreditApproved Mr. Bond No Yes Negative Blond Yes Mr. Grey No No Positive Grey No Miss White Yes No Positive Blond Yes Miss Silver Yes Yes Positive Blond Yes Mrs. Brown Yes No Negative Brown No Name RegularJob Rich AccountStatus HairColor CreditApproved Mr. Bond No Yes Negative Blond Yes Mr. Grey No No Positive Grey No Miss White Yes No Positive Blond Yes Miss Silver Yes Yes Positive Blond Yes Mrs. Brown Yes No Negative Brown No Name RegularJob Rich AccountStatus HairColor CreditApproved Mr. Bond No Yes Negative Blond Yes Mr. Grey No No Positive Grey No Miss White Yes No Positive Blond Yes Miss Silver Yes Yes Positive Blond Yes Mrs. Brown Yes No Negative Brown No Name RegularJob Rich AccountStatus HairColor CreditApproved Mr. Bond No Yes Negative Blond Yes Mr. Grey No No Positive Grey No Miss White Yes No Positive Blond Yes Miss Silver Yes Yes Positive Blond Yes Mrs. Brown Yes No Negative Brown No
Formal definition of  Argumented Examples ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
ABCN2 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Interactions between expert and ABML ,[object Object],[object Object],[object Object],[object Object],[object Object],Argument ,[object Object],[object Object],ABML critical example learn  data set
Interactions between expert and ABML ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Knowledge Acquisition of Chess Concepts  used in  a Chess Tutoring Application Case Study: Bad Bishop
The Concept of the Bad Bishop ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Data set ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],wGM Jana Krivec GM Garry Kasparov FM Matej Guid
Standard M achine  L earning  M ethods'  P erformance with  CRAFTY 's features only ,[object Object],[object Object],[object Object],[object Object],Method CA Brier score AUC Decision trees  ( C4.5) 73% 0,49 0,74 Logistic regression 70% 0,43 0,84 Rule learning (CN2) 72% 0,39 0,80
First Critical Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
Introducing new attributes into the domain and adding arguments to an example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
C ounter example ,[object Object],[object Object],[object Object],Counter example : “bad”, although  IMPR OVED_BISHOP_MOBILITY  is high. ,[object Object],Critical example : “not bad”,  IMPR OVED_BISHOP_MOBILITY  is high.
I mproving  A rguments  with Counter Examples ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Assesing “bad” pawns ,[object Object],BAD_PAWNS_AHEAD  =  16 + 24 + 2 =  42
After the Final Iteration... ,[object Object],[object Object],[object Object],Attribute Description BAD_PAWNS pawns  on the color of the square of the bishop  - weighted according to their squares ( bad pawns )  BAD_PAWNS_AHEAD  bad pawns  ahead of the bishop BAD_PAWNS _BLOCK_BISHOP_DIAGONAL  bad pawns that  block the bishop's (front) diagonals   BLOCKED_BAD_PAWNS  bad pawns  blocked by opponent's pawns or pieces   IMPROVED_BISHOP_MOBILITY  n umber of squares accessible to the bishop  taking into account only pawns of both opponents 
Classification Accuracy Through Iterations Results on the final dataset Method CA Brier score AUC Decision trees  ( C4.5) 89% 0,21 0,86 Logistic regression 88% 0,19 0,96 Rule learning (CN2) 88% 0,19 0,94 ABCN2 95% 0,11 0,97
Classification Accuracy Through Iterations ,[object Object],[object Object],A rguments  suggested useful attributes  AND  lead to even more accurate models.
Advantages of ABML for Knowledge Acquisition explain single example easier for experts to articulate knowledge more knowledge from experts critical examples expert provide only relevant knowledge time of experts' involvent is decreased
Advantages of ABML for Knowledge Acquisition counter examples detect deficiencies in expert's explanations even more knowledge from experts arguments constrain learning hypotheses are consistent with expert knowledge hypotheses comprehensible to expert more accurate hypotheses
Conclusions ,[object Object],[object Object],[object Object],[object Object],ABML-based Knowledge Acquisition process provides: A rgument  Based Machine Learning enables  better knowledge acquisition

More Related Content

Viewers also liked

Knowledge extraction and visualisation using rule-based machine learning
Knowledge extraction and visualisation using rule-based machine learningKnowledge extraction and visualisation using rule-based machine learning
Knowledge extraction and visualisation using rule-based machine learningjaumebp
 
Introduction to Knowledge Management
Introduction to Knowledge ManagementIntroduction to Knowledge Management
Introduction to Knowledge ManagementMiera Idayu
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine LearningLior Rokach
 
Types of knowledge management systems
Types of knowledge management systemsTypes of knowledge management systems
Types of knowledge management systemsNitin Reddy Katkam
 
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Xavier Amatriain
 
Introduction to Big Data/Machine Learning
Introduction to Big Data/Machine LearningIntroduction to Big Data/Machine Learning
Introduction to Big Data/Machine LearningLars Marius Garshol
 

Viewers also liked (6)

Knowledge extraction and visualisation using rule-based machine learning
Knowledge extraction and visualisation using rule-based machine learningKnowledge extraction and visualisation using rule-based machine learning
Knowledge extraction and visualisation using rule-based machine learning
 
Introduction to Knowledge Management
Introduction to Knowledge ManagementIntroduction to Knowledge Management
Introduction to Knowledge Management
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
Types of knowledge management systems
Types of knowledge management systemsTypes of knowledge management systems
Types of knowledge management systems
 
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
 
Introduction to Big Data/Machine Learning
Introduction to Big Data/Machine LearningIntroduction to Big Data/Machine Learning
Introduction to Big Data/Machine Learning
 

Similar to Fighting Knowledge Acquisition Bottleneck with Argument Based ...

Machine learning and_nlp
Machine learning and_nlpMachine learning and_nlp
Machine learning and_nlpankit_ppt
 
Query Linguistic Intent Detection
Query Linguistic Intent DetectionQuery Linguistic Intent Detection
Query Linguistic Intent Detectionbutest
 
Cs221 lecture5-fall11
Cs221 lecture5-fall11Cs221 lecture5-fall11
Cs221 lecture5-fall11darwinrlo
 
introducción a Machine Learning
introducción a Machine Learningintroducción a Machine Learning
introducción a Machine Learningbutest
 
introducción a Machine Learning
introducción a Machine Learningintroducción a Machine Learning
introducción a Machine Learningbutest
 
4. Classification.pdf
4. Classification.pdf4. Classification.pdf
4. Classification.pdfJyoti Yadav
 
Supervised_Learning.ppt
Supervised_Learning.pptSupervised_Learning.ppt
Supervised_Learning.pptHari629251
 
Data Science: The Product Manager's Primer
Data Science: The Product Manager's PrimerData Science: The Product Manager's Primer
Data Science: The Product Manager's PrimerProduct School
 
Learning Positional Features for Annotating Chess Games: A ...
Learning Positional Features for Annotating Chess Games: A ...Learning Positional Features for Annotating Chess Games: A ...
Learning Positional Features for Annotating Chess Games: A ...butest
 
Barga Data Science lecture 9
Barga Data Science lecture 9Barga Data Science lecture 9
Barga Data Science lecture 9Roger Barga
 
Computational Biology, Part 4 Protein Coding Regions
Computational Biology, Part 4 Protein Coding RegionsComputational Biology, Part 4 Protein Coding Regions
Computational Biology, Part 4 Protein Coding Regionsbutest
 
Chapter09.ppt
Chapter09.pptChapter09.ppt
Chapter09.pptbutest
 

Similar to Fighting Knowledge Acquisition Bottleneck with Argument Based ... (20)

Explainable AI
Explainable AIExplainable AI
Explainable AI
 
Machine learning and_nlp
Machine learning and_nlpMachine learning and_nlp
Machine learning and_nlp
 
Query Linguistic Intent Detection
Query Linguistic Intent DetectionQuery Linguistic Intent Detection
Query Linguistic Intent Detection
 
Cs221 lecture5-fall11
Cs221 lecture5-fall11Cs221 lecture5-fall11
Cs221 lecture5-fall11
 
introducción a Machine Learning
introducción a Machine Learningintroducción a Machine Learning
introducción a Machine Learning
 
introducción a Machine Learning
introducción a Machine Learningintroducción a Machine Learning
introducción a Machine Learning
 
Explainable AI
Explainable AIExplainable AI
Explainable AI
 
4. Classification.pdf
4. Classification.pdf4. Classification.pdf
4. Classification.pdf
 
Supervised_Learning.ppt
Supervised_Learning.pptSupervised_Learning.ppt
Supervised_Learning.ppt
 
Data Science: The Product Manager's Primer
Data Science: The Product Manager's PrimerData Science: The Product Manager's Primer
Data Science: The Product Manager's Primer
 
Learning Positional Features for Annotating Chess Games: A ...
Learning Positional Features for Annotating Chess Games: A ...Learning Positional Features for Annotating Chess Games: A ...
Learning Positional Features for Annotating Chess Games: A ...
 
Barga Data Science lecture 9
Barga Data Science lecture 9Barga Data Science lecture 9
Barga Data Science lecture 9
 
Computational Biology, Part 4 Protein Coding Regions
Computational Biology, Part 4 Protein Coding RegionsComputational Biology, Part 4 Protein Coding Regions
Computational Biology, Part 4 Protein Coding Regions
 
MyStataLab Assignment Help
MyStataLab Assignment HelpMyStataLab Assignment Help
MyStataLab Assignment Help
 
Supervised algorithms
Supervised algorithmsSupervised algorithms
Supervised algorithms
 
Chapter09.ppt
Chapter09.pptChapter09.ppt
Chapter09.ppt
 
evaluation and credibility-Part 2
evaluation and credibility-Part 2evaluation and credibility-Part 2
evaluation and credibility-Part 2
 
Naive Bayes Presentation
Naive Bayes PresentationNaive Bayes Presentation
Naive Bayes Presentation
 
LR2. Summary Day 2
LR2. Summary Day 2LR2. Summary Day 2
LR2. Summary Day 2
 
Matching Conceptual Models Using Multivariate Analysis
Matching Conceptual Models Using Multivariate AnalysisMatching Conceptual Models Using Multivariate Analysis
Matching Conceptual Models Using Multivariate Analysis
 

More from butest

EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEbutest
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jacksonbutest
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer IIbutest
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazzbutest
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.docbutest
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1butest
 
Facebook
Facebook Facebook
Facebook butest
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...butest
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...butest
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTbutest
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docbutest
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docbutest
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.docbutest
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!butest
 

More from butest (20)

EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBE
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jackson
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer II
 
PPT
PPTPPT
PPT
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.doc
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1
 
Facebook
Facebook Facebook
Facebook
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENT
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.doc
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.doc
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.doc
 
hier
hierhier
hier
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!
 

Fighting Knowledge Acquisition Bottleneck with Argument Based ...

  • 1. Fighting Knowledge Acquisition Bottleneck with Argument Based Machine Learning Martin Mozina, Matej Guid, Jana Krivec, Aleksander Sadikov and Ivan Bratko ECAI 2008 Faculty of Computer and Information Science University of Ljubljana, Slovenia
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12. Knowledge Acquisition of Chess Concepts used in a Chess Tutoring Application Case Study: Bad Bishop
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22. Classification Accuracy Through Iterations Results on the final dataset Method CA Brier score AUC Decision trees ( C4.5) 89% 0,21 0,86 Logistic regression 88% 0,19 0,96 Rule learning (CN2) 88% 0,19 0,94 ABCN2 95% 0,11 0,97
  • 23.
  • 24. Advantages of ABML for Knowledge Acquisition explain single example easier for experts to articulate knowledge more knowledge from experts critical examples expert provide only relevant knowledge time of experts' involvent is decreased
  • 25. Advantages of ABML for Knowledge Acquisition counter examples detect deficiencies in expert's explanations even more knowledge from experts arguments constrain learning hypotheses are consistent with expert knowledge hypotheses comprehensible to expert more accurate hypotheses
  • 26.