Data Mining OPtimization Ontology and its application to meta-mining of knowledge discovery processes

Data Mining OPtimization Ontology and its application
to meta-mining of knowledge discovery processes
Agnieszka Lawrynowicz
collaboration with C. Maria Keet, Melanie Hilario, Claudia d’Amato,
Jedrzej Potoniec and others - see acknowledgements
Poznan University of Technology, Poland
25th September 2014
OEG group seminar at Universidad Polit´ecnica de Madrid (UPM)
Agnieszka Lawrynowicz collaboration with C. Maria Keet, Melanie Hilario, Claudia d’Amato, Jedrzej Potoniec and others - see acknowleData Mining OPtimization Ontology and its application to meta-mining of knowledge disco
25th September 2014 OEG group sem
/ 31

Outline
Overview of DMOP: purpose, scope, core classes
Modeling issues
▸ meta-modeling in DMOP;
▸ alignment of DMOP with the DOLCE foundational ontology;
▸ qualities and attributes;
▸ property chains in DMOP;
▸ other modeling considerations;
Meta-mining of KDD processes
▸ RapidMiner
▸ RMOnto
▸ Fr-ONT-Qu
▸ experimental evaluation
/ 31

Data Mining OPtimization Ontology (DMOP)
the primary goal of DMOP is to support all decision-making steps
that determine the outcome of the data mining process;
development started in EU FP7 project e-LICO (2009-2012);
DMOP v5.4: ∼ 750 classes, ∼ 200 properties, ∼ 3200 axioms;
highly axiomatized;
using almost all of OWL 2 features;
/ 31

Overview of meta-learning
Meta-learning: learning to learn
application of machine learning techniques to meta-data about past
machine learning experiments;
the goal: to modify some aspect of the learning process to improve
the performance of the resulting model;
meta-mining: meta-learning applied to full data mining process
/ 31

Overview of the e-LICO system
.,+1B/0DF'4;)<'<!=1)/*'.0!<!*1<'1;!'?)BB!0!*1'=/D./*!*1<'/B'1;!'!"#$%&'+0=;)1!=1>0!'G()E>0!'7H'+*?'
<;/I<';/I'1;!A')*1!0+=1'1/'+=;)!J!'1;!'><!0K<'L*/I,!?E!'?)<=/J!0A'E/+,F''
4;!'!"#$%&')*B0+<10>=1>0!'G?!.)=1!?')*'1;!'B)E>0!'>*?!0'1;!'?+<;!?',)*!H')<'1;!'D!+*<'MA'I;)=;'1;!'
?+1+"D)*)*E'.,+1B/0D')<'?!,)J!0!?'1/'<=)!*1)<1<F'4;!')**/J+1)J!'=/0!''/B'1;!'!"#$%&'.,+1B/0D')<'1;!'
!"#$%%&'$"#( )&*+,-$./( 0**&*#1"#' G$NOP' +M/J!' 1;!' ?+<;!?' ,)*!H' I)1;' )1<' .,+**!0' +*?' D!1+",!+0*!0F'
Q/I!J!0P'1/'?!,)J!0'1;!'?+1+"D)*)*E'.,+1B/0D'1/')1<'<=)!*1)<1'><!0<P'1;!0!'+0!'<!J!0+,'/1;!0'<!0J)=!<'
+*?'=/D./*!*1<F'()E>0!'7'<;/I<'+*'/J!0J)!I'/B'!"#$%&R<'=/D./*!*1<'+*?';/I'1;!A')*1!0+=1'I)1;'
!+=;'/1;!0F'
'
()E>0!'7F'&J!0J)!I'/B'1;!'!"#$%&'<A<1!DF''
4;!0!'+0!'1I/'><!0"B+=)*E'=/D./*!*1<'B/0'1;!'!"#$%&'.,+1B/0DS'1;!<!'+,,/I'<=)!*1)<1<'1/'+==!<<'?+1+"
D)*)*E' /.!0+1/0<' +*?T/0' /1;!0' ?+1+' .0/=!<<)*E' <!0J)=!<P' 1/' =/D./<!' 1;!D' )*1/' I/0LB,/I<' +*?'
!U!=>1!' 1;!DP' =/,,!=1)*E' 1;!' 0!<>,1<' B/0' )*1!0.0!1+1)/*' /0' B>01;!0' +*+,A<)<F' 4;!<!' 1I/' =!*10+,'
)*B0+<10>=1>0!'=/D./*!*1<'+0!V'Agnieszka Lawrynowicz collaboration with C. Maria Keet, Melanie Hilario, Claudia d’Amato, Jedrzej Potoniec and others - see acknowleData Mining OPtimization Ontology and its application to meta-mining of knowledge disco
/ 31

Competency questions
”Given a data mining task/data set, which of the valid or applicable
workflows/algorithms will yield optimal results (or at least better results
than the others)?”
”Given a set of candidate workflows/algorithms for a given task/data
set, which data set/workflow/algorithm characteristics should be
taken into account in order to select the most appropriate one?”
and others more fine-grained, e.g.:
”Which induction algorithms should I use (or avoid) when my dataset
has many more variables than instances?”
/ 31

Architecture of DMOP knowledge base and its satellite
triple stores
12 e-LICO
Figure 5: Architecture of DMOP knowledge base and its satellite triple stores
/ 31

The core concepts of DMOP
Fig. 1. The core concepts of DMOP.
/ 31

Meta-modeling in DMOP 1/4
only processes (executions of workﬂows) and operations (executions
of operators) consume inputs and produce outputs
DM algorithms (as well as operators and workﬂows) can only specify
the type of input or output
inputs and outputs (DM-Dataset and DM-Hypothesis class hierarchy,
respectively) are modeled as subclasses of IO-Object class
/ 31

DM algorithms: classes or individuals? Individuals.
Problem: expressing types of inputs/outputs associated with
algorithm
”C4.5 speciﬁesInputClass CategoricalLabeledDataSet”
Individual Class
(instance of DM-Algorithm) (subclass of DM-Hypothesis)
/ 31

Initial solution: one artificial class per each single algorithm with a
single instance corresponding to this particular algorithm
Problem: hasInput, hasOutput, specifiesInputClass,
specifiesOutputClass—assigned a common range—IO-Object
”C4.5 specifiesInputClass Iris” ?
Individual Individual
(instance of DM-Algorithm) (instance of DM-Hypothesis)
Iris is a concrete dataset. Clearly, any DM algorithm is not designed
to handle only a particular dataset.
/ 31

Final solution: weak form of punning available in OWL 2
IO-Class: meta-class—the class of all classes of input and output
objects
”C4.5 speciﬁesInputClass CategoricalLabeledDataSet”
Individual Individual
(instance of DM-Algorithm) (instance of IO-Class)
”DM-Process hasInput some CategoricalLabeledDataSet”
Class Class
(subclass of dolce:process) (subclass of IO-Object)
/ 31

Alignment of DMOP with DOLCE 1/2
Two main reasons to align DMOP with a foundational ontology:
considerations about attributes and data properties; extant
non-foundational ontology solutions were partial re-inventions of how
they are treated in a foundational ontology;
reuse of the ontology’s object properties;
/ 31

Alignment of DMOP with DOLCE 2/2
Perdurant: DM-Experiment and DM-Operation are subclasses of
dolce:process;
Endurant: most DM classes, such as algorithm, software, strategy,
task, and optimization problem, are subclasses of
dolce:non-physical-endurant;
Quality: characteristics and parameters of DM entities made
subclasses of dolce:abstract-quality;
Abstract: for identifying discrete values, classes added as subclasses
of dolce:abstract-region;
object properties: DMOP reuses mainly DOLCE’s parthood, quality,
and quale relations;
each of the four DOLCE main branches have been used.
/ 31

Qualities and attributes 1/4
How to handle ’attributes’ in OWL ontologies, and, in a broader context,
measurements?
easy way: attribute is a binary functional relation between a class and
a datatype
Elephant ⊑ =1 hasWeight.integer
Elephant ⊑ =1 hasWeightPrecise.real
Elephant ⊑ =1 hasWeightImperial.integer (in lbs)
building into one’s ontology application decisions about how to store
the data (and in which unit it is)
/ 31

How to handle ’attributes’ in OWL ontologies, and, in a broader context,
measurements?
more elaborate way: unfold the notion of an object’s property (e.g.
weight) from one attribute/OWL data property into at least two
properties: one OWL object property from the object to the ’reiﬁed
attribute’ (“quality property” represented as an OWL class) and
another property to the value(s)
▸ favoured in foundational ontologies;
▸ solves the problem of non-reusability of the ’attribute’ and prevents
duplication of data properties;
▸ neither ontology has any solution to represent actual values and units
of measurements;
measurements for DMOP more alike values for parameters;
/ 31

DM-Data
dolce:non-physical-endurant dolce:abstract
DataType DataFormat
dolce:quality
dolce:region
dolce:abstract-regiondolce:quale
dolce:abstract-quality
anyType
hasDataValue
Characteristic Parameter
hasDataType
hasDataType
dolce:has-quale
dolce:particular
dolce:has-quality
dolce:q-location
TableFormat
DataTable hasTableFormat
DataCharacteristic
has-quality
/ 31

ModelingAlgorithm ⊑ =1 has-quality.LearningPolicy
LearningPolicy is a dolce:quality
LearningPolicy ⊑ =1 has-quale.Eager-Lazy
Eager-Lazy is a subclass of dolce:abstract-region
Eager-Lazy ⊑ ≤ 1 hasDataValue.anyType
In this way, the ontology can be linked to many diﬀerent applications, who
even may use diﬀerent data types, yet still agree on the meaning of the
characteristics and parameters (’attributes’) of the algorithms, tasks, and
other DM endurants.
/ 31

Property chains
DMOP has 11 property chains;
principal issues in declaring safe property chains (guaranteed not to
cause unsatisﬁable classes or other undesirable deductions), are
declaring and choosing properties, and their domain and range axioms;
all investigated in detail in (Keet, EKAW ’2012) and adjusted were
necessary;
Example: hasMainTable ○ hasFeature ⊑ hasFeature
/ 31

Other modeling considerations
several other OWL 2 features were used;
ObjectInverseOf;
“object property characteristics” used sparingly, and only the basic
‘functional’ characteristic asserted;
local reflexivity investigated on a subsumes property for instances in
DMOP v5.2, but eventually modeled differently with classes and
metamodeling/punning;
DOLCE’s parthood is transitive, should be transitive in DMOP; but it
was discovered after the release of v5.3 that the object property copy
function in Protégé does not copy any property characteristics;
/ 31

What is RapidMiner? 1/2
/ 31

What is RapidMiner? 2/2
/ 31

RMonto - plugin to RapidMiner
/ 31

Fr-ONT-Qu
algorithm for mining patterns in RDF(s) data
patterns expressed as SPARQL queries
2 thresholds: for keeping good enough patterns and for refining best
patterns
several quality measures to select for thresholds (e.g. support on KB)
for classification task outperformed state-of-art approaches to
classification of Semantic Web data on tasks with available results
and datasets (see: ”Pattern based feature construction in semantic
data mining” by A. Lawrynowicz, J. Potoniec, IJSWIS 10(1), 2014):
▸ kernel methods Bloehdorn et al. (2007), Loesch et al. (ESWC 2012
best paper) on SWRC AIFB dataset,
▸ statistical relational classifier SPARQL-ML by Kiefer et al (ESWC 2008
best paper) on SWRC AIFB dataset and OWLS-TC v2.1 dataset,
▸ concept learning algorithms DL-FOIL by Fanizzi et al (2008),
DL-Learner cutting-edge CELOE variant by Lehmann (2009) on all
measures on datasets BioPax, NTN, Financial
/ 31

Fr-ONT-Qu - pattern based classiﬁcation
/ 31

Fr-ONT-Qu - trie data structure
/ 31

Semantic meta-mining experimental setup
baseline DM experiment set: 1581 RapidMiner workﬂows solving a
predictive modeling task on 11 UCI datasets
dataset characteristics meta-data stored in DMEX-DB containing
over 85 million of RDF triples
workﬂow patterns represented as SPARQL queries using DMOP
entities
/ 31

Semantic meta-mining results
McNemar’s test for pairs of classifiers performed with the null
hypothesis that a classifier built using dataset characteristics and a
mined pattern set has the same error rate as the baseline that used
dataset characteristics and only the names of the machine learning
DM operators
Test confirmed that classifiers trained using workflow patterns
performed significantly better (accuracy 0.927) than the baseline
(accuracy 0.890)
/ 31

Acknowledgements
EU FP7 ICT-2007.4.4 (No 231519) ”e-LICO: An e-Laboratory for
Interdisciplinary Collaborative Research in Data Mining and
Data-Intensive Science”
Foundation for Polish Science under the PARENT/BRIDGE
programme, coﬁnanced from European Union, Regional Development
Fund (No POMOST/2013-7/8)
Contributors to the development of DMOP and/or other e-LICO
infrastructure used in the research described in this presentation:
Claudia d’Amato, Huyen Do, Simon Fischer, Dragan Gamberger,
Melanie Hilario, Lina Al-Jadir, Simon Jupp, Alexandros Kalousis, C.
Maria Keet, Joerg Uwe-Kietz, Petra Kralj Novak, Babak Mougouie,
Phong Nguyen, Raul Palma, Jedrzej Potoniec, Floarea Serban, Robert
Stevens, Anze Vavpetic, Jun Wang, Derry Wijaya, Adam Woznica
RMonto and Meta-mining experiments done jointly with Jedrzej
Potoniec
Thanks to Veli Bicer for sharing the AIFB dataset
/ 31

Bibliography
Keet, C.M., Lawrynowicz, A., dAmato, C., Hilario, M.: Modeling issues, choices in the data mining optimization ontology.
In Rodriguez-Muro, M., et al., eds.: OWLED. Volume 1080 of CEUR Workshop Proceedings., CEUR-WS.org (2013)
Hilario, M., Nguyen, P., Do, H., Woznica, A., Kalousis, A. (2011). Ontology-Based Meta-Mining of Knowledge Discovery
Workﬂows. In N. Jankowski, W. Duch, K. Grabczewski (Ed.), Meta-Learning in Computational Intelligence (pp.
273-316). Springer.
Potoniec, J., Lawrynowicz, A. (2011b). RMonto: Ontological extension to RapidMiner. Poster and Demo Session of the
ISWC 2011 - 10th International Semantic Web Conference.
Lawrynowicz, A., Potoniec, J.:Pattern Based Feature Construction in Semantic Data Mining. IJSWIS 10(1) (2014)
Keet, C.M, Detecting and Revising Flaws in OWL Object Property Expressions. EKAW 2012: 252-266
Serban, F., Vanschoren, J., Kietz, J.-U., Bernstein, A. (2012). A survey of intelligent assistants for data analysis. ACM
Computing Surveys
/ 31

Data Mining OPtimization Ontology and its application to meta-mining of knowledge discovery processes

Recommended

Recommended

More Related Content

Similar to Data Mining OPtimization Ontology and its application to meta-mining of knowledge discovery processes

Similar to Data Mining OPtimization Ontology and its application to meta-mining of knowledge discovery processes (20)

More from Agnieszka Ławrynowicz

More from Agnieszka Ławrynowicz (6)

Recently uploaded

Recently uploaded (20)

Data Mining OPtimization Ontology and its application to meta-mining of knowledge discovery processes