Similarity Features, and their Role in Concept Alignment Learning

Introduction Three classiﬁers Experiments and results Summary Similarity Features, and their Role in Concept Alignment Learning Shenghui Wang1 Gwenn Englebienne2 Christophe Gu´ret e 1 Stefan Schlobach1 Antoine Isaac1 Martijn Schut1 1 Vrije Universiteit Amsterdam 2 Universiteit van Amsterdam SEMAPRO 2010 Florence

Introduction Three classifiers Experiments and results Summary Outline 1 Introduction Classification of concept mappings based on instance similarity 2 Three classifiers Markov Random Field Multi-objective Evolution Strategy Support Vector Machine 3 Experiments and results 4 Summary

Introduction Three classiﬁers Experiments and results Summary Thesaurus mapping SemanTic Interoperability To access Cultural Heritage (STITCH) through mappings between thesauri Scope of the problem: Big thesauri with tens of thousands of concepts Huge collections (e.g., National Library of the Neterlands: 80km of books in one collection) Heterogeneous (e.g., books, manuscripts, illustrations, etc.) Multi-lingual problem Solving matching problems is one step to the solution of the interoperability problem. e.g., “plankzeilen” vs. “surfsport” e.g., “archeology” vs. “excavation”

Introduction Three classifiers Experiments and results Summary Automatic alignment techniques Lexical labels and textual information of entities Structural structure of the formal definitions of entities, position in the hierarchy Extensional statistical information of instances, i.e., objects indexed with entities Background knowledge using a shared conceptual reference to find links indirectly

Introduction Three classiﬁers Experiments and results Summary Instance-based techniques: common instance based

Introduction Three classiﬁers Experiments and results Summary Pros and cons Advantages Simple to implement Interesting results Disadvantages Requires suﬃcient amounts of common instances Only uses part of the available information

Introduction Three classiﬁers Experiments and results Summary Instance-based techniques: Instance similarity based

Introduction Three classiﬁers Experiments and results Summary Representing concepts and the similarity between them { Creator Creator Title Term 1: 4 Publisher Bag of words Term 2: 1 Term 3: 0 Concept 1 ... ... Creator Title Title Term 1: 0 Publisher Bag of words Term 2: 3 Term 3: 0 ... ... Creator Publisher Cos. dist. Title Term 1: 2 Bag of words Term 2: 1 Publisher Term 3: 3 ... ... f1 Cos. dist. f2 { Creator f3 Term 1: 2 ... Bag of words Term 2: 0 Creator Term 3: 0 Title ... Cos. dist. Concept 2 Publisher Title ... Term 1: 0 Bag of words Term 2: 4 Creator Term 3: 1 Title ... Publisher Publisher ... Term 1: 4 Bag of words Term 2: 1 Term 3: 1 ... { { { Instance features Concept features Pair features

Introduction Three classifiers Experiments and results Summary Classification of concept mappings based on instance similarity Classification based on instance similarity Each pair of concepts is treated as a point in a “similarity space” Its position is defined by the features of the pair. The features of the pair are the different measures of similarity between the concepts’ instances. Hypothesis: the label of a point — which represents whether the pair is a positive mapping or negative one — is correlated with the position of this point in this space. With already labelled points and the actual similarity values of concepts involved, it is possible to classify a point, i.e., to give it a right label, based on its location given by the actual similarity values.

Introduction Three classifiers Experiments and results Summary Classification of concept mappings based on instance similarity Research questions How do different classifiers perform on this instance-based mapping task? What are the benefits of using a machine learning algorithm to determine the importance of features? Are there regularities wrt. the relative importance given to specific features for similarity computation? Are these weights related to application data characteristics?

Introduction Three classiﬁers Experiments and results Summary Three classiﬁers used Markov Random Field (MRF) Evolutionary Strategy (ES) Support Vector Machine (SVM)

Introduction Three classiﬁers Experiments and results Summary Markov Random Field Markov Random Field Let T = { (x(i) , y (i) ) }N be the training set i=1 x(i) ∈ RK , the features y (i) ∈ Y = {positive, negative}, the label The conditional probability of a label given the input is modelled as K 1 p(y (i) |xi , θ) = exp λj φj (y (i) , x(i) ) , (1) Z (xi , θ) j=1 where θ = { λj }K are the weights associated to the feature j=1 functions φ and Z (xi , θ) is a normalisation constant

Introduction Three classifiers Experiments and results Summary Markov Random Field The classifier used: Markov Random Field (cont’) The likelihood of the data set for given model parameters p(T |θ) is given by: N p(T |θ) = p(y (i) |x(i) ) (2) i=1 During learning, our objective is to find the most likely values for θ for the given training data. The decision criterion for assigning a label y (i) to a new pair of concepts i is then simply given by: y (i) = argmax p(y |x(i) ) (3) y

Introduction Three classifiers Experiments and results Summary Multi-objective Evolution Strategy Multi-objective Evolution Strategy Evolutionary strategies (ES) have two characteristic properties: firstly, they are used for continuous value optimisation, and, secondly, they are self-adaptive. An ES individual is a direct model of the searched solution, defined by Λ and some evolution strategy parameters: Λ, Σ ↔ λ1 , . . . , λK , σ1 , . . . , σK (4) The fitness function is related to the decision criterion for the ES, which is sign-based: K 1 if j=1 λi Fij > 0 LES = i (5) 0 otherwise

Introduction Three classiﬁers Experiments and results Summary Multi-objective Evolution Strategy Multi-objective Evolution Strategy (cont’) Maximising the number of positive results and negative results are two opposite goals. K f1 (Λ | F , L) = #{Fi | λi Fij > 0 ∧ Li = 1} (6) j=1 K f2 (Λ | F , L) = #{Fi | λi Fij ≤ 0 ∧ Li = 0} (7) j=1

Introduction Three classiﬁers Experiments and results Summary Multi-objective Evolution Strategy Multi-objective Evolution Strategy (cont’) Evolution process Recombination: Two parent individuals are combined using diﬀerent weighting, producing two new individuals Mutation: One parent individual changes itself into a new child individual Survivor selection: NSGA-II

Introduction Three classifiers Experiments and results Summary Support Vector Machine Support Vector Machine Support Vector Machine (SVM) is used as a maximum margin classifier whose task consists in finding an hyperplane separating the two classes. The objective is to maximise the margin separating the two classes whilst minimizing classification error risk.

Introduction Three classiﬁers Experiments and results Summary Experiments and results Thesauri to match: GTT (35K) and Brinkman (5K) Instances: 1 million books GTT annotated books: 307K Brinkman annotated books: 490K Dually annotated books: 222K

Introduction Three classiﬁers Experiments and results Summary Feature slection for similarity calculation λj Feature λj Feature λj Feature 1 Lexical 11 author 21 issued 2 Jaccard 12 contributor 22 language 3 Date 13 creator 23 mods:edition 4 ISBN 14 dateCopyrighted 24 publisher 5 NBN 15 description 25 refNBN 6 PPN 16 extent 26 relation 7 SelSleutel 17 hasFormat 27 spatial 8 abstract 18 hasPart 28 subject 9 alternative 19 identiﬁer 29 temporal 10 annotation 20 isVersionOf 30 title Table: List of the features

Introduction Three classiﬁers Experiments and results Summary Quality of learning MRF 1-30 ES MRF 3-30 SVM 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 Precision Recall F-measure Precision Recall F-measure Figure: Precision, recall and F-Measure for mappings with a positive label (top) and a negative label (bottom). Error bars indicate one standard deviation over the 10 folds of cross-validation.

Introduction Three classiﬁers Experiments and results Summary Relative importance of features Which features of our instances are important for mapping? Figure: Mutual information between features and labels

Introduction Three classiﬁers Experiments and results Summary Relative importance of features ES lambdas are not really conclusive ES lambdas that are most inconclusive correspond to the least informative features

Introduction Three classiﬁers Experiments and results Summary Relative importance of features Important features in terms of mutual information are associated to large MRF weights

Introduction Three classiﬁers Experiments and results Summary A more detailed analysis Expected important features: Label similarity (1), instance overlap (2), subject (28), etc. Expected unimportant features: Size of the book (16), format description (17) and language (22), etc. Surprisingly important features: Date (14) Surprisingly unimportant features: Description (15) and abstract (8)

Introduction Three classifiers Experiments and results Summary Summary We tried three machine learning classifiers on the instance-based mapping task, among which MRF and ES can automatically identify meaningful features. The MRF and the ES, result in a performance in the neighbourhood of 90%, showing the validity of the approach. Our analysis suggests that when many different description features interact, there is no systematic correlation between what a learning method could find and what an application expert may anticipate.

Introduction Three classiﬁers Experiments and results Summary Thank you

Similarity Features, and their Role in Concept Alignment Learning

by Shenghui Wang on Oct 26, 2010

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Similarity Features, and their Role in Concept Alignment Learning Presentation Transcript