Towards a Universal Wordnet by Learning from Combined Evidence

Introduction
Existing Lexical Knowledge Bases
Building a Multilingual Wordnet
Results and Experiments
Summary and Future Work
Towards a Universal Wordnet
by Learning from Combined Evidence
Gerard de Melo and Gerhard Weikum
Max Planck Institute for Informatics
Saarbr¨ucken, Germany
2009-11-03
Gerard de Melo and Gerhard Weikum Towards a Universal Wordnet 1/29

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Lexical Knowledge
What meanings does
a word have?
How do those meanings
relate to the meanings
of other words? person who
gives a talk
“speaker”
device that
produces
sounds

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Lexical Knowledge
What meanings does
a word have?
of other words?
flat piece
of wood
“board”
committee
panel for writing
with chalk
to enter a
transportation
vehicle

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Lexical Knowledge
What meanings does
a word have?
of other words?
someone who
studies
“student”
“pupil”

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Lexical Knowledge
What meanings does
a word have?
of other words?
faculty
professor
member
part

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Lexical Knowledge
What meanings does
a word have?
of other words?
entity
institution
educational
institution
university
...

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Lexical Knowledge
What meanings does
a word have?
of other words?
Many Applications
examples:
NLP, AI
question answering
query expansion
human consultation
entity
institution
educational
institution
university
...

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Multilinguality
the world is
multilingual
the Internet is also
increasingly
multilingual
Top 10 Languages by
Approx. No. of Speakers
Source: Ethnologue 2005

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
Multilinguality
the world is
multilingual
the Internet is also
increasingly
multilingual
Internet users by Region
Source:
Internet World Stats

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
person who
gives a talk
eng: “speaker”
jpn: “ ”話者
rus: “докладчик”
ces: “řečník”
... ......
Vision
universal index of word
meanings
large-scale semantic network
with class hierarchy
look up any word
in any language,
get a list of its meanings

Introduction
Lexical Knowledge
Multilinguality
Vision
Introduction
entitypor: “entidade”
cmn: “ ”制度 institution
educational
institution
university
heb: “‫ישות‬.”
deu: “Bildungs-
einrichtung”
cym: “prifysgol”
...
Vision
universal index of word
meanings
large-scale semantic network
with class hierarchy
meanings should be connected
via semantic relations

Introduction
Lexical Knowledge
Multilinguality
Vision
Outline
1 Existing Lexical Knowledge Bases
2 Building a Multilingual Wordnet
3 Results and Experiments
4 Summary and Future Work

Introduction
WordNet
Non-English Wordnets
Other Resources
Outline

Introduction
WordNet
Other Resources
WordNet
lexical database created at Princeton
enumerates meanings of English
words
meaning-to-meaning links
Miller, Fellbaum et al. (1990)
among most-cited papers
in computer science
(source: CiteseerX)

Introduction
WordNet
Other Resources
WordNet
words

Introduction
WordNet
Other Resources
WordNet
words
hypernym hierarchy
meronymy (part of)
etc.

Introduction
WordNet
Other Resources
EuroWordNet, BalkaNet, Global WordNet Association
problem: many are small, incomplete
problem: diﬀerent identiﬁers, formats, etc.
problem: only ∼10 languages with freely available wordnets

Introduction
WordNet
Other Resources
EuroWordNet, BalkaNet, Global WordNet Association
problem: many are small, incomplete
problem: diﬀerent identiﬁers, formats, etc.
problem: only ∼10 languages with freely available wordnets
not a single, coherent resource

Introduction
WordNet
Other Resources
Other Resources
PANGLOSS Ontology: Knight & Luk (1994)
TransGraph system: Etzioni et al. (2007)
DBPedia, YAGO, OpenCyc
2 languages, around 70 000 entities

Introduction
WordNet
Other Resources
Other Resources
large translation graph
limited structure
e.g. no semantic hierarchy

Introduction
WordNet
Other Resources
Other Resources
class hierarchy not multilingual

Introduction
Strategy
Input Graph
Approach
Features
Outline

Introduction
Strategy
Input Graph
Approach
Features
Strategy
use existing wordnets as backbone
add new terms, link to meaning nodes
spa: “trayectoria”
academic course
part of a meal
route of travel
series of events
eng: “course”
eng: “class”
Existing Wordnets

Introduction
Strategy
Input Graph
Approach
Features
Strategy
add new terms, link to meaning nodes
academic course
part of a meal
route of travel
series of events
eng: “course”
eng: “class”
Existing Wordnets
−→
deu: “Reihe”
academic course
part of a meal
route of travel
series of events
ita: “piatto”
fra: “suite”
eng: “course”
deu: “Kurs”
eng: “class”
Desired Output

Introduction
Strategy
Input Graph
Approach
Features
Input Graph
add translations to graph
mainly English, Spanish, Catalan
academic course
part of a meal
route of travel
series of events
eng: “course”
eng: “class”
Input Graph G0

Introduction
Strategy
Input Graph
Approach
Features
Input Graph
add translations to graph
dictionaries (e.g. Wiktionary)
thesauri and ontologies
parallel corpora (word alignment)
also: predict new translations
deu: “Reihe”
academic course
part of a meal
route of travel
series of events
ita: “piatto”
fra: “suite”
eng: “course”
deu: “Kurs”
eng: “class”
Input Graph G0

Introduction
Strategy
Input Graph
Approach
Features
Approach: Link new words to meanings of their translations
Huge Challenge: Disambiguation!
academic course
part of a meal
route of travel
series of events
ita: “piatto”
eng: “course”
trans-
lation

Introduction
Strategy
Input Graph
Approach
Features
Approach: Link new words to meanings of their translations
Huge Challenge: Disambiguation!
academic course
part of a meal
route of travel
series of events
ita: “piatto”
eng: “course”
trans-
lation
?
?
?
?

Introduction
Strategy
Input Graph
Approach
Features
academic course
part of a meal
route of travel
series of events
ita: “piatto”
eng: “course”
trans-
lation
?
?
?
?
Approach
variety of features that analyse
previous graph Gi−1,
incorporate neighbourhood
information into an
edge’s feature vector
supervised learning: new edge
weights determined using
RBF-kernel SVM with posterior
probability estimation

Introduction
Strategy
Input Graph
Approach
Features
Example Feature:
fra: “suite” academic course
?
t m
Given term t
and meaning m
Question: Should they be linked?

Introduction
Strategy
Input Graph
Approach
Features
Example Feature:
?
t m
fra: “suite”
eng: “course”
part of a meal
academic course
route of travel
...
series of eventst'
m'm'
Given term t
and meaning m
Question: Should they be linked?
Look at neighbours t ∈ Γt

Introduction
Strategy
Input Graph
Approach
Features
Example Feature:
?
t m
fra: “suite”
eng: “course”
part of a meal
academic course
route of travel
...
series of eventst'
m'm'
t ∈Γ(t)
sim∗(t , m)
sim∗(t , m) + dissim(t , m)

Introduction
Strategy
Input Graph
Approach
Features
Example Feature:
?
t m
fra: “suite”
eng: “course”
part of a meal
academic course
route of travel
...
series of eventst'
m'm'
t ∈Γ(t)
sim∗(t , m)
sim∗(t ,m)= max
m ∈Γ(t )
sim(m ,m)
dissim(t ,m)=
P
m ∈Γ(t )
(1−sim(m ,m))

Introduction
Strategy
Input Graph
Approach
Features
Example Feature:
?
t m
fra: “suite”
eng: “course”
part of a meal
academic course
route of travel
...
series of eventst'
m'm'
t ∈Γ(t)
φ1(t, t ) sim∗(t , m)
sim∗(t ,m)= max
m ∈Γ(t )
φ2(t ,m )sim(m ,m)
dissim(t ,m)=
P
m ∈Γ(t )
φ2(t ,m )(1−sim(m ,m))
weighting based on:
part-of-speech
corpus frequency
...

Introduction
Strategy
Input Graph
Approach
Features
deu: “Reihe”
academic course
part of a meal
route of travel
series of events
ita: “piatto”
fra: “suite”
eng: “course”
deu: “Kurs”
eng: “class”
Other Features
cosine similarity of
translations with gloss
scores assessing polysemy by
looking at back-translations
many more
(see paper for details)

Introduction
Strategy
Input Graph
Approach
Features
deu: “Reihe”
academic course
part of a meal
route of travel
series of events
ita: “piatto”
fra: “suite”
eng: “course”
deu: “Kurs”
eng: “class”
Approach
use scores as features for
RBF-kernel SVM
multiple iterations:
each graphs Gi based on the
previous Gi−1
stop when F1 score plateau
is reached on a validation set

Introduction
Setup
Output
Evaluation
Application: Semantic Relatedness
Application: Cross-Lingual Text Classiﬁcation
Outline

Introduction
Setup
Output
Evaluation
Results
Setup
input graph G0:
448,069 pre-existing term-meaning links
10,805,400 translation edges
1.3 million term nodes with candidates
7.7 candidate meanings per new term
2,445 term-meaning links for training (French/German)
2,901 term-meaning links as validation set

Introduction
Setup
Output
Evaluation
Results
deu: “Schulgebäude”
school
(group of fish)
school
(institution)
school
(building)
deu: “Schulhaus”
deu: “Fischschwarm”
ces: “hejno”
fra: “banc”
ind: “sekolah”
jpn: “ ”学校
kor: “ ”학교
lao: “ໂຮງຮຽນ”
kat: “ ”სკოლა
Excerpt from ﬁnal UWN graph G3 after 3 iterations
retaining only edges with suﬃciently high weights (0.5 / 0.6)

Introduction
Setup
Output
Evaluation
Evaluation
Relation Precision1
Term-Meaning Links (French) 89.2% ± 3.4%
Term-Meaning Links (German) 85.9% ± 3.8%
Term-Meaning Links (Mandarin Chinese) 90.5% ± 3.3%
Generalization (Hypernymy) 87.1% ± 4.8%
Instance 89.3% ± 4.4%
Similarity 92.0% ± 3.8%
Category 93.3% ± 4.5%
Part (Meronymy) 94.4% ± 4.1%
Member (Meronymy) 92.7% ± 4.0%
Substance (Meronymy) 95.6% ± 3.5%
Opposite 94.3% ± 3.9%
1: Wilson score intervals for random samples

Introduction
Setup
Output
Evaluation
Coverage
Language Term-Meaning Links Distinct Terms
Overall 1,595,763 822,212
German 132,523 67,087
French 75,544 33,423
Esperanto 71,247 33,664
Dutch 68,792 30,154
Spanish 68,445 32,143
Turkish 67,641 31,553
Czech 59,268 33,067
Russian 57,929 26,293
Portuguese 55,569 23,499
Italian 52,008 24,974
Hungarian 46,492 28,324
Thai 44,523 30,815

Introduction
Setup
Output
Evaluation
Experimental Setup
Example: “curriculum” considered closely related to
“school”, but not to “water”
compute term relatedness using UWN
sim(t1, t2) = max
s1∈σ(t1)
max
s2∈σ(t2)
sim(s1, s2) sim(s1, s2):
combined graph-/gloss-based method
compare with assessments of relatedness made by human
judges

Introduction
Setup
Output
Evaluation
Results for 3 German Datasets
Dataset GUR65 GUR350 ZG222
r Cov. r Cov. r Cov.
Inter-Annot. Agreement 0.81 (65) 0.69 (350) 0.49 (222)
Wikipedia (ESA*) 0.56 65 0.52 333 0.32 205
GermaNet (Lin*) 0.73 60 0.50 208 0.08 88
UWN 0.80 60 0.68 242 0.51 106
r: Pearson product-moment correlation coeﬃcient
Cov.: absolute coverage
∗: scores by Gurevych et al. (2007)

Introduction
Setup
Output
Evaluation
cross-lingual TC: train using documents in one language,
classify documents in another language
used bag-of-words/meanings TF-IDF vectors
Dataset: Reuters corpora (RCV1/2)
for each language pair:
105 binary classiﬁcation tasks, each using
200 training documents, 600 test documents
SVMlight

Introduction
Setup
Output
Evaluation
Language Pair Terms only Terms + Meanings
English-Italian 68.3% 76.3%
English-Russian 51.7% 71.2%
Italian-English 74.4% 78.1%
Italian-Russian 58.4% 73.2%
Russian-English 67.3% 76.8%
Russian-Italian 62.2% 71.8%
(all values are F1 scores)

Introduction
Summary
Future Work
Outline

Introduction
Summary
Future Work
Summary
large-scale multilingual wordnet:
85% accuracy, 800,000 terms, over 1.5 million links from
terms to meanings,
built by learning edge weights using graph-based evidence
useful for monolingual and cross-lingual tasks

Introduction
Summary
Future Work
Future Work
ongoing work: user interface incl. user contributions
techniques to automatically discover new word meanings
word sense disambiguation, query expansion using UWN

Introduction
Summary
Future Work
Thanks!
expression of
gratitude
eng: “thank you”
yue: “ ”唔該
cmn: “ ”谢谢
jap: “ ”ありがとう
spa: “gracias”
ara: “‫را‬ً ‫شك‬.”

Towards a Universal Wordnet by Learning from Combined Evidence

Towards a Universal Wordnet by Learning from Combined Evidence

Recommended

More Related Content

Similar to Towards a Universal Wordnet by Learning from Combined Evidence

Similar to Towards a Universal Wordnet by Learning from Combined Evidence(20)

More from Gerard de Melo

More from Gerard de Melo(15)

Recently uploaded

Recently uploaded(20)

Towards a Universal Wordnet by Learning from Combined Evidence