Learning Multilingual Semantics from Big Data on the Web

Learning Multilingual Semantics
from Big Data on the Web
Gerard de Melo
Assistant Professor, Tsinghua University
http://gerard.demelo.org
Learning Multilingual Semantics
from Big Data on the Web
Gerard de Melo
Assistant Professor, Tsinghua University
http://gerard.demelo.org

Big Data on the WebBig Data on the WebBig Data on the WebBig Data on the Web
Matej Kren: Idiom. Prague Municipal Library https://www.flickr.com/photos/ill-padrino/6437837857/

From Big Data toFrom Big Data to
Multilingual Semantics?Multilingual Semantics?
From Big Data toFrom Big Data to
Multilingual Semantics?Multilingual Semantics?
Image:
Brett Ryder

Manual Knowledge OrganizationManual Knowledge Organization
Image: http://commons.wikimedia.org/wiki/File:Mundaneum_Tir%C3%A4ng_Karteikaarten.jpg
Universal Bibliographic Repertory
(Repertoire Bibliographique Universel, RBU)
by Paul Otlet and Henri La Fontaine in 1895
index cards with answers to queries

Manual Knowledge OrganizationManual Knowledge Organization
Image: Mundaneum
Alex Wright: This was a sort of
“analog search engine”
Alex Wright: This was a sort of
“analog search engine”

Zipfian DistributionZipfian DistributionZipfian DistributionZipfian Distribution
https://commons.wikimedia.org/wiki/File:Moby_Dick_Words.gif

Big Data on the WebBig Data on the WebBig Data on the WebBig Data on the Web
+

Goal: Large YetGoal: Large Yet
Reasonably Clean KnowledgeReasonably Clean Knowledge
Goal: Large YetGoal: Large Yet
Reasonably Clean KnowledgeReasonably Clean Knowledge
Theological Hall, Strahov Monastery Library, Prague

OutlineOutline
Large-Scale Knowledge Graphs
Semantics in Action
Models for the Future

Lexical Knowledge
Portuguese-Chinese Dictionary by Ruggieri et al. (1580s)
The first European-Chinese dictionary
https://commons.wikimedia.org/wiki/File:Ricci-Ruggieri-Portuguese-Chinese-dictionary-page-1.png

Provides translations, antonyms, etc.
WiktionaryWiktionary

e.g. “salary” < Lat. “salarius” < Lat. “sal” (salt)
Etymological WordnetEtymological Wordnet

LREC 2014LREC 2014

Old English
Example
Old English
Example

Lexical AmbiguitiesLexical Ambiguities

Hipsters in London
Images:
https://www.flickr.com/photos/poisonbabyfood/4274634681
https://www.facebook.com/alexander.balabanov.82

Reunion

Reunion
Images:
https://commons.wikimedia.org/wiki/File:Reunions_Class_of_82_2007.jpg
https://commons.wikimedia.org/wiki/File:Riviere_Langevin_Trou_Noir_P1440224-35.jpg
and many more...and many more...

Multilingual Lexical Knowledge
UWN (de Melo & Weikum 2009)

UWN: Universal Wordnet
Before:
manual work over
two decades but not
many large wordnets
Before:
manual work over
two decades but not
many large wordnets
Our Approach:
● Exploit translation
resources on the Web
● Learn regression model
with sophisticated
graph-based features
Our Approach:
● Exploit translation
resources on the Web
● Learn regression model
with sophisticated
graph-based features
Gerard de Melo

Gerard de Melo

over 1,000,000 words in over 100 languages
CIKM 2009CIKM 2009 ICGL 2008ICGL 2008
Best Paper AwardBest Paper Award
ICGL 2008ICGL 2008
Gerard de Melo

UWN: Getting StartedUWN: Getting Started
Simple API for JVM Languages
val uwn = new UWN(new File("plugins/"))
for (m <- uwn.getMeanings("souris", "fra"))
println(m)
Or Just Download the TSV File
Simple API for JVM Languages
val uwn = new UWN(new File("plugins/"))
for (m <- uwn.getMeanings("souris", "fra"))
println(m)
Or Just Download the TSV File

Adding Other Sources
Gerard de Melo
Language-specific,Language-specific,
Domain-specific,Domain-specific,
Arbitrary DatabasesArbitrary Databases
Language-specific,Language-specific,
Domain-specific,Domain-specific,
Arbitrary DatabasesArbitrary Databases

Adding Other SourcesAdding Other SourcesAdding Other SourcesAdding Other Sources
https://commons.wikimedia.org/wiki/File:Encyclopedia_Britannica_in_the_library_of_The_Kings_School,_Goa.jpg

Adding Other SourcesAdding Other SourcesAdding Other SourcesAdding Other Sources
Rob Matthews: printed small sample of Wikipedia
Actually, a printed
Wikipedia corresponds to
2000 Britannica volumes
Source:
http://www.labnol.org/internet/wikipedia-printed-book/9136/
Actually, a printed
Wikipedia corresponds to
2000 Britannica volumes
Source:
http://www.labnol.org/internet/wikipedia-printed-book/9136/

ACL 2010
AAAI 2013
ACL 2010
AAAI 2013
Use
Identity Links
to connect
What is
equivalent
Merging Structured DataMerging Structured DataMerging Structured DataMerging Structured Data

ACL 2010
AAAI 2013
ACL 2010
AAAI 2013

Merging Structured DataMerging Structured Data
Trentino Trentino-
Alto Adige

One bad link isOne bad link is
enough to make aenough to make a
connected componentconnected component
inconsistentinconsistent
One bad link isOne bad link is
enough to make aenough to make a
connected componentconnected component
inconsistentinconsistent
ACL 2010
AAAI 2013
ACL 2010
AAAI 2013

Source: Peter Mika
Entity Integration:
Challenges
Entity Integration:
Challenges

Merging Structured DataMerging Structured Data
Distinctness Assertions
Di
=
({en: Province of Trento,
en:Trentino},
{en:Trentino-South Tyrol,
en:Trentino-Alto Adige/Südtirol})
Distinctness Assertions
Di
=
({en: Province of Trento,
en:Trentino},
{en:Trentino-South Tyrol,
en:Trentino-Alto Adige/Südtirol})
ACL 2010
AAAI 2013
ACL 2010
AAAI 2013

How to reconcileHow to reconcile
equivalenceequivalence
andand
distinctnessdistinctness
evidence?evidence?
How to reconcileHow to reconcile
equivalenceequivalence
andand
distinctnessdistinctness
evidence?evidence?
a) ignore somea) ignore some
equivalence informationequivalence information
(delete certain edges)(delete certain edges)
a) ignore somea) ignore some
equivalence informationequivalence information
(delete certain edges)(delete certain edges)
b) ignore someb) ignore some
distinctness informationdistinctness information
(remove node from(remove node from
distinctness assertion)distinctness assertion)
b) ignore someb) ignore some
distinctness informationdistinctness information
(remove node from(remove node from
distinctness assertion)distinctness assertion)
ACL 2010
AAAI 2013
ACL 2010
AAAI 2013

Min. cost solution:Min. cost solution:
NP-hardNP-hard
APX-hardAPX-hard
Min. cost solution:Min. cost solution:
NP-hardNP-hard
APX-hardAPX-hard
ACL 2010
AAAI 2013
ACL 2010
AAAI 2013

Finally, use region growingFinally, use region growing
algorithm in the spiritalgorithm in the spirit
of Leighton & Rao 1988of Leighton & Rao 1988
Finally, use region growingFinally, use region growing
algorithm in the spiritalgorithm in the spirit
of Leighton & Rao 1988of Leighton & Rao 1988
Linear Program RelaxationLinear Program RelaxationLinear Program RelaxationLinear Program Relaxation
Approximation Guarantee:Approximation Guarantee:
4ln(nq+1)4ln(nq+1)
for n distinctness assertions,for n distinctness assertions,
q=max |Dq=max |Di,ji,j
||
but independent of |Dbut independent of |Dii
| !| !
Approximation Guarantee:Approximation Guarantee:
4ln(nq+1)4ln(nq+1)
for n distinctness assertions,for n distinctness assertions,
q=max |Dq=max |Di,ji,j
||
but independent of |Dbut independent of |Dii
| !| !

Linear Program RelaxationLinear Program RelaxationLinear Program RelaxationLinear Program Relaxation
Nice:Nice:
This generalizes theThis generalizes the
Hungarian AlgorithmHungarian Algorithm
to various advancedto various advanced
types of non-standardtypes of non-standard
matchingsmatchings
(cf. de Melo. AAAI 2013)(cf. de Melo. AAAI 2013)
Nice:Nice:
This generalizes theThis generalizes the
Hungarian AlgorithmHungarian Algorithm
to various advancedto various advanced
types of non-standardtypes of non-standard
matchingsmatchings
(cf. de Melo. AAAI 2013)(cf. de Melo. AAAI 2013)

Separated ConceptsSeparated Concepts
(Multilingual Wikipedia)(Multilingual Wikipedia)
Separated ConceptsSeparated Concepts
(Multilingual Wikipedia)(Multilingual Wikipedia)

Application:
Lexvo.org
Semantic WebSemantic Web
Journal 2014Journal 2014

Lexvo.orgLexvo.org

InterdisciplinaryInterdisciplinary
Work, e.g. inWork, e.g. in
Digital HumanitiesDigital Humanities
InterdisciplinaryInterdisciplinary
Work, e.g. inWork, e.g. in
Digital HumanitiesDigital Humanities
Lexvo.orgLexvo.org

Taxonomic Organization
a user wants
a list of
„Art Schools in
Europe“

Multilingual Taxonomies
a Swedish user
wants
a list of
„Konstskolor i
Europa“

De Melo & Weikum (2010).
CIKM Best Interdisciplinary Paper Award
Taxonomic Integration:Taxonomic Integration:
MENTA ApproachMENTA Approach
MENTA ApproachMENTA Approach

Predict Individual
Taxonomic Links:
Article → Category
Category → WordNet
Predict Individual
Taxonomic Links:
MENTAMENTA
MENTAMENTA

Predict Individual
Taxonomic Links:
Predict Individual
Taxonomic Links:
MENTAMENTA
MENTAMENTA

MENTAMENTA
MENTAMENTA

MENTAMENTA
MENTAMENTA
Image: https://de.wikipedia.org/wiki/Datei:Bersntol_palae.jpg
Fersental
(Bersntol, Valle dei Mòcheni)
Fersental

MENTAMENTA
MENTAMENTA
https://de.wikipedia.org/wiki/Datei:Language_distribution_Trentino_2011.png
Fersental
Fersental

Use Identity Constraint
Algorithm to form
equivalence classes
Use Identity Constraint
Algorithm to form
equivalence classes
Markov Chain Random
Walk with Restarts
to Rank Parents
Markov Chain Random
Walk with Restarts
to Rank Parents
MENTAMENTA
MENTAMENTA

Bansal et al.Bansal et al.
ACL 2014. Best Paper Runner-UpACL 2014. Best Paper Runner-Up
Belief PropagationBelief Propagation
exploiting Kirchhoff’sexploiting Kirchhoff’s
Matrix Tree TheoremMatrix Tree Theorem
for efficient handling offor efficient handling of
tree factortree factor
Belief PropagationBelief Propagation
exploiting Kirchhoff’sexploiting Kirchhoff’s
Matrix Tree TheoremMatrix Tree Theorem
for efficient handling offor efficient handling of
tree factortree factor
Chu-Liu-EdmondsChu-Liu-Edmonds
directed spanning treedirected spanning tree
algorithm for decodingalgorithm for decoding
Chu-Liu-EdmondsChu-Liu-Edmonds
directed spanning treedirected spanning tree
algorithm for decodingalgorithm for decoding
New Algorithm:
Structured Output Prediction
New Algorithm:
Structured Output Prediction

UWN/MENTA
CIKM 2010CIKM 2010
CIKM 2010CIKM 2010
Biggest (ontological)Biggest (ontological)
taxonomytaxonomy
Biggest (ontological)Biggest (ontological)
taxonomytaxonomy

UWN/MENTA
multilingual extension of WordNet for
word senses and taxonomical information over 200 languages
Gerard de Melo

Language EducationLanguage EducationLanguage EducationLanguage Education

UWNUWNUWNUWN
http://www.lexvo.org/uwn/http://www.lexvo.org/uwn/http://www.lexvo.org/uwn/http://www.lexvo.org/uwn/

Application: Sense-DisambiguatedApplication: Sense-Disambiguated
Example SentencesExample Sentences
Application: Sense-DisambiguatedApplication: Sense-Disambiguated
Example SentencesExample Sentences

Application:
Monolingual Language Users
Application:
Monolingual Language Users

ThesauriThesauri
See also: de Melo & Weikum (2008).

ThesauriThesauri
Borin, Allwood, de Melo. LREC 2014.

Application: Machine TranslationApplication: Machine Translation
OpenWN-PT:
Used by Google Translate
OpenWN-PT:
Used by Google Translate

Machine LearningMachine Learning
Examples
Incorrect
Correct

Examples
LearningLearning
Incorrect
Correct
ClassifierModel

Examples
LearningLearning
Incorrect
Correct
ClassifierModel
???

Examples
Probably
Incorrect!
LearningLearning PredictionPrediction
Incorrect
Correct
ClassifierModel

Better Machine LearningBetter Machine Learning
Examples
Probably
Incorrect!
LearningLearning PredictionPrediction
Incorrect
Correct
ClassifierModel
Better
Classifier!
+ Better
Labels
for Test
Data

UWN Senses in MT?
Issue: Senses
should be less
fine-grained
Issue: Senses
should be less
fine-grained

No Word Left Behind
Web page: http://www.buzzfeed.com/paulf24/24-signs-youre-in-a-pretty-rad-relationship-b5ra#.txpBGq4p4

Similar: Part-Of-Speech TaggingSimilar: Part-Of-Speech Tagging
● British fans gathered at the stadium to...
ADJECTIVE
“Didgeridoo”
is similar to:
“horn” (NOUN)
“drums” (NOUN)
“accordion” (NOUN)
“Didgeridoo”
is similar to:
“horn” (NOUN)
“drums” (NOUN)
“accordion” (NOUN)
Didgeridoo fans gathered at the park to...
???

Similar: Part-Of-Speech TaggingSimilar: Part-Of-Speech Tagging
● British fans gathered at the stadium to...
ADJECTIVE
Gaelic “didiridiú”
translates to
“didgeridoo” (NOUN)
in English
Gaelic “didiridiú”
translates to
“didgeridoo” (NOUN)
in English
...Astrálach is ea an didiridiú
???

What about
Document-Level Tasks?
What about
Document-Level Tasks?
Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/

“new” 1.0
“york” 1.0
“jaguar” 1.0
“automobile” 0.0
“car” 0.0
“10th” 1.0
“street” 1.0
“show” 1.0
... ...
New_York 1.0
Jaguar (car) 0.0
Jaguar (animal) 1.0
Automobile/Car 0.0
10th Street 1.0
Performance 1.0
... ...
“10th street new york jaguar show”
Similar:
“10th New show in York”
“New Jaguar show”
“Show New Street in York”
Similar:
“10th street nyc jaguar show”
Document LevelDocument Level

“new” 1.0
“york” 1.0
“jaguar” 1.0
“automobile” 0.0
“car” 0.0
“10th” 1.0
“street” 1.0
“show” 1.0
... ...
New_York 1.0
Jaguar (car) 0.0
Jaguar (animal) 1.0
Automobile/Car 0.0
10th Street 1.0
Performance 1.0
... ...
Animal 0.5
Vehicle 0.0
Similar:
“10th New show in York”
“New Jaguar show”
“Show New Street in York”
Similar:
“10th street nyc jaguar show”
“10th street nyc animal show”
“Exposición de jaguares Nueva York”
Expansion
(de Melo &
Siersdorfer
2007)
Document LevelDocument Level

Given: training documents with class labels
Goal: guess class labels for test documents in
some other language
Result: better than plain machine translation.
See de Melo & Siersdorfer 2007.
Multilingual Tasks:
Cross-Lingual Text Classification
Multilingual Tasks:
Cross-Lingual Text Classification

Underlying frame:
Commercial transfer
Capture the “who-did-what-to-whom”
Microsoft bought the patent from Nokia.
Nokia sold the patent to Microsoft.
The patent was acquired by Microsoft [from Nokia].
The patent was sold [by Nokia] to Microsoft.
Sentence-Level SemanticsSentence-Level Semantics
Buyer: Microsoft
Seller: Nokia
Product: The patent

FrameBase.org
Bringing knowledge into a standard form
based on natural language (FrameNet)
Bringing knowledge into a standard form
based on natural language (FrameNet)
ESWC 2015
Best Student
Paper Nominee
ESWC 2015
Best Student
Paper Nominee

Relation IntegrationRelation Integration
X isAuthorOf Y
Y writtenBy X
X wrote Y
Y writtenInYear Z
ESWC 2015
Best Student
Paper Nominee
ESWC 2015
Best Student
Paper Nominee

Relation IntegrationRelation Integration
YAGO: isMarriedTo predicateYAGO: isMarriedTo predicate
Freebase: Marriage EntityFreebase: Marriage Entity
Challenge:
Modelling
Differences
Challenge:
Modelling
Differences

Search Interfaces
“Which companies were created during the
last century in Silicon Valley ?”
YAGO2:
WWW 2011
Best Demo Award
YAGO2:
WWW 2011
Best Demo Award
Gerard de Melo

Answering Questions
IBM's Jeopardy!-winning Watson
system
Gerard de Melo

What Goes into Word Vectors?What Goes into Word Vectors?What Goes into Word Vectors?What Goes into Word Vectors?
Jiaqiang Chen and Gerard de Melo 2015

The Roman Empire was remarkably
multicultural, with ”a rather astonishing
cohesive capacity” to create a sense
of shared identity while encompassing
diverse peoples within its political
system over a long span of time.

syntactic

syntactic semantic!

semantic!syntactic syntactic?

semantic!syntactic syntactic? ?

semantic!syntactic ?
Word2Vec Solution:
Subsampling
Word2Vec Solution:
Subsampling
syntactic?

Word2Vec ApproachWord2Vec ApproachWord2Vec ApproachWord2Vec Approach
Alexandre Duret-Lutz
https://www.flickr.com/photos/gadl/110845690/
Take everything
we can get
Take everything
we can get

Our Proposal:Our Proposal:
Extract the Most Valuable PartsExtract the Most Valuable Parts
Theological Hall, Strahov Monastery Library, Prague

…Greek and Roman mythology...
semantic!
look for semantically
salient contexts in text!
look for semantically
salient contexts in text!

Two WorldsTwo Worlds
Distributional Semantics:
Use all available text
(Symbolic) Information Extraction:
Look for valuable connections

Proposed Research Program:
Joint Training
Joint Training
Better
Word Embeddings
Joint Training
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Joint Training
Joint Training
Better
Word Embeddings
Joint Training
Use parallel
threads
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Preliminary Experiments:Preliminary Experiments:
Joint TrainingJoint Training
Recently lots of related work: E.g.
Faruqui et al., Hill & Korhonen,
Wang et al., Johansson & Nieto Piña
Recently lots of related work: E.g.
Faruqui et al., Hill & Korhonen,
Wang et al., Johansson & Nieto Piña

Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Use negative samplingUse negative sampling
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Information ExtractionInformation Extraction
Variant 1: Definition ExtractionVariant 1: Definition Extraction
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Definitions
befuddle: to becloud and confuse as with liquor
befuddled: dazed by alcoholic drink
befuddled: confused and vague used especially of thinking
beg: to ask earnestly for, to entreat or supplicate for, to
beseech
Source: GCIDE
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Synonyms
effectual: effectual efficacious effective
effectuality: effectiveness effectivity effectualness
efficacious: effectual
efficaciousness: efficacy
Source: GCIDE
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Variant 2: List ExtractionVariant 2: List Extraction
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

● Look for repeated occurrences of commas
● Short units of roughly equal length
● noun phrases, adjectives
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

● Look for repeated occurrences of commas
● Short units of roughly equal length
● noun phrases, adjectives
● Also: Hearst patterns, e.g.
“cities such as New York, London, ...”
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Extracted Lists
player captain manager director vice-chairman
group race culture religion organisation person person
Italian Mexican Chinese Creole French
Self-Portraits Portraits iris Still-Lives with Sunflowers view
from the Asylum Works after Millet Vineyards
ballscrews leadscrews worm gear screwjacks linear
actuator
Cleveland Essex Lincolnshire Northamptonshire
Nottinghamshire Thames Valley South Wales
ant.py dimdriver.py dimdriverdatafile.py
dimdriverdatasetdef.py dimexception.py dimmaker.py
dimoperators.py dimparser.py dimrex.py dimension.py

SetupSetup
SetupSetup
Wikipedia 2010
normalize to lower case and remove special characters
Contain 1,205,009,010 words
Select words appearing at least 50 times
Vocabulary size 220,521
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

SetupSetup
SetupSetup
Wikipedia 2010
normalize to lower case and remove special characters
Contain 1,205,009,010 words
Select words appearing at least 50 times
Vocabulary size 220,521
Balance Components
simply by controlling
starting learning rates:
0.05 for CBOW, varying
rates for extracted information
Balance Components
simply by controlling
starting learning rates:
0.05 for CBOW, varying
rates for extracted information
Vector dim. 300
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

Results on WS353Results on WS353
Results on WS353Results on WS353
Positive effect from
0.001 until around 0.04
Positive effect from
0.001 until around 0.04
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

ExampleExample
ExampleExample
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop
Best Paper Award
at NAACL 2015
Vector Space
Modeling Workshop

History Repeating?History Repeating?History Repeating?History Repeating?
SMTSMT NMTNMT
Phrase-Based SMT
Hierarchical Phrases
WSD, MEANT etc.
Phrase-Based SMT
Hierarchical Phrases
WSD, MEANT etc.
Extended NMT?Extended NMT?

Well-Known IssuesWell-Known IssuesWell-Known IssuesWell-Known Issues

Source: The New Yorker
Future:Future:
Learning Common-SenseLearning Common-Sense
Future:Future:
Learning Common-SenseLearning Common-Sense

Learning Common-SenseLearning Common-SenseLearning Common-SenseLearning Common-Sense
WebChild
AAAI 2014
WSDM 2014
AAAI 2011
WebChild
AAAI 2014
WSDM 2014
AAAI 2011

Lexical Intensity OrderingsLexical Intensity Orderings
hothot
warmwarm
fieryfiery
scorchingscorching
<
<
<
weak
strong
TACL 2013TACL 2013

Knowlywood: Human ActivitiesKnowlywood: Human Activities
CIKM 2015CIKM 2015

Extension to RelationshipsExtension to RelationshipsExtension to RelationshipsExtension to Relationships
http://www.wikihow.com/Read-a-Book-to-a-Baby-or-Infant#/Image:Read-a-Book-to-a-Baby-or-Infant-Step-5.jpg

x x
x x
petronia
sparrow
parched
arid
xdry
x bird

x x
x x
petronia
sparrow
parched
arid
xdry
x bird
Should account for
relationships
(incl. affordances,
causality, etc.)
Should account for
relationships
(incl. affordances,
causality, etc.)

Assume that she
is learning
just from text
Assume that she
is learning
just from text

1. Gather large amounts of Patterns
2. Use Web-Scale Data (Google N-Grams,
derived from 10^12 words of text)
Hearst-style
Bootstrapping with
large
numbers of seeds
Gerard de Melo
Information Extraction from TextInformation Extraction from Text

Extension to RelationshipsExtension to Relationships
Commonsense word relationships
extracted from Google 1T n-grams
24 relations bootstrapped via ConceptNet
→ 1,158,141 triples
Jiaqiang Chen, Niket Tandon, Gerard de Melo. WI 2015

Extension to RelationshipsExtension to Relationships
earring hasProperty gorgeous
concept definedAs theory
sonar partOf submarine
predator desires food
Commonsense word relationships
extracted from Google 1T n-grams
24 relations bootstrapped via ConceptNet
→ 1,158,141 triples

What causes cancer?

Can cats fly?

Summary
Large-Scale Knowledge Graphs
► Universal WordNet/MENTA:
large multilingual taxonomy
► Etymological WordNet
Semantics in Action, e.g.
► Lexvo.org
► Question Answering
with YAGO
Future Perspectives
► Vector Representations
► Common-Sense for NLU
More Information:
www.demelo.org
gdm@demelo.org
More Information:
www.demelo.org
gdm@demelo.org
Gerard de Melo

Learning Multilingual Semantics from Big Data on the Web

More Related Content

What's hot

Similar to Learning Multilingual Semantics from Big Data on the Web

More from Gerard de Melo

Recently uploaded

Learning Multilingual Semantics from Big Data on the Web