The document discusses word sense disambiguation and induction using Python. It describes using graphs and clustering algorithms to group terms that frequently occur together to induce senses of ambiguous entities. It also discusses evaluating the approach and provides a demo link. Packages like NumPy, NetworkX, and NLTK are used to process text, build graphs, and classify word senses.

1. Using Python for industrial prototypes
Word Sense Induction and Disambiguation
Artem Revenko
Head of Research
semantic-web.at
PyDays Vienna
May 5, 2017

2. Problem Statement Method Implementation
Controlled Vocabularies
Coffee Taxonomy
Coffee
Filter Coffee Espresso-based
CappuccinoMelange Espresso
Concepts First class citizens,
2 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

3. Problem Statement Method Implementation
Controlled Vocabularies
Coffee Taxonomy
Coffee
Filter Coffee Espresso-based
CappuccinoMelange Espresso
Concepts First class citizens,
Hierarchy Broader / narrower relations,
2 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

4. Problem Statement Method Implementation
Controlled Vocabularies
Coffee Taxonomy
Coffee
Filter Coffee Espresso-based
CappuccinoMelange Espresso
Concepts First class citizens,
Hierarchy Broader / narrower relations,
Auxiliary Alternative forms, custom relations, axioms, . . .
2 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

5. Problem Statement Method Implementation
Entity Extraction for Enabling Search
3 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

6. Problem Statement Method Implementation
Exploitation of Developments
Plan
1. Take existing tool with target functionality;
2. Use APIs to get the results / data;
3. Improve the results / data;
4. Push it back or push it further.
Data Market Austria
Flagship project for commersializing data and services.
Call for startups out soon!
Check datamarket.at
4 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

7. Problem Statement Method Implementation
Problem Statement
Method
Implementation
5 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

8. Problem Statement Method Implementation
Finding Correct Concept
Taxonomy
Thing
Car
BMW Jaguar
Animal
Jaguar
BMW has designed a car that is going to drive Jaguar X1 out
of the market.
6 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

9. Problem Statement Method Implementation
Finding Correct Concept
Taxonomy
Thing
Car
BMW Jaguar
Animal
Jaguar
BMW has designed a car that is going to drive Jaguar X1 out
of the market.
6 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

10. Problem Statement Method Implementation
Exact Semantic Search
Coffee Taxonomy
Coffee
Filter Coffee Espresso-based
CappuccinoMelange Espresso
Search 1
Query Coffee;
In Document Espresso;
Result Found.
7 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

11. Problem Statement Method Implementation
Exact Semantic Search
Coffee Taxonomy
Coffee
Filter Coffee Espresso-based
CappuccinoMelange Espresso
Search 2
Query Espresso;
In Document Coffee;
Result ?
7 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

12. Problem Statement Method Implementation
Exact Semantic Search
Coffee Taxonomy
Coffee
Filter Coffee Espresso-based
CappuccinoMelange Espresso
Search 2
Query Espresso;
In Document Coffee;
Result Not Found.
7 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

13. Problem Statement Method Implementation
Exact Semantic Search
Coffee Taxonomy
Coffee
Filter Coffee Espresso-based
CappuccinoMelange Espresso
Document
One alternative to the Node.js
utility is the Coffee Maven
Plugin, a plugin for the popular
Apache Maven build system.
Search 3
Query Coffee;
In Document Coffee;
Result ?
7 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

14. Problem Statement Method Implementation
Problem Statement
Word Sense Disambiguation (and Induction)
WSD is identifying which sense of a word (i.e. meaning)
is used in a sentence, when the word has multiple
meanings.
We go even further and say that the task is to induce the senses of
the entity and to find the correct sense for each occurrence.
Conventions
The word of interest: entity,
The surrounding words: terms.
8 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

15. Problem Statement Method Implementation
Problem Statement
Method
Implementation
9 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

16. Problem Statement Method Implementation
Describing Senses
Describing Senses
You shall know a word by the company it keeps.
Firth, J. R. 1957
10 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

17. Problem Statement Method Implementation
Describing Senses
Describing Senses
You shall know a word by the company it keeps.
Firth, J. R. 1957
Co-occurrence Measures
Dice scores: 2 AB
A+B ,
Statistical significance of co-occurrence (z score),
. . .
10 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

18. Problem Statement Method Implementation
Co-occurrence Graph
11 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

19. Problem Statement Method Implementation
Graph Clustering Algorithms
Chinese whispers
Propagate the signal.
HyperLex
Find most important hubs (pageranky).
Triangulation
How many cliques (complete graphs).
12 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

20. Problem Statement Method Implementation
2nd Order Co-occurrence Graph
Entity
Term1Term2
Term3
Term4
0.2
0.15
0.12 0.1
13 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

21. Problem Statement Method Implementation
2nd Order Co-occurrence Graph
Entity
Term1Term2
Term3
Term4
Term5
Term6
0.2
0.15
0.12 0.1
0.22
0.16
0.2
0.1
0.2
13 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

22. Problem Statement Method Implementation
2nd Order Co-occurrence Graph
Entity
Term1Term2
Term3
Term4
Term5
Term6
0.2
0.15
0.12 0.1
0.22
0.16
0.2
0.1
0.2
Sense1
Sense2
13 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

23. Problem Statement Method Implementation
Problem Statement
Method
Implementation
14 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

24. Problem Statement Method Implementation
Packages
numpy
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

25. Problem Statement Method Implementation
Packages
numpy
networkx for graph operations, in particular pagerank;
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

26. Problem Statement Method Implementation
Packages
numpy
networkx for graph operations, in particular pagerank;
rdflib for queries and rdf;
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

27. Problem Statement Method Implementation
Packages
numpy
networkx for graph operations, in particular pagerank;
rdflib for queries and rdf;
requests for HTTP requests;
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

28. Problem Statement Method Implementation
Packages
numpy
networkx for graph operations, in particular pagerank;
rdflib for queries and rdf;
requests for HTTP requests;
nltk stemming, stopwords;
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

29. Problem Statement Method Implementation
Packages
numpy
networkx for graph operations, in particular pagerank;
rdflib for queries and rdf;
requests for HTTP requests;
nltk stemming, stopwords;
sklearn classification, vectorization;
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

30. Problem Statement Method Implementation
Packages
numpy
networkx for graph operations, in particular pagerank;
rdflib for queries and rdf;
requests for HTTP requests;
nltk stemming, stopwords;
sklearn classification, vectorization;
matplotlib, plotly visualizations;
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

31. Problem Statement Method Implementation
Packages
numpy
networkx for graph operations, in particular pagerank;
rdflib for queries and rdf;
requests for HTTP requests;
nltk stemming, stopwords;
sklearn classification, vectorization;
matplotlib, plotly visualizations;
django showcase.
15 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

32. HyperLex
1 for node , score in ranking :
2 p r s c o r e = pr [ node ] / max pr
3 i f p r s c o r e > th hub :
4 o r i g i n a l n e i g h b o r s =set (G. neighbors ( node ) )
5 l e f t n e i g h b o r s = l e f t n o d e s &
→ o r i g i n a l n e i g h b o r s
6 value = ( l e f t n e i g h b o r s /
→ o r i g i n a l n e i g h b o r s * p r s c o r e )
7 i f value > th hub :
8 l e f t n o d e s −= {node}
9 hubs . append ( node )
10 c l u s t e r = {node : pr [ node ]}
11 for x in l e f t n e i g h b o r s :
12 c l u s t e r [ x ] = pr [ x ] * c o s c o r e [ x ] [ node ]
13 l e f t n o d e s −= set ( c l u s t e r )
14 c l u s t e r s . append ( c l u s t e r )

33. Problem Statement Method Implementation
Evaluation and Demo
Method Macro average Micro average
Baseline 0.447 0.324
Our method 0.784 0.821
Demo: https://artem.semantic-web.at/wsd
Thank you!Thank you!
⇒⇒ poolparty.biz ⇐⇐
17 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY

34. Problem Statement Method Implementation
Evaluation and Demo
Method Macro average Micro average
Baseline 0.447 0.324
Our method 0.784 0.821
Demo: https://artem.semantic-web.at/wsd
Thank you!Thank you!
⇒⇒ poolparty.biz ⇐⇐
17 / 17
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY
SEMANTIC WEB COMPANY
school • consulting • projects • events • media
SEMANTIC WEB COMPANY
SEMANTIC
WEB COMPANY