Knowledgent Big Data-palooza:
Aspects of Semantic Processing
Na’im R. Tyson, PhD
February 6, 2014
Discussion Topics

• Semantic Processing
– What is Semantics?
– What is Pragmatics?
• Lexical Semantics
– Computing Semant...
Semantic Processing

• What is Semantics?
– Study of literal meanings of words and sentences
∗ Lexical Semantics - word me...
Lexical Semantics
WordNet: Description
• Word relation database
• Created by George Miller & Christiane Fellbaum (Miller, ...
– Example: ’color’ is a hypernym of ’red’
Question: What’s the relationship between a hyponym and a hypernym?
• 150K words...
Lexical Semantics

• Adapted from Python Text Processing with NLTK 2.0 Cookbook (Perkins,
2010)
>>> from nltk.corpus impor...
Lexical Semantics

• Antonymy:
>>> ga1 = wn.synset(’good.a.01’)
>>> ga1.definition
’having desirable or positive qualities...
Lexical Semantics

• Hyponymy & Hypernymy:
>>> word_synset.hyponyms()
>>> word_synset.hypernyms()

7
Computing Similarity by WordNet

• Similarity by Path Length (see Perkins, 2010, p. 19)
>>> from nltk.corpus import wordne...
Advantages & Disadvantages

• Advantages
Quality: developed and maintained by researchers
Practice: applications can use W...
Computing Word Similarity by Vector Space Modeling

• Computing Similarity from a Document Corpus
Goal: determine distribu...
11
Similarity Measure: Cosine
Cosine cos(⃗ , ⃗ ) =
x y

⃗ ∗⃗
x y
|⃗ ||⃗|
x y

n

=

i=1
n
i=1

xi yi
n

x2

i=1

y2

cosmonau...
13
Similarity Measure: Euclidean
n
i=1 (xi

Euclidean |⃗ , ⃗ | = |⃗ − ⃗ | =
x y
x y

− yi )2

cosmonaut

astronaut

moon

car...
Cosine & Euclidean Similarity in Python

>>> import numpy as np
>>> from scipy.spatial import distance as dist
>>> cosm = ...
Computing Word Similarity by Vector Space Modeling

• Advantages & Disadvantages
– Requires no database lookups
– Semantic...
Ontology Basics

• Semantic Web Technologies
–
–
–
–

Data Models
Ontology Language
Distributed Query Language
Application...
Ontology Basics

Figure 3: Cambridge Semantics’ simplified view of Semantic Web solutions.

18
Ontology Basics
• W3C Semantic Web
– RDF - Resource Description Framework
∗ Data model w/ identifiers and named relations b...
OWL: Expressive relation definitions (symmetry, transitivity, etc.)
RIF: Rules Interchange Form - representation for exchan...
Text Mining Basics

• What people think Text Mining is?
– Automated discovery of new previously unknown information, by
au...
Text Mining Basics
• What text mining really is?

Data Mining

Information Retrieval

Text Mining
Statistics

Web Mining

...
Text Mining Basics
• A General Approach — ignore Process
Text Mining the cloud!

• Document Clustering
• Text Characterist...
Text Mining Basics

• Application - Document Clustering
Goal: Group large amounts of textual data
Techniques: High Level
–...
THANK YOU!!

25
References
Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and
Jenifer C. Lai. Class-based n-...
George A. Miller. Wordnet: A lexical database for english. Communications of the
ACM, 38(11):39–41, 1995.
Jason Perkins. P...
Upcoming SlideShare
Loading in...5
×

Big Data Palooza Talk: Aspects of Semantic Processing

453

Published on

This inaugural, Meetup talk, sponsored by the Knowledgent Group, discussed aspects of semantic processing, and emphasized using python for lexical semantics. Slides cite example code snippets for computing the relationships between words using the Natural Language Toolkit (NLTK) in Python. There is also a small overview of the technologies underlying the Semantic Web and text mining.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
453
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Big Data Palooza Talk: Aspects of Semantic Processing

  1. 1. Knowledgent Big Data-palooza: Aspects of Semantic Processing Na’im R. Tyson, PhD February 6, 2014
  2. 2. Discussion Topics • Semantic Processing – What is Semantics? – What is Pragmatics? • Lexical Semantics – Computing Semantic Similarity ∗ WordNet ∗ Vector Space Modeling • Ontology Basics • Text Mining: Basics 1
  3. 3. Semantic Processing • What is Semantics? – Study of literal meanings of words and sentences ∗ Lexical Semantics - word meanings & word relations – Sometimes stated formally using some logical form ∗ Example: ∀x∃yloves(x, y) • What is Pragmatics? – Study of language use and its situational contexts (discourse, deixis, presupposition, etc.) 2
  4. 4. Lexical Semantics WordNet: Description • Word relation database • Created by George Miller & Christiane Fellbaum (Miller, 1995; Fellbaum, 1998) @ Princeton University • Types of Relationships Synonymy - word pair similarity Antonymy - word pair dissimilarity Meronymy - part-of relation – Example: ’engine’ and ’car’ Hyponymy - subordinate relation between words (i.e., a type-of relation) – Example: ’red’ is a hyponym of ’color’ (’red’ is a type of color) Hypernymy - superordinate relation between words 3
  5. 5. – Example: ’color’ is a hypernym of ’red’ Question: What’s the relationship between a hyponym and a hypernym? • 150K words w/ 115k synsets and approx. 200k word-sense pairs 4
  6. 6. Lexical Semantics • Adapted from Python Text Processing with NLTK 2.0 Cookbook (Perkins, 2010) >>> from nltk.corpus import wordnet as wn >>> word_synset = wn.synsets(’cookbook’)[0] >>> word_synset.name ’cookbook.n.01’ >>> word_synset.definition ’a book of receipes and cooking directions’ 5
  7. 7. Lexical Semantics • Antonymy: >>> ga1 = wn.synset(’good.a.01’) >>> ga1.definition ’having desirable or positive qualities especially those suitable for a thing specified’ >>> bad = ga1.lemmas[0].antonyms()[0] >>> bad.name ’bad’ >>> bad.synset.definition ’having undesirable or negative qualities’ 6
  8. 8. Lexical Semantics • Hyponymy & Hypernymy: >>> word_synset.hyponyms() >>> word_synset.hypernyms() 7
  9. 9. Computing Similarity by WordNet • Similarity by Path Length (see Perkins, 2010, p. 19) >>> from nltk.corpus import wordnet as wn >>> cb = wn.synset(’cookbook.n.01’) >>> ib = wn.synset(’instruction_book.n.01’) >>> cb.wup_similarity(ib) # Wu-Palmer Similarity 0.91666666666666663 • For path similarity explanations, see Jaganadhg (2010) 8
  10. 10. Advantages & Disadvantages • Advantages Quality: developed and maintained by researchers Practice: applications can use WordNet Software: SenseRelate (Perl) - http://senserelate.sourceforge.net • Disadvantages Coverage: technical terms may be missing Irregularity: path lengths can be irregular across hierarchies Relatedness: related terms may not be in the same hierarchies Example: Tennis Problem – ’player’, ’racquet’, ’ball’ and ’net’ 9
  11. 11. Computing Word Similarity by Vector Space Modeling • Computing Similarity from a Document Corpus Goal: determine distributional properties of a word Steps: In general... – Create vector of size n for each word of interest – Think of them as points in some n-dimensional space – Use a similarity metric to compute distance Algorithm: Brown et al. (1992) – C(x) - vector with properties of x (context of ’x’) – C(w) = #(w1), #(w2), ..., #(wk ) , where #(wi) is the number of times wi followed w in a corpus 10
  12. 12. 11
  13. 13. Similarity Measure: Cosine Cosine cos(⃗ , ⃗ ) = x y ⃗ ∗⃗ x y |⃗ ||⃗| x y n = i=1 n i=1 xi yi n x2 i=1 y2 cosmonaut astronaut moon car truck Soviet 1 0 0 1 1 American 0 1 0 1 1 spacewalking 1 1 0 0 0 red 0 0 0 1 1 full 0 0 1 0 0 old 0 0 0 1 1 , xn ) cos(cosm, astr) = 1∗0+0∗1+1∗1+0∗0+0∗0+0∗0 12 +02 +12 +02 +02 +02 02 +12 +12 +02 +02 +02 Figure 1: Cosine Similarity Comparison from Collins (2007) Outline 12
  14. 14. 13
  15. 15. Similarity Measure: Euclidean n i=1 (xi Euclidean |⃗ , ⃗ | = |⃗ − ⃗ | = x y x y − yi )2 cosmonaut astronaut moon car truck Soviet 1 0 0 1 1 American 0 1 0 1 1 spacewalking 1 1 0 0 0 red 0 0 0 1 1 full 0 0 1 0 0 old 0 0 0 1 1 • • • euclidian(cosm, astr) = (1 − 0)2 + (0 − 1)2 + (1 − 1)2 + (0 − 0)2 + (0 − 0)2 + (0 − 0)2 Figure 2: Euclidean Similarity Comparison from Collins (2007) 14
  16. 16. Cosine & Euclidean Similarity in Python >>> import numpy as np >>> from scipy.spatial import distance as dist >>> cosm = np.array([1,0,1,0,0,0]) >>> astr = np.array([0,1,1,0,0,0]) >>> dist.cosine(cosm, astr) 1.0 >>> dist.euclidean(cosm, astr) 2.4494897427831779 15
  17. 17. Computing Word Similarity by Vector Space Modeling • Advantages & Disadvantages – Requires no database lookups – Semantic similarity doesn’t imply synonymy, antonymy, meronymy, hyponymy, hypernymy, etc. 16
  18. 18. Ontology Basics • Semantic Web Technologies – – – – Data Models Ontology Language Distributed Query Language Applications ∗ Large knowledge bases ∗ Business Intelligence 17
  19. 19. Ontology Basics Figure 3: Cambridge Semantics’ simplified view of Semantic Web solutions. 18
  20. 20. Ontology Basics • W3C Semantic Web – RDF - Resource Description Framework ∗ Data model w/ identifiers and named relations b/t resource pairs ∗ Represented as directed graphs b/t resources and literal values · Done w/ collections of triples · triple: subject, predicate and object 1. Na’im Tyson born in 197x 2. Na’im Tyson works for Knowledgent 3. Knowledgent headquartered Warren – SPARQL - SPARQL Protocol And RDF Query Language ∗ Query language of Semantic Web ∗ Queries RDF stores over HTTP ∗ Very similar to SQL – Capturing Relationships RDF Schema: Vocabulary (term definitions), Schema (class definitions) and Taxonomies (defining hierarchies) 19
  21. 21. OWL: Expressive relation definitions (symmetry, transitivity, etc.) RIF: Rules Interchange Form - representation for exchanging sets of logical and business rules 20
  22. 22. Text Mining Basics • What people think Text Mining is? – Automated discovery of new previously unknown information, by automatically extracting information from a usually amount of different unstructured textual resources (Wasilewska, 2014) 21
  23. 23. Text Mining Basics • What text mining really is? Data Mining Information Retrieval Text Mining Statistics Web Mining Computational Linguistics & Natural Language Processing Figure 4: Venn Diagram of Text Mining (Wasilewska, 2014). 22
  24. 24. Text Mining Basics • A General Approach — ignore Process Text Mining the cloud! • Document Clustering • Text Characteristics Interpretation / Evaluation Data Mining / Pattern Discovery Attribute Selection Text Transformation (Attribute Generation) Text Preprocessing Text Figure 5: General Approaches to Text Mining Process (Wasilewska, 2014). 23
  25. 25. Text Mining Basics • Application - Document Clustering Goal: Group large amounts of textual data Techniques: High Level – k-means - top down ∗ cluster documents into k groups using vectors and distance metric – agglomerative hierarchical clustering - bottom up ∗ Start with each document being a single cluster ∗ Eventually all documents belong to the same cluster ∗ Documents represented as a hierarchy (dendogram) Reference: Taming Text (see Ingersoll et al., 2013, chap. 6) • Final Remarks 24
  26. 26. THANK YOU!! 25
  27. 27. References Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. Class-based n-gram models of natural language. Computational Linguistics, 18:467–479, 1992. Michael Collins. Lexical Semantics: Similarity Measures and Clustering, November 2007. URL http://www.cs.columbia.edu/∼mcollins/6864/slides/wordsim.4up.pdf. Christiane Fellbaum. WordNet: An Electronic Lexical Database. MIT Press, 1998. Grant S. Ingersoll, Thomas S. Morton, and Andrew L. Farris. Taming Text: How to Find, Organize, and Manipulate It. Manning Publications Co., January 2013. Jaganadhg. Wordnet sense similarity with nltk: some basics, October 2010. URL http://jaganadhg.freeflux.net/blog/archive/tag/WSD/. 26
  28. 28. George A. Miller. Wordnet: A lexical database for english. Communications of the ACM, 38(11):39–41, 1995. Jason Perkins. Python Text Processing with NLTK 2.0 Cookbook. Packt Publishing, 2010. Anita Wasilewska. CSE 634 - Data Mining: Text Mining, January 2014. URL http://www.cs.sunysb.edu/ cse634/presentations/TextMining.pdf. 27

×