A Distributional Semantics Approach for Selective Reasoning on Commonsense Graph Knowledge Bases

883 views

Published on

Tasks such as question answering and semantic search are dependent
on the ability of querying & reasoning over large-scale commonsense knowledge
bases (KBs). However, dealing with commonsense data demands coping with
problems such as the increase in schema complexity, semantic inconsistency, incompleteness
and scalability. This paper proposes a selective graph navigation
mechanism based on a distributional relational semantic model which can be applied
to querying & reasoning over heterogeneous knowledge bases (KBs). The
approach can be used for approximative reasoning, querying and associational
knowledge discovery. In this paper we focus on commonsense reasoning as the
main motivational scenario for the approach. The approach focuses on addressing
the following problems: (i) providing a semantic selection mechanism for facts
which are relevant and meaningful in a specific reasoning & querying context
and (ii) allowing coping with information incompleteness in large KBs. The approach
is evaluated using ConceptNet as a commonsense KB, and achieved high
selectivity, high scalability and high accuracy in the selection of meaningful nav-
igational paths. Distributional semantics is also used as a principled mechanism
to cope with information incompleteness.

Published in: Science, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
883
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
21
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

A Distributional Semantics Approach for Selective Reasoning on Commonsense Graph Knowledge Bases

  1. 1. A Distributional Semantics Approach for Selective Reasoning on Commonsense Graph Knowledge Bases André Freitas, João C. Pereira Da Silva, Edward Curry, Paul Buitelaar Insight Centre for Data Analytics NLDB 2014 Montpellier, France
  2. 2. Applying Distributional Semantics to Commonsense Reasoning André Freitas, João C. Pereira Da Silva, Edward Curry, Paul Buitelaar Insight Centre for Data Analytics NLDB 2014 Montpellier, France
  3. 3. Outline  Motivation  Distributional Semantics  Distributional Navigational Algorithm (DNA)  Evaluation  Take-away message
  4. 4. Motivation 4
  5. 5. Semantic Systems & Commonsense Knowledge Bases Knowledge Representation Model Commonsense Data Expected Result: Intelligent behavior Semantic flexibility, predictive power, automation ... Acquisition Inference Model Scalability Consistency 5
  6. 6. Formal Representation of Meaning 6
  7. 7.  Most semantic models have dealt with particular types of constructions, and have been carried out under very simplifying assumptions, in true lab conditions.  If these idealizations are removed it is not clear at all that modern semantics can give a full account of all but the simplest models/statements. Formal World Real World Baroni et al. 2013 Semantics for a Complex World 7
  8. 8. Commonsense Reasoning  Coping with KB incompleteness - Supporting semantic approximation  Selective reasoning - Selecting the relevant facts in the context of the inference Acquisition Scalability Strategy: Using distributional semantics to solve both the acquisition and scalability problems 10
  9. 9. Example Does John Smith have a degree? 11
  10. 10. Example Does John Smith have a degree? 12
  11. 11. Example Does John Smith have a degree? Selective reasoning Coping with KB Incompleteness 13
  12. 12. Applications  Semantic search  Question answering  Approximate semantic inference  Word sense disambiguation  Paraphrase detection  Text entailment  Semantic anomaly detection ... 14
  13. 13. Distributional Semantics 15
  14. 14. Distributional Hypothesis “Words occurring in similar (linguistic) contexts tend to be semantically similar”  He filled the wampimuk with the substance, passed it around and we all drunk some 16
  15. 15. Distributional Semantic Models (DSMs) car dog cat bark run leash 17 Context
  16. 16. Semantic Similarity & Relatedness θ car dog cat bark run leash 18
  17. 17. DSMs as Commonsense Reasoning Commonsense is here θ car dog cat bark run leash 19
  18. 18. DSMs as Commonsense Reasoning θ car dog cat bark run leash ... vs. Semantic best-effort
  19. 19. Distributional Navigational Algorithm (DNA) 21
  20. 20. Approach Overview Distributional Navigational Algorithm (DNA) Ƭ-Space Large-scale unstructured data Unstructured Commonsense KB Structured Commonsense KB Distributional semantics Reasoning Context 22
  21. 21. Ƭ-Space Distributional heuristics 23
  22. 22. Distributional semantic relatedness as a Selectivity Heuristics Distributional heuristics 24 target source
  23. 23. Distributional heuristics 25 Distributional semantic relatedness as a Selectivity Heuristics target source
  24. 24. Distributional heuristics 26 Distributional semantic relatedness as a Selectivity Heuristics targettarget source
  25. 25. Distributional Navigational Algorithm (DNA) Input: Reasoning context: Source and target word pairs Structured Knowledge Base (KB) Distributional Semantic Model (DSM) Output: Meaningful paths in the KB connecting source and target 27
  26. 26. Distributional Navigational Algorithm (DNA) 28
  27. 27. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB John Smith 29 Step: Resoning context = <John Smith, degree>
  28. 28. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB 30 occupation Step: Get neighboring relations engineer John Smith John Smith catholic religion ...
  29. 29. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB 31 Step: Calculate the distributional semantic relatedness between the target term and the neighboring entities John Smith John Smith catholicoccupation engineer religion ... sem rel (catholic, degree) = 0.004 sem rel (engineer, degree) = 0.07
  30. 30. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB 32 John Smith John Smith catholicoccupation engineer religion ... sem rel (catholic, degree) = 0.004 sem rel (engineer, degree) = 0.01 Step: Filter the elements below the threshold
  31. 31. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB 33 John Smith John Smith occupation engineer Step: Navigate to the next nodes
  32. 32. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB 34 John Smith John Smith occupation engineer Step: redefine the reasoning context: <engineer, degree>
  33. 33. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB Step: Get neighboring relations John Smith engineer learn subjectof bridge a rivercapableof dam creates 35 occupation
  34. 34. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB sem rel (dam, degree) = 0.002 Step: Calculate distributional semantic relatedness between the target term and the neighboring entities sem rel (brdge a river, degree) = 0.004 sem rel (learn, degree) = 0.01 John Smith engineer learn subjectof bridge a rivercapableof dam creates 36 occupation
  35. 35. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB sem rel (dam, degree) = 0.002 Step: Filter the elements below the threshold sem rel (brdge a river, degree) = 0.004 sem rel (learn, degree) = 0.01 John Smith engineer learn subjectof bridge a rivercapableof dam creates 37 occupation
  36. 36. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB Step: Search highly related entities in the KB not connected (distributional semantics) John Smith engineer learn subjectof Reasoning context: ‘learn degree’ 38 occupation
  37. 37. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB Step: Navigate to the elements above the threshold John Smith engineer learn subjectof 39 occupation
  38. 38. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB Step: Repeat the steps John Smith engineer learn subjectof education have or involve 40 occupation
  39. 39. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB Step: Repeat the steps John Smith engineer learn subjectof education have or involve at location university 41 occupation
  40. 40. Distributional Navigational Algorithm (DNA) Does John Smith have a degree? Structured Commonsense KB Distributional Commonsense KB Step: Search highly related entities in the KB not connected (distributional semantics) John Smith engineer learn subjectof education have or involve at location university Reasoning context: ‘university degree’ 42 occupation
  41. 41. Distributional Navigational Algorithm (DNA) Structured Commonsense KB Distributional Commonsense KB John Smith engineer learn subjectof education have or involve at location universitycollege Does John Smith have a degree? Step: Search highly related entities in the KB not connected (distributional semantics) Reasoning context: ‘university degree’ 43 occupation
  42. 42. Distributional Navigational Algorithm (DNA) Structured Commonsense KB Distributional Commonsense KB John Smith engineer learn subjectof education have or involve at location universitycollege Does John Smith have a degree? Step: Repeat the steps degree gives 44 occupation
  43. 43. Examples of Selected Paths Reasoning context: < battle, war > 45
  44. 44. Examples of Selected Paths 46
  45. 45. Improving the DNA Algorithm: Semantic Differential Δ 47 Closer to the target
  46. 46. Evaluation 48
  47. 47. Evaluating Semantic Selectivity How does the semantic selectivity scale with the increase in the number of candidate paths? How does the accuracy of the semantic selectivity scale with the increase in the number of candidate paths? 49
  48. 48. Experimental Setup  Query set: 102 word pairs (derived from Question Answering over Linked Data queries 2011/2012) E.g. - What is the highest mountain? - Mount Everest elevation 8848.0  Distributional Semantic Model: ESA  Threshold: η = 0.05  Dataset: ConceptNet  Gold standard: Manual validation with two independent annotators 50
  49. 49. ConceptNet  Number of clauses x per relation type: x = 1 (45,311) 1 < x < 10 (11,804) 10 <= x < 20 (906) 20<= x < 500 (790) x >= 500 (50) 51
  50. 50. Semantic Selectivity total number of paths (path length n) number of paths selected selectivity =  The semantic selectivity for the DNA approach scales with the increasing in the number of candidate paths  How does the semantic selectivity scale with the increase in the number of candidate paths? 52
  51. 51. Semantic Relevance number of returned paths number of relevant paths accuracy =  What is the semantic relevance of the returned paths?  How does the accuracy of the semantic selectivity scale with the increase in the number of candidate paths?  There is a significant reduction in the accuracy with the increase in the number of paths. However the accuracy value remains high. 53
  52. 52. Evaluating Semantic Incompleteness How does distributional semantics support increasing the KB completeness? 54
  53. 53. Incompleteness  39 <source, target> query pairs  Over all ConceptNet entities  Example: - Query: < mayor, city > - Returned entities: council municipality downtown ward incumbent borough reelected metropolitan elect candidate politician democratic 55
  54. 54. Incompleteness  Avg. KB completion precision = 0.568  Avg. # of strongly related entities returned per query = 19.21 number of retrieved entities number of strongly related entities KB completion precision =  How does distributional semantics support increasing the KB completeness?  Distributional semantics supports improving the completeness of the KB  However, further investigation is necessary to improve the precision of distributional models 56
  55. 55. Take-away message  Distributional Semantics provides in the selection of meaningful paths: - high selectivity - high selectivity scalability - medium-high accuracy  Distributional semantics supports improving the completeness of the KB - However, further investigation is necessary to improve the precision of distributional models in this context 57
  56. 56. EasyESA: Do-it-yourself http://treo.deri.ie/easyesa/ 58

×