We investigate two specific forms of linguistic ambiguities - polysemy, which is the multiplicity of meanings for a specific word, and tautology, which are seemingly uninformative and ambiguous phrases used in conversations. Both phenomena are widely-known manifestations of linguistic ambiguity at the lexical and pragmatic level, respectively. The first part of the thesis focuses on addressing this challenge by proposing a new method for quantifying the degree of polysemy in words, which refers to the number of distinct meanings that a word can have. The proposed approach is a novel, unsupervised framework to compute and estimate polysemy scores for words in multiple languages, infusing syntactic knowledge in the form of dependency structures. The proposed framework is tested on curated datasets controlling for different sense distributions of words in three typologically diverse languages - English, French, and Spanish. The framework leverages contextual language models and syntactic structures to empirically support the widely held theoretical linguistic notion that syntax is intricately linked to ambiguity/polysemy. The second part of the thesis explores how language models handle colloquial tautologies, a type of redundancy commonly used in conversational speech. We first present a dataset of colloquial tautologies and evaluate several state-of-the-art language models on this dataset using perplexity scores. We conduct probing experiments while controlling for the noun type, context and form of tautologies. The results reveal that BERT and GPT2 perform better with modal forms and human nouns, which aligns with previous literature and human intuition.

1. BEYOND THE SURFACE: A COMPUTATIONAL
EXPLORATION OF LINGUISTIC AMBIGUITY
Thesis Defense
Anmol Goel
Advisor: Prof. Ponnurangam Kumaraguru
June 27, 2023

2. WHY STUDY AMBIGUITY?
1

3. Ambiguity seems to be an essential,
indispensable element for the transfer of
information from one place to another by
words. — Lewis Thomas (1974)
WHY STUDY AMBIGUITY?
1

4. Ambiguity seems to be an essential,
indispensable element for the transfer of
information from one place to another by
words. — Lewis Thomas (1974)
WHY STUDY AMBIGUITY?
when a phrase, statement, or
resolution is not explicitly defined,
making several interpretations
plausible.
1

5. AMBIGUITY: THE GOOD, THE BAD AND THE UGLY
Ambiguity improves communication
Piantadosi, S. T., Tily, H., & Gibson, E. (2012). The communicative function of ambiguity in language. Cognition, 122(3), 280-291. 2

6. AMBIGUITY: THE GOOD, THE BAD AND THE UGLY
Ambiguity improves communication, makes language efficient
https://news.mit.edu/2012/ambiguity-in-language-0119 2

7. AMBIGUITY: THE GOOD, THE BAD AND THE UGLY
Ambiguity improves communication, makes language efficient, can lead to humor and wordplay
Mittal, A., Tian, Y., & Peng, N. (2022). Ambipun: Generating humorous puns with ambiguous context. NAACL. 2

8. AMBIGUITY: THE GOOD, THE BAD AND THE UGLY
Ambiguity can lead to confusion and misunderstanding
https://venturebeat.com/ai/amazon-scientist-explains-how-alexa-resolves-ambiguous-requests/ 3

9. AMBIGUITY: THE GOOD, THE BAD AND THE UGLY
Ambiguity can lead to confusion and misunderstanding, particularly in situations where
clarity is important, such as legal documents.
https://www.nbcnews.com/news/us-news/think-commas-don-t-matter-omitting-one-cost-maine-dairy-n847151
https://www.lexology.com/library/detail.aspx?g=d999319c-8125-4326-bce6-88669c654da5 3

10. AMBIGUITY: THE GOOD, THE BAD AND THE UGLY
Ambiguity can lead to strange behaviours in generative models like DALL-E
White, J. C., & Cotterell, R. (2022). Schr"{o} dinger's Bat: Diffusion Models Sometimes Generate Polysemous Words in Superposition. arXiv preprint arXiv:2211.13095. 4

11. AMBIGUITY: THE GOOD, THE BAD AND THE UGLY
Ambiguity can lead to strange behaviours in generative models like DALL-E and chatGPT
Ortega-Martín, M., García-Sierra, Ó., Ardoiz, A., Álvarez, J., Armenteros, J. C., & Alonso, A. (2023). Linguistic ambiguity analysis in ChatGPT. arXiv preprint arXiv:2302.06426. 4

12. TYPES OF AMBIGUITY
Typically, three main types of ambiguity [Fromkin et al., 2018] in language are considered -
lexical,
syntactic,
and semantic.
Fromkin, V., Rodman, R., & Hyams, N. (2018). An Introduction to Language (w/MLA9E Updates). Cengage Learning. 5

13. LEXICAL AMBIGUITY
Arises when a word has multiple
meanings
https://wstyler.ucsd.edu/101/ 6

14. SYNTACTIC AMBIGUITY
Arises when a sentence can be
parsed in multiple ways.
https://wstyler.ucsd.edu/101/ 7

15. SEMANTIC AMBIGUITY
Arises when a word or phrase can
be interpreted in multiple ways
based on the meaning of the words
used.
https://wstyler.ucsd.edu/101/ 8

16. Typically, three main types of ambiguity [Fromkin et al., 2018] in language are considered -
lexical,
syntactic,
and semantic.
TYPES OF AMBIGUITY
Foci of this thesis
Fromkin, V., Rodman, R., & Hyams, N. (2018). An Introduction to Language (w/MLA9E Updates). Cengage Learning. 9

17. THIS DISSERTATION
To alleviate the issues raised by linguistic ambiguities in
natural language systems, it is imperative to:
1) measure ambiguity
10

18. THIS DISSERTATION
To alleviate the issues raised by linguistic ambiguities in
natural language systems, it is imperative to:
1) measure ambiguity
2) analyze language models
10

19. THIS DISSERTATION
focuses on two manifestations of linguistic ambiguities...
11

20. POLYSEMY
The multiplicity of meanings (or senses) for a
specific word
A form of lexical ambiguity
THIS DISSERTATION
11

21. POLYSEMY
The multiplicity of meanings (or senses) for a
specific word
A form of lexical ambiguity
The word “bank” can refer to a financial
institution, the side of a river, or a place where
airplanes park, among multiple other
meanings.
THIS DISSERTATION
11

22. POLYSEMY TAUTOLOGY
The multiplicity of meanings (or senses) for a
specific word
A form of lexical ambiguity
The word “bank” can refer to a financial
institution, the side of a river, or a place where
airplanes park, among multiple other
meanings.
Seemingly uninformative and ambiguous
phrases used in conversations
A form of semantic (or pragmatic) ambiguity
THIS DISSERTATION
11

23. POLYSEMY TAUTOLOGY
The multiplicity of meanings (or senses) for a
specific word
A form of lexical ambiguity
The word “bank” can refer to a financial
institution, the side of a river, or a place where
airplanes park, among multiple other
meanings.
Seemingly uninformative and ambiguous
phrases used in conversations
A form of semantic (or pragmatic) ambiguity
Colloquial tautologies like “boys will be boys” can
create ambiguity by relying on vague or
imprecise language that can be interpreted in
different ways depending on the context.
THIS DISSERTATION
11

24. THIS DISSERTATION
asks two research questions:
12

25. RQ1
Can we use linguistically-motivated methods
to measure the polysemy of words?
THIS DISSERTATION
asks two research questions:
12

26. RQ1 RQ2
Can we use linguistically-motivated methods
to measure the polysemy of words?
Can current language models like BERT and GPT
interpret tautological sentences?
THIS DISSERTATION
asks two research questions:
12

27. RQ1:
POLYSEMY QUANTIFICATION
13

28. ATTEMPTS AT POLYSEMY QUANTIFICATION
EACL 2021
-Requires carefully tuned
hyperparameters to get good results
-Dimensionality reduction can lead to
embedding distortion
+Measures polysemy by iteratively
binning the contextual embedding space.
Xypolopoulos, C., Tixier, A. J. P., & Vazirgiannis, M. (2020). Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings.. 14

29. ATTEMPTS AT POLYSEMY QUANTIFICATION
EACL 2021 EMNLP 2020
-Relies on large amount of data
-more than 10,000 sentences required
-Requires carefully tuned
hyperparameters to get good results
-Dimensionality reduction can lead to
embedding distortion
+Measures polysemy by iteratively
binning the contextual embedding space.
+ Measures polysemy using mutual
information approximations
Xypolopoulos, C., Tixier, A. J. P., & Vazirgiannis, M. (2020). Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings..
Pimentel, T., Maudslay, R. H., Blasi, D., & Cotterell, R. (2020). Speakers fill lexical semantic gaps with context. arXiv preprint arXiv:2010.02172. 14

30. THE SYNERGISTIC THEORY OF
LANGUAGE
It is hypothesized that syntax influences the polysemy of a word.
This leads us to consider incorporating syntax and networks in our approach.
Čech, R., Mačutek, J., Žabokrtský, Z., & Horák, A. (2017). Polysemy and synonymy in syntactic dependency networks. Digital Scholarship in the Humanities, 32(1), 36-49. 15

31. DESIDERATA
Our measure should be:
Unsupervised
Syntax-aware
Work in low-resource settings.
16

32. Multiple sentences
containing the
same word
HOW TO MEASURE POLYSEMY?
17

33. Multiple sentences
containing the
same word
Model/Pipeline
HOW TO MEASURE POLYSEMY?
17

34. Multiple sentences
containing the
same word
Model/Pipeline
A real number
signifying
polysemy
HOW TO MEASURE POLYSEMY?
17

35. Multiple sentences
containing the
same word
Model/Pipeline
A real number
signifying
polysemy
HOW TO MEASURE POLYSEMY?
17
f(mouth) = 3.2

36. HOW TO MEASURE POLYSEMY?
17

37. HOW TO MEASURE POLYSEMY?
18

38. HOW TO MEASURE POLYSEMY?
19

39. HOW TO MEASURE POLYSEMY?
Ricci curvature is a Wasserstein-based
metric to identify bridges and clusters in
a network.
20

40. HOW TO MEASURE POLYSEMY?
21

41. HOW TO MEASURE POLYSEMY?
22

42. HOW TO MEASURE POLYSEMY?
23

43. HOW TO MEASURE POLYSEMY?
Low-resource
24

44. HOW TO MEASURE POLYSEMY?
Low-resource
Unsupervised
25

45. HOW TO MEASURE POLYSEMY?
Low-resource
Unsupervised
Syntax-aware
26

46. EXPERIMENTAL SETUP
DATA
Covers English, French and
Spanish
introduced by (Garí Soler and
Apidianaki, 2021)
Garí Soler, A., & Apidianaki, M. (2021). Let’s play mono-poly: BERT can reveal words’ polysemy level and partitionability into senses. TACL 27

47. EXPERIMENTAL SETUP
DATA MODEL
Covers English, French and
Spanish
introduced by (Garí Soler and
Apidianaki, 2021)
Huggingface
bert-base-uncased for English,
flaubert-baseuncased for French
and bert-basespanish-wwm-
uncased for Spanish
Garí Soler, A., & Apidianaki, M. (2021). Let’s play mono-poly: BERT can reveal words’ polysemy level and partitionability into senses. TACL 27

48. EXPERIMENTAL SETUP
DATA MODEL EVALUATION
Covers English, French and
Spanish
Following previous literature in
polysemy quantification
(Xypolopoulos et al., 2021), we
utilised Spearman correlation as
our evaluation metric.
introduced by (Garí Soler and
Apidianaki, 2021)
Huggingface Spearman correlation
bert-base-uncased for English,
flaubert-baseuncased for French
and bert-basespanish-wwm-
uncased for Spanish
Garí Soler, A., & Apidianaki, M. (2021). Let’s play mono-poly: BERT can reveal words’ polysemy level and partitionability into senses. TACL
Xypolopoulos, C., Tixier, A. J. P., & Vazirgiannis, M. (2020). Unsupervised Word Polysemy Quantification with Multiresolution Grids of Contextual Embeddings.. 27

49. RESULTS
Garí Soler, A., & Apidianaki, M. (2021). Let’s play mono-poly: BERT can reveal words’ polysemy level and partitionability into senses. TACL 28

50. RESULTS
Garí Soler, A., & Apidianaki, M. (2021). Let’s play mono-poly: BERT can reveal words’ polysemy level and partitionability into senses. TACL 28

51. DISCUSSION
The poly-bal data setting shows
consistently strong correlations as
compared to poly-rand setting which
is quite intuitive due to the carefully
controlled sense distribution in poly-
bal sentences.
Multilingual
29

52. DISCUSSION
The poly-bal data setting shows
consistently strong correlations as
compared to poly-rand setting which
is quite intuitive due to the carefully
controlled sense distribution in poly-
bal sentences.
Multilingual
These results suggest that studies in
ambiguity should investigate syntax
along with semantics of an utterance.
Ablation
29

53. LIMITATIONS
We rely on the availability of good quality language-specific language models
Any errors in the language model may propagate into our score.
Extrinsic tasks like Word Sense Disambiguation haven't been explored yet.
30

54. IMPLICATIONS
Our method can be used to extend existing word inventories like WordNet by
discovering new polysemantic relations.
It can be used with generative models to produce less ambiguous texts.
31

55. RQ2:
TAUTOLOGIES
There are two competing views of tautological constructions:
Pragmatic View
Semantic View
32

56. PRAGMATIC VIEW
[Grice, 1975] proposed a pragmatic model on how listeners and speakers communicate and cooperate in
conversations.
Information is implied rather than asserted.
Proposes the four maxims of conversation
Grice, H. P. (1975). Logic and conversation. In Speech acts, pages 41–58. Brill. 33

57. The four maxims of conversation:
Maxim of Quantity
speakers should give as much information as necessary, but no more
PRAGMATIC VIEW
Grice, H. P. (1975). Logic and conversation. In Speech acts, pages 41–58. Brill. 34

58. The four maxims of conversation:
Maxim of Quantity
speakers should give as much information as necessary, but no more
Maxim of Quality
speakers should be truthful and should provide accurate information.
PRAGMATIC VIEW
Grice, H. P. (1975). Logic and conversation. In Speech acts, pages 41–58. Brill. 34

59. The four maxims of conversation:
Maxim of Quantity
speakers should give as much information as necessary, but no more
Maxim of Quality
speakers should be truthful and should provide accurate information.
Maxim of Relation
speakers should stay on topic and should provide relevant information
PRAGMATIC VIEW
Grice, H. P. (1975). Logic and conversation. In Speech acts, pages 41–58. Brill. 34

60. The four maxims of conversation:
Maxim of Quantity
speakers should give as much information as necessary, but no more
Maxim of Quality
speakers should be truthful and should provide accurate information.
Maxim of Relation
speakers should stay on topic and should provide relevant information
Maxim of Manner
speakers should be clear and concise in their communication
PRAGMATIC VIEW
Grice, H. P. (1975). Logic and conversation. In Speech acts, pages 41–58. Brill. 34

61. SEMANTIC VIEW
Argues that the interpretation of tautologies is not solely based on their pragmatic implications, but
rather also on the syntactic patterns and nominal classifications of the phrases [Wierzbicka, 1987].
For example, tautologies of the form "N will be N" generally convey negative aspects of the topic with an
indulgent undertone.
The way that the words are arranged in a sentence can impact the interpretation of the tautology.
Wierzbicka, A. (1987). Boys will be boys:’radical semantics’ vs.’radical pragmat- ics’. Language, pages 95–114 35

62. PRAGMATIC VIEW SEMANTIC VIEW
The interpretation of nominal tautologies is
context-dependent. Same tautology can take
on different meanings depending on the
conversational context
Suggests that syntactic form of phrases
contribute semantic information to the
interpretation of tautologies
DICHOTOMY OF TAUTOLOGIES
36

63. EXPERIMENTAL SETUP
DATA
216 sentences Controlling for
noun type, syntax and context.
Methodology of [Gibbs
and McCarrell, 1990]
Gibbs, R. W. and McCarrell, N. S. (1990). Why boys will be boys and girls will be girls: Understanding colloquial tautologies. Journal of Psycholinguistic Research, 19:125–145. 37

64. EXPERIMENTAL SETUP
DATA MODEL
216 sentences Controlling for
noun type, syntax and context.
Methodology of [Gibbs
and McCarrell, 1990]
Huggingface
Pretrained BERT and GPT2
37

65. EXPERIMENTAL SETUP
DATA MODEL EVALUATION
216 sentences Controlling for
noun type, syntax and context.
Sequence log probability
scores are a measure of how
likely a sequence of words is
according to a transformer-
based language model.
Methodology of [Gibbs
and McCarrell, 1990]
Huggingface Acceptability scores
Pretrained BERT and GPT2
37

66. DATA
[Gibbs and McCarrell, 1990] describes a blueprint to create datasets for
tautology acceptability studies.
We use few-shot prompting with GPT-3.5 to synthetically generate data.
Gibbs, R. W. and McCarrell, N. S. (1990). Why boys will be boys and girls will be girls: Understanding colloquial tautologies. Journal of Psycholinguistic Research, 19:125–145. 38

67. DATA
38

68. For autoregressive models like GPT:
For masked language models like BERT:
ACCEPTABILITY SCORES
39

69. RESULTS
Acceptability of Tautologies without context
GPT is better at handling tautologies than BERT
BERT GPT
40

70. RESULTS
Acceptability of Tautologies without context
GPT is better at handling tautologies than BERT
Human nouns receive higher scores than concrete and abstract nouns
BERT GPT
40

71. RESULTS
Acceptability of Tautologies without context
GPT is better at handling tautologies than BERT
Human nouns receive higher scores than concrete and abstract nouns
Surprisingly, LLMs seem to prefer plural tautological constructions, contrary to previous literature
on humans’ preference for modal forms [Gibbs and McCarrell, 1990]
BERT GPT
40

72. RESULTS
Acceptability of Tautologies with context
In general, the scores are higher for negative contexts and for plural and modal syntactic forms.
BERT GPT
41

73. RESULTS
Acceptability of Tautologies with context
In general, the scores are higher for negative contexts and for plural and modal syntactic forms.
This suggests that models encode negative factual connotations for tautological constructions,
similar to human behaviour [Gibbs and McCarrell, 1990].
BERT GPT
41

74. IMPLICATIONS
We find evidence for both - Pragmatic and Semantic View.
We hope that a syncretic approach taking into account both factors may be
necessary to fully interpret tautologies.
A more nuanced approach is needed to understand language model behaviour.
Aligning LLMs using theories like Gricean Maxims can be fruitful.
42

75. LIMITATIONS
Log Likelihood scores are sensitive to noise
We did not evaluate the language model acceptability scores with human
acceptability ratings.
43

76. SUMMARY
In this dissertation, we propose a novel and linguistically-motivated metric to quantify
polysemy.
Additionally, we investigate the pragmatic competence of large language models on
tautological sentences.
This work has important implications within linguistics and NLP to develop state-of-
the-art conversational models.
44

77. PUBLICATION
A pic with Sebastian Ruder (Google) who was interested in the polysemy
work at EMNLP, Abu Dhabi.
An Unsupervised, Geometric and Syntax-aware Quantification of Polysemy, EMNLP 2022
45

78. Prof. Ponnurangam Kumaraguru for his supervision during my Masters.
Prof. Pawan Goyal and Prof. Parameshwari for the insightful reviews.
Prof. Charu for her insights while developing this work.
Multiple collaborators - Prof. Saptarshi Ghosh, Prof. Ashutosh Modi and Prof. Ravi.
iHub Data @ IIITH for funding my Masters.
Friends and the Precog group for countless memories, encouragement and stimulating
discussions.
Family - for everything
ACKNOWLEDGEMENTS
46

79. THANK YOU!
Questions?
47