The document discusses techniques for inducing interpretable word senses from text for word sense disambiguation and enrichment of lexical resources. It describes related work on inducing word sense representations through methods like retrofitting word embeddings to learn sense embeddings, inducing synsets through graph clustering, and inducing semantic classes. The document also discusses making the induced senses interpretable and linking them to existing lexical resources.
Graph's not dead: from unsupervised induction of linguistic structures from t...Alexander Panchenko
In this invited talk, presented at the Dialogue'2018 conference, I argue for the usefulness of graph representations for NLP in the deep learning era. In the lecture, it is described how to extract symbolic linguistic structures, such as word senses and semantic frames in an unsupervised way from text corpora using graph-based algorithms and distributional semantics.
Word sense disambiguation using wsd specific wordnet of polysemy wordsijnlc
This paper presents a new model of WordNet that is used to disambiguate the correct sense of polysemy
word based on the clue words. The related words for each sense of a polysemy word as well as single sense
word are referred to as the clue words. The conventional WordNet organizes nouns, verbs, adjectives and
adverbs together into sets of synonyms called synsets each expressing a different concept. In contrast to the
structure of WordNet, we developed a new model of WordNet that organizes the different senses of
polysemy words as well as the single sense words based on the clue words. These clue words for each sense
of a polysemy word as well as for single sense word are used to disambiguate the correct meaning of the
polysemy word in the given context using knowledge based Word Sense Disambiguation (WSD) algorithms.
The clue word can be a noun, verb, adjective or adverb.
Dr. Preslav Nakov — Combining, Adapting and Reusing Bi-texts between Related ...Yandex
Bilingual sentence-aligned parallel corpora, or bitexts, are a useful resource for solving many computational linguistics problems including part-of speech tagging, syntactic parsing, named entity recognition, word sense disambiguation, sentiment analysis, etc.; they are also a critical resource for some real-world applications such as statistical machine translation (SMT) and cross-language information retrieval. Unfortunately, building large bi-texts is hard, and thus most of the 6,500+ world languages remain resource-poor in bi-texts. However, many resource-poor languages are related to some resource-rich language, with whom they overlap in vocabulary and share cognates, which offers opportunities for using their bi-texts.
We explore various options for bi-text reuse: (i) direct combination of bi-texts, (ii) combination of models trained on such bi-texts, and (iii) a sophisticated combination of (i) and (ii).
We further explore the idea of generating bitexts for a resource-poor language by adapting a bi-text for a resource-rich language. We build a lattice of adaptation options for each word and phrase, and we then decode it using a language model for the resource-poor language. We compare word- and phrase-level adaptation, and we further make use of cross-language morphology. For the adaptation, we experiment with (a) a standard phrase-based SMT decoder, and (b) a specialized beam-search adaptation decoder.
Finally, we observe that for closely-related languages, many of the differences are at the subword level. Thus, we explore the idea of reducing translation to character-level transliteration. We further demonstrate the potential of combining word- and character-level models.
SCORE-BASED SENTIMENT ANALYSIS OF BOOK REVIEWS IN HINDI LANGUAGEkevig
Sentiment analysis has been performed in different languages and in various domains, such as movie reviews, product reviews and tourism reviews. However, not much work has been done in the area of books considering the high availability of book reviews on Hindi blogs and online forums. In this paper, a scorebased sentiment mining system for Hindi language is discussed, which captures the sentiment behind the words of book review sentences. We conducted three experiments using scores from the HindiSentiWordNet (H-SWN), first using parts-of-speech tags of opinion words to extract their potential scores. Then, we focused on word-sense disambiguation (WSD) to increase the accuracy of system. Finally, the classification results were improved by handling morphological variations. The results were validated against human annotations achieving an overall accuracy of 86.3%. The work was extended further using Hindi Subjective Lexicon (HSL). We also developed an annotated corpus of book reviews in Hindi.
SCORE-BASED SENTIMENT ANALYSIS OF BOOK REVIEWS IN HINDI LANGUAGEijnlc
Sentiment analysis has been performed in different languages and in various domains, such as movie reviews, product reviews and tourism reviews. However, not much work has been done in the area of books considering the high availability of book reviews on Hindi blogs and online forums. In this paper, a scorebased sentiment mining system for Hindi language is discussed, which captures the sentiment behind the
words of book review sentences. We conducted three experiments using scores from the HindiSentiWordNet
(H-SWN), first using parts-of-speech tags of opinion words to extract their potential scores. Then, we focused on word-sense disambiguation (WSD) to increase the accuracy of system. Finally, the classification results were improved by handling morphological variations. The results were validated against human annotations achieving an overall accuracy of 86.3%. The work was extended further using Hindi Subjective Lexicon (HSL). We also developed an annotated corpus of book reviews in Hindi.
Building a Web-Scale Dependency-Parsed Corpus from Common CrawlAlexander Panchenko
We present DepCC, the largest-to-date linguistically analyzed corpus in English including 365 million documents, composed of 252 billion tokens and 7.5 billion of named entity occurrences in 14.3 billion sentences from a web-scale crawl of the Common Crawl project. The sentences are processed with a dependency parser and with a named entity tagger and contain provenance information, enabling various applications ranging from training syntax-based word embeddings to open information extraction and question answering. We built an index of all sentences and their linguistic meta-data enabling quick search across the corpus. We demonstrate the utility of this corpus on the verb similarity task by showing that a distributional model trained on our corpus yields better results than models trained on smaller corpora, like Wikipedia. This distributional model outperforms the state of art models of verb similarity trained on smaller corpora on the SimVerb3500 dataset.
http://www.lrec-conf.org/proceedings/lrec2018/summaries/215.html
Improving Hypernymy Extraction with Distributional Semantic ClassesAlexander Panchenko
http://www.lrec-conf.org/proceedings/lrec2018/pdf/234.pdf
In this paper, we show how distributionally-induced semantic classes can be helpful for extracting hypernyms. We present methods for inducing sense-aware semantic classes using distributional semantics and using these induced semantic classes for filtering noisy hypernymy relations. Denoising of hypernyms is performed by labeling each semantic class with its hypernyms. On the one hand, this allows us to filter out wrong extractions using the global structure of distributionally similar senses. On the other hand, we infer missing hypernyms via label propagation to cluster terms. We conduct a large-scale crowdsourcing study showing that processing of automatically extracted hypernyms using our approach improves the quality of the hypernymy extraction in terms of both precision and recall. Furthermore, we show the utility of our method in the domain taxonomy induction task, achieving the state-of-the-art results on a SemEval'16 task on taxonomy induction.
The paper was presented at the LREC'2018 conference in Miyazaki, Japan.
Graph's not dead: from unsupervised induction of linguistic structures from t...Alexander Panchenko
In this invited talk, presented at the Dialogue'2018 conference, I argue for the usefulness of graph representations for NLP in the deep learning era. In the lecture, it is described how to extract symbolic linguistic structures, such as word senses and semantic frames in an unsupervised way from text corpora using graph-based algorithms and distributional semantics.
Word sense disambiguation using wsd specific wordnet of polysemy wordsijnlc
This paper presents a new model of WordNet that is used to disambiguate the correct sense of polysemy
word based on the clue words. The related words for each sense of a polysemy word as well as single sense
word are referred to as the clue words. The conventional WordNet organizes nouns, verbs, adjectives and
adverbs together into sets of synonyms called synsets each expressing a different concept. In contrast to the
structure of WordNet, we developed a new model of WordNet that organizes the different senses of
polysemy words as well as the single sense words based on the clue words. These clue words for each sense
of a polysemy word as well as for single sense word are used to disambiguate the correct meaning of the
polysemy word in the given context using knowledge based Word Sense Disambiguation (WSD) algorithms.
The clue word can be a noun, verb, adjective or adverb.
Dr. Preslav Nakov — Combining, Adapting and Reusing Bi-texts between Related ...Yandex
Bilingual sentence-aligned parallel corpora, or bitexts, are a useful resource for solving many computational linguistics problems including part-of speech tagging, syntactic parsing, named entity recognition, word sense disambiguation, sentiment analysis, etc.; they are also a critical resource for some real-world applications such as statistical machine translation (SMT) and cross-language information retrieval. Unfortunately, building large bi-texts is hard, and thus most of the 6,500+ world languages remain resource-poor in bi-texts. However, many resource-poor languages are related to some resource-rich language, with whom they overlap in vocabulary and share cognates, which offers opportunities for using their bi-texts.
We explore various options for bi-text reuse: (i) direct combination of bi-texts, (ii) combination of models trained on such bi-texts, and (iii) a sophisticated combination of (i) and (ii).
We further explore the idea of generating bitexts for a resource-poor language by adapting a bi-text for a resource-rich language. We build a lattice of adaptation options for each word and phrase, and we then decode it using a language model for the resource-poor language. We compare word- and phrase-level adaptation, and we further make use of cross-language morphology. For the adaptation, we experiment with (a) a standard phrase-based SMT decoder, and (b) a specialized beam-search adaptation decoder.
Finally, we observe that for closely-related languages, many of the differences are at the subword level. Thus, we explore the idea of reducing translation to character-level transliteration. We further demonstrate the potential of combining word- and character-level models.
SCORE-BASED SENTIMENT ANALYSIS OF BOOK REVIEWS IN HINDI LANGUAGEkevig
Sentiment analysis has been performed in different languages and in various domains, such as movie reviews, product reviews and tourism reviews. However, not much work has been done in the area of books considering the high availability of book reviews on Hindi blogs and online forums. In this paper, a scorebased sentiment mining system for Hindi language is discussed, which captures the sentiment behind the words of book review sentences. We conducted three experiments using scores from the HindiSentiWordNet (H-SWN), first using parts-of-speech tags of opinion words to extract their potential scores. Then, we focused on word-sense disambiguation (WSD) to increase the accuracy of system. Finally, the classification results were improved by handling morphological variations. The results were validated against human annotations achieving an overall accuracy of 86.3%. The work was extended further using Hindi Subjective Lexicon (HSL). We also developed an annotated corpus of book reviews in Hindi.
SCORE-BASED SENTIMENT ANALYSIS OF BOOK REVIEWS IN HINDI LANGUAGEijnlc
Sentiment analysis has been performed in different languages and in various domains, such as movie reviews, product reviews and tourism reviews. However, not much work has been done in the area of books considering the high availability of book reviews on Hindi blogs and online forums. In this paper, a scorebased sentiment mining system for Hindi language is discussed, which captures the sentiment behind the
words of book review sentences. We conducted three experiments using scores from the HindiSentiWordNet
(H-SWN), first using parts-of-speech tags of opinion words to extract their potential scores. Then, we focused on word-sense disambiguation (WSD) to increase the accuracy of system. Finally, the classification results were improved by handling morphological variations. The results were validated against human annotations achieving an overall accuracy of 86.3%. The work was extended further using Hindi Subjective Lexicon (HSL). We also developed an annotated corpus of book reviews in Hindi.
Building a Web-Scale Dependency-Parsed Corpus from Common CrawlAlexander Panchenko
We present DepCC, the largest-to-date linguistically analyzed corpus in English including 365 million documents, composed of 252 billion tokens and 7.5 billion of named entity occurrences in 14.3 billion sentences from a web-scale crawl of the Common Crawl project. The sentences are processed with a dependency parser and with a named entity tagger and contain provenance information, enabling various applications ranging from training syntax-based word embeddings to open information extraction and question answering. We built an index of all sentences and their linguistic meta-data enabling quick search across the corpus. We demonstrate the utility of this corpus on the verb similarity task by showing that a distributional model trained on our corpus yields better results than models trained on smaller corpora, like Wikipedia. This distributional model outperforms the state of art models of verb similarity trained on smaller corpora on the SimVerb3500 dataset.
http://www.lrec-conf.org/proceedings/lrec2018/summaries/215.html
Improving Hypernymy Extraction with Distributional Semantic ClassesAlexander Panchenko
http://www.lrec-conf.org/proceedings/lrec2018/pdf/234.pdf
In this paper, we show how distributionally-induced semantic classes can be helpful for extracting hypernyms. We present methods for inducing sense-aware semantic classes using distributional semantics and using these induced semantic classes for filtering noisy hypernymy relations. Denoising of hypernyms is performed by labeling each semantic class with its hypernyms. On the one hand, this allows us to filter out wrong extractions using the global structure of distributionally similar senses. On the other hand, we infer missing hypernyms via label propagation to cluster terms. We conduct a large-scale crowdsourcing study showing that processing of automatically extracted hypernyms using our approach improves the quality of the hypernymy extraction in terms of both precision and recall. Furthermore, we show the utility of our method in the domain taxonomy induction task, achieving the state-of-the-art results on a SemEval'16 task on taxonomy induction.
The paper was presented at the LREC'2018 conference in Miyazaki, Japan.
Fighting with Sparsity of the Synonymy Dictionaries for Automatic Synset Indu...Alexander Panchenko
Presentation at the AIST'17 conference by Dmitry Ustalov. Authors of the original paper: Dmitry Ustalov, Mikhail Chernoskutov, Chris Biemann, Alexander Panchenko.
Using Linked Disambiguated Distributional Networks for Word Sense DisambiguationAlexander Panchenko
We introduce a new method for unsupervised knowledge-based word sense disambiguation (WSD) based on a resource that links two types of sense-aware lexical networks: one is induced from a corpus using distributional semantics, the other is manually constructed. The combination of two networks reduces the sparsity of sense representations used for WSD. We evaluate these enriched representations within two lexical sample sense disambiguation benchmarks. Our results indicate that (1) features extracted from the corpus-based resource help to significantly outperform a model based solely on the lexical resource; (2) our method achieves results comparable or better to four state-of-the-art unsupervised knowledge-based WSD systems including three hybrid systems that also rely on text corpora. In contrast to these hybrid methods, our approach does not require access to web search engines, texts mapped to a sense inventory, or machine translation systems.
See the full paper at: http://www.aclweb.org/anthology/W/W17/W17-1909.pdf
Panchenko A., Faralli S., Ponzetto S. P., and Biemann C. (2017): Using Linked Disambiguated Distributional Networks for Word Sense Disambiguation. In Proceedings of the Workshop on Sense, Concept and Entity Representations and their Applications (SENSE) co-located with the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL'2017). Valencia, Spain. Association for Computational Linguistics
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction...Alexander Panchenko
The current trend in NLP is the use of highly opaque models, e.g. neural networks and word embeddings. While these models yield state-of-the-art results on a range of tasks, their drawback is
poor interpretability. On the example of word sense induction and disambiguation (WSID), we show that it is possible to develop an interpretable model that matches the state-of-the-art models in accuracy. Namely, we present an unsupervised, knowledge-free WSID approach, which is interpretable at three levels: word sense inventory, sense feature representations, and disambiguation procedure. Experiments show that our model performs on par with state-of-the-art word sense embeddings and other unsupervised systems while offering the possibility to justify
its decisions in human-readable form.
Noun Sense Induction and Disambiguation using Graph-Based Distributional Sema...Alexander Panchenko
We introduce an approach to word sense
induction and disambiguation. The method
is unsupervised and knowledge-free: sense
representations are learned from distributional
evidence and subsequently used to
disambiguate word instances in context.
These sense representations are obtained
by clustering dependency-based secondorder
similarity networks. We then add
features for disambiguation from heterogeneous
sources such as window-based and
sentence-wide co-occurrences, and explore
various schemes to combine these context
clues. Our method reaches a performance
comparable to the state-of-the-art unsupervised
word sense disambiguation systems
including top participants of the SemEval
2013 word sense induction task and two
more recent state-of-the-art neural word
sense induction systems
Full paper:
https://www.lt.informatik.tu-darmstadt.de/fileadmin/user_upload/Group_LangTech/publications/konvens2016panchenko.pdf
Ayush Kumar, Sarah Kohail, Amit Kumar, Asif Ekbal, Chris Biemann
IIT Patna, India
TU Darmstadt, Germany
Presented by: Alexander Panchenko, TU Darmstadt, Germany
A sentiment index measures the average emotional level in a corpus. We introduce four such indexes and use them to gauge average “positiveness” of a population during some period based on posts in a social network. This article for the first time presents a text-, rather than word-based sentiment index. Furthermore, this study presents the first large-scale study of the sentiment index of the Russian-speaking Facebook. Our results are consistent with the prior experiments for English language.
Semantic relations, such as synonyms, hypernyms and co-hyponyms proved to be useful for text processing applications, including text similarity, query expansion, question answering and word sense disambiguation. Such relations are practical because of the gap between lexical surface of the text and its meaning. Indeed, the same concept is often represented by different terms. However, existing resources often do not cover a vocabulary required by a given system. Manual resource construction is prohibitively expensive for many projects.
On the other hand, precision of the existing extractors still do not meet quality of the handcrafted resources. All these factors motivate the development of novel extraction methods. In this work we developed several similarity measures for semantic relation extraction. The main research question we address, is how to improve precision and coverage of such measures. First, we perform a large-scale study the baseline techniques. Second, we propose four novel measures. One of them significantly outperforms the baselines, the others perform comparably to the state-of-the-art techniques. Finally, we successfully apply one of the novel measures in two text processing systems.
Detecting Gender by Full Name: Experiments with the Russian LanguageAlexander Panchenko
This paper describes a method that detects gender of a person by his/her full name. While some approaches were proposed for English language, little has been done so far for Russian. We fill this gap and present a large-scale experiment on a dataset of 100,000 Russian full names from Facebook. Our method is based on three types of features (word endings, character $n$-grams and dictionary of names) combined within a linear supervised model. Experiments show that the proposed simple and computationally efficient approach yields excellent results achieving accuracy up to 96\%.
Вычислительная лексическая семантика: метрики семантической близости и их при...Alexander Panchenko
Вычислительная лексическая семантика: метрики семантической близости и их приложения
Серия лекций в НИУ ВШЭ, факультет бизнес-информатики и прикладной математики (Нижний Новгород)
Nutraceutical market, scope and growth: Herbal drug technologyLokesh Patil
As consumer awareness of health and wellness rises, the nutraceutical market—which includes goods like functional meals, drinks, and dietary supplements that provide health advantages beyond basic nutrition—is growing significantly. As healthcare expenses rise, the population ages, and people want natural and preventative health solutions more and more, this industry is increasing quickly. Further driving market expansion are product formulation innovations and the use of cutting-edge technology for customized nutrition. With its worldwide reach, the nutraceutical industry is expected to keep growing and provide significant chances for research and investment in a number of categories, including vitamins, minerals, probiotics, and herbal supplements.
The ability to recreate computational results with minimal effort and actionable metrics provides a solid foundation for scientific research and software development. When people can replicate an analysis at the touch of a button using open-source software, open data, and methods to assess and compare proposals, it significantly eases verification of results, engagement with a diverse range of contributors, and progress. However, we have yet to fully achieve this; there are still many sociotechnical frictions.
Inspired by David Donoho's vision, this talk aims to revisit the three crucial pillars of frictionless reproducibility (data sharing, code sharing, and competitive challenges) with the perspective of deep software variability.
Our observation is that multiple layers — hardware, operating systems, third-party libraries, software versions, input data, compile-time options, and parameters — are subject to variability that exacerbates frictions but is also essential for achieving robust, generalizable results and fostering innovation. I will first review the literature, providing evidence of how the complex variability interactions across these layers affect qualitative and quantitative software properties, thereby complicating the reproduction and replication of scientific studies in various fields.
I will then present some software engineering and AI techniques that can support the strategic exploration of variability spaces. These include the use of abstractions and models (e.g., feature models), sampling strategies (e.g., uniform, random), cost-effective measurements (e.g., incremental build of software configurations), and dimensionality reduction methods (e.g., transfer learning, feature selection, software debloating).
I will finally argue that deep variability is both the problem and solution of frictionless reproducibility, calling the software science community to develop new methods and tools to manage variability and foster reproducibility in software systems.
Exposé invité Journées Nationales du GDR GPL 2024
Fighting with Sparsity of the Synonymy Dictionaries for Automatic Synset Indu...Alexander Panchenko
Presentation at the AIST'17 conference by Dmitry Ustalov. Authors of the original paper: Dmitry Ustalov, Mikhail Chernoskutov, Chris Biemann, Alexander Panchenko.
Using Linked Disambiguated Distributional Networks for Word Sense DisambiguationAlexander Panchenko
We introduce a new method for unsupervised knowledge-based word sense disambiguation (WSD) based on a resource that links two types of sense-aware lexical networks: one is induced from a corpus using distributional semantics, the other is manually constructed. The combination of two networks reduces the sparsity of sense representations used for WSD. We evaluate these enriched representations within two lexical sample sense disambiguation benchmarks. Our results indicate that (1) features extracted from the corpus-based resource help to significantly outperform a model based solely on the lexical resource; (2) our method achieves results comparable or better to four state-of-the-art unsupervised knowledge-based WSD systems including three hybrid systems that also rely on text corpora. In contrast to these hybrid methods, our approach does not require access to web search engines, texts mapped to a sense inventory, or machine translation systems.
See the full paper at: http://www.aclweb.org/anthology/W/W17/W17-1909.pdf
Panchenko A., Faralli S., Ponzetto S. P., and Biemann C. (2017): Using Linked Disambiguated Distributional Networks for Word Sense Disambiguation. In Proceedings of the Workshop on Sense, Concept and Entity Representations and their Applications (SENSE) co-located with the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL'2017). Valencia, Spain. Association for Computational Linguistics
Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction...Alexander Panchenko
The current trend in NLP is the use of highly opaque models, e.g. neural networks and word embeddings. While these models yield state-of-the-art results on a range of tasks, their drawback is
poor interpretability. On the example of word sense induction and disambiguation (WSID), we show that it is possible to develop an interpretable model that matches the state-of-the-art models in accuracy. Namely, we present an unsupervised, knowledge-free WSID approach, which is interpretable at three levels: word sense inventory, sense feature representations, and disambiguation procedure. Experiments show that our model performs on par with state-of-the-art word sense embeddings and other unsupervised systems while offering the possibility to justify
its decisions in human-readable form.
Noun Sense Induction and Disambiguation using Graph-Based Distributional Sema...Alexander Panchenko
We introduce an approach to word sense
induction and disambiguation. The method
is unsupervised and knowledge-free: sense
representations are learned from distributional
evidence and subsequently used to
disambiguate word instances in context.
These sense representations are obtained
by clustering dependency-based secondorder
similarity networks. We then add
features for disambiguation from heterogeneous
sources such as window-based and
sentence-wide co-occurrences, and explore
various schemes to combine these context
clues. Our method reaches a performance
comparable to the state-of-the-art unsupervised
word sense disambiguation systems
including top participants of the SemEval
2013 word sense induction task and two
more recent state-of-the-art neural word
sense induction systems
Full paper:
https://www.lt.informatik.tu-darmstadt.de/fileadmin/user_upload/Group_LangTech/publications/konvens2016panchenko.pdf
Ayush Kumar, Sarah Kohail, Amit Kumar, Asif Ekbal, Chris Biemann
IIT Patna, India
TU Darmstadt, Germany
Presented by: Alexander Panchenko, TU Darmstadt, Germany
A sentiment index measures the average emotional level in a corpus. We introduce four such indexes and use them to gauge average “positiveness” of a population during some period based on posts in a social network. This article for the first time presents a text-, rather than word-based sentiment index. Furthermore, this study presents the first large-scale study of the sentiment index of the Russian-speaking Facebook. Our results are consistent with the prior experiments for English language.
Semantic relations, such as synonyms, hypernyms and co-hyponyms proved to be useful for text processing applications, including text similarity, query expansion, question answering and word sense disambiguation. Such relations are practical because of the gap between lexical surface of the text and its meaning. Indeed, the same concept is often represented by different terms. However, existing resources often do not cover a vocabulary required by a given system. Manual resource construction is prohibitively expensive for many projects.
On the other hand, precision of the existing extractors still do not meet quality of the handcrafted resources. All these factors motivate the development of novel extraction methods. In this work we developed several similarity measures for semantic relation extraction. The main research question we address, is how to improve precision and coverage of such measures. First, we perform a large-scale study the baseline techniques. Second, we propose four novel measures. One of them significantly outperforms the baselines, the others perform comparably to the state-of-the-art techniques. Finally, we successfully apply one of the novel measures in two text processing systems.
Detecting Gender by Full Name: Experiments with the Russian LanguageAlexander Panchenko
This paper describes a method that detects gender of a person by his/her full name. While some approaches were proposed for English language, little has been done so far for Russian. We fill this gap and present a large-scale experiment on a dataset of 100,000 Russian full names from Facebook. Our method is based on three types of features (word endings, character $n$-grams and dictionary of names) combined within a linear supervised model. Experiments show that the proposed simple and computationally efficient approach yields excellent results achieving accuracy up to 96\%.
Вычислительная лексическая семантика: метрики семантической близости и их при...Alexander Panchenko
Вычислительная лексическая семантика: метрики семантической близости и их приложения
Серия лекций в НИУ ВШЭ, факультет бизнес-информатики и прикладной математики (Нижний Новгород)
Nutraceutical market, scope and growth: Herbal drug technologyLokesh Patil
As consumer awareness of health and wellness rises, the nutraceutical market—which includes goods like functional meals, drinks, and dietary supplements that provide health advantages beyond basic nutrition—is growing significantly. As healthcare expenses rise, the population ages, and people want natural and preventative health solutions more and more, this industry is increasing quickly. Further driving market expansion are product formulation innovations and the use of cutting-edge technology for customized nutrition. With its worldwide reach, the nutraceutical industry is expected to keep growing and provide significant chances for research and investment in a number of categories, including vitamins, minerals, probiotics, and herbal supplements.
The ability to recreate computational results with minimal effort and actionable metrics provides a solid foundation for scientific research and software development. When people can replicate an analysis at the touch of a button using open-source software, open data, and methods to assess and compare proposals, it significantly eases verification of results, engagement with a diverse range of contributors, and progress. However, we have yet to fully achieve this; there are still many sociotechnical frictions.
Inspired by David Donoho's vision, this talk aims to revisit the three crucial pillars of frictionless reproducibility (data sharing, code sharing, and competitive challenges) with the perspective of deep software variability.
Our observation is that multiple layers — hardware, operating systems, third-party libraries, software versions, input data, compile-time options, and parameters — are subject to variability that exacerbates frictions but is also essential for achieving robust, generalizable results and fostering innovation. I will first review the literature, providing evidence of how the complex variability interactions across these layers affect qualitative and quantitative software properties, thereby complicating the reproduction and replication of scientific studies in various fields.
I will then present some software engineering and AI techniques that can support the strategic exploration of variability spaces. These include the use of abstractions and models (e.g., feature models), sampling strategies (e.g., uniform, random), cost-effective measurements (e.g., incremental build of software configurations), and dimensionality reduction methods (e.g., transfer learning, feature selection, software debloating).
I will finally argue that deep variability is both the problem and solution of frictionless reproducibility, calling the software science community to develop new methods and tools to manage variability and foster reproducibility in software systems.
Exposé invité Journées Nationales du GDR GPL 2024
This presentation explores a brief idea about the structural and functional attributes of nucleotides, the structure and function of genetic materials along with the impact of UV rays and pH upon them.
Professional air quality monitoring systems provide immediate, on-site data for analysis, compliance, and decision-making.
Monitor common gases, weather parameters, particulates.
Toxic effects of heavy metals : Lead and Arsenicsanjana502982
Heavy metals are naturally occuring metallic chemical elements that have relatively high density, and are toxic at even low concentrations. All toxic metals are termed as heavy metals irrespective of their atomic mass and density, eg. arsenic, lead, mercury, cadmium, thallium, chromium, etc.
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Sérgio Sacani
We characterize the earliest galaxy population in the JADES Origins Field (JOF), the deepest
imaging field observed with JWST. We make use of the ancillary Hubble optical images (5 filters
spanning 0.4−0.9µm) and novel JWST images with 14 filters spanning 0.8−5µm, including 7 mediumband filters, and reaching total exposure times of up to 46 hours per filter. We combine all our data
at > 2.3µm to construct an ultradeep image, reaching as deep as ≈ 31.4 AB mag in the stack and
30.3-31.0 AB mag (5σ, r = 0.1” circular aperture) in individual filters. We measure photometric
redshifts and use robust selection criteria to identify a sample of eight galaxy candidates at redshifts
z = 11.5 − 15. These objects show compact half-light radii of R1/2 ∼ 50 − 200pc, stellar masses of
M⋆ ∼ 107−108M⊙, and star-formation rates of SFR ∼ 0.1−1 M⊙ yr−1
. Our search finds no candidates
at 15 < z < 20, placing upper limits at these redshifts. We develop a forward modeling approach to
infer the properties of the evolving luminosity function without binning in redshift or luminosity that
marginalizes over the photometric redshift uncertainty of our candidate galaxies and incorporates the
impact of non-detections. We find a z = 12 luminosity function in good agreement with prior results,
and that the luminosity function normalization and UV luminosity density decline by a factor of ∼ 2.5
from z = 12 to z = 14. We discuss the possible implications of our results in the context of theoretical
models for evolution of the dark matter halo mass function.
What is greenhouse gasses and how many gasses are there to affect the Earth.moosaasad1975
What are greenhouse gasses how they affect the earth and its environment what is the future of the environment and earth how the weather and the climate effects.
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...University of Maribor
Slides from:
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Track: Artificial Intelligence
https://www.etran.rs/2024/en/home-english/
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...Sérgio Sacani
Since volcanic activity was first discovered on Io from Voyager images in 1979, changes
on Io’s surface have been monitored from both spacecraft and ground-based telescopes.
Here, we present the highest spatial resolution images of Io ever obtained from a groundbased telescope. These images, acquired by the SHARK-VIS instrument on the Large
Binocular Telescope, show evidence of a major resurfacing event on Io’s trailing hemisphere. When compared to the most recent spacecraft images, the SHARK-VIS images
show that a plume deposit from a powerful eruption at Pillan Patera has covered part
of the long-lived Pele plume deposit. Although this type of resurfacing event may be common on Io, few have been detected due to the rarity of spacecraft visits and the previously low spatial resolution available from Earth-based telescopes. The SHARK-VIS instrument ushers in a new era of high resolution imaging of Io’s surface using adaptive
optics at visible wavelengths.
2. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 2/54
Overview
3. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 3/54
Inducing word sense representations:
word sense embeddings via retrofitting
[Pelevina et al., 2016, Remus & Biemann, 2018];
inducing synsets [Ustalov et al., 2017b, Ustalov et al., 2017a,
Ustalov et al., 2018]
inducing semantic classes [Panchenko et al., 2018]
Overview
Overview
4. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 3/54
Inducing word sense representations:
word sense embeddings via retrofitting
[Pelevina et al., 2016, Remus & Biemann, 2018];
inducing synsets [Ustalov et al., 2017b, Ustalov et al., 2017a,
Ustalov et al., 2018]
inducing semantic classes [Panchenko et al., 2018]
Making induced senses interpretable
[Panchenko et al., 2017b, Panchenko et al., 2017c]
Overview
Overview
5. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 3/54
Inducing word sense representations:
word sense embeddings via retrofitting
[Pelevina et al., 2016, Remus & Biemann, 2018];
inducing synsets [Ustalov et al., 2017b, Ustalov et al., 2017a,
Ustalov et al., 2018]
inducing semantic classes [Panchenko et al., 2018]
Making induced senses interpretable
[Panchenko et al., 2017b, Panchenko et al., 2017c]
Linking induced word senses to lexical
resources [Panchenko, 2016, Faralli et al., 2016,
Panchenko et al., 2017a, Biemann et al., 2018]
Overview
Overview
6. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 4/54
Inducing word sense representations
7. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 5/54
Inducing word sense representations
Word vs sense embeddings
8. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 6/54
Inducing word sense representations
Word vs sense embeddings
9. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 7/54
Inducing word sense representations
Related work
10. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 8/54
AutoExtend [Rothe & Schütze, 2015]
* image is reproduced from the original paper
Inducing word sense representations
Related work: knowledge-based
11. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 9/54
Adagram [Bartunov et al., 2016]
Multiple vector representations θ for each word:
Inducing word sense representations
Related work: knowledge-free
12. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 9/54
Adagram [Bartunov et al., 2016]
Multiple vector representations θ for each word:
p(Y, Z, β|X, α, θ) =
V∏
w=1
∞∏
k=1
p(βwk|α)
N∏
i=1
[p(zi|xi, β)
C∏
j=1
p(yij|zi, xi, θ)],
zi – a hidden variable: a sense index of word xi in context C;
α – a meta-parameter controlling number of senses.
Inducing word sense representations
Related work: knowledge-free
13. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 9/54
Adagram [Bartunov et al., 2016]
Multiple vector representations θ for each word:
p(Y, Z, β|X, α, θ) =
V∏
w=1
∞∏
k=1
p(βwk|α)
N∏
i=1
[p(zi|xi, β)
C∏
j=1
p(yij|zi, xi, θ)],
zi – a hidden variable: a sense index of word xi in context C;
α – a meta-parameter controlling number of senses.
See also: [Neelakantan et al., 2014] and [Li and Jurafsky, 2015]
Inducing word sense representations
Related work: knowledge-free
14. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 10/54
Word sense induction (WSI) based on graph clustering:
[Lin, 1998]
[Pantel and Lin, 2002]
[Widdows and Dorow, 2002]
Chinese Whispers [Biemann, 2006]
[Hope and Keller, 2013]
Inducing word sense representations
Related work: word sense induction
15. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 11/54
* source of the image: http://ic.pics.livejournal.com/blagin_anton/33716210/2701748/2701748_800.jpg
Inducing word sense representations
Related work: Chinese Whispers#1
16. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 12/54
Inducing word sense representations
Related work: Chinese Whispers#2
17. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 13/54
Inducing word sense representations
Related work: Chinese Whispers#2
18. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 14/54
Inducing word sense representations
Related work: Chinese Whispers#2
19. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 15/54
RepL4NLP@ACL’16 [Pelevina et al., 2016], LREC’18 [Remus & Biemann, 2018]
Prior methods:
Induce inventory by clustering of word instances
Use existing sense inventories
Our method:
Input: word embeddings
Output: word sense embeddings
Word sense induction by clustering of word ego-networks
Inducing word sense representations
Sense embeddings using retrofitting
20. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 16/54
From word embeddings to sense embeddings
Calculate Word
Similarity Graph
Learning Word Vectors
Word Sense Induction
Text Corpus
Word Vectors
Word Similarity Graph
Pooling of Word Vectors
Sense Inventory
Sense Vectors
1 2
4 3
Inducing word sense representations
Sense embeddings using retrofitting
21. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 17/54
Word sense induction using ego-network clustering
Inducing word sense representations
Sense embeddings using retrofitting
22. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 18/54
Neighbours of Word and Sense Vectors
Vector Nearest Neighbors
table tray, bottom, diagram, bucket, brackets, stack, basket,
list, parenthesis, cup, saucer, pile, playfield, bracket,
pot, drop-down, cue, plate
Inducing word sense representations
Sense embeddings using retrofitting
23. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 18/54
Neighbours of Word and Sense Vectors
Vector Nearest Neighbors
table tray, bottom, diagram, bucket, brackets, stack, basket,
list, parenthesis, cup, saucer, pile, playfield, bracket,
pot, drop-down, cue, plate
table#0 leftmost#0, column#1, tableau#1, indent#1, bracket#3,
pointer#0, footer#1, cursor#1, diagram#0, grid#0
table#1 pile#1, stool#1, tray#0, basket#0, bowl#1, bucket#0,
box#0, cage#0, saucer#3, mirror#1, pan#1, lid#0
Inducing word sense representations
Sense embeddings using retrofitting
24. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 19/54
Word Sense Disambiguation
1 Context extraction: use context words around the target
word
2 Context filtering: based on context word’s relevance for
disambiguation
3 Sense choice in context: maximise similarity between a
context vector and a sense vector
Inducing word sense representations
Sense embeddings using retrofitting
25. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 20/54
Inducing word sense representations
Sense embeddings using retrofitting
26. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 21/54
Inducing word sense representations
Sense embeddings using retrofitting
27. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 22/54
Inducing word sense representations
Sense embeddings using retrofitting
28. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 23/54
Inducing word sense representations
Sense embeddings using retrofitting
29. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 24/54
Unsupervised WSD SemEval’13, ReprL4NLP [Pelevina et al., 2016]:
comparable to SOTA, incl. Adagram sense embeddings.
Inducing word sense representations
Sense embeddings using retrofitting
30. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 24/54
Unsupervised WSD SemEval’13, ReprL4NLP [Pelevina et al., 2016]:
comparable to SOTA, incl. Adagram sense embeddings.
Semantic relatedness, LREC’2018 [Remus & Biemann, 2018]:
autoextend
adagram
SGNS
.
glove
.
sympat
.
LSAbow
.
LSAhal
.
paragramSL
.
SimLex999 0.45 0.29 0.44 0.37 0.54 0.30 0.27 0.68
MEN 0.72 0.67 0.77 0.73 0.53 0.67 0.71 0.77
SimVerb 0.43 0.27 0.36 0.23 0.37 0.15 0.19 0.53
WordSim353 0.58 0.61 0.70 0.61 0.47 0.67 0.59 0.72
SimLex999-N 0.44 0.33 0.45 0.39 0.48 0.32 0.34 0.68
MEN-N 0.72 0.68 0.77 ____ 0.76 ____ 0.57 ____ 0.71 ____ 0.73 ____ 0.78 ____
Inducing word sense representations
Sense embeddings using retrofitting
32. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 26/54
Word and sense embeddings
of words iron and vitamin.
LREC’18 [Remus & Biemann, 2018]
Inducing word sense representations
Sense embeddings using retrofitting
33. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 27/54
ACL’17 [Ustalov et al., 2017b]
Examples of extracted synsets:
Size Synset
2 {decimal point, dot}
3 {gullet, throat, food pipe}
4 {microwave meal, ready meal, TV dinner, frozen dinner}
5 {objective case, accusative case, oblique case, object
case, accusative}
6 {radiotheater, dramatizedaudiobook, audiotheater, ra-
dio play, radio drama, audio play}
Inducing word sense representations
Synset induction
34. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 28/54
Outline of the ’Watset’ method:
Background Corpus
Synonymy Dictionary
Learning
Word
Embeddings
Graph
Construction
Synsets
Word
Similarities
Ambiguous
Weighted Graph
Local
Clustering:
Word Sense Induction
Global
Clustering:
Synset Induction
Sense Inventory
Disambiguation
of
Neighbors
Disambiguated
Weighted Graph
Local-Global
Fuzzy
Graph
Clustering
Inducing word sense representations
Synset induction
35. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 29/54
Inducing word sense representations
Synset induction
36. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 30/54
Inducing word sense representations
Synset induction
37. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 31/54
Inducing word sense representations
Synset induction
38. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 32/54
CW MCL MaxMax ECO CPM Watset
0.0
0.1
0.2
0.3
WordNet (English)
F−score
CW MCL MaxMax ECO CPM Watset
0.0
0.1
0.2
0.3
BabelNet (English)
F−score
Inducing word sense representations
Synset induction
39. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 32/54
CW MCL MaxMax ECO CPM Watset
0.0
0.1
0.2
0.3
WordNet (English)
F−score
CW MCL MaxMax ECO CPM Watset
0.0
0.1
0.2
0.3
BabelNet (English)
F−score
CW MCL MaxMax ECO CPM Watset
0.00
0.05
0.10
0.15
0.20
RuWordNet (Russian)
F−score
CW MCL MaxMax ECO CPM Watset
0.0
0.1
0.2
0.3
0.4
YARN (Russian)
F−score
Inducing word sense representations
Synset induction
40. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 33/54
Examples of semantic classes:
ID Sense Cluster Hypernyms
1 peach#1, banana#1, pineapple#0, berry#0,
blackberry#0, grapefruit#0, strawberry#0, blue-
berry#0, grape#0, melon#0, orange#0, pear#0,
plum#0, raspberry#0, watermelon#0, apple#0,
apricot#0, …
fruit#0, crop#0, ingredi-
ent#0, food#0, ·
2 C#4, Basic#2, Haskell#5, Flash#1, Java#1, Pas-
cal#0, Ruby#6, PHP#0, Ada#1, Oracle#3, Python#3,
Apache#3, Visual Basic#1, ASP#2, Delphi#2, SQL
Server#0, CSS#0, AJAX#0, the Java#0, …
programming language#3,
technology#0, language#0,
format#2, app#0
Inducing word sense representations
Induction of semantic classes
41. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 34/54
Text
Corpus
Representing
Senses
with
Ego
Networks
Semantic
Classes
Word
Sense
Induction
from
Text
Corpus
Sense
Graph
Construction
Clustering
of
Word
Senes
Labeling
Sense
Clusters
with
Hypernyms
Induced Word Senses Sense Ego-Networks Global Sense Graph
s
Noisy
Hypernyms
Cleansed
Hypernyms
Induction
of
Semantic
Classes
Global Sense Clusters
Inducing word sense representations
Induction of semantic classes
42. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 35/54
Filtering noisy hypernyms with semantic classes
LREC’18 [Panchenko et al., 2018]:
fruit#1
food#0
apple#2 mango#0 pear#0
Hypernyms,
Sense Cluster,
mangosteen#0
city#2
Removed
Wrong
Added
Missing
Inducing word sense representations
Induction of sense semantic classes
43. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 36/54
Filtering of a noisy hypernymy database with semantic classes.
LREC’18 [Panchenko et al., 2018]
Precision Recall F-score
Original Hypernyms (Seitner et al., 2016) 0.475 0.546 0.508
Semantic Classes (coarse-grained) 0.541 0.679 0.602
Inducing word sense representations
Induction of sense semantic classes
44. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 37/54
Making induced senses interpretable
45. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 38/54
Knowledge-based sense representations are interpretable
Making induced senses interpretable
Making induced senses interpretable
46. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 39/54
Most knowledge-free sense representations are uninterpretable
Making induced senses interpretable
Making induced senses interpretable
47. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 40/54
Making induced senses interpretable
Making induced senses interpretable
48. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 41/54
Hypernymy prediction in context. EMNLP’17 [Panchenko et al., 2017b]
Making induced senses interpretable
Making induced senses interpretable
49. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 42/54
11.702 sentences, 863 words with avg.polysemy of 3.1.
WSD Model Accuracy
Inventory Features Hypers HyperHypers
Word Senses Random 0.257 0.610
Word Senses MFS 0.292 0.682
Word Senses Cluster Words 0.291 0.650
Word Senses Context Words 0.308 0.686
Making induced senses interpretable
Making induced senses interpretable
50. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 42/54
11.702 sentences, 863 words with avg.polysemy of 3.1.
WSD Model Accuracy
Inventory Features Hypers HyperHypers
Word Senses Random 0.257 0.610
Word Senses MFS 0.292 0.682
Word Senses Cluster Words 0.291 0.650
Word Senses Context Words 0.308 0.686
Super Senses Random 0.001 0.001
Super Senses MFS 0.001 0.001
Super Senses Cluster Words 0.174 0.365
Super Senses Context Words 0.086 0.188
Making induced senses interpretable
Making induced senses interpretable
51. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 43/54
Linking induced senses to resources
52. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 44/54
Text Corpus
Linking
Induced
Senses
to
Senses
of
the
LR
Induce
a
Graph
of
Sem.
Related
Words
Enriched
Lexical Resource
Graph of Related Words
Word
Sense
Induction
Labeling
Senses
with
Hypernyms
Disambiguation
of
Neighbours
Typing
of
the
Unmapped
Induced
Senses
Word Sense Inventory Labeled Word Senses
PCZ
Construction
of
Proto-Conceptualization (PCZ)
Linking
Proto-Conceptualization
to
Lexical
Resource
Part. Linked Senses to the LR
Lexical Resource (LR):
WordNet, BabelNet, ...
Construction
of
sense
feature
representations
Graph of Related Senses
LREC’16 [Panchenko, 2016], ISWC’16 [Faralli et al., 2016],
SENSE@EACL’17 [Panchenko et al., 2017a],
NLE’18 [Biemann et al., 2018]
Linking induced senses to resources
Linking induced senses to resources
55. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 47/54
Evaluation of linking accuracy:
Linking induced senses to resources
Linking induced senses to resources
56. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 48/54
Evaluation of enriched representations based on WSD:
Linking induced senses to resources
Linking induced senses to resources
57. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 49/54
Conclusion
58. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 50/54
Conclusion
Vectors + Graphs = ♡
59. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 51/54
We can induce word senses, synsets and semantic classes in
a knowledge-free way using graph clustering and
distributional models.
Conclusion
Take home messages
60. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 51/54
We can induce word senses, synsets and semantic classes in
a knowledge-free way using graph clustering and
distributional models.
We can make the induced word senses interpretable in a
knowledge-free way with hypernyms, images, definitions.
Conclusion
Take home messages
61. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 51/54
We can induce word senses, synsets and semantic classes in
a knowledge-free way using graph clustering and
distributional models.
We can make the induced word senses interpretable in a
knowledge-free way with hypernyms, images, definitions.
We can link induced senses to lexical resources to
improve performance of WSD;
enrich lexical resources with emerging senses.
Conclusion
Take home messages
62. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 52/54
Participate in an ACL SIGSLAV sponsored shared task on
word sense induction and disambiguation for Russian!
More details: http://russe.nlpub.org/2018/wsi
Conclusion
An ongoing shared task on WSI&D
63. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 53/54
Thank you! Questions?
This research was supported by
Conclusion
Acknowledgments
65. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 54/54
Bartunov, S., Kondrashkin, D., Osokin, A., & Vetrov, D. (2016).
Breaking sticks and ambiguities with adaptive skip-gram.
In Artificial Intelligence and Statistics (pp. 130–138).
Biemann, C., Faralli, S., Panchenko, A., & Ponzetto, S. P. (2018).
A framework for enriching lexical semantic resources with
distributional semantics.
In Journal of Natural Language Engineering (pp. 56–64).:
Cambridge Press.
Faralli, S., Panchenko, A., Biemann, C., & Ponzetto, S. P. (2016).
Linked disambiguated distributional semantic networks.
In International Semantic Web Conference (pp. 56–64).:
Springer.
Panchenko, A. (2016).
Best of both worlds: Making word sense embeddings
interpretable.
In LREC.
66. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 54/54
Panchenko, A., Faralli, S., Ponzetto, S. P., & Biemann, C.
(2017a).
Using linked disambiguated distributional networks for word
sense disambiguation.
In Proceedings of the 1st Workshop on Sense, Concept and
Entity Representations and their Applications (pp. 72–78).
Valencia, Spain: Association for Computational Linguistics.
Panchenko, A., Marten, F., Ruppert, E., Faralli, S., Ustalov, D.,
Ponzetto, S. P., & Biemann, C. (2017b).
Unsupervised, knowledge-free, and interpretable word sense
disambiguation.
In Proceedings of the 2017 Conference on Empirical Methods in
Natural Language Processing: System Demonstrations (pp.
91–96). Copenhagen, Denmark: Association for
Computational Linguistics.
Panchenko, A., Ruppert, E., Faralli, S., Ponzetto, S. P., &
Biemann, C. (2017c).
67. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 54/54
Unsupervised does not mean uninterpretable: The case for
word sense induction and disambiguation.
In Proceedings of the 15th Conference of the European Chapter
of the Association for Computational Linguistics: Volume 1,
Long Papers (pp. 86–98). Valencia, Spain: Association for
Computational Linguistics.
Panchenko, A., Ustalov, D., Faralli, S., Ponzetto, S. P., &
Biemann, C. (2018).
Improving hypernymy extraction with distributional
semantic classes.
In Proceedings of the LREC 2018 Miyazaki, Japan: European
Language Resources Association.
Pelevina, M., Arefiev, N., Biemann, C., & Panchenko, A. (2016).
Making sense of word embeddings.
In Proceedings of the 1st Workshop on Representation
Learning for NLP (pp. 174–183). Berlin, Germany: Association
for Computational Linguistics.
Remus, S. & Biemann, C. (2018).
68. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 54/54
Retrofittingword representations for unsupervised sense
aware word similarities.
In Proceedings of the LREC 2018 Miyazaki, Japan: European
Language Resources Association.
Rothe, S. & Schütze, H. (2015).
Autoextend: Extending word embeddings to embeddings for
synsets and lexemes.
In Proceedings of the 53rd Annual Meeting of the Association
for Computational Linguistics and the 7th International Joint
Conference on Natural Language Processing (Volume 1: Long
Papers) (pp. 1793–1803). Beijing, China: Association for
Computational Linguistics.
Ustalov, D., Chernoskutov, M., Biemann, C., & Panchenko, A.
(2017a).
Fighting with the sparsity of synonymy dictionaries for
automatic synset induction.
In International Conference on Analysis of Images, Social
Networks and Texts (pp. 94–105).: Springer.
69. Jan 11, 2018 Inducing Interpretable Word Senses for WSD and Enrichment of Lexical Resources, Alexander Panchenko 54/54
Ustalov, D., Panchenko, A., & Biemann, C. (2017b).
Watset: Automatic induction of synsets from a graph of
synonyms.
In Proceedings of the 55th Annual Meeting of the Association
for Computational Linguistics (Volume 1: Long Papers) (pp.
1579–1590). Vancouver, Canada: Association for
Computational Linguistics.
Ustalov, D., Teslenko, D., Panchenko, A., Chernoskutov, M., &
Biemann, C. (2018).
Word sense disambiguation based on automatically induced
synsets.
In LREC 2018, 11th International Conference on Language
Resources and Evaluation : 7-12 May 2018, Miyazaki (Japan)
(pp. tba). Paris: European Language Resources Association,
ELRA-ELDA.
Accepted for publication.