Download as PDF, PPTX
![Issues in Syntax
Chunking: identify basic structures (phrases)
[the dog]-NP [ate]-VP [my homework]-NP
Shallow parsing
the dog->subject ate->predicate my homework->object
“the dog ate my homework”](https://image.slidesharecdn.com/analisidiacronica-161121165739/75/Diachronic-Analysis-13-2048.jpg)
Uploaded byPierpaolo Basile
Diachronic Analysis
The document discusses the field of natural language processing (NLP) and its applications, including information retrieval, machine translation, and personal assistants. It also explores diachronic linguistics, which studies language change over time, and introduces tools like the Google N-gram Viewer for analyzing language evolution. The author emphasizes the importance of semantics in NLP, particularly in the context of computational linguistics and distributional semantic models.
More Related Content
Diachronic Analysis
- 1.
- 2. Hello! Pierpaolo Basile Researcher inComputer Science Natural Language Processing, Distributional Semantic Models, Information Retrieval pierpaolo.basile@gmail.com http://www.di.uniba.it/~basilepp/
- 3. Basic Concepts Natural LanguageProcessing Computational Linguistics Diachronic Linguistics
- 4. Natural Language Refers tothe language spoken by people, e.g. English, Japanese, Swahili, Italian, as opposed to artificial languages, like C++, Java, etc.
- 5. …Processing Applications that dealwith natural language in a way or another
- 6. NLP Applications ▪ Classifytext into categories ▪ Index and search large texts ▪ Automatic translation ▪ Speech understanding ▪ Information extraction ▪ Automatic summarization ▪ Question answering ▪ Knowledge acquisition ▪ Text generations / dialogues
- 7. Why NLP? ▪ Google,Yahoo!, Bing (3,37%), Baidu (0,79%) -> Information Retrieval ▪ LinkedIn -> Information Extraction + Information Retrieval ▪ Google Translate, Babelfish, Systran -> Machine Translation ▪ Ask, IBM Watson -> Question Answering ▪ Myspace, Facebook, Twitter -> Social Networks, Processing of User- Generated Content ▪ All “Big Guys” have (several) strong NLP research labs: IBM, Microsoft, AT&T, Xerox, ORACLE-Sun Microsystems, etc. ▪ Academia: research in a university environment
- 8.
- 9. NLP Applications: MachineTranslation
- 10. NLP Applications: PersonalAssistant Apple Siri (2011) Microsoft Cortana (2014)Google Now (2013)
- 11. Linguistics Levels of Analysis ▪Speech ▪ Written language ▫ Phonology: sounds / letters / pronunciation ▫ Morphology: the structure of words ▫ Syntax: how these sequences are structured ▫ Semantics: meaning of the strings ▪ Interaction between levels
- 12. Issues in Syntax Identifythe part of speech (POS) dog = noun ate = verb homework = noun “the dog ate my homework”
- 13. Issues in Syntax Chunking:identify basic structures (phrases) [the dog]-NP [ate]-VP [my homework]-NP Shallow parsing the dog->subject ate->predicate my homework->object “the dog ate my homework”
- 14. Issues in Syntax Fullparsing S NP VP VBP NPDT NN PRP$ NN The dog ate my homework
- 15. Issues in Semantics “Theyare producing about 1,000 automobiles in the new plant” “The sea flora consists in 1,000 different plant species” Understand language! How? • “plant” = industrial plant • “plant” = living organism Words are ambiguous Importance of semantics? • Machine Translation: wrong translations • Information Retrieval: wrong information
- 16. Computational Linguistics Doing linguisticson computers More on the linguistic side than NLP, but closely related (interdisciplinary field)
- 17. Diachronic Linguistics The scientificstudy of language change over time also called Historical Linguistics
- 18. Marty, in 2015 peoplewill surf on the web!!!
- 19.
- 20. Diachronic Linguistics Why? ▪ Observechanges in particular languages ▪ Reconstruct the pre-history of languages ▪ Develop general theories about how and why language changes ▪ Describe the history of speech communities ▪ Etymology
- 21. Synchronic It describes thelanguage rules at a specific point of time without taking its history into account. Synchronic vs. Diachronic Diachronic It considers the evoluation of a language over time.
- 22. Google N-gram Viewer ▪Searchand visualize n-gram statistics from Google Books ▪N-gram: sequence of n words ▪Google Books digitalizes millions of books
- 23. N-gram “Google Books digitalizesmillions of books” 1-gram Google, Books, digitalizes, millions, of, books 2-gram Google Books, Books digitalizes, digitalizes millions, millions of, of books 3-gram Google Books digitalizes, Books digitalizes millions, digitalizes millions of, millions of books
- 24.
- 25. CULTUROMICSA form ofcomputational lexicology that studies human behavior and cultural trends through the quantitative analysis of digitized texts. J.-B. Michel, Y. K. Shen, A. P. Aiden, A. Veres, M. K. Gray, T. G. B. Team, J. P. Pickett, D. Hoiberg, D. Clancy, P. Norvig, J. Orwant, S. Pinker, M. A. Nowak e E. L. Aiden, Quantitative Analysis of Culture Using Millions of Digitized Books
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44. Google N-gram Issues Overabundanceof Scientific Literature
- 45. Google N-gram Issues OCRErrors long s in old books looks a lot like a f
- 46. Google N-gram Issues PopularityContests a book only appears once, whether it’s been read once or millions of times
- 47. “ Moving awayfrom mere frequentist approaches…
- 48.
- 49. What’s Tezguno? A bottleof Tezguno is on the table. Everyone likes Tezguno. Tezguno makes you drunk. We make Tezguno out of corn.
- 50.
- 51.
- 52. John Rupert Firth Youshall know a word by the company it keeps!
- 53. Ludwig Wittgenstein Meaning ofa word is determined by its usage.
- 54. Zellig Harris Methodsin structural linguistics Distributional structure Mathematical structures of language
- 55. Distributional Semantic Models Analysis ofword-usage statistics over huge corpora Geometric space of concepts Similar words are represented close in the space
- 56. Distributional Semantics Count co-occurrences dogcat bread pasta meat mouse dog 40 7 1 0 1 5 cat 7 32 0 1 0 8 bread 1 0 22 15 8 0 pasta 0 1 15 24 10 1 meat 1 0 8 10 30 2 mouse 5 8 0 1 2 31
- 57. Distributional Semantics Word similarity dogcat bread pasta meat mouse dog 40 7 1 0 1 5 cat 7 32 0 1 0 8 bread 1 0 22 15 8 0 pasta 0 1 15 24 10 1 meat 1 0 8 10 30 2 mouse 5 8 0 1 2 31
- 58. Distributional Semantics Word Similarity dogcat bread pasta meat mouse dog 40 7 1 0 1 5 cat 7 32 0 1 0 8 bread 1 0 22 15 8 0 pasta 0 1 15 24 10 1 meat 1 0 8 10 30 2 mouse 5 8 0 1 2 31
- 59. Distributional Semantics Word Similarity dogcat bread pasta meat mouse dog 40 7 1 0 1 5 cat 7 32 0 1 0 8 bread 1 0 22 15 8 0 pasta 0 1 15 24 10 1 meat 1 0 8 10 30 2 mouse 5 8 0 1 2 31
- 60.
- 61. Geometric space WordSpace dog pasta bread mouse cat catand mouse are close in the space
- 62. “ A WordSpaceis a snapshot of a specific corpus it does not take into account temporal information
- 63.
- 64. Temporal Random Indexing (TRI) ▪Corpus with temporal information: split the corpus in several time periods ▪ Build a WordSpace for each time period ▪ Words in different WordSpaces are comparable! P. Basile, A. Caputo, G. Semeraro. Temporal random indexing: A system for analysing word meaning over time. IJCoL vol. 1: Emerging Topics at the First Italian Conference on Computational Linguistics, Accademia University Press.
- 65. Similarity between wordscan change over time WordSpace 1910 WordSpace 1920 WordSpace 1930 chiamare chiamare telefonare chiamare telefonare
- 66.
- 67. Change point detection ▪Track theword meaning change over time ▪Build a time series by taking into account the semantic shift of each word ▪Find significant change: Mean shift model telefonare -> 0,25 0,3 0,7 0,8 0,75 1900 1910 1920 1930 1940 change point
- 68. Evaluation ▪Build TRI byrelying on the Italian Google Ngram corpus ▪Build a standard benchmarking for meaning shift detection for the Italian language (“Dizionario Sabatino Coletti”, “Dizionario Etimologico Zanichelli”) ▪Evaluate the performance of TRI: compare the system output with manual annotations provided by experts ▪Future work: extend the evaluation to the English language P. Basile, A. Caputo, G. Semeraro. Diachronic Analysis of the Italian Language exploiting Google Ngram. CLIC-it 2016: Third Italian Conference on Computational Linguistics.
- 69. Thanks! ! Any questions? You canfind me at pierpaolo.basile@gmail.com