NLP 2020: What Works and What's Next
•Download as PPT, PDF•
1 like•287 views
Presentation to the Chicago chapter of the American Statistical Association, November 20, 2020, by Seth Grimes.
Recommended
More Related Content
Similar to NLP 2020: What Works and What's Next
Similar to NLP 2020: What Works and What's Next (20)
More from Seth Grimes
More from Seth Grimes (20)
Recently uploaded
Recently uploaded (20)
NLP 2020: What Works and What's Next
- 1. NLP 2020: What Works and What's Next Recent Advances in Natural Language Processing Seth Grimes Alta Plana Corporation @SethGrimes November 20, 2020
- 4. Disclaimer I use A LOT of commercial product materials in the slides that follow. These are illustrations and not recommendations.
- 5. Natural Language Processing Natural Language Understanding (NLU) • OCR, language detection, tokenization, parsing • Information extraction: parts of speech, chunks , entities, aspects, topics/themes, relations, attributes, events, … • Speech processing Natural Language Generation (NLG) NLU + NLG together, for example: • Summarization • Machine translation • Conversational interfaces • Question answering
- 8. Empirical Methods in Natural Language Processing (EMNLP2020)
- 9. Early Days (1958) Transcribing Encoding Abstracting Who needs to know? Who knows what? What is known? Hans Peter Luhn “A Business Intelligence System” IBM Journal, October 1958
- 10. “Statistical information derived from word frequency and distribution is used by the machine to compute a relative measure of significance, first for individual words and then for sentences. Sentences scoring highest in significance are extracted and printed out to become the auto-abstract.” -- H.P. Luhn, The Automatic Creation of Literature Abstracts, IBM Journal, 1958.
- 11. “All models are wrong, but some are useful.” -- George Box
- 13. Skipping A Lot of Stuff… Rules Taxonomies Booleans …
- 14. Word2Vec (2013) https://code.google.com/p/word2vec/ https://developers.google.com/machine-learning/crash-course/embeddings/translating-to-a-lower-dimensional-space “You shall know a word by the company it keeps.” – J.R. Firth, 1957
- 15. Word2Vec: Key Concepts Continuous bag-of- words (CBOW) predicts a word from a window of surrounding words. Skip-gram uses a word to predict a window of surrounding words.
- 17. Sense2Vec (2015) https://arxiv.org/abs/1511.06388 “Sense2vec (Trask et. al, 2015) is a new twist on word2vec that lets you learn more interesting, detailed and context- sensitive word vectors.”
- 27. Back To The Garden
- 28. NLP Libraries
- 33. Hugging Face Pipeline Example
- 34. Hugging Face Pipeline Examples
- 35. Cloud Services
- 36. Amazon Comprehend Medical https://aws.amazon.com/comprehend/medical/ “Amazon Comprehend Medical is a natural language processing service that makes it easy to use machine learning to extract relevant medical information from unstructured text. Using Amazon Comprehend Medical, you can quickly and accurately gather information, such as medical condition, medication, dosage, strength, and frequency from a variety of sources like doctors’ notes, clinical trial reports, and patient health records. Amazon Comprehend Medical can also link the detected information to medical ontologies such as ICD-10-CM or RxNorm so it can be used easily by downstream healthcare applications.”
- 38. Training
- 39. Services and Solutions: Examples
- 43. Emotion AI
- 44. The Expressions Charles Le Brun
- 46. “If we want computers to be genuinely intelligent, to adapt to us and to interact naturally with us, then they will need to ability to recognize and express emotions, and to have what has come to be called ‘emotional intelligence.’” -- Rosalind Picard, Affective Computing, 1997 Affective Computing
- 47. 1997
- 48. « Le cœur a ses raisons que la raison ne connaît point. » “The heart has its reasons that reason knows nothing of.” -- Blaise Pascal
- 50. What about Sentiment Analysis? Text sourced… https://dl.acm.org/doi/10.1145/945645.945658, October 2003 https://dl.acm.org/doi/10.1561/1500000011, January 2008
- 51. Bing Liu, 2010
- 52. Bing Liu, 2015, 2020
- 53. Emotion AI “Emotion mining is the science of detecting, analyzing, and evaluating humans' feelings towards different events, issues, services, or any other interest.” Emotion synthesis enhances the ability of a machine to provide meaningful, contextual responses, by conveying an appropriate emotional state through words, voice, and expression. Emotion induction aims to evoke a certain emotional response or affective state.
- 56. “Parsing Text for Emotion Terms: Analysis & Visualization Using R” Emotion in Text: Parsing, Stats https://datascienceplus.com/parsing-text-for-emotion-terms-analysis-visualization-using-r/
- 57. Bing Liu, 2020
- 58. Emotion in Text
- 59. Words aren’t everything, and they’re not enough: • Idiom • Medium/media • Personality • Communicative act • Social act (sharing etc.) • Linguisitics…
- 60. Conversation: Text and Voice
- 64. Speech https://www.phon.ucl.ac.uk/courses/spsci/iss/week9.php “Voice cues are commonly divided into those related to: (a) fundamental frequency (F0, a correlate of the perceived pitch), (b) vocal perturbation (short- term variability in sound production), (c) voice quality (a correlate of the perceived ‘timbre’), (d) intensity (a correlate of the perceived loudness), and (e) temporal aspects of speech (e.g., speech rate), as well as various combinations of these aspects (e.g., prosodic features).” http://www.scholarpedia.org/article/Speech_emotion_analysis
- 68. “Conversational Intelligence & Behavioral Prediction Insights from Voice: Our Oliver engine offers ASR+ with a sophisticated layer of emotion recognition metrics & behavioral KPIs, not only from what is being said but also from the how it is said.”
- 69. NLP 2020: What Works and What's Next Recent Advances in Natural Language Processing Seth Grimes Alta Plana Corporation @SethGrimes November 20, 2020