Download as PDF, PPTX
Uploaded byVanessa Camilleri
ICS1020 NLP 2020
The document provides an introduction to natural language processing (NLP) and discusses key NLP tasks such as text classification, information extraction, and machine translation. It explains the challenges of NLP including ambiguity and diversity across languages. Different approaches to NLP are covered, including heuristics-based, machine learning, and deep learning methods.
More Related Content
Similar to ICS1020 NLP 2020
ICS1020 NLP 2020
- 1. Dr V. Camilleri- vanessa.camilleri@um.edu.mt December 2020 Foundations of Ai Introduction to Natural Language Processing
- 3. Natural language processing (NLP)is the automatic extraction, analysis, and generation of human language. a branch of artificial intelligence
- 5.
- 6. • Language Modelling •Text Classification • Information Extraction • Information Retrieval • Conversational Agent • Text Summarisation • Question Answering • Machine Translation • Topic Modelling Tasks and their Difficulty Natural Language Processing Easy Medium Hard Spell Check Keyword Information Retrieval Topic Modelling Text Classification Information Extraction Closed Domain Conversational Agent Text Summarisation Question Answering Machine Translation Open Domain Conversational Agent
- 7.
- 8. • Phonemes • Morphemesand Lexemes • Syntax • Context An example of a parsing tree
- 9. • Lexical Analysis • Syntactic Analysis (Parsing) • Semantic Analysis • Discourse Integration • Pragmatic Analysis 5 general steps Steps in NLP
- 10. • Ambiguity • CommonKnowledge • Creativity • Diversity across Languages Why is NLP Challenging ?
- 11. Time for somequestions …
- 12. • What isthe goal of Machine Learning? • List the three paradigms of ML • What is the difference between them? • Give me an example of a supervised learning problem related to language • Explain the main concept behind deep learning
- 13. • Heuristics-Based •Machine Learning • Deep Learning Three approaches to NLP
- 14. • Rule-based systemsfor words (Thesaurus/Dictionaries/Word relationships) • Regular expressions (REGEX) • Context-free grammar (CFG) Heuristics-based
- 15. 3 steps: •Extracting feature from the text • Using the feature representation to learn a model • Evaluating and improving the model Supervised Machine Learning Model • Naive Bayes • Support Vector Machine • Hidden Markov Model • Conditional Random Fields Machine Learning
- 16. • Recurrent NeuralNetworks • Long Short-Term Memory • Convolutional Neural Networks • Transformers Deep Learning Example of CNN for Sentiment Analysis
- 17. • For textclassification LSTM and CNN-based models have surpassed the performance of standard ML techniques like Naive Bayes & SVM; • For sequence labelling tasks LSTMs have performed better than CRF models; • For most NLP tasks, ranging from text classification to sequence labelling, transformer models have become state of the art. Deep Learning - a recap
- 18. • Overfitting onsmall datasets • Few shot learning (learning from few training examples) and synthetic data generation • Domain Adaptation • Interpretable models • Common sense and world knowledge • Cost • On-device deployment Deep Learning Still many challenges in NLP
- 19. Activity time Work ingroups (use chat/social media/…) So now let us think back at our Self-driving ambulance scenario; describe one possible scenario in detail and discuss the major NLP components that might be involved. Describe these components in more detail.