Introduction to NLP Lecture 05 B.Tech IT

Introduction to NLP: Definition and importance of NLP in AI. Applications in industries: chatbots, search engines, machine translation, recommendation systems, healthcare, finance, and social media analytics. NLP vs. traditional programming paradigm. NLP pipeline: text acquisition, preprocessing, feature extraction, model building, and evaluation. Levels of NLP: Morphology, syntax, semantics, pragmatics, and discourse analysis. Machine Translation in NLP: Need of MT, process and types of MT, Ambiguity in NLP: Lexical ambiguity, syntactic ambiguity, semantic ambiguity. Text Preprocessing: Sentence segmentation, tokenization, stop-word removal, punctuation handling, stemming vs. lemmatization, normalization: lowercasing, Unicode handling, contraction expansion. Part-of-Speech Tagging: Concept of grammatical tagging and its applications. Challenges in NLP: Multilingual processing, sarcasm, code-mixed languages, noisy text data, context, and morphological complexity.