bbbnmm .gv ggj yghjfrft tddrdv tfyhj yyfuykg ygoygj gfiuyg. uyfyj fyutegjh uyryutk 6tuoyfiyteiyf 868t88yiyrffhg 766r766r 6rtfkgd.

1. UNIT 5
NATURAL LANGUAGE PROCESSING

2. INTRODUCTION TO NATURAL LANGUAGE
PROCESSING
• NLP -Natural Language Processing is the branch of artificial intelligence that deals with
training a computer, processing, and generating language.
• It includes search engines, voice assistants, and machine translation services powered by
the technology. NLP mainly works through machine learning, where the system stores
words and finds ways to come together like other data forms.
• The phrases, sentences, and other books are fed into ML engines, where they are
processed into grammatical rules by adding real-life linguistic habits.

3. WHY IS NLP IMPORTANT?
• Uses Large Volume of textual data
• The natural language process helps the computer communicate with humans in its
language, which scales other language-related tasks. These times, machines can analyze
more language-based data without a fridge and in a consistent, unbiased way.
• Structured high unstructured data source
• Human language is complex and diverse. They can express themselves in infinite ways
verbally and in writing. It is not like hundreds of languages within each language that are
unique in terms of rules and grammar. When we speak or have regional accents, we stutter
and borrow terms from other languages. NLP is essential as it helps resolve ambiguity in
language and adds numeric structure to the data by adding downstream apps.

4. BENEFITS OF NATURAL LANGUAGE
PROCESSING(NLP)
• There are many benefits of Natural Language Processing which improves the way humans
behave and computers communicate. It enables computers to understand the human
language, interact with them, and become more intuitive. Few advantages include:
Ability to automatically make a readable summary of a larger or more complex text.
Add improved accuracy and efficiency of documentation.
Perform with ease and perform sentiment analysis.
Perform advanced insights from analytics that can be unreachable due to data volume.

5. NATURAL LANGUAGE PROCESSING
TECHNIQUES
• There are mainly two main techniques like Syntax and Semantic, that are used with natural language processing. The syntax is used
as the arrangement of words to make grammatical sense.
• The syntax technique adds:
• Parsing: It comes with grammatical sentence analysis and is helpful for complex downstream processing tasks.
• Word Segmentation: It acts by talking a string of text and deriving word forms from it.
• Sentence Breaking: It comes with boundaries in significant texts where the algorithm recognizes the period that splits the sentence.
• Morphological Segmentation: This technique words into smaller parts called morphemes. It is mainly used in machine translation
and speech recognition.
• Stemming is used to divide words with infection into terms to root them. The essential algorithm uses the same word where the letters
are different.

6. SEMANTICS TECHNIQUE
• The Semantics technique adds:
• Word Sense disambiguation: The technique derives the word-based meaning on context
and uses a word pen that refers to a fenced area.
• Named Entity Recognition: It determines the word that can be categorized into groups.
The algorithm used here can recognize the two instances which separate the entity

7. ROLE OF THE MACHINE LEARNING IN NLP

8. ROLE OF THE MACHINE LEARNING IN NLP
• Machine learning act as an essential value that comes with processes that's easy to understand. Let's know the role of machine
learning for Natural language processing.
• 1) Morphological Analysis- It comes with a computing system in the form of 0s and 1s. Later, it can be converted into alphabets that
use ASCII code. It is said that the machine receives a bunch of characters that a sentence or a paragraph offers. Machine learning and
deep learning algorithms that employ tokenization. It supports vector machines and recurrent neural networks.
• 2) Syntactic Analysis- It is one of the other natural language processing tasks that use grammar rules. The words were firstly tagged
using machine learning and deep learning. It uses a machine-learning algorithm such as k-nearest neighbour used for implementation
syntactic.
• 3) Sentiment Analysis-In this stage, the word meaning is mainly identified using word-meaning dictionaries. The problem primarily
identifies with different meanings in the context of sentences. It plays an integral part in customer relationship management.
Additionally, a single negative opinion comes with disastrous consequences for the product.

9. ROLE OF THE MACHINE LEARNING IN NLP
• 4) Chatbots System
• The system comes with conventional agents that engage the user in a conversation. The conversation uses text and voice.
Furthermore, it uses personal assistants like Amazon Alexa and Google assistant. The current framework covers IBM, Google
dialogue, and Amazon Alexa that provide an easy way to develop a chatbot system.
• 5) Information Retrieval Systems
• It's one crucial role that offers apps to retrieve information. It uses an intelligent system process that requires queries with extensive
data to retrieve data. It comes with the most basic way that helps in retrieving and determining the data. The smart system process
required deep learning techniques.

10. ROLE OF THE MACHINE LEARNING IN NLP
• 6) Question Answering Systems
• In recent times, an answering question system tries to answer the user's question, whereas the
thin line separates the dialogue system. Machine learning and deep learning play a vital role in
all components. The task used covers the classification problem that comes with experienced
and better resolving techniques.
• 7) Machine Translation
• A machine translation technique translates a text from one language to another by adding
minimal human intervention. The application is used to solve the word following the subject-
verb-object format. Apart from this, it supports different rules followed by machine translation.

11. ROLE OF AI (ARTIFICIAL INTELLIGENCE) IN NLP
• AI is used to intricate the logic using advanced analytical methods to perform the simple task
at a grander scale. AI is used to perform simple tasks at a greater scale, allowing them to focus
on what humans are best with, handling complex exceptions.
• AI is mainly used as a computerized simulation of human intelligence. Using AI, one can
accomplish the task by using the right strategy.
• NLP has roots in linguistics, enabling computers and processing both machine learning and
deep learning. It effectively injects the process of adding unstructured speech and data text.
• AI helps generate learning models by improving its performance in turning tests.

12. DIFFERENT ISSUE INVOLVED IN NLP
• NLP is a powerful tool with huge benefits, but there are still a number of Natural Language Processing limitations and problems:
• Contextual words and phrases and homonyms
• Synonyms
• Irony and sarcasm
• Ambiguity
• Errors in text or speech
• Colloquialisms and slang
• Domain-specific language
• Low-resource languages
• Lack of research and development

13. EXPERT SYSTEM
• An expert system, as the name suggests, is a complex computer application that can solve
a majority of problems in any set domain. It was the brainchild of the researchers at
Stanford University.
• It gets its name because it learns from humans who are experts in their domains. This
gives the machines the highest level of cognition at this point.
• An expert system is a system that can solve any complex problem by taking inputs such as
facts and heuristics and producing a meaningful solution to the problem.

14. CHARACTERISTICS OF EXPERT SYSTEMS
• High performance
• Understandable
• Reliable
• Highly responsive

15. CAPABILITIES OF EXPERT SYSTEMS
• CAPABLE
• Advising
• Instructing and assisting human in decision making
• Demonstrating
• Deriving a solution
• Diagnosing
• Explaining
• Interpreting input
• Predicting results
• Justifying the conclusion
• INCAPABLE
• Substituting human decision makers
• Possessing human capabilities
• Producing accurate output for inadequate knowledge base
• Refining their own knowledge

16. ADVANTAGES OF EXPERT SYSTEMS
• Storage: They have the ability to store large amounts of data easily and add levels of access.
• Training: Expert systems can be mapped to help train new employees or incoming employees, thereby, saving a lot of money
involved in training and preparing employees for their roles.
• Decision-making: The ability to implement computing power to derive expert skills and use it to drive decisions is very much
possible. This is leaps and bounds ahead of the conventional ‘educated-guess’ approach.
• Efficiency: In the process of problem-solving, most of the time, the requirement is time-sensitive. Cutting the process short by
making everything more efficient, thereby, solving the problem quickly is a possibility with the use of expert systems.
• Error handling: Expert systems not only reduce human errors but also make sure that errors can be handled and solved in a
structured way, which is an important advantage in most situations.
• Wrangling: With the usage of expert systems, one can look at data or any process and understand it in a way that would’ve been
incomprehensible to the naked eye.

17. COMPONENTS OF EXPERT SYSTEMS
• Knowledge Hub- This is also called the knowledge base, which is basically a huge container of data. The processed part of the
data and the output of the learning equate to the data creation called ‘knowledge.’ Be it man or machine, only using the
knowledge you can validate the accuracy of the solution to a problem.
• Memory Units-It is the storage for the raw data, which is used as an input for models to train and function. The important
aspect here is the variety of methodologies and techniques used to store the data for immediate access when required.
• Inference Engines-Inference engines are the heart of Expert Systems. These processes power through the knowledge hub and
analyze the problem at hand to arrive at a solution.
• If it is rule-based, then the engine ensures to iterate through facts, apply the knowledge obtained, and resolve conflicts where
required (in case of multiple rules).
• Explanation Systems-These systems are put into place to supply the information that helps in clarifying the problem domain
and the structure. This has multiple use cases not only in the field of expert systems but otherwise as well.

18. BUILDING AN EXPERT SYSTEM
• First, you need to begin by determining the characteristics of the Expert Systems and the
requirements of the problem at hand. Doing this gives you a thorough understanding of the problem.
• The second step is to use the expertise of the domain experts and the knowledge engineers to create
and define a structured problem statement.
• Next, the knowledge engineers help convert the knowledge packages into high-level, easy-to-
understand verbiage. The inference engine and a structured approach to the solution are designed at
this point.
• Lastly, the knowledge experts will ensure how these raw knowledge sources can be integrated with
the reasoning system, and they assess the segregation of the explanations that would help solve the
problem.

19. LIMITATIONS OF EXPERT SYSTEMS
• Creativity: No matter how good an Expert System is designed to be, there are many situations where it
lacks finesse and brings out a response that can be deemed as plain and commonplace.
• Result validation: If there is a situation where the input data to the knowledge hub is not accurate, the
validation engine might not be able to figure this out and, therefore, might furnish inaccurate results.
• Running cost: As we might expect, designing, maintaining, and running these expert systems require a
lot of finances. Once set up, it will serve for longer durations efficiently than human experts driving in
more return on investment overall.
• Conflict of solutions: When you consider the human approach to solving a problem, each person can
come up with a unique solution. This is not the case when working with Expert Systems, which happen to
be one of the biggest limitations today.

20. APPLICATIONS OF EXPERT SYSTEMS
• Information technology
• Hospitals and healthcare
• Loan provision analytics
• Virology
• Planning and scheduling tasks
• Warehouse optimization
• Stock trading
• Process control and automation
• Airline scheduling

21. ROBOTICS
• Artificial intelligence (AI) is becoming more popular in robotic solutions, bringing
flexibility and learning capabilities to formerly rigid applications.
• AI helps to :-
a. made modern robotics conceivable
b. more versatile and adaptable
c. perform increasingly complex tasks
Robotics, on the other hand, introduces artificial intelligence into the actual world and
allows it to interact with materials in real-time.

22. ROBOTICS
• Robotics is a branch of AI, which is composed of Electrical Engineering,
Mechanical Engineering, and Computer Science for designing, construction, and
application of robots.
• Aim-Robots are aimed at manipulating the objects by perceiving, picking,
moving, modifying the physical properties of object, destroying it, or to have an
effect thereby freeing manpower from doing repetitive functions without getting
bored, distracted, or exhausted.