Essay Writing Service http://StudyHub.vip/Artificial-Intelligence-Simplified-(Tex 👈

1. Artificial Intelligence
Simplified

2. iii
Artificial Intelligence
Simplified
Understanding Basic Concepts
Second Edition
Authors:
Binto George
Gail Carmichael
Editor:
Susan S. Mathai

3. iv
Publisher’s Cataloging-In-Publication Data
(Prepared by The Donohue Group, Inc.)
Names: George, Binto, 1970- author. | Carmichael, Gail, author. | Mathai,
Susan S., editor.
Title: Artificial intelligence simplified : understanding basic concepts /
authors: Binto George, Gail Carmichael ; editor: Susan S. Mathai.
Description: Second edition. | [Bettendorf, Iowa] : [CSTrends LLP],
[2021] | Includes bibliographical references and index.
Identifiers: ISBN 9781944708030 (paperback) | ISBN 1944708030
(paperback) | ISBN 9781944708054 (hardback) | ISBN
9781944708047 (ebook)
Subjects: LCSH: Artificial intelligence.
Classification: LCC Q335 .G46 2021 (print) | LCC Q335 (ebook) | DDC
006.3--dc23
Library of Congress Control Number: 2020950789
Copyright ©2021 CSTrends LLP.
All rights reserved.
Publisher’s Website:
https://www.cstrends.com
Contact Email: ai@cstrends.com
ISBN: 1-944708-05-7
ISBN-13: 978-1-944708-05-4

4. 5
DISCLAIMER
The authors and the publisher make no representations or
warranties on accuracy or completeness with respect to the
information provided in this book. Although the authors and
publisher have made efforts to ensure that the information in
this book was correct at the time of original publication, the
authors and publisher do not assume and hereby disclaim any
liability to any party for any loss, damage, or disruption caused
by errors or omissions, whether such errors or omissions result
from negligence, accident, or any other cause.
No implied warranties of fitness or merchantability are
made for specific purposes. No one is authorized to offer any
expressed or implied warranties on our behalf. Always seek
appropriate professional advice before using any information
provided in this book. NEITHER THE AUTHORS NOR THE
PUBLISHER WILL BE LIABLE FOR ANY LOSSES OR
DAMAGES, INCLUDING, WITHOUT LIMITATION,
INDIRECT OR CONSEQUENTIAL LOSS OR DAMAGE.
Product and company names mentioned in this book
may be trademarks or registered trademarks of their respective
holders. The use or appearance of these trademarks does not
imply any affiliation with or endorsement by their holders.

5. 7
ABOUT THE SECOND EDITION
Artificial Intelligence (AI) impacts our lives in more
ways than we can imagine. This book is for anyone wanting
to learn AI concepts without in-depth prior knowledge in the
field. You can use this book if you have basic computer
science knowledge, particularly data structures and
algorithms.
The book introduces key AI concepts in an easy-to-
read format with examples and illustrations. Our goal is to
keep our explanations as simple as possible. While we have
included some algorithms, flowcharts, and technical details,
you should be able to get an idea of basic AI concepts even if
you don’t go in-depth into those contents. If you are a
beginner, you may want to skip the depth content in your first
reading.
If you are a professional and wish to get an overview
of AI, this book will provide you with some essential
background to begin. If you are a robotics enthusiast wanting
to understand the broader aspects of AI, you may find this
book useful. If you are a student taking an AI course, you can
use this as an introductory textbook to develop a solid
understanding of the basic AI concepts. This book includes
numerous bibliographical references and resources that you
may find useful for delving into more profound aspects of AI.

6. 8
Our first edition was very well received all over the
world. We received fascinating reviews and encouraging
feedback from readers, professionals, and well-wishers. Based
on inputs received, we prepared this second edition, including
more relevant topics such as Generative Adversarial Network
(GAN), Recurrent Neural Network (RNN), Support Vector
Machine (SVM), and artificial vision. A few readers wanted
more depth content. So we added more advanced material
without severely compromising the simplicity. A beginner can
skim over the depth content for your first reading.
We appreciate you choosing this book. We are
delighted to be part of your incredible AI journey.

7. 9
TABLE OF CONTENTS
1. Introduction ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐ 13
1.1. Organization of This Book ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐16
2. Search Methods for Problem‐Solving ‐‐‐‐‐‐‐‐‐‐‐‐‐ 21
2.1. Modeling Operating Room Scheduling Problem 22
2.2. Generate and Test‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐28
2.3. Making Search More Efficient‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐31
2.4. Blind Search Methods ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐40
2.5. Heuristic Search Methods ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐52
2.5.1 Hill Climbing ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐54
2.5.2 Best First Search‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐68
2.6. Best Path Methods ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐71
3. Handling Competing Goals with Game Trees ‐‐‐ 79
3.1. Minimax algorithm‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐83
3.2. Horizon effect‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐85
3.3. Alpha‐Beta Pruning‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐85
3.4. Progressive Deepening‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐88
4. Planning Techniques ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐ 91
4.1. Forward Planning ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐93
4.2. Backward Planning ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐96
4.3. Partial‐Order Planning‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐97
4.4. Planning Under Uncertainty ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐99

8. 10
5. Evolutionary Computing‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐101
5.1. Crossover‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐104
5.2. Mutation‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐105
5.3. Fitness function‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐108
5.4. Genetic Programming‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐108
6. Expert Systems ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐111
6.1. Knowledge Representation ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐112
6.2. Expert Systems‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐114
6.3. Expert System Types ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐115
6.3.1 Forward chaining‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐115
6.3.2 Backward chaining ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐116
6.3.3 Hybrid chaining ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐118
6.3.4 Deduction and reaction systems ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐118
6.3.5 Explanation Facility ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐119
6.4. Inference under uncertainty ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐119
6.5. Fuzzy Expert Systems ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐126
6.5.1 Fuzzification ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐131
7. Learning from Experience ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐137
7.1. Gradient Descent Training Algorithm‐‐‐‐‐‐‐‐‐‐‐‐‐147
7.2. Regression and Classification ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐150
7.3. Multi‐layer neural networks ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐155
7.3.1 Backpropagation ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐156
7.4. Convolutional Neural Network (CNN) ‐‐‐‐‐‐‐‐‐‐‐‐159
7.5. Recurrent Neural Network (RNN) ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐163
7.6. Generative Adversarial Network (GAN)‐‐‐‐‐‐‐‐‐‐164
7.7. Support Vector Machine (SVM) ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐166

9. 11
7.8. Bayesian Neural Networks (BNN) ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐168
7.9. Applications of Machine Learning‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐169
7.9.1 ANN Deployment‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐170
8. Human Interaction and Robotics ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐173
8.1. Natural Language Understanding ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐178
8.2. Speech Synthesis‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐181
8.3. Artificial Vision ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐182
9. Evaluating Intelligence‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐185
9.1. Identifying Intelligence‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐188
9.2. Partial Intelligence‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐190
9.3. Bringing up Intelligence‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐193
10. Conclusions and Where to Go From Here‐‐‐‐‐‐197
10.1. How is AI deployed today? ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐198
10.2. AI and Other Disciplines‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐200
10.3. The Future of Artificial Intelligence ‐‐‐‐‐‐‐‐‐‐‐‐‐201
11. Bibliography‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐211
INDEX‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐233

10. 12
Chapter 1

11. 13
1. INTRODUCTION
Objectives
 Introduce AI
 Explore applications of AI
 Discuss the organization of the book

12. 1. Introduction
14
Modern applications of Artificial Intelligence (AI)
range from searching for asteroids to self-driving cars. AI
touches every walk of our lives that we can imagine, in one
way or the other. If you have ever checked the weather, talked
to a digital personal assistant, snapped a picture with your
smartphone, or even just searched for information online, you
have used AI. Financial institutions regularly deploy AI for
fraud monitoring, investment decisions support, credit risk
assessment, data mining of customer behaviors, and economic
forecasting. The military uses AI for target discrimination,
setting up intelligent missile defense shields, and robot
steering. In healthcare, AI helps with diagnosis, medical
treatment, and robot-assisted surgery. Law enforcement uses
robots for rescue, bomb disposal, and hostage situations. The
fire force has successfully used robots to extinguish major
fires. Not all robots require AI, but some increasingly use AI
to sense their surroundings, intelligently make decisions, and
act appropriately.
Many businesses with access to a massive volume of
data are aggressively developing AI applications to help them
make smart decisions and offer improved products and
services. AI can work hand in hand with big data to solve
challenging problems by discovering exciting patterns from
massive data volume.
Our purpose here is to introduce important AI concepts
that can empower one to succeed in such an environment.
While the book provides an excellent overview of AI, it does
not quite delve into the intricacies of the complex realm of all
AI applications. We hope that the information provided in this

13. 1. Introduction
15
book will help you start your own AI journey armed with some
core ideas that we present in an easily relatable context.
Historically, AI's initial objective was to create
systems that perceive, think, and act like humans (Winston,
1992). Years of research have shown us that this is a more
challenging proposition than initially anticipated. While a few
researchers continue their work towards achieving this
objective, most others pursue developing systems that perform
as good as or better than humans in a focused area such as
chess-playing, stock trading, or disease diagnosis. AI helps
machines to be smarter and better, often by examining how
humans solve real-life problems. AI also helps us understand
ourselves better by allowing us to gain insight into human
intelligence's inner-workings.
AI is not a “cure-all.” Challenging problems don’t
always require human intelligence to solve them. In some
instances, AI may not be the best tool to use at all. Many
successful systems operate based on mathematical models
such as Bayesian networks rather than using human reasoning.
We will explore how and where we can use AI to solve
interesting problems.
Researchers have tried various methods to build AI. A
few believe that symbolic systems hold the key to AI.
Heuristic searches are another way to go, according to some.
Others think that expert systems can adequately capture
human intelligence. Some researchers focus on recreating the
human brain by emulating its structure. In this book, we
explain some of these methods in the context of practical
applications.

14. 1. Introduction
16
Some of our readers are looking for an overview of AI
to understand how it works. Others are interested in choosing
a career in AI. What are some of the valuable skills for an AI
career? First, we need to develop an understanding of basic AI
concepts. It is also a good idea to build some mathematical
background useful for AI. For an AI developer, some
programming skills will also be invaluable as well.
Knowledge of computer hardware, such as CPU, GPU, and
memory, will also be helpful to design, develop, and deploy
AI systems.
1.1. Organization of This Book
We tackle several exciting problem areas that AI
techniques can address. For example, AI can help job
scheduling in manufacturing and service industries by
automatically generating high-quality schedules while
considering various parameters and meeting potentially
conflicting objectives. AI can help find driving directions
from one location to another through a network of roads with
real-time updates based on dynamic events such as detours,
constructions, and traffic delays. If you are traveling to many
cities and want to return after visiting all cities exactly once,
which route is the best for you? (traveling salesperson
problem) We can solve questions like these with the search
methods described in Chapter 2.
In 1999, Deep Blue (Hsu, 2002), a computer program,
won a chess game with the then world chess master Garry
Kasparov. The chess program used a particular type of search
called game tree search. We can also implement other board

15. 1. Introduction
17
games using game trees. Game trees can also help devise a
winning strategy by weighing conflicting options such as non-
invasive or invasive treatments and developing a treatment
plan. We learn more about game trees in Chapter 3.
Planning helps us decide the sequence of actions to be
performed for reaching one of the goal states starting from the
current state. Planning assists us in dealing with surprise
events that occur during the otherwise ordinary course of
operations. We can apply planning techniques to such
domains as logistics, military campaigns, space exploration,
and operating procedure synthesis. Chapter 4 explains
planning.
There is an ingenious AI method used for gaming,
cryptographic code-breaking, freight routing, data packet
routing, market prediction, hardware design, signal filtering,
and signal processing. That method is called genetic
algorithms that have origins in evolutionary biology, in which
organisms evolve and adapt to thrive in environmental
conditions. Learn more about genetic algorithms and
evolutionary computing in Chapter 5.
Expert systems can assist or even replace human
experts with specialized knowledge. Expert systems for
disease diagnosis include MYCIN (Shortliffe, 1977) and
PUFF (Aikins, Kunz, Shortliffe, & Fallat, 1983). Expert
systems can help as diagnostic tools for automobiles,
airplanes, and machinery. LITHIAN, an expert system, guides
archeologists in exploring stone tools. DENDRAL (Lindsay,
Feigenbaum, Buchanan, & Lederberg, 1980) can identify
chemical compounds' structure. We can use expert systems in

16. 1. Introduction
18
auditing, taxation, and accounting. One type of expert system,
called a fuzzy expert system, is used to avoid automobile
collisions, auto-piloting airplanes, controlling Unmanned
Area Vehicle (UAV), detecting cancer, diagnosing heart
disease, evaluating insurance fraud, and predicting credit risk.
See how expert systems work in Chapter 6.
Learning from experience is a sign of intelligence.
Neural networks simulate the learning capacity of biological
neurons in the brain. We can use Neural networks for gesture
recognition, speech recognition, handwriting recognition,
fraud detection, cancer cell detection, petroleum exploration,
and much more. In particular, big data systems handle a vast
volume, and a variety of data moved at extremely high speeds,
where conventional data processing methods are not
sufficient. Neural networks can predict patterns or devise
processing strategies to sufficiently deal with big data. See
Chapter 7 for more information on neural networks. This
chapter also talks about Support Vector Machine (SVM),
Generative Adversarial Network (GAN), Recurrent Neural
Network (RNN), and Bayesian Neural Network (BNN).
Two highly regarded Jeopardy! (a TV quiz show)
players lost to IBM’s Watson, a computer, in a landmark
Jeopardy! game in 2011 (Brown, 2012). Microsoft researchers
developed an artificial humor system called CAHOOTS
(Wen, et al., 2015) that observes chat sessions and suggests
funny pictures that users can use to respond. Neither project
would be possible without computers understanding everyday
conversation. Digital assistants such as Siri, Alexa, Cortana
can also interact with us using natural language. Chapter 8

17. 2. Search Methods for Problem-Solving
28
2.2. Generate and Test
A simple approach to solving the scheduling problem
is to generate schedules and test them to see if the schedules
satisfy our rules (Russell & Norvig, 2009). As seen in Figure
2-5, the general idea is to develop a possible schedule by
randomly or systematically allocating patients to timeslots,
checking whether it satisfies the rules, and continuing to
generate new schedules until one of them passes the test.
Figure 2-5: Generate and test.

18. 2. Search Methods for Problem-Solving
33
Figure 2-9: A partial example of a surgery schedule tree.
We can represent relationships between schedules
using the data structure tree. See Figure 2-9 for the tree
representation of an OR schedule. For those who are not

19. 2. Search Methods for Problem-Solving
38
Figure 2-12: Types of graphs
Figure 2-12 summarizes various types of graphs.
Besides the graphs we discussed so far, the figure also shows
multigraph, in which there can be more than one edge
between nodes,

20. 2. Search Methods for Problem-Solving
40
out to be disconnected. Anyway, for simplicity, in this chapter,
we illustrate various find search methods on trees.
2.4. Blind Search Methods
Blind search (Cormen, Leiserson, Rivest, & Stein,
2009) involves exhaustively looking through all nodes in the
search space until we find the schedule that best satisfies all of
our rules. Blind searching is very inefficient (Aho, Hopcroft,
& Ullman, 1974) because we are not using any domain-
specific knowledge to be smart about how we search.
Figure 2-13: Blind searching is like trying to get to the peak
without using knowledge of the mountain’s geography; you would be
hoping to find the peak by chance.
Blind searching is only useful when the size of the tree
is relatively small. With a huge tree, figuratively, searching for

21. 2. Search Methods for Problem-Solving
41
a goal state is like climbing to the peak of Mount Everest with
a blindfold on (See Figure 2-13) – in-fact, climbing Mount
Everest may turn out to be easier!
For performing a blind search, we can either search the
tree randomly or systematically (Knuth D. E., 1973). For
example, with the Depth First Search (DFS) technique, we
check nodes by working our way down a tree until we reach
the bottom-most node on a branch. Once the search comes to
the leaf node, there is no way further down if it needs to
continue searching. So the search needs to backtrack a step
and work down other branches.
Figure 2-14: An example of searching for node N
in a tree using a depth-first search.
In the example search tree shown in Figure 2-14,
suppose we are searching for the node labeled N. The DFS
would begin its search at the root, A. It would continue to B

22. 2. Search Methods for Problem-Solving
42
and then to F, which is the bottom-most node in that branch.
Since F is still not the node we are looking for, the search tries
to continue, but F has no children. So the search finishes the
node F and backtracks to B, then continues down the next
branch to G. The search continues in this fashion, eventually
reaching the goal node N.
Now that we understood how DFS works let us see
how we can write an algorithm to perform the DFS from a
given start node to a specified goal node on a given graph. We
introduce some terminology first. We define a neighbor of a
node as the set of other nodes directly connected and reachable
from it. We consider a node ‘closed’ if the search algorithm
already finished examining it. An open node is a node that the
search still has to explore fully.
A
OUT
IN
B
C
Figure 2-15: Stack (LIFO)
The DFS algorithm uses the data structure stack
(Figure 2-15) for storing open nodes. Stack employs the Last
in First Out (LIFO) paradigm – the last element that goes into

23. 2. Search Methods for Problem-Solving
43
the stack comes out first. The algorithm uses two operations
on the stack: push and pop. When the algorithm pushes a node
to the stack, it goes to the top of the stack. Figure 2-16 shows
the stack before, and after pushing the element D. The pop
operation removes a node from the top of the stack.
A
OUT
IN
B
C
D
OUT
IN
A
B
C
Figure 2-16: Before and after the push operation
Here is the pseudocode for the DFS algorithm.
def dfs(start_node, goal_node)
stack.push(start_node);
while stack is not empty
u = stack.pop(); #pop the top element
if u is the goal_node
return success;
if u not visited
mark u as visited
for each w that is
the unvisited neighbor of u
stack.push(w);
return failure; # stack empty; goal not found
.

24. 2. Search Methods for Problem-Solving
44
Start
Push the start node
into stack
Is stack
empty?
Yes
Pop the first node from
the stack
No
Is thisthe
goal node?
Was the node
visited
before?
Mark the node as visited
Push all unvisited
neighborsto thestack
Return
Failure
Return
Success
No
No
Yes
Yes
Figure 2-17: DFS Flowchart
We show the algorithm visually using a flow chart in
Figure 2-17.

25. 2. Search Methods for Problem-Solving
45
The DFS starts by pushing the start_node onto the
stack. The while loop keeps popping one node from the top of
the stack to expand until the stack is empty.
The expansion process works as follows. If the popped
node (u) is the goal node, the search succeeds. Otherwise, the
algorithm marks a node as ‘visited’ on visiting it the first time.
If the search has not visited the node before, it expands the
node by pushing all of its unvisited neighbors (w) onto the
stack. The search continues until finding the goal node or the
stack becomes empty. In the latter case, the search fails
without finding the goal node since there are no more nodes to
be searched.
Note that instead of marking a node as visited as in the
above algorithm, the search can maintain a list of visited
nodes, which may be preferred in some scenarios. For
verifying whether a node has been visited before, the search
then has to check if the node is in the list of visited nodes.
How efficient is DFS? To determine an algorithm's
efficiency, we can use the notion of time complexity and
space complexity. Time complexity quantifies the
computational time taken by an algorithm as a function of the
size (or number) of inputs. For example, assume that the
algorithm takes five units of time to search a patient in a
database. That means, if the search is for ten patients, the
algorithm will take 50 units (i.e., 10 x 5 = 50) of time. Say, the
algorithm also takes a constant time of 4 units for
housekeeping regardless of inputs. We can express the
runtime for the algorithm as T(n) = 5n+4. As the value of n
grows, the constant 4 units become less and less significant.

26. 2. Search Methods for Problem-Solving
46
So the runtime can be expressed as T(n) = 5n omitting the
constant 4. If we are trying to compare the algorithm, it also
makes sense to ignore the multiplier five as the execution
speed can vary based on the hardware. The only thing that is
sure in the above case is that the execution time linearly grows
with the number of inputs. In other words, an order function
O(n) can represent how fast computational time grows with
respect to the number of inputs. The order function is also
called Big Θ (pronounced Big-O) notation or Landau’s
symbol. The Big-O notation is useful for comparing
algorithms. For example, a space complexity Θ(n) is
considerably better than Θ(n5
).
Just like time complexity, space complexity keeps
track of space usage. Say an algorithm uses 1000 units of
memory regardless of the size of inputs. Additionally, the
algorithm needs 32 units per patient name in the input list.
That means if there are ten names, the algorithm needs a total
of 1320 units of memory (i.e., 32 x 10 + 1000). Again, on
generalization, for n names, the algorithm needs 32n +1000
units of memory. This expression represents the space
complexity of the algorithm. Ignoring constants in 32n +1000
as we did before, we can say that the space complexity is Θ(n).
What are the time and space complexities of DFS?
Say, the height of the tree is h, and the maximum branching
factor is b. In that case, the worst-case time complexity will
be proportional to the number of nodes in the tree, which is
Θ(bh
). Well, strictly the number of nodes is Θ(1 + b+ b2
+ b3 +
… +
bh
). For the root node, there are b children. Each of those
b children will have b children and so on. For accuracy, we

27. 2. Search Methods for Problem-Solving
47
should also include the number of edges the search should
traverse through. None of these change the fact that the
algorithm has exponential time complexity, Θ (bh
).
What about the worst-case space complexity of DFS?
How much space does the algorithm take? The stack has to
store the path from the root to the current node at any time. So
the stack's maximum size is proportional to the tree’s height
times the maximum branching factor (bh). Since we ignore
constants as above, so the space complexity of DFS is Θ(h).
The table below summarizes the time and space complexity of
DFS.
DFS Complexity
Time Complexity Θ (bh
)
Space Complexity Θ(h)
As we saw, Big-O notation helps us compare
algorithms. Given that AI problems are challenging, using an
algorithm with exponential complexity (Θ (bh
)) will take too
much time to compute and give us any useful results. So we
need to develop algorithms that can provide us with answers
at a reasonable time, even if we may have to compromise on
the accuracy of the answers a little.

28. 2. Search Methods for Problem-Solving
66
Figure 2-31: The hiker finds it hard to follow the path that
goes up as many paths go sharply down.
As a result, the search slows down, as seen in Figure
2-31, where the hiker finds himself on a ridge trying to climb
through the sharp edge. Despite these limitations (Konolige,
1994), hill climbing does help find the goal much faster than
blind searches in most scenarios.

29. 2. Search Methods for Problem-Solving
72
In any case, the actual distance will be equal to or more than
the straight line distance.
Figure 2-34: Example of an A* search from the source S to
destination D.
Figure 2-34 shows an A* search in progress. The nodes
represent street intersections, while the edges represent
roadways connecting the nodes. The number shown inside the
node is the estimated remaining distance to the destination.
For example, B is estimated to be 13 units away from the
destination. What is the estimated distance from F? In this
case, eight units. The number shown against an edge is the
actual distance observed when traversing that edge. For
instance, the edge SC has an actual distance of 15 units. The
distance for edge FD is nine units
The thick lines show the paths already explored by the
end of the following description: the search begins at source S

30. 4. Planning Techniques
95
apply, the planning process fails. On success, the solver
returns the plan ‘P,’ which is the sequence of operators that
can make the successful transition(s) from the initial state ‘S’
to the goal state ‘G.’
Once the plan is prepared in advance, we can apply it
on demand to transition to the goal state(s) when the system
enters a prespecified start state.
Figure 4-2: A high-level view of the planning process. The
goal is to find a series of actions that take us from the source state to
destinations or desirable goals.
Coming back to the OR scheduling example, when we
have to accommodate an emergency surgical procedure, our
current schedule is in an undesirable state, shown as the source
in Figure 4-2. New schedules that effectively accommodate
both the emergency and existing procedures are the desirable
goals, labeled D1 through D4. These schedules can be
described by a set of constraints or specifically picked out

31. 5. Evolutionary Computing
106
Linda
1
John
6
Matt
4
Warren
3
Jenna
7
Parent 1 Parent 2
Linda
1
John
6
Phill
5
John
6
Jenna
7
Sue
8
Bill
2
Matt
4
Warren
3
Jenna
7
Child 1 Child 2
Sue
8
Bill
2
Phill
5
John
6
Jenna
7
X
Duplicate Gene
Figure 5-4: An example of a defective schedule where John
is scheduled twice.
Crossover and mutation don’t always go smoothly, as
illustrated in Figure 5-4. If a crossover happens at the dotted
line, we will end up with a defective chromosome (Child 1)
because John is scheduled for surgery twice. For combating
this issue, genetic algorithms have mechanisms to detect
defective chromosomes and repair or regenerate them.

32. 6. Expert Systems
132
Fuzzy Inference
Once we have fuzzified the inputs, the rules then work
concurrently on fuzzified inputs to generate multiple results as
described below.
`
Figure 6-10: Fuzzy inference.
Let us look at the first rule:
R1: if age is Old
risk = High
The rule says that if someone is “Old,” then the risk is
High. As per the fuzzification we did before, the membership
of the person in “Old” is 0.8, so the rule infers that the degree
of truth of the risk being “High” is also 0.8 (see Figure 6-10).
Now consider the second rule:
R2: if systolic BP is HighBP
risk = High
Fuzzification of systolic BP gives us a membership
value of 0.33 in HighBP. Therefore, the rule R2 infers that the
risk is “High” with a degree of truth 0.33.

33. 6. Expert Systems
134
the shaded area. We show the centroid with a downward arrow
as above. The centroid has an x-coordinate of 2.85. So,
according to the expert system, the stroke risk is 2.85 annual
occurrences per 1000 seventy-year-olds with a systolic BP of
140.
In the previous examples, we used only one fuzzy set
for the risk, “High.” Can we use more than one fuzzy set? Yes,
for example, we may use three fuzzy sets (“Low,” “Medium,”
and “High”) to define risk. This way, when creating rules, an
expert can quantify the risk as “Low,” “Medium,” or “High”
based on the domain knowledge.
Say, we have three rules: Rule A says the risk is
“Low”; Rule B says the risk is “Medium,” and Rule C says the
risk is “High.” So the shaded areas in Figure 6-12 represent
the membership values determined by each rule.
Figure 6-12: General Defuzzification
Just like before, to defuzzify the risk into a crisp value,
we can calculate the centroid of the combined shaded area

34. 7. Learning from Experience
162
Convolution +
Activation
Pooling
Convolution +
Activation
Pooling
Fully
Connected
Fully
Connected
Output
Tree
Lawn Mower
Girl
Layers for Feature Extraction
Layers for Classification
Figure 7-18: CNN for object recognition
Figure 7-18 shows how CNN performs object
recognition in image processing. Say, the input to CNN is a
picture, and the first layer may detect basic curves from a
group of pixels. The pooling layer reduces the size of the
input. The next layer would then combine these curves to
transform them into higher-layer features such as basic shapes.
The next layer may put together the features obtained from the
previous layer into representations of various objects in the
picture.
The final stage of the CNN contains a few fully
connected layers and an output layer. These layers can be
trained to classify input object representations into assigning
labels such as a tree, lawnmower, girl, etc. As the data
propagates from the input layer to the output layer, each
convolution layer in the CNN creates increasingly abstract

35. 7. Learning from Experience
166
Figure 7-20: Portrait Created by Artbreeder
Figure 7-20 shows the portrait created by Artbreeder
(Simon J. , 2020) program that uses BigGAN, a type of GAN.
The software provides the ability to crossbreed various images
and edit the “genes” such as age, gender, ethnicity, etc. The
picture above is generated from some already existing images.
However, creating a new portrait from raw images by
combining their features still needs some degree of ingenuity.
7.7. Support Vector Machine (SVM)
We know that a perceptron can learn a decision
boundary between outputs 0s and 1s if those are linearly
separable. Perceptron is unable to learn anything more
complex than linearly separable functions. Support Vector

36. 8. Human Interaction and Robotics
177
Figure 8-2: How symbolic reduction works.
After our request has been located, we expect a natural
response in return. Whenever somebody asks something, to
avoid responding the same way, the chatbot can randomize
how the next request is phrased (Livingstone, 2006).
In some cases, Alice may not be able to locate what we
are asking. There are default categories to address this
problem. For instance, if the input does not match with any of
the built-in categories, we can set the response as, “I am sorry,
I didn’t understand. Please say that again.”
Chatbots can help by interacting with customers in
natural language to find products they are looking for or
asking for product recommendations, or initiating a product
return. We can also integrate a chatbot to work with social
media. Based on what a customer does on the website, AI can
detect if the customer needs help and deploy a chatbot for
assistance.

37. 211
11. BIBLIOGRAPHY
Adelson-Velsky, G. M., Arlazarov, V. L., Bitman, A. R.,
Zhivotovsky, A. A., & Uskov, A. V. (1970).
Programming a computer to play chess. Russian
Mathematical Surveys, 25, 221-262.
Aggarwal, C. C. (2018). Neural networks and deep learning:
a textbook. Springer.
Aho, A. V., Hopcroft, J. E., & Ullman, J. D. (1974). The
design and analysis of computer algorithms. Addison-
Wesley.
Aikins, J. S., Kunz, J. C., Shortliffe, E. H., & Fallat, R. J.
(1983). PUFF: An expert system for interpretation of
pulmonary function data. Computers in Biomedical
Research, 199-208.
AIML Foundation. (2020, July 20). Retrieved from
www.aiml.foundation
Alander, J. T. (1994). An indexed bibliography of genetic
algorithms: years 1957--1993. Vaasa, Finland: Art of
CAD ltd.
Allen, J. (1995). Natural Language Understanding (2Nd Ed.).
Redwood City, CA, USA: Benjamin-Cummings.
Anderson, M., & Anderson, S. L. (2007). Machine ethics:
Creating an ethical intelligent agent. AI Magazine.

38. 233
INDEX
A* algorithm, 71
activation function, 141
AI infrastructure, 203
artificial neural networks. See
neural networks
artificial vision, 182
primal sketch, 182
best first search, 68
best path methods, 71
dynamic programming principle,
73
Big Θ, 46
blind search, 40
breadth‐first search, 48
chatbot, 174
closed world assumption, 112
defuzzification, 133
Depth First Search, 41
DFS, 41
evidence variables, 123
evolutionary computing. See
genetic algorithms
evolutionary process, 107
exoskeletons, 187
expert system, 114
inference engine, 114
expert systems
backward chaining, 116
conflict resolution strategy, 115
deduction systems, 118
hybrid chaining, 118
knowledge base, 114
reaction system, 118
working memory, 114
find method, 39
forest, 36
fuzzification, 131
fuzzy expert systems, 126
defuzzification, 133
fuzzy inference, 132
membership function, 127
fuzzy inference, 132
game trees, 80
alpha‐beta pruning, 85
horizon effect, 85
progressive deepening, 88
general intelligence, 193
generate and test, 28
genetic algorithms, 102
crossover, 104
fitness function, 103
mutation, 105
selection, 103
genetic programming, 108
graph, 36
connected graph, 36

39. 234
DAG, 37
directed acyclic graph, 37
disconnected graph, 36
heuristic function, 53
hidden variables, 124
hill climbing, 54
foothill, 63
limitations of, 61
plateau, 65
ridge, 65
simple hill climbing, 54
simulated annealing, 67
steepest ascent hill climbing, 54
with restart, 67
humanoid, 187
intelligent agents, 186
knowledge base, 114
Landau's symbol, 46
Loebner prize, 192
membership function, 127
mini‐max trees. See game trees
natural language processing, 174
symbolic reduction, 176
natural language understanding,
178
semantic interpretation, 179
syntax analysis, 179
world knowledge
interpretation, 179
neural network
binary classification, 153
epoch, 147
gradient descent, 147
learning rate, 148
linear classification, 152
linear regression, 150
logistic regression, 153
loss function, 148
multi classification, 153
neural networks, 139
activation function, 141
convolutional layer, 160
convolutional network, 159
deep learning, 159
deep learning, 155
generative adversarial network,
164
linearly separable, 146
pooling layer, 161
recurrent neural network, 163
reinforced learning, 158
stride length, 160
supervised learning, 157
support vector machine
kernel trick, 168
support vector machine, 167
unsupervised learning, 157
partial intelligence, 190
partial‐order planning, 97
planning, 92
backward planning, 96
forward planning, 93
goal protection, 96
planning under uncertainty, 99
progression planning, 93
regression planning, 96
query variables, 123
robot, 186
scheduling problem, 22
space complexity, 45
speech synthesis, 181
state space search, 31
static evaluation function, 81
superintelligence, 194
supervised learning, 157
support vector machine, 167
symbolic reduction, 176
tensor processing units, 199

40. 235
time complexity, 45
tree, 33
branching factor, 34
height, 34
leaf node, 34
Turing test, 188
uncanny valley effect, 188

41. 237
ADDITIONAL RESOURCES
We plan to make any corrections, additional resources, and
materials available from:
https://www.cstrends.com/AI
If you have any comments or suggestions for
improvement, you may let us know. We like to hear from you.
ai@cstrends.com

42. 238
ABOUT THE AUTHORS
Dr. Binto George is a professor in the School of Computer
Sciences at Western Illinois University (WIU), Macomb, IL, USA.
Before joining WIU, he worked at Rutgers University. Dr. George
received his Ph.D. from the Indian Institute of Science, Bangalore.
He has authored several journal articles, conference papers, book
chapters, and books. As the principal investigator, Dr. George has
led the National Science Foundation (NSF) funded research to
incorporate usable security into the computer science curriculum.
He loves teaching and developing new courses. Dr. George is a
partner of the CSTrends LLP, an organization committed to making
Computer Science accessible for all. Dr. George is a member of the
IEEE, IEEE Computer Science Society, and the Association for
Computing Machinery (ACM). Dr. George actively participates in
community service and curriculum development activities.
Gail Carmichael is currently a technical educator at Shopify, where
she led the design and launch of the work-integrated learning
program Dev Degree. She previously worked as a full-time
instructor at Carleton University, where she taught both majors and
non-majors a variety of computer science courses. She is
particularly passionate about teaching beginners and enticing them
to fall in love with computer science, whether as a major or as a
tool to help them in their own fields. She co-founded Carleton
University's Women in Science and Engineering, helped launch the
now Ontario wide Go Code Girl high school outreach program, and
has developed and taught many computing workshops and courses
for folks of all ages.