This document provides an introduction to machine learning and intelligent agents. It defines machine learning as programs that automatically improve performance through experience without human assistance. Examples of machine learning problems include optical character recognition, spam filtering, and medical diagnosis. Learning models define the learning problem formally. Intelligent agents can perceive their environment and act upon it. Rational agents aim to perform useful actions. The document discusses properties of environments including observable vs partially observable. It outlines different types of agents from simple reflex agents to goal-based and utility-based agents.

1. Introduction to machine learning BY miss saba sumreen
What is machine learning?
How many people heard, know and using about machine learning?
Hebert Alexander simon:
 ”learning is any process by which a system improves performance from experience”
 “machine learning is concerned with computer programs that automatically improve their
performance through experience”. The emphasis of machine learning is on automatic
methods. In other words, the goal is to devise learning algorithms that do the learning
automatically without human intervention or assistance. The machine learning paradigm
can be viewed as “programming by example.” Often we have a specific task in mind,
such as spam filtering. But rather than program the computer to solve the task directly, in
machine learning, we seek methods by which the computer will come up with its own
program based on examples that we provide. Machine learning is a core subarea of
artificial intelligence. It is very unlikely that we will be able to build any kind of
intelligent system capable of any of the facilities that we associate with intelligence, such
as language or vision, without using learning to get there. These tasks are otherwise
simply too difficult to solve. Further, we would not consider a system to be truly
intelligent if it were incapable of learning since learning is at the core of intelligence.
Although a subarea of AI, machine learning also intersects broadly with other fields,
especially statistics, but also mathematics, physics, theoretical computer science and
more.
Examples of Machine Learning Problems
There are many examples of machine learning problems. Much of this course will focus on
classification problems in which the goal is to categorize objects into a fixed set of
categories. Here are several examples:
optical character recognition: categorize images of handwritten characters by the letters
represented • face detection: find faces in images (or indicate if a face is present)
spam filtering: identify email messages as spam or non-spam
topic spotting: categorize news articles (say) as to whether they are about politics, sports,
entertainment, etc
spoken language understanding: within the context of a limited domain, determine the
meaning of something uttered by a speaker to the extent that it can be classified into one of a
fixed set of categories
medical diagnosis: diagnose a patient as a sufferer or non-sufferer of some disease
customer segmentation: predict, for instance, which customers will respond to a particular
promotion

2. fraud detection: identify credit card transactions (for instance) which may be fraudulent in
nature
weather prediction: predict, for instance, whether or not it will rain tomorrow (In this last
case, we most likely would actually be more interested in estimating the probability of rain
tomorrow.) Although much of what we will talk about will be about classification problems,
there are other important learning problems. In classification, we want to categorize objects
into fixed categories. In regression, on the other hand, we are trying to predict a real value.
For instance, we may wish to predict how much it will rain tomorrow. Or, we might want to
predict how much a house will sell for.
A richer learning scenario is one in which the goal is actually to behave intelligently, or to
make intelligent decisions. For instance, a robot needs to learn to navigate through its
environment without colliding with anything. To use machine learning to make money on the
stock market, we might treat investment as a classification problem (will the stock go up or
down) or a regression problem (how much will the stock go up), or, dispensing with these
intermediate goals, we might want the computer to learn directly how to decide to make
investments so as to maximize wealth. A final example is game playing where the goal is for
the computer to learn to play well through experience.
Learning Models
To study machine learning mathematically, we need to formally define the learning problem.
This precise definition is called a learning model. A learning model should be rich enough to
capture important aspects of real learning problems, but simple enough to study the problem
mathematically. As with any mathematical model, simplifying assumptions are unavoidable.
A learning model should answer several questions:
• What is being learned?
• How is the data being generated? In other words, where does it come from?
• How is the data presented to the learner? For instance, does the learner see all the data at
once, or only one example at a time?
• What is the goal of learning in this model
3 Intelligent agents
In computer science, a software agent is a computer program that acts for a user or
other program in a relationship of agency, which derives from the Latin agere (to do): an
agreement to act on one's behalf. Such "action on behalf of" implies the authority to
decide which, if any, action is appropriate.
 Agents include humans, robots, softbots, thermostats, etc.
 The agent function maps from percept histories to actions: f : P∗ → A
 The agent program runs on the physical architecture to produce f

3. An agent is anything that can perceive its environment through sensors and acts upon that
environment through effectors.
 A human agent has sensory organs such as eyes, ears, nose, tongue and skin parallel to
the sensors, and other organs such as hands, legs, mouth, for effectors.
 A robotic agent replaces cameras and infrared range finders for the sensors, and various
motors and actuators for effectors.
 A software agent has encoded bit strings as its programs and actions
Rationality
Rationality is nothing but status of being reasonable, sensible, and having good sense of
judgment.
Rationality is concerned with expected actions and results depending upon what the agent
has perceived. Performing actions with the aim of obtaining useful information is an important
part of rationality.
PEAS
To design a rational agent, we must specify the task environment
Consider, e.g., the task of designing an automated taxi:
Performance measure?? safety, destination, profits, legality, comfort, ...
Environment?? US streets/freeways, traffic, pedestrians, weather, ...
Actuators?? steering, accelerator, brake, horn, speaker/display, ...
Sensors?? video, accelerometers, gauges, engine sensors, keyboard, GPS, ...

4. Internet shopping agent
Performance measure?? price, quality, appropriateness, efficiency
Environment?? current and future WWW sites, vendors, shippers
Actuators?? display to user, follow URL, fill in form
Sensors?? HTML pages (text, graphics, scripts)
Vacuum-cleaner world:
Consider a Vacuum cleaner world
Imagine that our intelligent agent is a robot vacuum cleaner.
Let's suppose that the world has just two rooms. The robot can be in either room and there can be
dirt in zero, one, or two rooms.
Goal formulation: intuitively, we want all the dirt cleaned up. Formally, the goal is
{ state 7, state 8 }.
Percepts: location and contents, e.g., [A,Dirty]
Actions: Left, Right, Suck, NoOp

5. A vacuum-cleaner agent
Percept sequence Action
[A,Clean] Right
[A,Dirty] Suck
[B,Clean] Left
[B,Dirty] Suck
[A,Clean], [A,Clean] Right
[A,Clean], [A,Dirty] Suck . . . . .
function Reflex-Vacuum-Agent([location,status]) returns an action
if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left
Properties ofEnvironment
The environment has multifold properties −

6.  Discrete / Continuous − If there are a limited number of distinct, clearly defined, states
of the environment, the environment is discrete (For example, chess); otherwise it is
continuous (For example, driving).
 Observable / Partially Observable − If it is possible to determine the complete state of
the environment at each time point from the percepts it is observable; otherwise it is
only partially observable.
 Static / Dynamic − If the environment does not change while an agent is acting, then it
is static; otherwise it is dynamic.
 Single agent / Multiple agents − The environment may contain other agents which may
be of the same or different kind as that of the agent.
 Accessible / Inaccessible − If the agent’s sensory apparatus can have access to the
complete state of the environment, then the environment is accessible to that agent.
 Deterministic / Non-deterministic − If the next state of the environment is completely
determined by the current state and the actions of the agent, then the environment is
deterministic; otherwise it is non-deterministic.
 Episodic / Non-episodic − In an episodic environment, each episode consists of the
agent perceiving and then acting. The quality of its action depends just on the episode
itself. Subsequent episodes do not depend on the actions in the previous episodes.
Episodic environments are much simpler because the agent does not need to think
ahead.
Solitaire Backgammon Internet shopping Taxi
Observable?? Yes Yes No No
Deterministic?? Yes No Partly No
Episodic?? No No No No
Static?? Yes Semi Semi No
Discrete?? Yes Yes Yes No
Single-agent?? Yes No Yes (except auctions) No

7. Lecture 6:
Agent types
Four basic types in order of increasing generality: –
 simple reflex agents
 reflex agents with state
 goal-based agents
 utility-based agents
All these can be turned into learning agents
Simple Reflex Agents
 They choose actions only based on the current percept.
 They are rational only if a correct decision is made only on the basis of current precept.
 Their environment is completely observable.
Condition-Action Rule − It is a rule that maps a state (condition) to an action.
Model BasedReflex Agents
They use a model of the world to choose their actions. They maintain an internal state.

8. Model − The knowledge about how the things happen in the world.
Internal State − It is a representation of unobserved aspects of current state depending on
percept history.
Updating the state requires the information about −
 How the world evolves.
 How the agent’s actions affect the world.
Goal BasedAgents
They choose their actions in order to achieve goals. Goal-based approach is more flexible than
reflex agent since the knowledge supporting a decision is explicitly modeled, thereby allowing
for modifications.
Goal − It is the description of desirable situations.

9. Utility BasedAgents
They choose actions based on a preference (utility) for each state. Goals are inadequate when −
 There are conflicting goals, out of which only few can be achieved.
 Goals have some uncertainty of being achieved and you need to weigh likelihood of
success against the importance of a goal.