An agent is anything that perceives its environment through sensors and acts upon the environment through actuators. Intelligent agents can be human, robots with cameras and motors, or thermostats detecting room temperature. An agent's behavior is described by its agent function, which maps percept sequences to actions. Rational agents select actions expected to maximize a performance measure given the agent's knowledge and percept sequence. Learning agents can improve their performance over time by experiencing examples.

1. Click to add Text
Chapter 2
Intelligent Agents

2. Chapter 2
Intelligent Agents
What is an agent ?
 An agent is anything that perceiving its
environment through sensors and acting
upon that environment through actuators
 Example:
 Human is an agent
 A robot is also an agent with cameras and motors
 A thermostat detecting room temperature.

3. Intelligent Agents

4. Diagram of an agent
What AI should fill

5. Simple Terms
Percept
 Agent’s perceptual inputs at any given instant
Percept sequence
 Complete history of everything that the agent
has ever perceived.

6. Agent function & program
Agent’s behavior is mathematically
described by
 Agent function
 A function mapping any given percept
sequence to an action
Practically it is described by
 An agent program
 The real implementation

7. Vacuum-cleaner world
Perception: Clean or Dirty? where it is in?
Actions: Move left, Move right, suck, do
nothing

8. Vacuum-cleaner world

9. Program implements the agent
function tabulated in Fig. 2.3
Function Reflex-Vacuum-Agent([location,status])
return an action
If status = Dirty then return Suck
else if location = A then return Right
else if location = B then return left

10. Concept of Rationality
Rational agent
 One that does the right thing
 = every entry in the table for the agent
function is correct (rational).
What is correct?
 The actions that cause the agent to be
most successful
 So we need ways to measure success.

11. Performance measure
Performance measure
 An objective function that determines
 How the agent does successfully
 E.g., 90% or 30% ?
An agent, based on its percepts
  action sequence :
if desirable, it is said to be performing well.
 No universal performance measure for all
agents

12. Performance measure
A general rule:
 Design performance measures according to
 What one actually wants in the environment
 Rather than how one thinks the agent should
behave
E.g., in vacuum-cleaner world
 We want the floor clean, no matter how the
agent behave
 We don’t restrict how the agent behaves

13. Rationality
What is rational at any given time depends
on four things:
 The performance measure defining the criterion
of success
 The agent’s prior knowledge of the environment
 The actions that the agent can perform
 The agents’s percept sequence up to now

14. Rational agent
For each possible percept sequence,
 an rational agent should select
 an action expected to maximize its performance
measure, given the evidence provided by the
percept sequence and whatever built-in knowledge
the agent has
E.g., an exam
 Maximize marks, based on
the questions on the paper & your knowledge

15. Example of a rational agent
Performance measure
 Awards one point for each clean square
 at each time step, over 10000 time steps
Prior knowledge about the environment
 The geography of the environment
 Only two squares
 The effect of the actions

16. Actions that can perform
 Left, Right, Suck and NoOp
Percept sequences
 Where is the agent?
 Whether the location contains dirt?
Under this circumstance, the agent is
rational.
Example of a rational agent

17. An omniscient agent
 Knows the actual outcome of its actions in
advance
 No other possible outcomes
 However, impossible in real world
An example
 crossing a street but died of the fallen
cargo door from 33,000ft  irrational?
Omniscience

18. Based on the circumstance, it is rational.
As rationality maximizes
 Expected performance
Perfection maximizes
 Actual performance
Hence rational agents are not
omniscient.
Omniscience

19. Learning
Does a rational agent depend on only
current percept?
 No, the past percept sequence should also
be used
 This is called learning
 After experiencing an episode, the agent
 should adjust its behaviors to perform better
for the same job next time.

20. Autonomy
If an agent just relies on the prior knowledge of
its designer rather than its own percepts then
the agent lacks autonomy
A rational agent should be autonomous- it
should learn what it can to compensate for
partial or incorrect prior knowledge.
E.g., a clock
 No input (percepts)
 Run only but its own algorithm (prior knowledge)
 No learning, no experience, etc.

21. Sometimes, the environment may not be
the real world
 E.g., flight simulator, video games, Internet
 They are all artificial but very complex
environments
 Those agents working in these environments
are called
 Software agent (softbots)
 Because all parts of the agent are software
Software Agents

22. Task environments
Task environments are the problems
 While the rational agents are the solutions
Specifying the task environment
 PEAS description as fully as possible
 Performance
 Environment
 Actuators
 Sensors
In designing an agent, the first step must always be to
specify the task environment as fully as possible.
Use automated taxi driver as an example

23. Task environments
Performance measure
 How can we judge the automated driver?
 Which factors are considered?
 getting to the correct destination
 minimizing fuel consumption
 minimizing the trip time and/or cost
 minimizing the violations of traffic laws
 maximizing the safety and comfort, etc.

24. Environment
 A taxi must deal with a variety of roads
 Traffic lights, other vehicles, pedestrians,
stray animals, road works, police cars, etc.
 Interact with the customer
Task environments

25. Actuators (for outputs)
 Control over the accelerator, steering, gear
shifting and braking
 A display to communicate with the
customers
Sensors (for inputs)
 Detect other vehicles, road situations
 GPS (Global Positioning System) to know
where the taxi is
 Many more devices are necessary
Task environments

26. A sketch of automated taxi driver
Task environments

27. Properties of task environments
Fully observable vs. Partially observable
 If an agent’s sensors give it access to the
complete state of the environment at each
point in time then the environment is
effectively and fully observable
 if the sensors detect all aspects
 That are relevant to the choice of action

28. Partially observable
An environment might be Partially observable
because of noisy and inaccurate sensors or
because parts of the state are simply missing
from the sensor data.
Example:
 A local dirt sensor of the cleaner cannot tell
 Whether other squares are clean or not

29. Deterministic vs. stochastic
 next state of the environment Completely
determined by the current state and the actions
executed by the agent, then the environment is
deterministic, otherwise, it is Stochastic.
 Strategic environment: deterministic except for
actions of other agents
-Cleaner and taxi driver are:
 Stochastic because of some unobservable aspects 
noise or unknown
Properties of task environments

30. Episodic vs. sequential
 An episode = agent’s single pair of perception & action
 The quality of the agent’s action does not depend on
other episodes
 Every episode is independent of each other
 Episodic environment is simpler
 The agent does not need to think ahead
Sequential
 Current action may affect all future decisions
-Ex. Taxi driving and chess.
Properties of task environments

31. Static vs. dynamic
 A dynamic environment is always changing
over time
 E.g., the number of people in the street
 While static environment
 E.g., the destination
Semidynamic
 environment is not changed over time
 but the agent’s performance score does
Properties of task environments

32. Discrete vs. continuous
 If there are a limited number of distinct
states, clearly defined percepts and actions,
the environment is discrete
 E.g., Chess game
 Continuous: Taxi driving
Properties of task environments

33. Single agent VS. multiagent
 Playing a crossword puzzle – single agent
 Chess playing – two agents
 Competitive multiagent environment
 Chess playing
 Cooperative multiagent environment
 Automated taxi driver
 Avoiding collision
Properties of task environments

34. Properties of task environments
Known vs. unknown
This distinction refers not to the environment itslef but to
the agent’s (or designer’s) state of knowledge about
the environment.
-In known environment, the outcomes for all actions are
given. ( example: solitaire card games).
- If the environment is unknown, the agent will have to
learn how it works in order to make good decisions.(
example: new video game).

35. Examples of task environments

36. Click to add Text
Structure of agents

37. Structure of agents
Agent = architecture + program
 Architecture = some sort of computing
device (sensors + actuators)
 (Agent) Program = some function that
implements the agent mapping = “?”
 Agent Program = Job of AI

38. Agent programs
Input for Agent Program
 Only the current percept
Input for Agent Function
 The entire percept sequence
 The agent must remember all of them
Implement the agent program as
 A look up table (agent function)

39. Agent programs
Skeleton design of an agent program

40. Agent Programs
P = the set of possible percepts
T= lifetime of the agent
 The total number of percepts it receives
Size of the look up table
Consider playing chess
 P =10, T=150
 Will require a table of at least 10150 entries

T
t
t
P
1

41. Agent programs
Despite of huge size, look up table does
what we want.
The key challenge of AI
 Find out how to write programs that, to the
extent possible, produce rational behavior
 From a small amount of code
 Rather than a large amount of table entries
 E.g., a five-line program of Newton’s Method
 V.s. huge tables of square roots, sine, cosine,
…

42. Types of agent programs
Four types
 Simple reflex agents
 Model-based reflex agents
 Goal-based agents
 Utility-based agents

43. Simple reflex agents
It uses just condition-action rules
 The rules are like the form “if … then …”
 efficient but have narrow range of applicability
 Because knowledge sometimes cannot be
stated explicitly
 Work only
 if the environment is fully observable

44. Simple reflex agents

45. Simple reflex agents (2)

46. A Simple Reflex Agent in Nature
percepts
(size, motion)
RULES:
(1) If small moving object,
then activate SNAP
(2) If large moving object,
then activate AVOID and inhibit SNAP
ELSE (not moving) then NOOP
Action: SNAP or AVOID or NOOP
needed for
completeness

47. Model-based Reflex Agents
For the world that is partially observable
 the agent has to keep track of an internal state
 That depends on the percept history
 Reflecting some of the unobserved aspects
 E.g., driving a car and changing lane
Requiring two types of knowledge
 How the world evolves independently of the
agent
 How the agent’s actions affect the world

48. Example Table Agent
With Internal State
Saw an object ahead,
and turned right, and
it’s now clear ahead
Go straight
Saw an object Ahead,
turned right, and object
ahead again
Halt
See no objects ahead Go straight
See an object ahead Turn randomly
IF THEN

49. Example Reflex Agent With Internal State:
Wall-Following
Actions: left, right, straight, open-door
Rules:
1. If open(left) & open(right) and open(straight) then
choose randomly between right and left
2. If wall(left) and open(right) and open(straight) then straight
3. If wall(right) and open(left) and open(straight) then straight
4. If wall(right) and open(left) and wall(straight) then left
5. If wall(left) and open(right) and wall(straight) then right
6. If wall(left) and door(right) and wall(straight) then open-door
7. If wall(right) and wall(left) and open(straight) then straight.
8. (Default) Move randomly
start

50. Model-based Reflex Agents
The agent is with memory

51. Model-based Reflex Agents

52. Goal-based agents
Current state of the environment is
always not enough
The goal is another issue to achieve
 Judgment of rationality / correctness
Actions chosen  goals, based on
 the current state
 the current percept

53. Goal-based agents
Conclusion
 Goal-based agents are less efficient
 but more flexible
 Agent  Different goals  different tasks
 Search and planning
 two other sub-fields in AI
 to find out the action sequences to achieve its goal

54. Goal-based agents

55. Utility-based agents
Goals alone are not enough
 to generate high-quality behavior
 E.g. meals in Canteen, good or not ?
Many action sequences  the goals
 some are better and some worse
 If goal means success,
 then utility means the degree of success
(how successful it is)

56. Utility-based agents (4)

57. Utility-based agents
it is said state A has higher utility
 If state A is more preferred than others
Utility is therefore a function
 that maps a state onto a real number
 the degree of success

58. Utility-based agents (3)
Utility has several advantages:
 When there are conflicting goals,
 Only some of the goals but not all can be
achieved
 utility describes the appropriate trade-off
 When there are several goals
 None of them are achieved certainly
 utility provides a way for the decision-making

59. Learning Agents
After an agent is programmed, can it
work immediately?
 No, it still need teaching
In AI,
 Once an agent is done
 We teach it by giving it a set of examples
 Test it by using another set of examples
We then say the agent learns
 A learning agent

60. Learning Agents
Four conceptual components
 Learning element
 Making improvement
 Performance element
 Selecting external actions
 Critic
 Tells the Learning element how well the agent is doing with
respect to fixed performance standard.
(Feedback from user or examples, good or not?)
 Problem generator
 Suggest actions that will lead to new and informative
experiences.

61. Learning Agents