1. Types of Agent 2. Problem-solving agent 3. Problem formulation

1. Agents and Problem Formulation
CSI 341 Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

2. Agents
2
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

3. Agents >> Structure
agent = architecture + program
Agent program >> It implements the agent function – the mapping from percepts to actions.
Architecture >> It is some sort of computing device with physical sensors and actuators on which agent program will run.
Architecture makes the percepts from the sensors available to the program, runs the program, and feeds the program’s
action choices to the actuators as they are generated.
3
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

4. Agents >> Table-Driven-Agent program
4
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

5. Agents >> Table-Driven-Agent program
Problems>>
▸Let P be the set of possible percepts and let T be the lifetime of the agent. The lookup table will contain
σ 𝑡=1
𝑇
𝑃 𝑡
entries. So no physical agent in this universe will have the space to store the table.
▸The designer would not have time to create the table.
▸No agent could ever learn all the right table entries from its experience.
▸If the environment is simple enough to yield a feasible table size, the designer still has no guidance about
how to fill in the table entries.
5
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

6. Agents >> Types
>> The key challenge for AI is to find out how to write programs that to the extent possible, produce rational
behavior from a smallish program rather than from a vast table.
>> 5 kinds of agent programs that embody the principles underlying almost all intelligent systems:
▹Simple reflex agents
▹Model-based reflex agents
▹Goal-based agents
▹Utility-based agents
▹Learning agents
6
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

7. Simple Reflex Agents
▸ The Simple reflex agents are the simplest agents. These agents take decisions on the basis of the current
percepts and ignore the rest of the percept history.
▸ These agents only succeed in the fully observable environment.
▸ The Simple reflex agent does not consider any part of percepts history during their decision and action
process.
▸ The Simple reflex agent works on Condition-action rule, which means it maps the current state to action.
Such as a Room Cleaner agent, it works only if there is dirt in the room.
▸ Problems for the simple reflex agent design approach:
▹ They have very limited intelligence
▹ They do not have knowledge of non-perceptual parts of the current state
▹ Not adaptive to changes in the environment.
7
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

8. Simple Reflex Agents
8
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

9. Model-based Reflex Agents
▸The Model-based agent can work in a partially observable environment, and track the situation.
▸A model-based agent has two important factors:
Model: It is knowledge about "how things happen in the world," so it is called a Model-based agent.
Internal State: It is a representation of the current state based on percept history.
▸These agents have the model, "which is knowledge of the world" and based on the model they perform
actions.
▸Updating the agent state requires information about:
▹How the world evolves
▹How the agent's action affects the world.
9
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

10. Model-based Reflex Agents
10
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

11. Goal-based Agents
▸The knowledge of the current state environment is not always sufficient to decide for an agent to what to do.
▸The agent needs to know its goal which describes desirable situations.
▸Goal-based agents expand the capabilities of the model-based agent by having the "goal" information.
▸They choose an action, so that they can achieve the goal.
▸These agents may have to consider a long sequence of possible actions before deciding whether the goal is
achieved or not. Such considerations of different scenario are called searching and planning, which makes an
agent proactive.
11
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

12. Goal-based Agents
12
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

13. Utility-based Agents
▸These agents are similar to the goal-based agent but provide an extra component of utility measurement
which makes them different by providing a measure of success at a given state.
▸Utility-based agent act based not only goals but also the best way to achieve the goal.
▸The Utility-based agent is useful when there are multiple possible alternatives, and an agent has to choose in
order to perform the best action.
▸The utility function maps each state to a real number to check how efficiently each action achieves the goals.
13
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

14. Utility-based Agents
14
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

15. Learning Agents
▸A learning agent in AI is the type of agent which can learn from its past experiences, or it has learning
capabilities.
▸It starts to act with basic knowledge and then able to act and adapt automatically through learning.
▸A learning agent has mainly four conceptual components, which are:
- Learning element: It is responsible for making improvements by learning from environment
- Critic: Learning element takes feedback from critic which describes that how well the agent is
doing with respect to a fixed performance standard.
- Performance element: It is responsible for selecting external action
- Problem generator: This component is responsible for suggesting actions that will lead to new
and informative experiences.
Hence learning agents are able to learn, analyze performance, and look for new ways to improve the
performance.
15
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

16. Learning Agents
16
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

17. 17
Solving Problems by Searching
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

18. Problem-Solving Agents [Goal-based Agent]
Problem solving steps:
1. Goal Formulation
- Formulate goals based on the current situation and the agent’s performance.
2. Problem Formulation
- It is the process of deciding what actions and states to consider, given a goal.
3. Search for Solution
- The process of looking for a sequence of actions that reaches the goal is called search.
- A search algorithm takes a problem as input and returns a solution in the form of an action sequence.
4. Execution
- The process of executing the first action from the solution’s action sequence.
Assumptions:
Fully observable, Single agent, Discrete, Deterministic, Static
18
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

19. Example >> Traveling in Romania
19
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

20. Example >> Traveling in Romania
▸Formulate goal:
- be in Bucharest
▸Formulate problem:
- states: various cities
- actions/operators: drive between cities
▸Find solution
- By searching through states to find a goal
- sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharest
▸Execute states that lead to a solution
20
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

21. Well-defined Problems >> Traveling in Romania
A problem can be defined formally by 5 components:
▸Initial State:
- The state from which the agent starts; here Initial state= In(Arad)
▸Actions:
- A description of the possible actions available to the agent.
- ACTIONS(s) = the set of actions that can be executed in s
- For this example, ACTIONS( In(Arad) ) = { GO(Sibiu), GO(Timisoara), GO(Zerind) }
▸Transition Model:
- A description of what each action does
- It is specified by a function RESULT(s, a) that returns the state that results from doing action a in state s.
- For this example, RESULT( IN(Arad), GO(Zerind) ) = IN(Zerind)
- The initial state, actions, and transition model implicitly define the state space of the problem – the set of all states
reachable from the initial state by any sequence of actions.
- The state space forms a graph in which the nodes are states and the links between nodes are actions.
▸Goal Test
- It determines whether a given state is a goal state; for this example, Goal States = { IN(Bucharest) }
▸Path cost
- Sum of the costs of the individual actions along the path.
21
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

22. Example >> Vacuum World Problem
States >>
The agent is one of two locations and each of which might or might not contain dirt. So there are 2 x 22 = 8 world states.
A larger environment with n locations has n.2n states.
Initial State >> any state can be the initial state
Actions >> { Left, Right, Suck }
Transition Model >> It is represented as a state-space graph shown in the next slide.
Goal Test >> This checks whether all the squares are clean?
Path Cost >> Each step cost is 1, so the path cost is the number of steps in the path.
22
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

23. Example >> Vacuum World Problem
23
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

24. Example >> 8-puzzle
States >> each state describes the location of each of the 8 tiles and the blank. So in total 9! no of states.
Initial State>> Any state can be the initial state
Actions >> Left, Right, Up or Down
Transition Model >> If we apply Left to the start state the resulting state has the 5 and the blank switched.
Goal Test >> Checks whether the current state matched the goal state?
Path Cost >> Each step costs 1, so the path cost is the number of steps in the path.
24
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

25. Example >> 8-queen
States >> any arrangements of 0 to 8 queens on the board, total = 64.63. … … .57 = 1.8 x 1014 possibilities
or,
all possible arrangements of n queens one per column in the leftmost n columns, with no queen attacking another
Initial State >> No queens on the board
Actions >> Add a queen to any empty square
Transition Model >> Returns the board with a queen added to the specified
square.
Goal Test >> 8 queens are on the board, none attacked.
Path Cost >> 1 per move.
25
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

26. Example >> Water Jug problem
▸You have a 4-gallon and a 3-gallon water jug
▸You have a faucet with an unlimited amount of water
▸You need to get exactly 2 gallons in 4-gallon jug
26
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

27. Example >> Water Jug problem
States >> Each state is represented as (x,y) where x =contents of 4 gallon and y = contents of 3 gallon, in total=5x4=20 states.
Initial State >> (0,0)
Actions and Transition Model >> following, Goal Test>> state = (2, n) ?? Path Cost >> 1 per step
27
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU
(x,y)  (4,y) if x < 4 Fill the 4-gallon jug
(x,y)  (x,3) if y < 3 Fill the 3-gallon jug
(x,y)  (0,y) if x > 0 Empty 4-gallon jug on ground
(x,y)  (x,0) if y > 0 Empty 3-gallon jug on ground
(x, y)  (4, y-(4-x) ) if y>0 and x+y>=4 Pour water from 3-gallon to fill 4-gallon
(x, y)  (x-(3-y), 3) if x>0 and x+y>=3 Pour water from 4-gallon to 3-gallon to fill it
(x, y)  (x+y, 0) if x+y<=4 and y>0 Pour all of water from 3-gallon to 4-gallon
(x, y)  (0, x+y) if x+y<=3 and x>0 Pour all of water from 4-gallon to 3-gallon

28. Example >> Water Jug problem
One Solution>>
28
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

29. Previous Questions
29
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

30. Previous Questions
30
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

31. Previous Questions
31
Mohammad Imam Hossain | Lecturer, Dept. of CSE | UIU

32. 32
THANKS!
Any questions?
You can find me at imam@cse.uiu.ac.bd