There are several types of environments that can affect an agent's design: Fully observable vs partially observable environments determine whether the agent can see the complete state or not. Deterministic vs stochastic environments determine whether the next state is completely determined or random. Episodic vs sequential environments divide the agent's experience into independent episodes or dependent ongoing decisions. The document provides examples of different agent tasks and classifies them based on whether their environment is fully/partially observable, deterministic/stochastic, episodic/sequential, and other characteristics like static/dynamic, discrete/continuous, and single/multi-agent. Real world environments tend to be more complex, with partial observability, stochasticity, sequential decisions,

1. Environment types
1
July 16, 2015

2. Objectives
• List and define the different types of
environment
• Determine the different types of task
environment of each agent
• Give example of each type
2

3. Environment types
Fully observable (vs. partially observable):
• An agent's sensors give it access to the
complete state of the environment at each point
in time, then we say that the task environment
is fully observable.
• Example of Fully Observable: Chess with a
clock
• Example of partially observable: Automated Taxi
3

4. Deterministic (vs. stochastic):
• If the the next state of the environment is
completely determined by the current state and
the action executed by the agent. Otherwise, it
is stochastic
• If the environment is partially observable, then
is could appear to be stochastic.
• If the environment is deterministic except for the
actions of other agents, then the environment is
strategic
• Deterministic (e.g. Crossword Puzzle),
Stochastic (e.g. Taxi Driver), Strategic (e.g.
Chess with a clock)
4

5. Episodic (vs. sequential):
• In episodic environment, the agent's experience
is divided into atomic "episodes" .
• Each episode consists of the agent perceiving
and then performing a single action.
• the choice of action in each episode depends
only on the episode itself.
• Episodic (e.g. Part Picking Robot, Image-
analysis)
• Sequential environment, the current decision
could affect all future decisions (e.g. chess,
automated taxi)
5

6. Static (vs. dynamic):
• The environment is unchanged while an
agent is deliberating, then, it is static.
(The environment is semidynamic if the
environment itself does not change with
the passage of time but the agent's
performance score does)
• Static (e.g. cross-word puzzle)
• Semidynamic (e.g. chess played with
clock)
• Dynamic (e.g. Automated Taxi)
6

7. Discrete (vs. continuous):
• The discrete/continuous can be applied to
the state of the environment, to the way
time is handled, and to the percepts and
actions of the agents.
• Discrete (e.g. Chess, Crossword Puzzle)
• Continuous (e.g. Automated Taxi,
Refinery Controller)
7

8. Single agent (vs. multiagent):
• An agent operating by itself in an
environment.
• Single agent (e.g. Crossword Puzzle,
Medical Diagnosis, Refinery Controller)
• Multiagent (e.g. Chess with a clock, Taxi
Driving)
8

9. Environment types
Chess with Chess without Taxi driving
a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No
 The environment type largely determines the agent
design
 The real world is (of course) partially observable,
stochastic, sequential, dynamic, continuous, multi-agent
9