This document provides an introduction to reinforcement learning, which is a machine learning method that allows an agent to learn how to take actions in an environment by interacting with it. The agent receives rewards or punishments that shape its behavior toward achieving goals. Through trial and error, the agent learns a policy that maps states to actions in a way that maximizes rewards over time without being explicitly programmed.

1. Artificial Intelligence
Reinforcement Learning
Introduction
Portland Data Science Group
Created by Andrew Ferlitsch
Community Outreach Officer
July, 2017

2. Introduction
• A machine learning method for adaptation to an
environment.
• An agent (e.g., robot) interacts with a dynamic
environment.
• An agent learns from interacting with the environment
the best actions to take.
• Concepts include:
• Markov Principles
• Exploration / Exploitation
• Dynamic Programming

3. Actions / Environment
Reflex Agent
Environment
Action Observation
Continuous Cycle:
Observe Environment,
Take Action,
Observe Environment,
Take Action
Actions are determined
based on predefined
rules.
Preprogrammed

4. Intelligent Agent
Wikipedia: An Intelligent Agent is an autonomous entity which observes through sensors
and acts upon an environment using actuators (i.e. it is an agent) and directs its activity
towards achieving goals (i.e. it is "rational", as defined in economics).
Intelligent Agent
Sensors
Actuators
Environment
Senses the environment (e.g., camera,
audio, LIDAR, GPS, ultrasonic)
Room, Street,
Warehouse, etc.
Modifies the environment
(e.g., walk, pickup, drive)
State
Actions
Current State of the Agent
relative to the environment.
Possible Actions the Agent
can take.
Policy
A set of rules that
map a state to an
action.

5. State / Reward
Intelligent Agent
Environment
Action Observation
State Reward
How the action
effected the agent
and environment.
How positive or
negative is the
new state.
LEARN
What was
learned from
the reward.
Policy
Learned set of
rules of:
States -> Actions
Example Positive Reward:
Robot Stands Up,
Closer to Destination
Example Negative Reward:
Robot Falls Down,
Further from Destination
Reinforcement Learning

6. Learn by Trial and Error
Un-programmed
Predefined set
of Actions
Reward
Function
Delivered From Factory
Trial and Error
Programmed
Predefined set
of Actions
Reward
Function
Reinforcement Learning
Policy
States ->
Actions

7. Simple Reinforcement Learning Example
Actions:
Stand
Walk
Run
Rewards:
Stand -> 0
Walk -> 1
Run -> 2
Fall -> -2
Un-programmed
Initial State -> Fall State->Action
Actions: Stand -> 0 Fall -> Stand
Walk -> Robot Falls Down -> -2
Run -> Robot Falls Down -> -2
State->Stand
Stand -> 0
Walk -> 1 Stand->Walk
Run -> Robot Falls Down -> -2
State->Walk
Stand -> 0
Walk -> 1
Run -> 2 Walk->Run
Learned PolicyTrials
Highest Reward