This document summarizes an experiment using Particle Filter on Episode (PFoE) to teach and replay behaviors on a mobile robot. PFoE allows a robot to directly make decisions from recorded episodes of sensor values and actions without needing a map. In the experiment, a trainer used a gamepad to control the robot and teach it behaviors over three laps, which were recorded as an episode. During replay, the robot was able to reperform the taught behaviors in different sensory situations and properly alternate its goal, demonstrating PFoE can enable teach-and-replay with a simple algorithm.

1. recovery from
interruptions
choosing different
actions in sensory
equivalent situations
Teach-and-Replay of Mobile Robot with
Particle Filter on Episode
1Chiba Institute of Technology, Chiba, Japan
Ryuichi Ueda1, Masahiro Kato1, Atsushi Saito1, and Ryo Okazaki1
WeAICRA 535
1. Introduction
u Use of a long memory for decision making of robots
u Memory (“episode”): a sequence of sensor values and
actions
u Easy for robots to keep long and correct memory
u Different from animals that must summarize it
u Direct decision making from an episode
u Advantage: no need of a map (different from [Nitsche 2014], for example)
u Disadvantage: perceptual aliasing, computational costs, ...
6. Conclusion
u PFoE for teach-and-replay:
u Replayed given behaviors in spite of its simplicity
u Replayed the wall following task after only one trial
of training
u Chose different actions properly in some
(short time) sensory equivalent situations
alternation of the goal
2. Related works
u Reinforcement leaning with Particle Filter on
Episode (PFoE) [Ueda 2017]
u Learned an alternation task
in T-maze applied to mice
in neuroscience
u Calculated a particle filter
in real-time
u Needed to be improved for
difficult tasks
3. Purpose
u To modify PFoE for teach-and-replay
u To utilize instructions from a trainer to perform more
practical tasks
inaccuracy in the
distance of wall
following (the next task)
Teaching (3 laps) Replay
..........
time
next
action
recalling
event
........
time
shifting particles
(with noise and random placements)
[Nitsche 2014] M. Nitsche, et al.: “Monte Carlo Localization for Teach-and-Repeat
Feature-Based Navigation,” M. Mistry, et al. (eds.), Advances in Autonomous
Robotics Systems, pp. 13-24, Springer, 2014.
[Ueda 2017] R. Ueda, et al.: “Particle Filter on Episode for Learning Decision Making
Rule,” W. Chen, et al. (eds.), Intelligent Autonomous Systems 14, pp. 737-754,
Springer, 2017.
References
..........
time
episode
observation
action
event
A trainer controls the robot
and the robot simply records
its action and sensor values.
An episode is stored after the
teaching.
..........
time
observation
Bayes’ theorem
........
time
particles
probability
observation
action
Particles are randomly
placed on the episode.
The next action of the mode
is chosen after a resampling.
The time variables of the particles
proceed with simulated noises.
(return to )
5. Experiment
u With a small mobile robot
u Four range sensors / two stepping motors
u Stand-alone (PFoE works on Raspberry Pi 3 in real-time)
u Teaching with a gamepad
u Frequency of event recording: 10[Hz]
u Number of particles: 1000
Teaching
Replay
only one trial
Feedback
motion
Teaching Replay
4. PFoE for teach-and-replay
u Teaching phase
u Replay phase
The sensor values change the
weights with Bayes theorem.
(continued on next column)
observation
time
action
controlled
by a trainer
computer memory
real world
mode