The recent burst in progress for machine learning has enabled more sophisticated algorithms for complex tasks. The combination of deep learning (which uses machine learning algorithms such as neural nets) and reinforcement learning has been quite effective at robotics control tasks. However, proper toolkits for building these algorithms are lacking, alienating beginners and making it hard to prototype or even compare results of algorithms developed by different teams. This project deals with building such a toolkit. The requirements were considered and the final product is a fairly easy-to-use toolkit that can assist in building robot learning algorithms.

1. An Experimentation Toolkit for
Robotics Control and Manipulation
Tasks using Reinforcement Learning
Algorithms: A Robot Learning Gym
ASHWIN REDDY
THE HARKER SCHOOL
APRIL 15, 2017

2. Introduction/Motivation
• How to program robots to interact with real world objects intelligently?
• Hand-engineered methods are fragile/unreliable
• Recent advances in machine learning have made
“robot learning” more tractable
• However, there aren’t many tools for
machine-learning based robotics
• Goal: create a toolkit for researchers/developers to
train robots

3. Background: Reinforcement Learning
• Formalizes the concept of the robot accomplishing a task
• State information through sensors (may be noisy)
• Actuators/motors to manipulate environment
3

4. Background: Machine Learning
• Algorithms that learn to predict/mimic dataset
• Applicable to many problems
• Can extrapolate for unseen conditions
• Requires tools to visualize performance
• Recent spike (especially neural nets)
• Lots of frameworks and tools

5. A Toolkit for Experimentation
• Requirements:
• Minimal setup barriers
• Be able to test quickly
• Include tools to visualize learned performance
• Incorporate popular tools
• Should be able to choose any combo of robot and task
• Decided to use simulation environment

6. Building the Toolkit
• Learn current tools and choose which ones to use
• MuJoCo (simulation engine)
• Google TensorFlow (machine learning)
• OpenAI Gym (environments for RL tasks)
• Incorporate a few algorithms
• Collect robot “models” for simulator

7. Using the Toolkit
• Use cases
• Benchmarking
• Prototyping/testing
• Visualizing

8. Visualization
8

9. Video Demonstration

10. Environments

11. Conclusion/Future Work
• Available on GitHub (can be used for free with fairly minimal setup!)
• Enables experts and amateurs alike to experiment with robot learning algorithms
• Future Work
• Write documentation to make it easy to get started
• Experiment with neural network architectures
• Create public benchmarks for various robot learning algorithms
• Include more environments, tasks, and robots

12. References/Citations
https://web.stanford.edu/group/pdplab/pdphandbook/suttonbarto_rl.png
https://www.smashingrobotics.com/wp-content/uploads/2012/10/jce39n01q2031123.png
https://i.ytimg.com/vi/eFopkAceTGs/hqdefault.jpg
http://cs231n.github.io/assets/nn1/neural_net.jpeg
Chelsea Finn, Marvin Zhang, Justin Fu, Xin Yu Tan, Zoe McCarthy, Emily Scharff, Sergey Levine. Guided Policy Search Code
Implementation. 2016. Software available from rll.berkeley.edu/gps.

13. Questions?