Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lecture (EGH444, QUT)
•
1 like•322 views
These slides were used in the guest lecture for QUT's Image processing class. The two part presentation consists of our Amazon Robotics Challenge robot #Cartman and some introduction to (deep) reinforcement learning.
Recommended
More Related Content
Similar to Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lecture (EGH444, QUT)
Similar to Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lecture (EGH444, QUT) (20)
More from Juxi Leitner
More from Juxi Leitner (20)
Recently uploaded
Recently uploaded (20)
Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lecture (EGH444, QUT)
- 1. Juxi Leitner arc centre of excellence for robotic vision queensland university of technology <j.leitner@qut.edu.au> http://Juxi.net reinforcement learning Juxi (deep)
- 2. Juxi Leitner arc centre of excellence for robotic vision queensland university of technology <j.leitner@qut.edu.au> http://Juxi.net reinforcement learning Juxi (deep)
- 4. Dalle Molle Institute for AI (IDSIA) Work Juxi Leitner PhD Informatics / Intelligent Systems MSc Space Robotics & Automation BSc Information & Software Engineering Intelligent (Space) Robots European Space Agency (ESA) Erasmus Intelligent Systems Work (Humanoid) Robot Vision Instituto Superior Técnico (IST) Mobility Intelligent Space Systems Laboratory About Me Current Robotic Vision and Actions Queensland University of Technology (QUT) arc centre of excellence for robotic vision | qut juxi.net | roboticvision.org | bne-robotics.net | brisbane.ai
- 5. BRISBANE ARTIFICIAL INTELLIGENCE @BrisbaneAI #brai sponsors Event hosts
- 7. http://roboticvision.org/ create agent that see and interact with the world http://Juxi.net/aboutme
- 10. BRISBANE.AI defining AI study of "intelligent agents”: any device that perceives its environment and takes actions that maximize its chance of success at some goal
- 12. http://roboticvision.org/ IM-CLeVeR Teaser https://www.youtube.com/watch?v=OyfonCDxUiU full video: https://vimeo.com/51011081
- 18. http://roboticvision.org/ robot RL Konidaris et al. Autonomous Skill Acquisition on a Mobile Manipulator https://www.youtube.com/watch?v=yUICAkSQTZY
- 19. http://roboticvision.org/ Jan Peters et al. Motor Skill Learning from Demonstration robot RL https://www.youtube.com/watch?v=qtqubguikMk
- 20. http://roboticvision.org/ foundations a policy, a reward signal, a value func,on, and, op,onally, a model of the environment http://cs.stanford.edu/people/karpathy/reinforcejs/http://karpathy.github.io/2016/05/31/rl/
- 21. http://roboticvision.org/ policy Policy defines the actions to be taken per state
- 22. http://roboticvision.org/ value function Value function is a prediction of future reward per state
- 23. http://roboticvision.org/ mdp An information state (a.k.a. Markov state) contains all useful information from the history. i.e. the state is a sufficient statistic of the future pomdp what if: robot with camera vision isn’t told its absolute location agent state != environment state Formally this is a partially observable Markov decision process (POMDP)
- 28. http://roboticvision.org/ on/off policy Q-learning (Watkins, 1989), h7ps://studywolf.wordpress.com/2013/07/01/reinforcement-learning-sarsa-vs-q-learning/
- 29. BRISBANE.AI deephype Dartmouth1956 Initial Hype AI Winter AI Winter Expert Systems Hype AI Spring Deep Learning Hype
- 30. http://roboticvision.org/ deep RL Sergey Levine / Google Brain DeepMind QUT
- 31. http://roboticvision.org/ deep RL DeepMind idea: learn a policy to play a computer game using only visual information state: ? actions: ? reward: ? game screen button press game score problems?
- 32. understanding limita,ons of deep nets, reinforcement learning and transfer of knowledge deep learning visual control [Tow et al, ACRA 2015]
- 33. http://roboticvision.org/ [Zhang et al, ACRA 2015] deep learning visual control
- 34. http://roboticvision.org/ [Zhang et al, arxiv.org] deep learning visual control understanding limita,ons of deep nets, reinforcement learning and transfer of knowledge
- 35. http://roboticvision.org/ transfer visual control from simulation to real world reward issue percep,on issue noise issue explora,on issue
- 36. deep learning visual servoing Perception Module Control Module Conv1 Conv2 Conv3 FC_c2 FC_c3FC_c1 Q-values 7×7conv+ReLU stride2 4×4conv+ReLU stride2 3×3conv+ReLU stride1 64 lters 64 lters 64 lters fullyconn. 300units 9units 84×84 400units fullyconn. fullyconn.+ReLU fullyconn.+ReLU I BN 5units θ Bottleneck Or Occlusion A B C ED Occlusion Occlusion Occlusion [Zhang et al, arxiv.org] understanding limita,ons of deep nets, reinforcement learning and transfer of knowledge
- 37. ARC Centre of Excellence for Robotic Vision roboticvision.org limita,ons of current robo,c systems reproducible research on TASKS not datasets picking benchmark http://Juxi.net/dataset/acrv-picking-benchmark/ https://arxiv.org/abs/1609.05258
- 38. http://roboticvision.org/ deep RL Sergey Levine, Peter Pastor et al. (Google Brain)
- 39. http://roboticvision.org/ artificial curiosity Reward the reward-optimizing controller for actions yielding data that cause improvements of the adaptive predictor or data compressor! idea: similar: intrinsic motivation
- 41. http://roboticvision.org/ challenges Curiosity? Exploration/Exploitation? Oracles? discreet —> continuous? real-world reward? https://github.com/aikorea/awesome-rl#theory
- 42. BRISBANE.AI new developments arxiv-sanity, twitter & get your hands dirty come to Brisbane.AI meetups! :) how to keep in the loop? http://Juxi.net/workshop/deep-learning-rss-2017/ Tools and toolboxes Neuroscience vs Deep Learning & Evolutionary approaches Generative Adversarial Networks Unsupervised Learning, Embodied Learning
- 43. BRISBANE.AI Jürgen ‘Juxi’ Leitner arc centre of excellence for robotic vision | qut juxi.net | roboticvision.org | bne-robotics.net | brisbane.ai In which we try to explain why we consider ar,ficial intelligence to be a subject most worthy of study, and in which we try to decide what exactly it is, this being a good thing to decide before embarking. TUTORIAL ONE BRISBANE ARTIFICIAL INTELLIGENCE http://Juxi.net <juxi.leitner@gmail.com>
- 45. Amazon Robotics Challenge Jürgen ‘Juxi’ Leitner arc centre of excellence for robotic vision queensland university of technology <j.leitner@qut.edu.au> http://Juxi.net
- 46. http://roboticvision.org/ object manipulation in clutter long term goal
- 47. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ #cartman
- 48. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Hardware #cartman
- 49. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Hardware #cartman
- 50. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Hardware #cartman
- 52. s m a r t . r o b o t s . SMRTRobots END- EFFECTOR
- 53. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Perception #cartman seman&c segmenta&on
- 54. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Perception #cartman rapid training
- 55. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Perception #cartman
- 56. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Perception #cartman grasp synthesis
- 57. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ in Action #cartman Videos https://www.youtube.com/watch?time_continue=5&v=p-WhO0LF4oY (ARC Pick failure) https://www.youtube.com/watch?v=BB5Pyh4dtxw (ARC Quick Learning of Items) https://www.youtube.com/watch?v=VEKanLH2gFY (ARC Finals) https://www.youtube.com/watch?v=a4_j6EAK3rs&feature=youtu.be (Reaching Learning)
- 58. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ in Action #cartman Papers / TechRep Cartman: The low-cost Cartesian Manipulator that won the Amazon Robotics Challenge. Douglas Morrison, et al https://arxiv.org/abs/1709.06283 Mechanical Design of a Cartesian Manipulator for Warehouse Pick and Place. M. McTaggart, et al http://juxi.net/papers/ACRV-TR-2017-02.pdf Design of a Multi-Modal End-Effector and Grasping System: How Integrated Design helped win the ARC. S. Wade- McCue, N. Kelly-Boxall, et al. http://juxi.net/papers/ACRV-TR-2017-03.pdf Semantic Segmentation from Limited Training Data. Anton Milan, et al. http://juxi.net/papers/ACRV-TR-2017-04.pdf Sim-to-real Transfer of Visuo-motor Policies for Reaching in Clutter: Domain Randomization & Adaptation with Modular Nets. Fangyi Zhang, et al. https://arxiv.org/abs/1709.05746 Training Deep Neural Networks for Visual Servoing. Quentin Bateux, et al https://arxiv.org/abs/1705.08940 h7p://Juxi.net/papers
- 59. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Nice Features #cartman
- 60. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Nice Features #cartman
- 61. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Nice Features #cartman
- 62. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Nice Features #cartman
- 63. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Nice Features #cartman
- 64. ARC Centre of Excellence for Robotic Vision roboticvision.orghttp://roboticvision.org/ Nice Features #cartman HW=12k project=42k travel=22k w/o salaries
- 65. http://roboticvision.org/ #teamACRVRoboticVisionAU Adam Tow Steve Mar&n Rohan Smith Jordan Erskine Anthony Gillespie Riccardo Grinover Alec Gurman Tom Hunn Darryl Lee Nathan Perkins Gerard Rallos Andrew Razjigaev Juxi Leitner, Ian Reid, Peter Corke http://facebook.com/TeamACRV Doug Morrison Ma7 McTaggert Zheyu Zhuang Norton Kelly-Boxall Sean Wade-McCue Thomas Rowntree Trung Pham Vijay Kumar Ming Cai Saroj Weerasekera Chris Lehnert Anton Milan Thank You!