Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

【Unite Tokyo 2018】Unity for ディープ・ラーニング:ツールキット『ML-Agents』のご紹介

2,872 views

Published on

講演者:Mike Geig(Unity Technologies)

こんな人におすすめ
・機械学習や深層学習に興味をお持ちの開発者

受講者が得られる知見
・ 『ML-Agents』ツールキットに含まれる最新の学習メソッド(カリキュラム学習、模倣学習など)
・ それらを使用してUnityでエージェントをトレーニングする方法

Published in: Technology
  • Be the first to comment

【Unite Tokyo 2018】Unity for ディープ・ラーニング:ツールキット『ML-Agents』のご紹介

  1. 1. Unity for Deep Learning: ML-Agents Explained
  2. 2. Mike Geig Head of Global Evangelism Content
  3. 3. Let’s start with one important question...
  4. 4. Why program system to complete a specific task when you can design it to learn? Let’s start with one important question...
  5. 5. Visual Complexity Cognitive ComplexityPhysical Complexity ML Training Environment Requirements
  6. 6. The Unity Ecosystem
  7. 7. ML-Agents v0.1 Components ● Learning Environments ● Flexible training scenarios (single agent, simultaneous single agent, adversarial self-play, cooperative multi-agent, competitive multi- agent, ecosystem ● Monitoring agent’s decision making ● Complex Visual observations ML-Agents v0.2 Components ● Additional environments (two new continuous control environments, plus two platforming environments) ● Curriculum Learning ● Broadcasting ● Flexible monitor ML-Agents v0.3 Components ● Imitation Learning ● Multi-Brain training ● On-demand decision-making ● Memory-enhanced agents
  8. 8. How does it work?
  9. 9. Unity ML-Agents Workflow Create Environment Train Agents Embed Agents
  10. 10. Create Environment (Unity) Observe & Act Decide Coordinate
  11. 11. Unity ML-Agents Workflow Create Environment Train Agents Embed Agents
  12. 12. Training Methods Reinforcement Learning ● Learn through rewards ● Trial-and-error ● Super-speed simulation ● Agent becomes “optimal” at task Imitation Learning ● Learn through demonstrations ● No rewards necessary ● Real-time interaction ● Agent becomes “human-like” at task
  13. 13. Unity ML-Agents Workflow Create Environment Train Agents Embed Agents
  14. 14. Embed Agents (Unity) ● Simply import a .bytes file (trained brain) into Unity project ● Set corresponding brain component to “Internal” mode. ● Support for Mac, Windows, Linux, iOS, and Android.
  15. 15. Let’s see it in action!
  16. 16. Learning Scenarios
  17. 17. Goal Balance ball as long as possible Observations Platform rotation, ball position and rotation Actions Platform rotation (in x and z) Rewards Bonus for keeping ball up Twelve Agents, One Brain, Independent Rewards
  18. 18. Goal Keep ball up as long as possible Observations Positions and velocities of racket and ball Actions Forward, backward, and upward movement Rewards +0.1 when sent over net by agent -0.1 when ball falls because of agent Two Agents, One Brain, Cooperative Rewards
  19. 19. Striker Goal Get the ball into the opponents goal Goalie Goal Defend own goal from opponents Observations Local ray-cast perception on nearby objects Actions Movement and rotation in x, z plane Striker Rewards +1 when its team scores goal -0.1 when opponent scores goal Goalie Rewards -1 when opponent scores goal +0.1 when its team scores goal Four Agents, Multi-Brain, Competitive Rewards
  20. 20. Multi-Stage Soccer Training Defense Train one brain with negative reward for ball entering their goal Offense Train one brain with positive reward for ball entering opponents goal Combined Train both brains together to play against opponent team
  21. 21. Learning Methods
  22. 22. Curriculum Learning
  23. 23. Curriculum Learning ● Bootstrap learning of difficult task with simpler task ● Utilize custom reset parameters ● Change environment task based on reward or fixed progress Easy Difficult
  24. 24. Imitation Learning
  25. 25. Imitation Learning Collect demonstrations from a teacher Learn policy via imitation
  26. 26. ML-Agents v0.1 Components ● Learning Environments ● Flexible training scenarios (single agent, simultaneous single agent, adversarial self-play, cooperative multi-agent, competitive multi- agent, ecosystem ● Monitoring agent’s decision making ● Complex Visual observations ML-Agents v0.2 Components ● Additional environments (two new continuous control environments, plus two platforming environments) ● Curriculum Learning ● Broadcasting ● Flexible monitor ML-Agents v0.3 Components ● Imitation Learning ● Multi-Brain training ● On-demand decision-making ● Memory-enhanced agents
  27. 27. We are hiring!
  28. 28. Get it Now github.com/Unity-Technologies/ml-agents Contact us https://unity3d.ai ML-Agents@Unity3d.com
  29. 29. Thank you! Mike Geig Mike@unity3d.com @MikeGeig

×