Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SageMaker로 강화학습(RL) 마스터링 :: 남궁선 - AWS Community Day 2019

1,710 views

Published on

SageMaker로 강화학습(RL) 마스터링
남궁선

Published in: Data & Analytics
  • Be the first to comment

SageMaker로 강화학습(RL) 마스터링 :: 남궁선 - AWS Community Day 2019

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker (RL) SageMaker and DeepRacer Seon Namkung Student Hanyang University ERICA
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reinforcement Learning) SageMaker RL Demo - SageMaker RL &Auto Scaling DeepRacer
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reinforcement Learning)
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. …(1)
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. …(2)
  6. 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reinforcement) . - Wikipedia -
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “ ” - Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction -
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. inAI Artificial
 Intelligence Machine
 Learning Reinforcement
 Learning Supervised
 Learning Unsupervised
 Learning
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Environment) ?
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Rewards RL Algorithm
  12. 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AUTONOMOUS CARS FINANCIAL TRADING DATACENTER COOLINGFLEET LOGISTICS
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. , RL … RL agent Environment
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker RL
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker Build - - - - , Train - - - - , Deploy - - API - -
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. RL RL toolkits that provide RL agent algorithm implementations RL-Coach RL-Ray RLLib Open AI Baselines … DQN PPO A3C Rainbow … DQN PPO A3CIMPALA … DQN PPO … Amazon SageMaker deep learning frameworks Tensorflow MxNet PyTorch Chainer SageMaker supported Customer BYO
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Common gym interfaces env.reset()
 env.render()
 env.step()
 env.action_space
 env.observation_space Simulation environments to model real-world problems AWS simulation 
 environments Open source 
 environments Custom developed environments Commercial simulators AWS RoboMaker EnergyPlus RoboSchool PyBullet … Bring Your Own SimuLink Amazon Sumerian
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simulation environment RL 
 agent SageMaker Training R O L L O U T Initial model
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Actions and Observations Simulation environment Actions Observation RL 
 agent Initial model SageMaker Training R O L L O U T
  21. 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training data Simulation environment Actions Observation RL 
 agent Training Data Observation, action, reward SageMaker Training
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simulation environment Actions Observation RL 
 agent Model updates Training data Observation, action, reward SageMaker Training
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Simulation environment Actions Observation Trained Model RL 
 agent Model updates Training data Training visualization Observation, action, reward Deploy Model SageMaker Training
  25. 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DEMO - SageMaker RL & Auto Scaling
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Example :Auto Scaling : • Environment : & • Action : / • State : ( , ) • Reward : R( , ) < >
  27. 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configures
  28. 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. (Reward)
  29. 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Observation, Reward
  30. 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Training
  31. 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  32. 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DeepRacer
  33. 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer 1/18 ( )
  34. 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer
  35. 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. DeepRacer RL ACTION GOALAGENT ENVIRONMENT
  36. 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AGENT STATE REWARD ENVIRONMENT ACTION VALUE FUNCTION POLICY FUNCTION
  37. 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker RL +AWS RoboMaker accelerate learning forAWS DeepRacer
  38. 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Samples
  39. 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon SageMaker https://aws.amazon.com/ko/sagemaker/ Amazon SageMaker https://aws.amazon.com/ko/sagemaker/pricing/ Amazon SageMaker RL https://aws.amazon.com/ko/blogs/korea/amazon-sagemaker-rl- managed-reinforcement-learning-with-amazon-sagemaker/ Amazon SageMaker RL https://docs.aws.amazon.com/ko_kr/sagemaker/latest/dg/ sagemaker-rl-workflow.html
  40. 40. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Seon Namkung whitesoil.kr@gmail.com

×