This document discusses deep reinforcement learning and how it was applied in AlphaGo to master the game of Go. It provides an overview of deep learning, reinforcement learning, and how AlphaGo combined the two approaches. AlphaGo used deep neural networks to mimic human expert moves and play games against itself to estimate win probabilities. It had a policy network to choose moves and a value network to estimate game outcomes. Through deep reinforcement learning, AlphaGo was able to achieve superhuman performance at the game of Go.

1. Behind AlphaGo
- Deep Reinforcement Learning
Houston Machine Learning
Yan Xu
08/05/2017

2. Roadmap
• Introduction and Feature Engineering (2 lectures)
• Supervised Learning (4 lectures)
• Unsupervised Learning (3 lectures)
• Deep Learning series (4 lectures)
• Optimization in Deep learning
• Behind AlphaGo
• Mastering the game of Go with deep neural networks and tree search
• Reinforcement learning 101 – Ravi
• Deep reinforcement learning
• Attention network
• Cuda Programming Hands-on - Martin
• Application of Deep Learning
• Object recognition
• Chatbot
Slides posted on:
http://www.slideshare.net/xuyangela

3. Outline
• Go game
• Deep learning recap
• Reinforcement learning 101
• AlphaGo system overview: Deep reinforcement
learning
More depth into deep reinforcement learning in upcoming
meetups

4. Go game

5. Go game

6. Why hard for computer to play

7. Deep Learning
+
Reinforcement Learning
=
Deep reinforcement learning

8. Deep Learning: Basic Component
Activation
function

9. Deep Learning: Representation

10. Deep Learning: Optimization

11. Deep Learning: Architecture
Convolutional neural network
feature
map
Feature map

12. Reinforcement Learning

13. Reinforcement Learning
Agent
Environment

14. Reinforcement Learning

15. Reinforcement Learning
https://www.youtube.com/watch?v=xWe58WGWmlk&t=64s

16. Deep Reinforcement Learning

17. AlphaGo: Deep Reinforcement Learning
http://techtalks.tv/talks/deep-reinforcement-learning/62360/
Mimic human
experts
Play against self
Estimate wins
Policy Network
Value Network

18. http://techtalks.tv/talks/deep-reinforcement-learning/62360/

19. http://techtalks.tv/talks/deep-reinforcement-learning/62360/

20. http://techtalks.tv/talks/deep-reinforcement-learning/62360/

21. http://techtalks.tv/talks/deep-reinforcement-learning/62360/

28. Wide application

29. More coming up!
• Behind AlphaGo
• Mastering the game of Go with deep neural networks and
tree search
• Reinforcement learning 101 – Ravi
• Deep reinforcement learning
• Attention network
• Cuda Programming Hands-on - Martin
• Application of Deep Learning
• Object recognition
• Chatbot
• Any proposal?

30. HML Speaker Hall of Fame
Recognize contribution to Houston Machine Learning Meetup

31. Thank you ~
Slides will be posted at:
http://www.slideshare.net/xuyangela
Leave a
group
review
please 