Advertisement
Check these out next
Ilya Sutskever at AI Frontiers : Progress towards the OpenAI mission
Nov. 15, 2018•0 likes•1,894 views
3 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Technology
I will present several advances in deep learning from OpenAI. First, I will present OpenAI Five, a neural network that learned to play on par with some of the strongest professional Dota 2 teams in the world in an 18-hero version of the game. Next, I will present Dactyl, a human-like robot hand trained entirely in simulation with reinforcement learning that has achieved unprecedented dexterity on a physical robot. I will also present our results on unsupervised learning in language, that show that pre-training and finetuning can achieve a significant improvement over state of the art. Finally, I will present an overview of the historical progress in the field.
AI FrontiersFollow
AI FrontiersAdvertisement
Advertisement
Advertisement
Recommended
Wei Xu at AI Frontiers : Language Learning in an Interactive and Embodied Set...AI Frontiers
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...AI Frontiers
Training at AI Frontiers 2018 - Lukasz Kaiser: Sequence to Sequence Learning ...AI Frontiers
Jeff Dean at AI Frontiers: Trends and Developments in Deep Learning ResearchAI Frontiers
Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...AI Frontiers
"Large-Scale Deep Learning for Building Intelligent Computer Systems," a Keyn...Edge AI and Vision Alliance
Ilya Sutskever at AI Frontiers : Progress towards the OpenAI mission
- Progress towards the OpenAI mission Ilya Sutskever Co-founder and Chief Scientist, OpenAI NOVEMBER 9, 2018
- OpenAI’s mission OpenAI’s mission is to ensure that artificial general intelligence (AGI) — by which we mean highly autonomous systems that outperform humans at most economically valuable work — benefits all of humanity. — The OpenAI Charter
- Technical progress from OpenAI
- OpenAI Five
- Dota
- Dota is hard Partial observability 120 heroes (we integrated 18) 20,000 actions per game, massive action space Pros dedicate their lives to the game, 10K+ hrs of deliberate practice
- Dota is popular Largest professional scene Annual prize pool of $40M+
- Dota
- Our approach Very large scale reinforcement learning millennia of practice LSTM policy = honeybee brain Self play Reward shaping
- Reinforcement learning (RL) actually works! Nearly all RL experts believed that RL can’t solve tasks as hard as Dota Horizon too long
- Results
- Dactyl
- Dexterity
- Diverse objects
- Strategy: Sim 2 Real
- Domain randomization Train in simulation: randomize perception and physics
- Domain randomization
- Curiosity-based exploration
- Core idea Novel states = reward Fix all bugs Very hard to do
- Montezuma’s Revenge
- Montezuma’s Revenge
- Mario
- Curiosity early in training
- Curiosity late in training
- The OpenAI Mission
- OpenAI’s mission OpenAI’s mission is to ensure that artificial general intelligence (AGI) — by which we mean highly autonomous systems that outperform humans at most economically valuable work — benefits all of humanity. — The OpenAI Charter
- Impact of AGI Generate massive wealth Potential to end poverty, achieve material abundance
- Impact of AGI Generate massive wealth Potential to end poverty, achieve material abundance Generate science and technology cure disease, extend life, superhuman healthcare mitigate global warming, clean the oceans, etc massively improve education and psychological well being
- Why is OpenAI’s mission relevant today? We review progress in the field over the past 6 years Our conclusion: near term AGI should be taken as a serious possibility
- Algorithms
- Deep Learning at the root of it all During the past 6 years, deep learning repeatedly and rapidly broke through “insurmountable” barriers A multilayered Perceptron trainable with backpropagation
- Vision (2012-2016) HOG feature (2005)—image by Antonio Torrabla
- Vision (2012-2016) ImageNet Classification Error (Top 5) Human 2015 (ResNet) 2016 (GoogleLeNet-v4) 0 5 10 15 20 25 30 3.13.6 5.0 2011 (XRCE) 2012 (AlexNet) 2013 (ZF) 2014 (VGG) 2014 (GoogleLeNet) 26.0 16.4 11.7 7.3 6.7
- Translation (2014-2018) BLEU score on EN to FR translation on the WMT dataset
- Image generation (2014-2018) GANs over the years: 2014 Goodfellow et al, 2014
- Image generation (2014-2018) GANs over the years: 2015 Radford et al, 2015
- Image generation (2014-2018) GANs over the years: 2017 Karras et al, 2017
- Image generation (2014-2018) GANs over the years: 2018 Brock et al, 2018
- Reinforcement Learning (2013-2018) DQN (2013) Mnih et al, 2013
- Reinforcement Learning (2013-2018) TRPO (2015) Schulman et al, 2015
- Reinforcement Learning (2013-2018) AlphaGo (2016) Silver et al, 2016
- Reinforcement Learning (2013-2018) OpenAI Five (2018) Very large scale: +100,000 CPU cores, +1000 GPUs
- Reinforcement Learning (2013-2018) Dactyl (2018)
- Compute
- Compute grows rapidly Neural networks can usefully consume all available compute Neural networks are extremely parallelizable 300,000x increase in neural net compute used for the largest neural net experiments over the past 6 years
- Petaflop/s-days to train Early production convnet
- Conclusion
- Formidable challenges remain Unsupervised learning Robust classification Reasoning Abstraction ???
- We’ve been breaking through barriers for 6 years Will this trend continue, or will it stop? And if so, when?
- Means proactively thinking about risks: Machines pursuing goals misspecified by their operator Malicious humans subverting deployed systems Out-of-control economy that grows without resulting in improvements to human lives THIS TALK’S GOAL IS TO PRESENT EVIDENCE THAT: While highly uncertain, near-term AGI should be taken as a serious possibility.
- Thank you
Advertisement