SlideShare a Scribd company logo
1 of 46
Download to read offline
Sebastian Ruder

PhD Candidate, Insight Centre

Research Scientist, AYLIEN
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NIPS 2016 Highlights
Agenda
1. NIPS overview
2. Generative Adversarial Networks
3. Building applications with Deep Learning
4. RNNs
5. Improving classic algorithms
6. Reinforcement Learning
7. Learning-to-learn / Meta-learning
8. General AI
9. NLP
10. Miscellaneous
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NIPS Overview
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Source: http://www.tml.cs.uni-tuebingen.de/team/luxburg/misc/nips2016/index.php
Generative Adversarial Networks
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Ian Goodfellow’s GAN tutorial1
• Soumith Chintala’s “How to train your GAN” talk2
1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf
2https://github.com/soumith/ganhacks
Generative Adversarial Networks
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Generative Adversarial Networks
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Ian Goodfellow’s GAN tutorial1
• Soumith Chintala’s “How to train your GAN” talk2
• Yann LeCun is very bullish on GANs; lots of work on
GANs from FAIR, etc.3
• Cool extensions, e.g. GA what-were network4
1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf
2https://github.com/soumith/ganhacks
3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view
4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf
Generative Adversarial Networks
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Generative Adversarial Networks
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Ian Goodfellow’s GAN tutorial1
• Soumith Chintala’s “How to train your GAN” talk2
• Yann LeCun is very bullish on GANs; lots of work on
GANs from FAIR, etc.3
• Cool extensions, e.g. GA what-were network4
• Mostly used in CV for image generation;
discriminator can be used as feature extractor
• Less work in NLP and other areas; promising
directions at GAN workshop (1, 2, 3, 4)
1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf
2https://github.com/soumith/ganhacks
3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view
4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf
5https://arxiv.org/abs/1611.01144
Building applications with DL
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Building applications with DL
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
RNNs
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• 20 year
anniversary of
LSTM…
RNNs
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• 20 year
anniversary of
LSTM…

being rejected
from NIPS 1996

— perseverance
pays off
RNNs
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• RNN symposium
• Many papers on improving RNNs:
• Handling different time scales1,2
1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf
2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf
RNNs
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• RNN symposium
• Many papers on improving RNNs:
• Handling different time scales1,2
1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf
2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-
long-or-event-based-sequences.pdf
RNNs
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• RNN symposium
• Many papers on improving RNNs:
• Handling different time scales1,2
• Modelling uncertainty3
1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf
2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf
3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf
RNNs
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• RNN symposium
• Many papers on improving RNNs:
• Handling different time scales1,2
• Modelling uncertainty3
RNNs
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• RNN symposium
• Many papers on improving RNNs:
• Handling different time scales1,2
• Modelling uncertainty3
•General DL improvements
• Weight normalisation4
• Multinomial dropout5
•Edward Grefenstette’s talk at NAMPI workshop6
1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf
2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf
3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf
4https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural-
networks.pdf
5https://papers.nips.cc/paper/6561-improved-dropout-for-shallow-and-deep-learning.pdf
6https://uclmr.github.io/nampi/talk_slides/grefenstette_limitations_of_rnns.pdf
Improving classic algorithms
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Matrix completion: Non-convex objective for
matrix completion has no spurious local minima1
• Clustering: Seedings for large-scale k-means++
clustering2
1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf
2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
Improving classic algorithms
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Matrix completion: Non-convex objective for
matrix completion has no spurious local minima1
• Clustering: Seedings for large-scale k-means++
clustering2
• Clustering: Using queries to convey domain
information and reduce computational complexity
1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf
2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
Improving classic algorithms
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Matrix completion: Non-convex objective for
matrix completion has no spurious local minima1
• Clustering: Seedings for large-scale k-means++
clustering2
• Clustering: Using queries to convey domain
information and reduce computational complexity3
1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf
2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
3https://papers.nips.cc/paper/6449-clustering-with-same-cluster-queries.pdf
Reinforcement Learning
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• RL tutorial by Pieter Abbeel and John Schulman:

Slides1; practical advice2
• Best paper: Value Iteration Networks3
• Learns to plan; contribution: differentiable
approximation to classic algorithm via CNN
1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf
2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf
3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
Reinforcement Learning
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Reinforcement Learning
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• RL tutorial by Pieter Abbeel and John Schulman:

Slides1; practical advice2
• Best paper: Value Iteration Networks3
• Learns to plan; contribution: differentiable
approximation to classic algorithm via CNN
• New research environments:
• OpenAI’s Universe
• Deep Mind Lab
• FAIR’s Torchcraft
1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf
2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf
3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
Learning-to-learn / Meta-learning
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Several papers, e.g.

Learning to learn by gradient descent by gradient
descent1
• Meta-learning panel at RNN symposium
• Sutskever: No good meta-learning models so far
• Schmidhuber: Focus on universal model
• Focus at Neural Abstract Machines workshop
1https://papers.nips.cc/paper/6461-learning-to-learn-by-gradient-descent-by-gradient-descent.pdf
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Topic in keynote talks
• Yann LeCun: Focus on unsupervised learning
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Topic in keynote talks
• Yann LeCun: Focus on unsupervised learning
General AI
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Topic in keynote talks
• Yann LeCun: Focus on unsupervised learning
• Drew Purves: AI for environment, ground for
General AI
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Topic in keynote talks
• Yann LeCun: Focus on unsupervised learning
• Drew Purves: AI for environment, ground for
General AI
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Source: Drew Purves
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Topic in keynote talks
• Yann LeCun: Focus on unsupervised learning
• Drew Purves: AI for environment, ground for
General AI
• Saket Navlakha: Borrowing engineering
principles from the brain
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Topic in keynote talks
• Yann LeCun: Focus on unsupervised learning
• Drew Purves: AI for environment, ground for
General AI
• Saket Navlakha: Borrowing engineering
principles from the brain
General Artificial Intelligence
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Topic in keynote talks
• Yann LeCun: Focus on unsupervised learning
• Drew Purves: AI for environment, ground for
General AI
• Saket Navlakha: Borrowing engineering
principles from the brain
• Machine Intelligence workshop:
• FAIR’s CommAI-env
• Hierarchical representations, identifying patterns
• Panel: Unclear if focus on specific tasks brings us
closer to General AI.
NLP
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Variations of Neural MT
• Dual learning for MT (with RL)1
• Structured prediction with bandit feedback2
• Improvements to existing methods
• Supervised Word Mover’s Distance3
1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf
2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf
3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf
NLP
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Variations of Neural MT
• Dual learning for MT (with RL)1
• Structured prediction with bandit feedback2
• Improvements to existing methods
• Supervised Word Mover’s Distance3
• Modelling user reviews4
1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf
2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf
3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf
4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
NLP
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
NLP
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Variations of Neural MT
• Dual learning for MT (with RL)1
• Structured prediction with bandit feedback2
• Improvements to existing methods
• Supervised Word Mover’s Distance3
• Modelling user reviews4
• Dialogue modelling workshop:
• End-to-end systems, linguistics, and ML methods
1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf
2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf
3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf
4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
Miscellaneous
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Schmidhuber is

everywhere!
Miscellaneous
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Schmidhuber is

everywhere!
• Robotics
Miscellaneous
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Schmidhuber is

everywhere!
• Robotics
• Apple starts
publishing
Miscellaneous
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Schmidhuber is

everywhere!
• Robotics
• Apple starts
publishing
• AI startups:
• Geometric
Intelligence
Miscellaneous
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
• Schmidhuber is

everywhere!
• Robotics
• Apple starts
publishing
• AI startups:
• Geometric
Intelligence
• RocketAI
“launch” party
@seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
Thanks for your attention!
Questions?

More Related Content

What's hot

Document Classification with Neo4j
Document Classification with Neo4jDocument Classification with Neo4j
Document Classification with Neo4jKenny Bastani
 
Intro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringIntro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringTraian Rebedea
 
From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?Constantin Orasan
 
Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Tasnim Ara Islam
 
Practical machine learning - Part 1
Practical machine learning - Part 1Practical machine learning - Part 1
Practical machine learning - Part 1Traian Rebedea
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesDeep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesMatthew Lease
 
Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Rich Heimann
 
Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Rich Heimann
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Rich Heimann
 
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Ahmed Magdy Ezzeldin, MSc.
 
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Association for Computational Linguistics
 
[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systems[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systemsQi He
 
Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)YerevaNN research lab
 
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systemsQi He
 
SF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonSF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonPaco Nathan
 
Recurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovRecurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovBhaskar Mitra
 
Text, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsText, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsFred Morstatter
 
Microsoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisMicrosoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisAlex Polozov
 

What's hot (20)

Document Classification with Neo4j
Document Classification with Neo4jDocument Classification with Neo4j
Document Classification with Neo4j
 
Intro to Deep Learning for Question Answering
Intro to Deep Learning for Question AnsweringIntro to Deep Learning for Question Answering
Intro to Deep Learning for Question Answering
 
From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?From TREC to Watson: is open domain question answering a solved problem?
From TREC to Watson: is open domain question answering a solved problem?
 
Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.Presentation of Domain Specific Question Answering System Using N-gram Approach.
Presentation of Domain Specific Question Answering System Using N-gram Approach.
 
Practical machine learning - Part 1
Practical machine learning - Part 1Practical machine learning - Part 1
Practical machine learning - Part 1
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesDeep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
 
Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)Data Tactics Data Science Brown Bag (April 2014)
Data Tactics Data Science Brown Bag (April 2014)
 
Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)Data Tactics Analytics Brown Bag (Aug 22, 2013)
Data Tactics Analytics Brown Bag (Aug 22, 2013)
 
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
Big Social Data: The Spatial Turn in Big Data (Video available soon on YouTube)
 
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
Arabic Question Answering: Challenges, Tasks, Approaches, Test-sets, Tools, A...
 
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
Philippe Langlais - 2017 - Users and Data: The Two Neglected Children of Bili...
 
[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systems[KDD 2018 tutorial] End to-end goal-oriented question answering systems
[KDD 2018 tutorial] End to-end goal-oriented question answering systems
 
Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)Sentence representations and question answering (YerevaNN)
Sentence representations and question answering (YerevaNN)
 
Aspects of NLP Practice
Aspects of NLP PracticeAspects of NLP Practice
Aspects of NLP Practice
 
Practical NLP with Lisp
Practical NLP with LispPractical NLP with Lisp
Practical NLP with Lisp
 
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
[AAAI 2019 tutorial] End-to-end goal-oriented question answering systems
 
SF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in PythonSF Python Meetup: TextRank in Python
SF Python Meetup: TextRank in Python
 
Recurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas MikolovRecurrent networks and beyond by Tomas Mikolov
Recurrent networks and beyond by Tomas Mikolov
 
Text, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical TopicsText, Topics, and Turkers: A Consensus Measure for Statistical Topics
Text, Topics, and Turkers: A Consensus Measure for Statistical Topics
 
Microsoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program SynthesisMicrosoft PROSE SDK: A Framework for Inductive Program Synthesis
Microsoft PROSE SDK: A Framework for Inductive Program Synthesis
 

Similar to NIPS 2016 Highlights - Sebastian Ruder

Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05tracykteal
 
Building a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result DiversificationBuilding a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result DiversificationKe Tao
 
DIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for LibrariansDIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for LibrariansDigCurV
 
DIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for LibrariansDIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for LibrariansEDINA, University of Edinburgh
 
DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14Carly Strasser
 
Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...LIBER Europe
 
Mid-term presentation.pdf
Mid-term presentation.pdfMid-term presentation.pdf
Mid-term presentation.pdfZixunZhou
 
Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24tracykteal
 
Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.Lynn Connaway
 
Librarians Learning Online to Teach Online
Librarians Learning Online to Teach OnlineLibrarians Learning Online to Teach Online
Librarians Learning Online to Teach OnlineArden Kirkland
 
Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013Shawna Reibling
 
OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.OCLC
 
Assessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finlandAssessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finlandPetri Linna
 
Building Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked DataBuilding Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked DataMarcia Zeng
 

Similar to NIPS 2016 Highlights - Sebastian Ruder (20)

Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05Data carpentry ndic-2015-05-05
Data carpentry ndic-2015-05-05
 
DMPTool Webinar 2: Data Management Resources You Can Use
DMPTool Webinar 2: Data Management Resources You Can UseDMPTool Webinar 2: Data Management Resources You Can Use
DMPTool Webinar 2: Data Management Resources You Can Use
 
Building a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result DiversificationBuilding a Microblog Corpus for Search Result Diversification
Building a Microblog Corpus for Search Result Diversification
 
DIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for LibrariansDIY’ Research Data Management Training Kit for Librarians
DIY’ Research Data Management Training Kit for Librarians
 
DIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for LibrariansDIY Research Data Management Training Kit for Librarians
DIY Research Data Management Training Kit for Librarians
 
Exploratory Analysis of User Data
Exploratory Analysis of User DataExploratory Analysis of User Data
Exploratory Analysis of User Data
 
DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14DMPTool for IMLS #WebWise14
DMPTool for IMLS #WebWise14
 
Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...Working With Researchers on Data Management and Workflows for Archiving and S...
Working With Researchers on Data Management and Workflows for Archiving and S...
 
Mid-term presentation.pdf
Mid-term presentation.pdfMid-term presentation.pdf
Mid-term presentation.pdf
 
Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24Data carpentry replicathon_2017-03-24
Data carpentry replicathon_2017-03-24
 
Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.Digging into assessment data: Tips, tricks, and tools of the trade.
Digging into assessment data: Tips, tricks, and tools of the trade.
 
Librarians Learning Online to Teach Online
Librarians Learning Online to Teach OnlineLibrarians Learning Online to Teach Online
Librarians Learning Online to Teach Online
 
Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013Research culture presentation Sept 4, 2013
Research culture presentation Sept 4, 2013
 
OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.OCLC Research Update at ALA Chicago. June 26, 2017.
OCLC Research Update at ALA Chicago. June 26, 2017.
 
DMPTool Webinar 3: Customizing the DMPTool
DMPTool Webinar 3: Customizing the DMPToolDMPTool Webinar 3: Customizing the DMPTool
DMPTool Webinar 3: Customizing the DMPTool
 
Research Data Management and Spatial Data
Research Data Management and Spatial DataResearch Data Management and Spatial Data
Research Data Management and Spatial Data
 
Assessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finlandAssessment of social media skills among vocational teachers in finland
Assessment of social media skills among vocational teachers in finland
 
Charleston usa
Charleston usaCharleston usa
Charleston usa
 
Building Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked DataBuilding Learning Modules around the Competency Index for Linked Data
Building Learning Modules around the Competency Index for Linked Data
 
DMPTool Webinar 11: Complementary Tools
DMPTool Webinar 11: Complementary ToolsDMPTool Webinar 11: Complementary Tools
DMPTool Webinar 11: Complementary Tools
 

More from Sebastian Ruder

Frontiers of Natural Language Processing
Frontiers of Natural Language ProcessingFrontiers of Natural Language Processing
Frontiers of Natural Language ProcessingSebastian Ruder
 
Strong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain ShiftStrong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain ShiftSebastian Ruder
 
On the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionOn the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionSebastian Ruder
 
Neural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain ShiftNeural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain ShiftSebastian Ruder
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep LearningSebastian Ruder
 
Human Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila CastilhoHuman Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila CastilhoSebastian Ruder
 
Machine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian MihaiMachine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian MihaiSebastian Ruder
 
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana IfrimHashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana IfrimSebastian Ruder
 
Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...Sebastian Ruder
 
Spoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer GilmartinSpoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer GilmartinSebastian Ruder
 
Modeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John GloverModeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John GloverSebastian Ruder
 
Multi-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer CalixtoMulti-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer CalixtoSebastian Ruder
 
Funded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIENFunded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIENSebastian Ruder
 
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...Sebastian Ruder
 
Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...Sebastian Ruder
 
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Sebastian Ruder
 
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...Sebastian Ruder
 
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment AnalysisA Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment AnalysisSebastian Ruder
 
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...Sebastian Ruder
 

More from Sebastian Ruder (19)

Frontiers of Natural Language Processing
Frontiers of Natural Language ProcessingFrontiers of Natural Language Processing
Frontiers of Natural Language Processing
 
Strong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain ShiftStrong Baselines for Neural Semi-supervised Learning under Domain Shift
Strong Baselines for Neural Semi-supervised Learning under Domain Shift
 
On the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionOn the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary Induction
 
Neural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain ShiftNeural Semi-supervised Learning under Domain Shift
Neural Semi-supervised Learning under Domain Shift
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep Learning
 
Human Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila CastilhoHuman Evaluation: Why do we need it? - Dr. Sheila Castilho
Human Evaluation: Why do we need it? - Dr. Sheila Castilho
 
Machine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian MihaiMachine intelligence in HR technology: resume analysis at scale - Adrian Mihai
Machine intelligence in HR technology: resume analysis at scale - Adrian Mihai
 
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana IfrimHashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
Hashtagger+: Real-time Social Tagging of Streaming News - Dr. Georgiana Ifrim
 
Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...Making sense of word senses: An introduction to word-sense disambiguation and...
Making sense of word senses: An introduction to word-sense disambiguation and...
 
Spoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer GilmartinSpoken Dialogue Systems and Social Talk - Emer Gilmartin
Spoken Dialogue Systems and Social Talk - Emer Gilmartin
 
Modeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John GloverModeling documents with Generative Adversarial Networks - John Glover
Modeling documents with Generative Adversarial Networks - John Glover
 
Multi-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer CalixtoMulti-modal Neural Machine Translation - Iacer Calixto
Multi-modal Neural Machine Translation - Iacer Calixto
 
Funded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIENFunded PhD/MSc. Opportunities at AYLIEN
Funded PhD/MSc. Opportunities at AYLIEN
 
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
FaDA: Fast document aligner with word embedding - Pintu Lohar, Debasis Gangul...
 
Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...Transformation Functions for Text Classification: A case study with StackOver...
Transformation Functions for Text Classification: A case study with StackOver...
 
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
 
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
Idiom Token Classification using Sentential Distributed Semantics (Giancarlo ...
 
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment AnalysisA Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis
 
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams...
 

Recently uploaded

Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Sérgio Sacani
 
igcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdf
igcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdfigcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdf
igcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdfTeenaSheikh
 
Understanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdfUnderstanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdfHabibouKarbo
 
Remote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationRemote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationfahad Alotaibiu
 
Physical pharmacy unit 1 solubility of drugs.pptx
Physical pharmacy unit 1 solubility of drugs.pptxPhysical pharmacy unit 1 solubility of drugs.pptx
Physical pharmacy unit 1 solubility of drugs.pptxchetanvgh
 
Environmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxEnvironmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxpriyankatabhane
 
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasBACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasChayanika Das
 
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...jana861314
 
Mycobacterium Mycobacterium tuberculosis
Mycobacterium Mycobacterium tuberculosisMycobacterium Mycobacterium tuberculosis
Mycobacterium Mycobacterium tuberculosisKARTHIK REDDY C A
 
DERMATOMYCOSIS Epidermophyton KKR.pptx
DERMATOMYCOSIS Epidermophyton   KKR.pptxDERMATOMYCOSIS Epidermophyton   KKR.pptx
DERMATOMYCOSIS Epidermophyton KKR.pptxKARTHIK REDDY C A
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxGiDMOh
 
FBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptxFBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptxPayal Shrivastava
 
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...Chayanika Das
 
Pharmaceutical warehouse audit using eAuditor Audits & Inspections .pptx
Pharmaceutical warehouse audit using eAuditor Audits & Inspections .pptxPharmaceutical warehouse audit using eAuditor Audits & Inspections .pptx
Pharmaceutical warehouse audit using eAuditor Audits & Inspections .pptxeAuditor Audits & Inspections
 
chapter 28 Reproductive System Power Point
chapter 28 Reproductive System Power Pointchapter 28 Reproductive System Power Point
chapter 28 Reproductive System Power Pointchelseakland
 
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsTotal Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsMarkus Roggen
 

Recently uploaded (20)

Introduction Classification Of Alkaloids
Introduction Classification Of AlkaloidsIntroduction Classification Of Alkaloids
Introduction Classification Of Alkaloids
 
TISSUE TYPING .pptx
TISSUE TYPING                       .pptxTISSUE TYPING                       .pptx
TISSUE TYPING .pptx
 
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
 
igcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdf
igcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdfigcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdf
igcse_edexcel_math-_gold_qp1_-igcse_9-1_.pdf
 
Understanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdfUnderstanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdf
 
Battery Reasearch Reagents from TCI Chemicals
Battery Reasearch Reagents from TCI ChemicalsBattery Reasearch Reagents from TCI Chemicals
Battery Reasearch Reagents from TCI Chemicals
 
Remote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationRemote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformation
 
Physical pharmacy unit 1 solubility of drugs.pptx
Physical pharmacy unit 1 solubility of drugs.pptxPhysical pharmacy unit 1 solubility of drugs.pptx
Physical pharmacy unit 1 solubility of drugs.pptx
 
Environmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxEnvironmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptx
 
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasBACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
 
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
 
Mycobacterium Mycobacterium tuberculosis
Mycobacterium Mycobacterium tuberculosisMycobacterium Mycobacterium tuberculosis
Mycobacterium Mycobacterium tuberculosis
 
DERMATOMYCOSIS Epidermophyton KKR.pptx
DERMATOMYCOSIS Epidermophyton   KKR.pptxDERMATOMYCOSIS Epidermophyton   KKR.pptx
DERMATOMYCOSIS Epidermophyton KKR.pptx
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptx
 
FBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptxFBI Profiling - Forensic Psychology.pptx
FBI Profiling - Forensic Psychology.pptx
 
TOPIC OF ELECTROMAGNETISM.PHYSICS NOTES FORM 4
TOPIC OF ELECTROMAGNETISM.PHYSICS NOTES FORM 4TOPIC OF ELECTROMAGNETISM.PHYSICS NOTES FORM 4
TOPIC OF ELECTROMAGNETISM.PHYSICS NOTES FORM 4
 
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
 
Pharmaceutical warehouse audit using eAuditor Audits & Inspections .pptx
Pharmaceutical warehouse audit using eAuditor Audits & Inspections .pptxPharmaceutical warehouse audit using eAuditor Audits & Inspections .pptx
Pharmaceutical warehouse audit using eAuditor Audits & Inspections .pptx
 
chapter 28 Reproductive System Power Point
chapter 28 Reproductive System Power Pointchapter 28 Reproductive System Power Point
chapter 28 Reproductive System Power Point
 
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsTotal Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
 

NIPS 2016 Highlights - Sebastian Ruder

  • 1. Sebastian Ruder
 PhD Candidate, Insight Centre
 Research Scientist, AYLIEN @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup NIPS 2016 Highlights
  • 2. Agenda 1. NIPS overview 2. Generative Adversarial Networks 3. Building applications with Deep Learning 4. RNNs 5. Improving classic algorithms 6. Reinforcement Learning 7. Learning-to-learn / Meta-learning 8. General AI 9. NLP 10. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 3. NIPS Overview @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Source: http://www.tml.cs.uni-tuebingen.de/team/luxburg/misc/nips2016/index.php
  • 4. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks
  • 5. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 6. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 • Yann LeCun is very bullish on GANs; lots of work on GANs from FAIR, etc.3 • Cool extensions, e.g. GA what-were network4 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks 3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view 4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf
  • 7. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 8. Generative Adversarial Networks @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Ian Goodfellow’s GAN tutorial1 • Soumith Chintala’s “How to train your GAN” talk2 • Yann LeCun is very bullish on GANs; lots of work on GANs from FAIR, etc.3 • Cool extensions, e.g. GA what-were network4 • Mostly used in CV for image generation; discriminator can be used as feature extractor • Less work in NLP and other areas; promising directions at GAN workshop (1, 2, 3, 4) 1http://www.iangoodfellow.com/slides/2016-12-04-NIPS.pdf 2https://github.com/soumith/ganhacks 3https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view 4https://papers.nips.cc/paper/6111-learning-what-and-where-to-draw.pdf 5https://arxiv.org/abs/1611.01144
  • 9. Building applications with DL @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 10. Building applications with DL @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 11. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • 20 year anniversary of LSTM…
  • 12. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • 20 year anniversary of LSTM…
 being rejected from NIPS 1996
 — perseverance pays off
  • 13. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf
  • 14. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for- long-or-event-based-sequences.pdf
  • 15. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf 3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf
  • 16. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3
  • 17. RNNs @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RNN symposium • Many papers on improving RNNs: • Handling different time scales1,2 • Modelling uncertainty3 •General DL improvements • Weight normalisation4 • Multinomial dropout5 •Edward Grefenstette’s talk at NAMPI workshop6 1https://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf 2https://papers.nips.cc/paper/6310-phased-lstm-accelerating-recurrent-network-training-for-long-or-event-based-sequences.pdf 3https://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf 4https://papers.nips.cc/paper/6114-weight-normalization-a-simple-reparameterization-to-accelerate-training-of-deep-neural- networks.pdf 5https://papers.nips.cc/paper/6561-improved-dropout-for-shallow-and-deep-learning.pdf 6https://uclmr.github.io/nampi/talk_slides/grefenstette_limitations_of_rnns.pdf
  • 18. Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
  • 19. Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 • Clustering: Using queries to convey domain information and reduce computational complexity 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf
  • 20. Improving classic algorithms @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Matrix completion: Non-convex objective for matrix completion has no spurious local minima1 • Clustering: Seedings for large-scale k-means++ clustering2 • Clustering: Using queries to convey domain information and reduce computational complexity3 1https://papers.nips.cc/paper/6048-matrix-completion-has-no-spurious-local-minimum.pdf 2https://papers.nips.cc/paper/6478-fast-and-provably-good-seedings-for-k-means.pdf 3https://papers.nips.cc/paper/6449-clustering-with-same-cluster-queries.pdf
  • 21. Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RL tutorial by Pieter Abbeel and John Schulman:
 Slides1; practical advice2 • Best paper: Value Iteration Networks3 • Learns to plan; contribution: differentiable approximation to classic algorithm via CNN 1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf 2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf 3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
  • 22. Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 23. Reinforcement Learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • RL tutorial by Pieter Abbeel and John Schulman:
 Slides1; practical advice2 • Best paper: Value Iteration Networks3 • Learns to plan; contribution: differentiable approximation to classic algorithm via CNN • New research environments: • OpenAI’s Universe • Deep Mind Lab • FAIR’s Torchcraft 1http://people.eecs.berkeley.edu/~pabbeel/nips-tutorial-policy-optimization-Schulman-Abbeel.pdf 2http://rll.berkeley.edu/deeprlcourse/docs/nuts-and-bolts.pdf 3https://papers.nips.cc/paper/6046-value-iteration-networks.pdf
  • 24. Learning-to-learn / Meta-learning @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Several papers, e.g.
 Learning to learn by gradient descent by gradient descent1 • Meta-learning panel at RNN symposium • Sutskever: No good meta-learning models so far • Schmidhuber: Focus on universal model • Focus at Neural Abstract Machines workshop 1https://papers.nips.cc/paper/6461-learning-to-learn-by-gradient-descent-by-gradient-descent.pdf
  • 25. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning
  • 26. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning
  • 27. General AI @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 28. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI
  • 29. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI
  • 30. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Source: Drew Purves
  • 31. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain
  • 32. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain
  • 33. General Artificial Intelligence @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Topic in keynote talks • Yann LeCun: Focus on unsupervised learning • Drew Purves: AI for environment, ground for General AI • Saket Navlakha: Borrowing engineering principles from the brain • Machine Intelligence workshop: • FAIR’s CommAI-env • Hierarchical representations, identifying patterns • Panel: Unclear if focus on specific tasks brings us closer to General AI.
  • 34. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf
  • 35. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 36. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 37. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 • Modelling user reviews4 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf 4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
  • 38. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 39. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup
  • 40. NLP @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Variations of Neural MT • Dual learning for MT (with RL)1 • Structured prediction with bandit feedback2 • Improvements to existing methods • Supervised Word Mover’s Distance3 • Modelling user reviews4 • Dialogue modelling workshop: • End-to-end systems, linguistics, and ML methods 1https://papers.nips.cc/paper/6469-dual-learning-for-machine-translation.pdf 2 https://papers.nips.cc/paper/6134-stochastic-structured-prediction-under-bandit-feedback.pdf 3 https://papers.nips.cc/paper/6139-supervised-word-movers-distance.pdf 4 https://papers.nips.cc/paper/6362-beyond-exchangeability-the-chinese-voting-process.pdf
  • 41. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere!
  • 42. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics
  • 43. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing
  • 44. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing • AI startups: • Geometric Intelligence
  • 45. Miscellaneous @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup • Schmidhuber is
 everywhere! • Robotics • Apple starts publishing • AI startups: • Geometric Intelligence • RocketAI “launch” party
  • 46. @seb_ruder | @_aylien |13.12.16 | 4th NLP Dublin Meetup Thanks for your attention! Questions?