Transfer Learning -- The Next Frontier for Machine Learning

Sebastian Ruder 
Research Scientist, AYLIEN
PhD Candidate, Insight Centre
@seb_ruder |01.03.17 | LinkedIn Tech Talk
Transfer Learning —
The Next Frontier for ML

Agenda
1. What is Transfer Learning?
2. Why Transfer Learning now?
3. Transfer Learning in practice
4. Transfer Learning for NLP
5. Our research
6. Opportunities and directions

What is Transfer Learning?
Model A Model B
Task / domain A
Task / domain B
Traditional ML
Training and
evaluation on the same
task or domain.

What is Transfer Learning?
@seb_ruder |
Knowledge
Model
Source task /
domain Target task /
domain
Transfer learning
Storing knowledge gained solving
one problem and applying it to a
different but related problem.
Model
01.03.17 | LinkedIn Tech Talk

“Transfer learning
will be the next
driver of ML
success.”
Andrew Ng,
NIPS 2016 keynote
@seb_ruder |

Why Transfer Learning now?
@seb_ruder |
Supervised learning
Transfer learning
Unsupervised learning
Reinforcement learning
2016Time
Commercial
success
Drivers of ML success in industry
- Andrew Ng, NIPS 2016 keynote

Why Transfer Learning now?
@seb_ruder |
1. Learn very accurate input-output mapping
2. Maturity of ML models
- Computer vision (5% error on ImageNet)
-Automatic speech recognition (3x faster than
typing, 20% more accurate1)
3. Large-scale deployment & adoption of ML models
-Google’s NMT System2
1 Ruan, S., Wobbrock, J. O., Liou, K., Ng, A., & Landay, J. (2016). Speech Is 3x Faster than Typing for English
and Mandarin Text Entry on Mobile Devices. arXiv preprint arXiv:1608.07323.
2 Wu, Y., Schuster, M., Chen, Z., Le, Q. V, Norouzi, M., Macherey, W., … Dean, J. (2016). Google’s Neural
Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv preprint
arXiv:1609.08144.
Huge reliance on labeled data  
Novel tasks / domains without (labeled) data

Transfer Learning in practice
@seb_ruder |
• Train new model on features
of large model trained on
ImageNet3
• Train model to confuse source
and target domains4
• Train model on domain-
invariant representations5,6
3 Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014). CNN features off-the-shelf: An astounding baseline for
recognition. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 512–519.
4 Ganin, Y., & Lempitsky, V. (2015). Unsupervised Domain Adaptation by Backpropagation. Proceedings of the 32nd
International Conference on Machine Learning., 37.
5 Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., & Erhan, D. (2016). Domain Separation Networks. NIPS 2016.
6 Sener, O., Song, H. O., Saxena, A., & Savarese, S. (2016). Learning Transferrable Representations for Unsupervised Domain
Adaptation. NIPS 2016.
Computer vision

Transfer Learning in practice
@seb_ruder |
• Progressive Neural 
Networks7 have
access to weights
from trained models
• PathNet8 learns
weight paths via a
genetic algorithm
7 Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., … Deepmind, G.
(2016). Progressive Neural Networks. arXiv preprint arXiv:1606.04671.
8 Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A. A., … Wierstra, D. (2017). PathNet:
Evolution Channels Gradient Descent in Super Neural Networks. In arXiv preprint arXiv:1701.08734.
Reinforcement learning

Transfer Learning for NLP
@seb_ruder |
• Task and domainT D
DS 6= DT TS 6= TT
A (slightly) more technical deﬁnition
• Domain where
- : feature space, e.g. BOW representations
- : e.g. distribution over terms in documents
D = {X, P(X)}
X
P(X)
• Task where
- : label space, e.g. true/false labels
- : learned mapping from samples to labels
T = {Y, P(Y |X)}
Y
P(Y |X)
• Transfer learning: 
Learning when or

@seb_ruder |
Transfer scenarios
1. : Different topics, text types, etc. 
2. : Different languages. 
3. : Unbalanced classes. 
4. : Different tasks.
P(XS) 6= P(XT )
XS 6= XT
P(YS|XS) 6= P(YT |XT )
YS 6= YT

@seb_ruder |
Current status
• Not as straightforward as in CV
- No universal deep features
• However: “Simple” transfer through word
embeddings is pervasive
• History of research for task-speciﬁc transfer, e.g.
sentiment analysis, POS tagging leveraging NLP
phenomena such as structured features, sentiment
words, etc.
• Few research on transfer between tasks
• More recently: representation-based research

Our research
@seb_ruder |
Research focus
Finding better ways to transfer knowledge to new
domains, tasks, and languages that
1. perform well in large-scale settings and real-
world applications;
2. are applicable to many tasks and models.
Current focus:
: Training and test distributions are
different.
P(XS) 6= P(XT )

Our research
@seb_ruder |
Training and test distributions are different.
Different text types. Different accents/ages.
Different topics/categories.
Performance drop or even collapse is inevitable.

Our research
@seb_ruder |
Transfer learning challenges in real-world applications
1. Domains are not well-deﬁned, but fuzzy and
conﬂate many factors. 
 
 
2. One-to-one adaptation is rare and many source
domains are generally available.
3. Models need to be adapted frequently as
conditions change, new data becomes available, etc.
Language
socialfactors
genre
topic

Our research
@seb_ruder |
• Idea: Use distillation + insights from semi-supervised
learning to transfer knowledge from a single (a) and
multiple teachers (b) to a student model9.
(a) (b)
9 Ruder, S., Ghaffari, P., & Breslin, J. G. (2017). Knowledge Adaptation: Teaching to Adapt. In arXiv preprint arXiv:1702.02052.
How to adapt from large source domains?

Our research
@seb_ruder |
• Idea: Take into account diversity of training data to
select subsets (c) rather than an entire domain (a) or
individual examples (b)10.
10 Ruder, S., Ghaffari, P., & Breslin, J. G. (2017). Data Selection Strategies for Multi-Domain Sentiment Analysis. In arXiv preprint arXiv:1702.02426.
How to select data for adaptation?
(a) (b) (c)

Our research
@seb_ruder |
Opportunities and future directions
• Learn from past adaptation scenarios and
generalise across domains and tasks.
• Robust adaptation to non-English and low-
resource languages.
• Adaptation for novel tasks and more sophisticated
models, e.g. QA and memory networks.
• Transfer across tasks and leveraging knowledge
from related tasks.

References
@seb_ruder |
Image credit
• Google Research blog post11
• Mikolov, T., Joulin, A., & Baroni, M. (2015). A Roadmap towards
Machine Intelligence. arXiv preprint arXiv:1511.08130.
• Google Research blog post12
Our papers
• Ruder, S., Ghaffari, P., & Breslin, J. G. (2017). Knowledge Adaptation:
Teaching to Adapt. In arXiv preprint arXiv:1702.02052.
• Ruder, S., Ghaffari, P., & Breslin, J. G. (2017). Data Selection Strategies
for Multi-Domain Sentiment Analysis. In arXiv preprint arXiv:
1702.02426.
11 https://research.googleblog.com/2016/10/how-robots-can-acquire-new-skills-from.html
12 https://googleblog.blogspot.ie/2014/04/the-latest-chapter-for-self-driving-car.html

@seb_ruder |
Thanks for your attention!
Questions?

Transfer Learning -- The Next Frontier for Machine Learning

In this document

More Related Content

What's hot

Similar to Transfer Learning -- The Next Frontier for Machine Learning

More from Sebastian Ruder

Recently uploaded

Transfer Learning -- The Next Frontier for Machine Learning