This document discusses transfer learning for deep learning applications. Transfer learning is the idea of utilizing knowledge gained from one task to help solve another related task. It can help overcome isolated learning and move towards artificial general intelligence. There are different strategies for transfer learning including inductive, unsupervised, and transductive transfer learning. Deep transfer learning techniques include fine-tuning pre-trained models on new tasks and domains. Transfer learning provides advantages such as helping solve complex problems with limited labeled data and making AI systems more generally applicable. However, challenges include avoiding negative transfer between domains and understanding transfer bounds.

1. Transfer Learning for Real-World
Applications using Deep Learning
Deep Learning on Steroids with the
Power of Knowledge Transfer!
Conducted by: Rahat Yasir
Microsoft MVP - AI
AI Engineer, Intact Financial Corp

2. Outline
• Session Agenda:
• Transfer Learning
• Strategies
• Type of transfer learning
• Transfer learning process
• Limitations and advantages

3. Human
learning
process
Know how to ride
a bicycle ⮫ Learn
how to ride a
motorbike
1
Know how to play
classic piano ⮫
Learn how to play
jazz piano
2
Know math and
statistics ⮫ Learn
machine learning
3

4. Motivation for
Transfer
Learning
• Transfer learning is the idea of
overcoming the isolated learning
paradigm and utilizing knowledge
acquired for one task to solve related
ones.

5. Artificial General Intelligence
– Instead of AI
• Given the craze for True Artificial General
Intelligence, transfer learning is something
which data scientists and researchers believe
can further our progress towards AGI –
Andrew NG
• After supervised learning — Transfer Learning
will be the next driver of ML commercial
success- Andrew NG

6. Traditional Learning vs Transfer Learning
diagram

7. Key approaches of Transfer Learning
1. What to
transfer
2. When
to transfer
3. How to
transfer

8. Transfer
Learning
Strategies
1. Inductive Transfer learning: Same
domain but different tasks.
2. Unsupervised Transfer Learning:
Source and target domains are
similar but tasks are different.
3. Transductive Transfer Learning:
Corresponding domains are different
but tasks are similar.

9. Transfer
Learning
Approaches
1. Instance transfer: Reusing knowledge from
the source domain to the target task .
2. Feature-representation transfer: This
approach aims to minimize domain
divergence and reduce error rates by
identifying good feature representations .
3. Parameter transfer: Models for related
tasks share some parameters or prior
distribution of hyperparameters
4. Relational-knowledge transfer: The
relational-knowledge transfer attempts to
handle non-IID data, such as data that is
not independent and identically
distributed, ex: social network data points.

10. Transfer Learning
for Deep Learning

11. Idea of Transfer Learning

12. Learning strategy for feature extraction

13. Fine tuning parameters for
CNN

14. freezing / fine tuning pre-trained models

15. Existing pre-
trained model use

16. Types of Deep Transfer Learning
Domain
Adaptation
Domain Confusion
Multitask Learning
– example and
diagram

17. Multitask Learning
• Example: Car accident
severity detection of next
weeks demo for insurance
purpose, car detection task,
damage detection task, car
model / brand detection,
car color, car damage
severity detection.

18. One-shot Learning

19. Zero-shot Learning

20. Applications of Transfer
Learning
• Transfer Learning for NLP
• Transfer Learning on Audio Data
• Transfer Learning for Generative Deep
Learning
• More complex Computer Vision problems
like Image Captioning

23. Transfer Learning Advantages
Helps solve
complex
problems
1
Helps to design
models with
little / no labeled
data
2
Knowledge
transfer is based
on domains and
tasks
3
Making AI more
general towards
Artificial General
Intelligence.
4

24. Transfer Learning Limitations and Challenges
Negative
Transfer
Transfer
Bounds

25. Thank You
Q &A