[DL輪読会]Taskonomy: Disentangling Task Transfer Learning

•

4 likes•3,379 views

Deep Learning JP

2018/07/06 Deep Learning JP: http://deeplearning.jp/seminar-2/

Technology

DEEPLEARNINGJP
[DL Papers]
Taskonomy: Disentangling Task Transfer Learning
(CVPR2018)
MasashiYokota, RESTAR inc.
http://deeplearning.jp/
1

• 2 CBCAL :BG B B 2 2 B : : B B
• A 0 A : B : 1 A 1 :B :CB
8 , G:B . 1 C 1 : :
• 1G B C B : GL B : GL C C B : : :L
• /0 : G / D: J
• GGD G CBCAL CB

• 4 - 4 -2>D
E 4 2
2D Segm
2D Edge4
24 2

• > 1
• ( )
() )
t: target task, s: source task
Es: encoder, D: decoder,
L: loss func, ft(I): grand truth

• 2 F 1 2 >
- 2 1 F 1 > F
• 1 ES F TE 2 1 1
1

) ( (
• ) ( 4 ( 1
4 )) ( s
r c
o
• 1 !"
!# g a t
1
gu t1 c L
e 4

) ( (
s1 s2 s3 s4
s1 1 0.8 0.5 0.9
s2 0.2 1 0.7 0.5
s3 0.5 0.3 1 0.8
s4 0.1 0.5 0.2 1
s1 s2 s3 s4
s1 1 0.2 0.5 0.1
s2 0.8 1 0.3 0.5
s3 0.5 0.7 1 0.2
s4 0.9 0.5 0.8 1
s1 s2 s3 s4
s1 1/1 0.8/
0.2
0.5/
0.5
0.9/
0.1
s2 0.2/
0.8
1/1 0.7/
0.3
0.5/
0.5
s3 0.5/
0.5
0.3/
0.7
1/1 0.8/
0.2
s4 0.1/
0.9
1 0.2/
0.8
1/1
s1 s2 s3 s4
s1 1 4 1 9
s2 0.25 1 7/3 1
s3 1 3/7 1 4
s4 1/9 1 0.25 1
Source target
1
source W W 5
W 5

) ( (
PE
s1 0.47
s2 0.20
s3 0.25
s4 0.08
s1 s2 s3 s4
t1 0.47 0.20 0.25 0.08
t2
t3
t4
s1 s2 s3 s4
s1 1 4 1 9
s2 0.25 1 7/3 1
s3 1 3/7 1 4
s4 1/9 1 0.25 1
6
6
1
Normalize
1
Target

) ( (
PE
s1 0.47
s2 0.20
s3 0.25
s4 0.08
PE
s1 0.47
s2 0.20
s3 0.25
s4 0.08
PE
s1 0.47
s2 0.20
s3 0.25
s4 0.08
PE
s1 0.47
s2 0.20
s3 0.25
s4 0.08
s1 s2 s3 s4
t1 0.47 0.20 0.25 0.08
t2 0.10 0.42 0.18 0.30
t3 0.21 0.11 0.33 0.35
t4 0.01 0.20 0.29 0.50
s1 s2 s3 s4
s1 1 4 1 9
s2 0.25 1 7/3 1
s3 1 3/7 1 4
s4 1/9 1 0.25 1
s1 s2 s3 s4
s1 1 4 1 9
s2 0.25 1 7/3 1
s3 1 3/7 1 4
s4 1/9 1 0.25 1
s1 s2 s3 s4
s1 1 4 1 9
s2 0.25 1 7/3 1
s3 1 3/7 1 4
s4 1/9 1 0.25 1
s1 s2 s3 s4
s1 1 4 1 9
s2 0.25 1 7/3 1
s3 1 3/7 1 4
s4 1/9 1 0.25 1
Target
7 1
1
Normalize
target

• V ( 35 :
45 D 2 65 5
7 45 D !"
# 2 65 5
• 2 65 6 3 8 U
0 ) ( TP U
• ,2 2 5
– 28 2 538683 5 UG
– 12:842 8 ( 2 65 6 3 8 UG
– 5 ) G

:
• 5 .5 D [ O
Q
. ).
• ( 2 G 5 .5
O Q G S
:

• 2 .3
.
• 6 .3 3 6
0 6 3
• 6 3 0 6
0
6 2

What's hot

【DL輪読会】Perceiver io a general architecture for structured inputs & outputs Deep Learning JP

Swin Transformer (ICCV'21 Best Paper) を完璧に理解する資料Yusuke Uchida

SSII2022 [SS2] 少ないデータやラベルを効率的に活用する機械学習技術〜足りない情報をどのように補うか？〜SSII

[DL輪読会]ICLR2020の分布外検知速報Deep Learning JP

[DL輪読会]AdaShare: Learning What To Share For Efficient Deep Multi-Task LearningDeep Learning JP

[DL輪読会]Control as Inferenceと発展Deep Learning JP

ConvNetの歴史とResNet亜種、ベストプラクティスYusuke Uchida

【DL輪読会】Factory: Fast Contact for Robotic AssemblyDeep Learning JP

Attentionの基礎からTransformerの入門までAGIRobots

【DL輪読会】Mastering Diverse Domains through World ModelsDeep Learning JP

深層生成モデルと世界モデルMasahiro Suzuki

[DL輪読会]World ModelsDeep Learning JP

[DL輪読会]Learning Transferable Visual Models From Natural Language SupervisionDeep Learning JP

【DL輪読会】A Time Series is Worth 64 Words: Long-term Forecasting with TransformersDeep Learning JP

畳み込みLstmtak9029

ゼロから始める深層強化学習（NLP2018講演資料）/ Introduction of Deep Reinforcement LearningPreferred Networks

backbone としての timm 入門Takuji Tahara

Layer Normalization@NIPS+読み会・関西Keigo Nishida

不老におけるOptunaを利用した分散ハイパーパラメータ最適化 - 今村秀明（名古屋大学 Optuna講習会）Preferred Networks

近年のHierarchical Vision TransformerYusuke Uchida

What's hot (20)

【DL輪読会】Perceiver io a general architecture for structured inputs & outputs

Swin Transformer (ICCV'21 Best Paper) を完璧に理解する資料

SSII2022 [SS2] 少ないデータやラベルを効率的に活用する機械学習技術〜足りない情報をどのように補うか？〜

[DL輪読会]ICLR2020の分布外検知速報

[DL輪読会]AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning

[DL輪読会]Control as Inferenceと発展

ConvNetの歴史とResNet亜種、ベストプラクティス

【DL輪読会】Factory: Fast Contact for Robotic Assembly

Attentionの基礎からTransformerの入門まで

【DL輪読会】Mastering Diverse Domains through World Models

深層生成モデルと世界モデル

[DL輪読会]World Models

[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision

【DL輪読会】A Time Series is Worth 64 Words: Long-term Forecasting with Transformers

畳み込みLstm

ゼロから始める深層強化学習（NLP2018講演資料）/ Introduction of Deep Reinforcement Learning

backbone としての timm 入門

Layer Normalization@NIPS+読み会・関西

不老におけるOptunaを利用した分散ハイパーパラメータ最適化 - 今村秀明（名古屋大学 Optuna講習会）

近年のHierarchical Vision Transformer

More from Deep Learning JP

【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving PlannersDeep Learning JP

【DL輪読会】事前学習用データセットについてDeep Learning JP

【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...Deep Learning JP

【DL輪読会】Zero-Shot Dual-Lens Super-ResolutionDeep Learning JP

【DL輪読会】BloombergGPT: A Large Language Model for Finance arxivDeep Learning JP

【DL輪読会】マルチモーダル LLMDeep Learning JP

【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...Deep Learning JP

【DL輪読会】AnyLoc: Towards Universal Visual Place RecognitionDeep Learning JP

【DL輪読会】Can Neural Network Memorization Be Localized?Deep Learning JP

【DL輪読会】Hopfield network　関連研究についてDeep Learning JP

【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )Deep Learning JP

【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...Deep Learning JP

【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"Deep Learning JP

【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "Deep Learning JP

【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat ModelsDeep Learning JP

【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"Deep Learning JP

【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...Deep Learning JP

【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...Deep Learning JP

【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...Deep Learning JP

【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...Deep Learning JP

More from Deep Learning JP (20)

【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners

【DL輪読会】事前学習用データセットについて

【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...

【DL輪読会】Zero-Shot Dual-Lens Super-Resolution

【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv

【DL輪読会】マルチモーダル LLM

【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...

【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition

【DL輪読会】Can Neural Network Memorization Be Localized?

【DL輪読会】Hopfield network　関連研究について

【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )

【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...

【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"

【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "

【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models

【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"

【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...

【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...

【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...

【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...

[DL輪読会]Taskonomy: Disentangling Task Transfer Learning

1. DEEPLEARNINGJP [DL Papers] Taskonomy: Disentangling Task Transfer Learning (CVPR2018) MasashiYokota, RESTAR inc. http://deeplearning.jp/ 1

2. • 2 CBCAL :BG B B 2 2 B : : B B • A 0 A : B : 1 A 1 :B :CB 8 , G:B . 1 C 1 : : • 1G B C B : GL B : GL C C B : : :L • /0 : G / D: J • GGD G CBCAL CB

3. • • 3 = 36 2

4. • 4 - 4 -2>D E 4 2 2D Segm 2D Edge4 24 2

5. • 5 5

6. ! 6

8. • 8 6 2

9. • 9 4 4 0 0

10. - • 1 - 0

11. • > 1 • ( ) () ) t: target task, s: source task Es: encoder, D: decoder, L: loss func, ft(I): grand truth

12. • 2 F 1 2 > - 2 1 F 1 > F • 1 ES F TE 2 1 1 1

13. ( ) ) :) ) ) • 1 A 1 3

14. ) ( ( • ) ( 4 ( 1 4 )) ( s r c o • 1 !" !# g a t 1 gu t1 c L e 4

15. ) ( ( s1 s2 s3 s4 s1 1 0.8 0.5 0.9 s2 0.2 1 0.7 0.5 s3 0.5 0.3 1 0.8 s4 0.1 0.5 0.2 1 s1 s2 s3 s4 s1 1 0.2 0.5 0.1 s2 0.8 1 0.3 0.5 s3 0.5 0.7 1 0.2 s4 0.9 0.5 0.8 1 s1 s2 s3 s4 s1 1/1 0.8/ 0.2 0.5/ 0.5 0.9/ 0.1 s2 0.2/ 0.8 1/1 0.7/ 0.3 0.5/ 0.5 s3 0.5/ 0.5 0.3/ 0.7 1/1 0.8/ 0.2 s4 0.1/ 0.9 1 0.2/ 0.8 1/1 s1 s2 s3 s4 s1 1 4 1 9 s2 0.25 1 7/3 1 s3 1 3/7 1 4 s4 1/9 1 0.25 1 Source target 1 source W W 5 W 5

16. ) ( ( PE s1 0.47 s2 0.20 s3 0.25 s4 0.08 s1 s2 s3 s4 t1 0.47 0.20 0.25 0.08 t2 t3 t4 s1 s2 s3 s4 s1 1 4 1 9 s2 0.25 1 7/3 1 s3 1 3/7 1 4 s4 1/9 1 0.25 1 6 6 1 Normalize 1 Target

17. ) ( ( PE s1 0.47 s2 0.20 s3 0.25 s4 0.08 PE s1 0.47 s2 0.20 s3 0.25 s4 0.08 PE s1 0.47 s2 0.20 s3 0.25 s4 0.08 PE s1 0.47 s2 0.20 s3 0.25 s4 0.08 s1 s2 s3 s4 t1 0.47 0.20 0.25 0.08 t2 0.10 0.42 0.18 0.30 t3 0.21 0.11 0.33 0.35 t4 0.01 0.20 0.29 0.50 s1 s2 s3 s4 s1 1 4 1 9 s2 0.25 1 7/3 1 s3 1 3/7 1 4 s4 1/9 1 0.25 1 s1 s2 s3 s4 s1 1 4 1 9 s2 0.25 1 7/3 1 s3 1 3/7 1 4 s4 1/9 1 0.25 1 s1 s2 s3 s4 s1 1 4 1 9 s2 0.25 1 7/3 1 s3 1 3/7 1 4 s4 1/9 1 0.25 1 s1 s2 s3 s4 s1 1 4 1 9 s2 0.25 1 7/3 1 s3 1 3/7 1 4 s4 1/9 1 0.25 1 Target 7 1 1 Normalize target

18. ( ) ) :) ) ) • P 8 : 1 2 P ) )0 ,), (

19. / a e I • 1 9 /d c9 • 1 2 / k • 1 on

20. • 0 target task AHP

21. • V ( 35 : 45 D 2 65 5 7 45 D !" # 2 65 5 • 2 65 6 3 8 U 0 ) ( TP U • ,2 2 5 – 28 2 538683 5 UG – 12:842 8 ( 2 65 6 3 8 UG – 5 ) G

22. • 2 2 ( • 2 2 2 2 • DC :E 2 )

23. • % ( 3 G G : G • 2 ) 3 G G G : G

24. E : • 4 1 • 6 4 ( 1) 0 ( 2 2 1

25. : • 5 .5 D [ O Q . ). • ( 2 G 5 .5 O Q G S :

26. - - : • 2 6 6

27. ) ) ) ( 7 7 7 2

28. & & & & N P & I E M • I 2M 8 N

29.

30. • 2 .3 . • 6 .3 3 6 0 6 3 • 6 3 0 6 0 6 2

[DL輪読会]Taskonomy: Disentangling Task Transfer Learning

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

More from Deep Learning JP

More from Deep Learning JP (20)

[DL輪読会]Taskonomy: Disentangling Task Transfer Learning