Presenting multi-task learning techniques for dense prediction tasks in computer vision

1. Multi-task Learning for
Dense Prediction Tasks
in Computer Vision
ARUN TALKAD

2. What is MTL?
Multi-Task Learning(MTL) is a family of techniques that are
designed to learn multiple tasks at the same time
MTL relies on identifying a common representation from
different datasets that can solve multiple tasks at different
layers of abstraction
Task specific layers are then applied on the data to ensure
that the model can solve multiple tasks at the same time

3. Dense Prediction and MTL
Dense Prediction refers to predicting different outputs at a
pixel level, thereby producing a ‘dense’ output.
Example – Model identifies Gleason pattern which indicates
abnormalities at a pixel level
MTL allows for such costly operations to be performed at a
much more cost-effective scale by learning a common
representation and building over with task specific layers

4. Soft and Hard Parameter Sharing
Soft Parameter Learning refers to
designs where each task is assigned
to its own set of parameters and a
feature sharing mechanism handles
the cross-talk between different
tasks.
Hard Parameter Learning refers to
the parameter set being divided
between shared and task-specific
parameters.

5. Encoder & Decoder-focused Models
Encoder-focused networks share the
task features in the encoding stage
while the processing is taken care of
in independent decoding heads.
Decoder-focused networks alleviate
the problems in encoder-focused
networks by utilizing a recursive
improvement approach to making
predictions obtained from a multi-
task network better.

6. Encoder-focused Architectures

7. Decoder-focused Architectures

8. Decoder-focused Architectures

9. Joint Task-Recursive Learning

10. Optimization – Task Balancing

11. Results

12. Thank You!

13. References
[1] An All-In-One Convolutional Neural Network for Face Analysis - Scientific Figure on
ResearchGate. Available from: https://www.researchgate.net/figure/A-general-multitask-
learning-framework-for-deep-CNN-architecture-The-lower-layers-are_fig1_309663347 [accessed
28 Nov, 2021]
[2]Vandenhende, S., Georgoulis, S., Van Gansbeke, W., Proesmans, M., Dai, D., & Gool, L.V.
(2021). Multi-Task Learning for Dense Prediction Tasks: A Survey. IEEE transactions on pattern
analysis and machine intelligence, PP.