(1) The paper presents a method called LASO for few-shot multi-label learning that combines pairs of examples in feature space to synthesize new examples corresponding to label set operations like intersection and union. (2) LASO trains a network to predict these synthesized label sets using losses that encourage the network to learn label set relationships. (3) The method is evaluated on multi-label datasets like COCO and CelebA, outperforming baselines on classification and retrieval metrics for few-shot learning settings.

1. CVPR 2019 oral
Laso: Label-Set Operations Networks for Multi-label Few-shot Learning
Amit Alfassy, Leonid Karlinsky, Amit Aides
IBM Research AI
Haifa, Israel
Source – https://slideplayer.com/slide/17422890/

2. 思想自由 兼容并包
Note – Paper is presented from the point of view of Few shot learning for multi-
label samples

3. 思想自由 兼容并包
Methodology
Combine pairs of given examples in feature space, so that the resulting synthesized feature vectors will
correspond to examples whose label sets are obtained through certain set operations on the label sets of the
corresponding input pairs.

4. 思想自由 兼容并包
More of an
expectation.
Takes content into
consideration
Training process is different. We take
intersection of the labels (albeit in
feature space) and not the content.
What would happen if intersection of
labels if empty set.

5. 思想自由 兼容并包
Methodology
𝒁𝒊𝒏𝒕 = 𝑴𝒊𝒏𝒕(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃(𝑭 𝒙, 𝑭 𝒚)

6. 思想自由 兼容并包
Binary Cross-Entropy Multi-Label Loss -

7. 思想自由 兼容并包
Methodology
𝐶𝑙𝑜𝑠𝑠
𝒁𝒊𝒏𝒕 = 𝑴𝒊𝒏𝒕(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃(𝑭 𝒙, 𝑭 𝒚)

8. 思想自由 兼容并包
Methodology
𝐿𝑎𝑠𝑜𝑙𝑜𝑠𝑠
𝒁𝒊𝒏𝒕 = 𝑴𝒊𝒏𝒕(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃(𝑭 𝒙, 𝑭 𝒚)
MLP with 3 or 4 blocks

9. 思想自由 兼容并包
Methodology
𝑅𝑙𝑜𝑠𝑠
𝑠𝑦𝑚
𝒁𝒊𝒏𝒕 = 𝑴𝒊𝒏𝒕(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃(𝑭 𝒙, 𝑭 𝒚)

10. 思想自由 兼容并包
Methodology
𝒁𝒊𝒏𝒕 = 𝑴𝒊𝒏𝒕(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒖𝒏𝒊 = 𝑴 𝒖𝒏𝒊(𝑭 𝒙, 𝑭 𝒚)
𝒁 𝒔𝒖𝒃 = 𝑴 𝒔𝒖𝒃(𝑭 𝒙, 𝑭 𝒚)
𝑅𝑙𝑜𝑠𝑠
𝑚𝑐 mc – mode
collapse

11. 思想自由 兼容并包

12. 思想自由 兼容并包
Experiments
Coco: object-based multi-label dataset (80 object categories)
Celeb-A: attribute-based multi-label dataset(40 facial attributes)

13. 思想自由 兼容并包
Coco Experiments: Classification
The Coco dataset is split into Coco-A(64 classes), Coco-B(16 classes)
in order to evaluate the model on the unseen
categories, they separately train a 16-way classifier
on them, used for evaluation. This is a standard
metric in domain generalization, but it means the
model can not confuse them with already seen
classes, which would be a likely source of errors

14. 思想自由 兼容并包
Coco Experiments: Retrieval
The Coco dataset is split into Coco-A(64 classes), Coco-B(16 classes)
Their backbone itself seems to be
okish. Would like to see its
performance numbers (guess based
on visual results)
Why IOU now, we
are only bothered
about labels, no?

15. 思想自由 兼容并包
Experiments
Celeb-A: attribute-based multi-label dataset
(40 facial attributes)
(No seen unseen in this scenario)

16. 思想自由 兼容并包
Experiments: Ablations
Coco: object-based multi-label dataset
Celeb-A: attribute-based multi-label dataset (40 facial attributes)
Tab. Analytic approximations to set operations
All numbers are mAP %
Inspired by DCGAN

17. 思想自由 兼容并包
Visual Results

18. 思想自由 兼容并包
Source - https://ameroyer.github.io/reading-notes/few-shot%20learning/2019/04/26/laso_label_set_operations_networks_for_multi_label_few_shot_learning.html