The document proposes a self-supervised method called SCOPS to learn co-part segmentation without using any semantic labels. It uses three key losses: 1) Geometric Concentration Loss encourages pixels of the same part to be spatially concentrated, 2) Equivariance Loss enforces consistency under random transformations, and 3) Semantic Consistency Loss avoids bias towards background parts by using image-level labels from a pretrained model. Together these losses allow SCOPS to segment objects into semantically consistent parts by leveraging only unlabeled images of the same category.

1. SCOPS: Self-Supervised Co-Part
Segmentation
UC Merced, NVIDIA
Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang, and
Jan Kautz

3. The problem
• Fully supervised
• Costly for the annotation work
• Hard to generalize to unseen categories
• Self-supervised
• Learn part segmentations that are semantically consistent across different
object instances, given only an image collection of the same object category.
• Class agnostic

4. Contributions
• Geometric concentration
• Geometric Concentration Loss
• Robustness to variations
• Equivariance Loss
• Semantic consistency
• Semantic Consistency Loss
• Objects as union of parts
• Saliency Constraint

5. Overall Framework
• Backbone: DeepLab-V2(ResNet50)
• Output: 𝑅 = 𝐹(𝐼; 𝜃𝑓) ∈ [0,1] 𝐾+1 ∗𝐻∗𝑊
• K is the number of parts

6. Geometric Concentration Loss
• Observation:
• Pixels belonging to the same object part are spatially concentrated or form a
connected component.
• Minimize the variance of spatial probability distribution
• The part center for a part k along axis u

7. Equivariance Loss
• Random spatial Transform Ts
• Random appearance perturbation
Ta
Transformed
part center

8. Semantic Consistency Loss
Pretrained
on ILSVRC
Avoid the bias towards
background