Most existing high-performance co-segmentation algorithms are usually complex due to the way of co-labelling a set of images as well as the common need of fine-tuning few parameters for effective co-segmentation. In this paper, instead of following the conventional way of co-labelling multiple images, we propose to first exploit inter-image information through cosaliency, and then perform single-image segmentation on each individual image. To make the system robust and to avoid heavy dependence on one single saliency extraction method, we propose to apply multiple existing saliency extraction methods on each image to obtain diverse salient maps. Our major contribution lies in the proposed method that fuses the obtained diverse saliency maps by exploiting the inter-image information, which we call saliency co-fusion. Experiments on five benchmark datasets with eight saliency extraction methods show that our saliency co-fusion based approach achieves competitive performance even without parameter fine-tuning when compared with the state-of-the-art methods.

1. Image Co-segmentation via Saliency
Co-fusion
- In IEEE Transactions on Multimedia, 2016
- By Koteswar Rao Jerripothula, Jianfei Cai and Junsong Yuan

2. Outline
• Objective: Image Co-segmentation
• Background
• Motivation: To fuse multiple saliency maps
• Saliency Co-fusion
• Results
• Conclusion

3. Image Co-segmentation: A task of extracting
common objects’ segments from a set of images
Objective

4. Multi image co-segmentation
Multi Foreground Co-segmentation Noisy Dataset Co-segmentation
[1] [2]
[3] [4]
[1] Cosegmentation of image pairs by histogram matching-incorporating a global constraint into mrfs, CVPR’06.
[2] Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion , ICCV’11.
[3] On Multiple Foreground Cosegmentation, CVPR’12.
[4] Unsupervised Joint Object Discovery and Segmentation in Internet Images, CVPR’13
Review

5. But..
• Complicated Co-labelling and Time consuming
• Require Parameter Tuning

6. Motivation
Can we fuse different saliency maps while benefiting from the association?
Advantage: Once we have such a fused saliency map, we just need to do
single image segmentation

7. Saliency Co-fusion
(our processing unit)

8. Objective Function
Element weights vector
Smoothness Coefficient Matrix (to
curb inconsistencies in weight
distribution)
Prior Coefficient Vector
(to respect different
recommendations)
Total number of elements
Image index Super-pixel index
Saliency map index
GOAL: To assign fusion weights to each of these
elements such that fused saliency highlights the
common objects
A Quadratic Programming Problem

9. Feature Space
Each element has 274 {2x(128+3+3+3)} dimensions descriptor using SIFT & color.

10. Recommendations
Cost for deviating from
the saliency value
recommended by
similar elements
Average of costs for
deviating from labels
recommended by the
similar elements
Idea: Lower the costs higher weights can be assigned

11. Smoothness
• Both in saliency and feature space
• Conventional Laplacian matrix is used
Idea: Elements emphasizing the
central regions can be
recommended for higher weights

12. Performance improves by fusion even with simple average and max functions, and further improved by ours.
fusedSaliency Maps

14. Additional Benefits
Repetitive
Case
Multi
Foreground
Case

15. Comparison
Coseg-Rep Dataset
Internet Images Dataset
[1] Cosegmentation and cosketch by unsupervised learning, ICCV’13.
[2] Automatic image co-segmentation using geometric mean saliency, ICIP’14
[3] Discriminative clustering for image co-segmentation, CVPR’10.
[4] Multiple-class co-segmentation, CVPR’12.
[5] Unsupervised joint object discovery in internet images, CVPR’13

16. Visual Comparison
Failure cases
Non-salient dog misses out
Insufficient background variations
[9] Cosegmentation by Composition, ICCV’13
Extra foregrounds in
the single common
object evaluation
datasets

17. Conclusion
• Saliency Co-fusion: taking help of the
association to fuse its own saliency maps
• By default setting itself, good performance.
• Effective on repetitive and multi-foreground
case as well.