[NS][Lab_Seminar_250825]OFFSET: Segmentation-based Focus Shift Revi...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

OFFSET: Segmentation-based
Focus Shift Revision for
Composed Image Retrieval
Tien-Bach-Thanh Do
Network Science Lab
Dept. ...

2
Problems
Figure 1: (a) gives an example of the CIR task. (b) demonstrates the phenomenon of inhomogeneity in visual samp...

3
Problems
● Noise indicative signal absence
○ Difficulty in identifying dominant and noise regions without explicit noise...

4
Proposed method
Figure 2: The proposed OFFSET consists of three key modules: (a) Dominant Portion Segmentation, (b) Dual...

5
Proposed method
Problem formulation
● denote a set of N triplets
reference image modification text target image

6
Proposed method
Dominant portion segmentation
● Generate dominant segmentation
● BLIP2
● CLIPSeg

7
Proposed method
Dual focus mapping
● Visual focus mapping: extract local features by CLIP
● Map focus information in dom...

8
Proposed method
Dual focus mapping
● Textual focus mapping:
● Focus-mapped global feature of modification text:
local fe...

9
Proposed method
Dual focus mapping
● Multi-grained focus projection
○ Unify varied channel numbers of multi-grained focu...

10
Proposed method
Textually guided focus revision
● Utilize textual modification semantics to guide the revision of focus...

11
Experiments
● Datasets
○ FashionIQ
○ CIRR
○ Shoes
● Implementation details
○ CLIP (ViT-H/14)
○ AdamW optimizer
○ LR=1e-...

12
Experiments
Table 1: Performance comparison on FashionIQ relative to R@k(%). The overall best results are in bold, whil...

13
Experiments
Table 2: Performance comparison on Shoes with respect to R@k(%)
Table 4: Ablation Studies of OFFSET with va...

14
Experiments
Table 3: Performance comparison on CIRR with respect to R@k(%) and Rsubset @k(%). The overall best results ...

15
Experiments
igure 3: Sensitivity to Focus Channel Number P and the hyper-parameter μ on (a) FashionIQ, (b) Shoes, and (...

16
Experiments
Figure 4: Case study on (a) FashionIQ and (b) CIRR.

17
Conclusion
● CIR community often overlook 2 phenomena:
○ Visual data inhomogeneity leading to query feature degradation...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1011447 views
Storytelling For The Web: Integrate... by Chiara Aliotta 919244 views
Artificial Intelligence, Data and C... by OECD Directorate ... 820562 views
How to Leverage AI to Boost Employe... by SocialHRCamp 342424 views
2024 State of Marketing Report – by... by Marius Sescu 195380 views
Everything You Need To Know About C... by Expeed Software 218594 views

View on Slideshare

1 of 17 Ad

View on Slideshare

1 of 17 Ad