Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Smoothed manifold

110 views

Published on

Yifan Sun

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Smoothed manifold

  1. 1. 1. S. Bai et al., Scalable Person Re-identification on Supervised Smoothed Manifold. CVPR2017 spotlight 2. Y. Sun et al., Deep Representation Learning: a Similarity Smoothing Perspective 3. Y. Shen et al., Person Re-identification with Deep Similarity-Guided Graph Neural Network. ECCV 2018 accepted Keywords: Smooth Similarity, Smoothed Manifold, Graph Neural Network 孙奕帆 ReID中的相似性平滑约束
  2. 2. Scalable Re-ID on Supervised Smoothed Manifold. Reference: 1. D. Zhou et al., Learning with local and global consistency, NIPS2003 2. S. Bai et al., Scalable Person Re-identification on Supervised Smoothed Manifold. CVPR2017 何为相似性不平滑 PA: A1 A2 B2 假设某特征空间中存在以下关系: A1 close to B1 A2 close to B2 现考察样本对内部的样本距离,假设: A1 close to A2 B1 far from B2 PB : B1 PA close to PB 互斥于 PA close to PB 一个极端的例子:假设2组样本对PA=(A1,A2)及PB=(B1,B2)
  3. 3. 如何施加相似性平滑约束 k i l j Qki: 样本k,i间的相似性 Qlj: 样本l,j间的相似性 P(ki→lj) 样本对ki到样本对lj的状态转移概率 Lki: 样本k,i是否属于同一ID 正样对为1,负样对为0,L视为硬化的相似性,故可与Q进行加法运算 直观意义:k,i之间的相似性Qk i“吸收”所有其它样本的相似性 Ql j, l,j∈{1,2,…,N} , 吸收强度由样本对{l,j}、{k,i}之间的状态转移概率P(ki→lj)决定 Scalable Re-ID on Supervised Smoothed Manifold.
  4. 4. 如何施加相似性平滑约束 k i l j Qki: 样本k,i间的相似性 Qlj: 样本l,j间的相似性 P(ki→lj) 样本对ki到样本对lj的状态转移概率 Lki: 样本k,i是否属于同一ID 正样对为1,负样对为0,L视为硬化的相似性,故可与Q进行加法运算 直观意义:k,i之间的相似性Qk i“吸收”所有其它样本的相似性 Ql j, l,j∈{1,2,…,N} , 吸收强度由样本对{l,j}、{k,i}之间的状态转移概率P(ki→lj)决定 Scalable Re-ID on Supervised Smoothed Manifold. 真的是 所有吗?
  5. 5. 相似性吸收强度 2 W expij dij            一个常用定义: 控制半径 越小,越关注局部:欧氏距离近的样本对,其W绝对占优,导致吸收仅在较小局部发生 越大,越关注全局:欧氏距离较远的样本对,其强度仍然能被吸收 Scalable Re-ID on Supervised Smoothed Manifold.
  6. 6. SSM的一些问题: 采用后处理方式,增加测试阶段时间。在整个样本空间 (train+test)进行相似性传递,复杂度接近O(N4) (N为样本数), 难以在大训练集上学习。且利用了gallery信息
  7. 7. Deep Representation Learning: a Similarity Smoothing Perspective Yifan Sun, Liang Zheng, Qin Xu, Zhongdao Wang, Shengjin Wang
  8. 8. Motivation----smooth similarity • Has been valued in semi-supervised learning or transductive inference • Has not been explored in deep representation learning under fully supervision A1 A2 B1 B2 Pair I Pair II similar dissimilar A1 A2 B1 B2 Pair I Pair II A1 A2 B1 B2 Pair I Pair II (a) smooth (b) smooth (c) unsmooth
  9. 9. Our Contribution • We introduce the smooth similarity constraint, which is traditionally utilized in semi-supervised learning, to deep representation learning under the fully supervised manner • We define an evaluation protocol to measure similarity smoothness and transform it to a Similarity Smoothing Regularizer (SSR). • We demonstrate though extensive experiments on four fine-grained retrieval datasets, that similarity smoothing is beneficial towards more discriminative representation.
  10. 10. Proposed Method • A revisit to Smooth Similarity Constraint • Similarity Smoothness Indicator • Similarity Smoothing Regularizer (SSR) • The similarity measure W • A light edition of SSR for efficient training
  11. 11. Proposed Method • A revisit to Smooth Similarity Constraint Affinity value which is initialized with W
  12. 12. Proposed Method • A revisit to Smooth Similarity Constraint Affinity value which is initialized with W Wij: the similarity value calculated with similarity measure which may be heuristically defined
  13. 13. Proposed Method • Similarity Smoothness Indicator
  14. 14. Proposed Method • Similarity Smoothness Indicator A weighted mean of the similarity variations between sample pairs
  15. 15. Proposed Method • Similarity Smoothing Regularizer (SSR) • Takes the same formula as the Similarity Smoothness Indicator • To be evaluated within the training mini-batch • Essentially enforces the similarity not to change too much between nearby pairs • May achieve a optimum Solution A, compromising the discriminative ability • So, it is important to combine a metric loss
  16. 16. Proposed Method • The similarity measure W • Cosine similarity • Gaussian similarity RBF kernel width
  17. 17. Proposed Method • The similarity measure W • Gaussian similarity RBF kernel width impacts on the optimization of SSR Inappropriate settings will lead SSR to approximate another optimum Solution B, decreasing the retrieval accuracy:
  18. 18. Proposed Method • A light edition of SSR for efficient training When adopting the N-pairs sampling strategy, the computational cost is reduced by V^4 (1/4096 when 8 instances for a same identity) while bringing little impact on the retrieval accuracy LSSR focuses on inter-class similarity smoothness
  19. 19. Task----Fine grained Retrieval • Person Re-identification • Birds & Cars Retrieval (王重道) CUB-200 CARS196
  20. 20. Experiments-reID
  21. 21. Experiments-birds and cars retrieval Baseline 4: contrastive loss alone
  22. 22. Experiments-Mechanism Study Impact of Similarity Measure W 1)under cosine similarity 2) The effectiveness of using Gaussian Simlarity Depends 3) A interesting observation: Only when SSR construct a competing effect with the cooperating metric loss, (Solution A), the accuracy increases
  23. 23. Experiments-Mechanism Study Visualization
  24. 24. SGGNN----propagating features within mini-batch Element-wise subtraction and square operation FC+sigmoid (positive pair1 negative pair0) Absorbing difference feature “d” from other pairs. The absorbing weight W is determined by the transition probability
  25. 25. SGGNN----propagating features within mini-batch The message (or feature) to propagate is a learnable t = 2FC+BN on “d”, rather than “d” itself
  26. 26. SGGNN----propagating features within mini-batch • Performance • The similarity scores are predicted by SGGNN ---time consuming
  27. 27. SGGNN----propagating features within mini-batch
  28. 28. Connection & Difference between SSR and SGGNN • Both method employ smooth similarity constraint on the training dataset (instead of on the training + testing) A1 A2 B1 B2 Pair I Pair II SGGNN: propagating features within triplet (special case of quadruple) SSR: propagating similarities between any sample pairs (quadruple) A1 A2 B2 Pair I Pair II

×