This is a poster material for the paper of "Symmetrical Synthesis for Deep Metric Learning" accepted in AAAI 2020. Written by Geonmo Gu*, Byungsoo Ko* (* Authors contributed equally.) @NAVER/LINE Vision - Arxiv: https://arxiv.org/abs/2001.11658 - Github: https://github.com/clovaai/symmetrical-synthesis

1. Symmetrical Synthesis for Deep Metric Learning
Geonmo Gu*, Byungsoo Ko*
{korgm403, kobiso62}@gmail.com
Association for the Advancement of
Artificial Intelligence
(a) CUB200-2011 (b) CARS196 (c) Stanford online products
Introduction
• We propose a novel method for synthetic hard sample generation:
Symmetrical Synthesis (Symm).
• In contrast to previous methods (DAML, HDML), it only requires simple
algebraic computation to generate synthetic hard samples.
• It is hyper-parameter free, plug-and-play, no need of network
modification, no influence to training speed and optimization difficulty.
• Our method outperforms over existing methods for a variety of losses on
clustering and image retrieval tasks.
Blue: original points / Red: synthetic points
Comparison with State-of-the-Art
Experiment
Link
Impact of Similarity and Norm
Github
Arxiv
Proposed Method
Label of SyntheticsRatio of Feature Points
Level of Hardness
Why Symmetric?
1. Symmetrical synthesis gives the same similarity among original and
synthesized features in same class.
-> It does not affect the positive side of the loss.
2. Symmetrical synthesis always have same norm.
-> it gives continuity of control over the norm during training process.
t-SNE Visualization of Embedding Space
Motivation
Contribution
• Hard sample generation methods have been proposed to improve metric
learning losses (triplet, N-pair, lifted structure, angular).
• Previous methods (DAML, HDML) uses generative networks, which leads
to more hyper-parameters, harder optimization, slower training speed.
Symmetrical Synthesis
* Authors contributed equally.
Step 1. Generate symmetric points Step 2. Hard negative pair mining
• Given original points, generate their
synthetic points with each other as an
axis of symmetry.
• We set 𝛼=2.0, 𝛽=1.0 for symmetrical
synthesis.
• Perform hard negative pair mining
between two different classes.
original points
synthetic points
• Maintaining similarity and norm is
essential for optimization.
• Synthetic points lie in meaningful spots
due to the increased clustering ability.
• The harder the pair selected, the higher
the performance.
• At the beginning: more original points
• After some steps: more synthetic points
original points
synthetic points
original points
synthetic points
Class A Class B
Metric Learning with Symm
• 100 iterations: synthetic points are generated on the meaningless place.
• After 3000 iterations: synthetic points are lying around the boundary of clusters.
• Synthetic points will work as hard negatives to push classes with stronger power.
• Combining Symm with metric learning losses leads to a large performance boost in both clustering and retrieval tasks.
• Our proposed method outperforms all hard sample generation methods for every loss and dataset except one.