이주영(michael.lee) / kakao corp. (멀티미디어처리파트) --- 이미지 분류 문제에서 매우 높은 수준의 성능을 가지는 CNN구조가 많이 소개되었습니다. 이미지 분류 문제의 성공을 바탕으로 유사 이미지 검색에서부터 쇼핑 상품 이미지 검색까지 이미지 유사도를 이용한 이미지 검색의 기본 구조와 카카오에서의 응용사례를 소개합니다.

1. 이미지로 이미지를 검색하기
(Query By Image in kakao)
이주영(michael.lee)
kakao corp. (멀티미디어처리파트)

4. Microsoft COCO: Common Objects in Context, ECCV 2016, http://cocodataset.org
a long train of model animals on display to show how noah's arc was filled.
a few people are standing near a large exhibit of animal statues.
a large group of wild animals standing next to each other.
a display of wild animals inside a building.
a museum display of assorted animals like zebras and giraffes.
id=55575

5. 어떻게 정확한 결과를 찾아줄 수 있을까?

6. Microsoft COCO: Common Objects in Context, ECCV 2016, http://cocodataset.org
a long train of model animals on display to show how noah's arc was filled.
a few people are standing near a large exhibit of animal statues.
a large group of wild animals standing next to each other.
a display of wild animals inside a building.
a museum display of assorted animals like zebras and giraffes.
방법 1.

7. Microsoft COCO: Common Objects in Context, ECCV 2016, http://cocodataset.org
a long train of model animals on display to show how noah's arc was filled.
a few people are standing near a large exhibit of animal statues.
a large group of wild animals standing next to each other.
a display of wild animals inside a building.
a museum display of assorted animals like zebras and giraffes.
방법 1.

9. Microsoft COCO: Common Objects in Context, ECCV 2016, http://cocodataset.org
방법 2.
query

10. anchor
positive
negative

11. anchor
positive
negative

12. anchor
positive
negative

16. Triplet Loss
CNN
CNN
CNN
Similarity
(Triplet Loss)

17. 성능비교
Recall@1 Recall@10 Recall@100
Triplet, Semi-Hard [1] 66.7 82.4 91.9
Lifted Struct [2] 62.5 80.8 91.9
Proxy NCA Loss [3] 73.7 - -
Sampling Matters [4] 72.7 86.2 93.8
DAML [5] 68.4 83.5 92.3
kakao 73.3 86.2 93.9
1. Learnable structured clustering framework for deep metric learning, arXiv 2016
2. Deep metric learning via lifted structured feature embedding, CVPR 2016
3. No Fuss Distance Metric Learning using Proxies, ICCV 2017
4. Sampling Matters in Deep Embedding Learning, ICCV 2017
5. Deep Adversarial Metric Learning, CVPR 2018
Stanford Online Product dataset [1]
120, 053 images (train : valid = 59,551 : 60,502)
22,634 classes (train : valid = 11,318 : 11316)

18. 서비스 응용 1

19. 서비스 응용 2

20. Q & A