The document outlines Scott Thompson's experience with applying transfer learning in TensorFlow to build a visual search and classifier for clothing items. It emphasizes using pre-trained models like Google Inception-v3 for efficiency and generalization, detailing techniques like edge detection and image feature extraction. Thompson also discusses performance metrics and methods for optimizing classification tasks, including nearest neighbor searches and caching features.

1. Applying Transfer Learning
in TensorFlow
Scott Thompson
Software Engineer
@KontorHQ

2. Lessons learned building a visual
search and classifier for Gilt

3. A guide for Software Engineers

4. Classification
Men - Short Sleeve Sport Shirts
Men - Long Sleeve Sport Shirts
Men - Activewear - Activewear Tops
Men - Polos

5. Visual Search

6. How to distill the ‘essence’ of an image?

7. Men - Short Sleeve Sport Shirts
Men - Long Sleeve Sport Shirts

8. Image Features
Source: http://opencv.org
Edge Detection Color Histogram Pixel Segmentation

9. How to train?
Google Inception-v3
Pre-trained on ImageNet
http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz
Convolutional Neural Nets
Source: https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html

10. 1,000,000 images, 1,000 categories
Business office Internet site President
Image Source: http://image-net.org/explore

11. Why use a pre-trained model?
It’s faster (it’s pre-trained)
It’s cheaper (no need for GPU farm)
It generalizes (avoid overfitting)
Image Source: http://image-net.org/explore

12. Transfer Learning

13. Source: https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html
‘Essence’Edge
Detector
Shape
Detector
Structure of the Network

14. t-SNE plot - CIFAR-10 pool_3 features
t-SNE Code: https://lvdmaaten.github.io/tsne/

15. Output Layer Input Layer
Source: https://github.com/sthomp/tensorflow_transfer_cifar10

16. Performance
About 1 second per image
Gilt has ~40,000 images
~12 hours
Embarrassingly Parallel -> Partition across threads / machines
Cache pool_3 features (numpy array)

17. Applying pool_3 features
Nearest Neighbor Search
Brute Force
Cosine Distance
Checkout https://github.com/spotify/annoy
Train a classifier
Nearest Neighbor Classifier

18. You don’t need to use TensorFlow for everything
Reading files / Work Queues
Image resizing
Serializing weights