Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak

5,224 views

Published on

PyData Seattle 2015

Published in: Data & Analytics
  • Check out the brain training for Dogs course now. It's great for eliminating any bad behaviors by tapping into your dog's hidden intelligence. ♣♣♣ http://t.cn/Aie4mTQb
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak

  1. 1. Learn to Build an App to Find Similar Images using Deep Learning Piotr Teterwak Dato, Machine Learning Engineer
  2. 2. 2 First things first, installation http://bit.ly/dato_pydata import graphlab sf = graphlab.Sframe() sa = graphlab.Sarray(range(100)).apply(lambda x: x)
  3. 3. 3 Who is Dato?
  4. 4. Graphlab Create: Production ML Pipeline DATA YourWebServiceor IntelligentApp ML Algorithm Data cleaning & feature eng Offline eval & Parameter search Deploy model Data engineering Data intelligence Deployment Goal: Platform to help implement, manage, optimize entire pipeline
  5. 5. Deep Learning
  6. 6. Today’s talk
  7. 7. Today’s tutorial Deep Learning Overview Training a model on MNIST Building a Similar Dress Recommender Deploying the Dress Recommender
  8. 8. Features are key to machine learning
  9. 9. 9 Simple example: Spam filtering • A user opens an email… - Will she thinks its spam? • What’s the probability email is spam? Text of email User info Source info Input: x MODEL Yes! No Output: Probability of y
  10. 10. 10 Feature engineering: the painful black art of transforming raw inputs into useful inputs for ML algorithm • E.g., important words, complex transformation of input,… MODEL Yes! No Output: Probability of y Feature extraction Features: Φ(x) Text of email User info Source info Input: x
  11. 11. Deep Learning for Learning Features
  12. 12. 12 Linear classifiers • Most common classifier - Logistic regression - SVMs - … • Decision correspond to hyperplane: - Line in high dimensional space w0 + w1 x1 + w2 x2 > 0 w0 + w1 x1 + w2 x2 < 0
  13. 13. 13 What can a simple linear classifier represent? AND 0 0 1 1
  14. 14. 14 What can a simple linear classifier represent? OR 0 0 1 1
  15. 15. 15 What can’t a simple linear classifier represent? XOR 0 0 1 1 Need non-linear features
  16. 16. 16 Non-linear feature embedding 0 0 1 1
  17. 17. 17 Graph representation of classifier: Useful for defining neural networks x 1 x 2 x d y … 1 w2 w0 + w1 x1 + w2 x2 + … + wd xd > 0, output 1 < 0, output 0 Input Output
  18. 18. 18 What can a linear classifier represent? x1 OR x2 x1 AND x2 x 1 x 2 1 y -0.5 1 1 x 1 x 2 1 y -1.5 1 1
  19. 19. Solving the XOR problem: Adding a layer XOR = x1 AND NOT x2 OR NOT x1 AND x2 z 1 -0.5 1 -1 z1 z2 z 2 -0.5 -1 1 x 1 x 2 1 y 1 -0.5 1 1 Thresholded to 0 or 1
  20. 20. 20 http://deeplearning.stanford.edu/wiki/images/4/40/Network3322.png Deep Neural Networks P(cat|x) P(dog|x)
  21. 21. 21 Deep Neural Networks • Can model any function with enough hidden units. • This is tremendously powerful: given enough units, it is possible to train a neural network to solve arbitrarily difficult problems. • But also very difficult to train, too many parameters means too much memory+computation time.
  22. 22. 22 Neural Nets and GPU’s • Many operations in Neural Net training can happen in parallel • Reduces to matrix operations, many of which can be easily parallelized on a GPU.
  23. 23. 24 Convolutional Neural Nets • Strategic removal of edges Input Layer Hidden Layer
  24. 24. 25 Convolutional Neural Nets • Strategic removal of edges Input Layer Hidden Layer
  25. 25. 26 Convolutional Neural Nets • Strategic removal of edges Input Layer Hidden Layer
  26. 26. 27 Convolutional Neural Nets • Strategic removal of edges Input Layer Hidden Layer
  27. 27. 28 Convolutional Neural Nets • Strategic removal of edges Input Layer Hidden Layer
  28. 28. 29 Convolutional Neural Nets • Strategic removal of edges Input Layer Hidden Layer
  29. 29. 30 Convolutional Neural Nets http://ufldl.stanford.edu/wiki/images/6/6c/Convolution_schematic.gif
  30. 30. 31 Pooling layer Ranzato, LSVR tutorial @ CVPR, 2014. www.cs.toronto.edu/~ranzato
  31. 31. 32 Pooling layer http://ufldl.stanford.edu/wiki/images/6/6c/Pooling_schematic.gif
  32. 32. 33 Final Network Krizhevsky et al. ‘12
  33. 33. Applications to computer vision
  34. 34. 35 Image features • Features = local detectors - Combined to make prediction - (in reality, features are more low-level) Face! Eye Eye Nose Mouth
  35. 35. 36 Standard image classification approach Input Computer$vision$features$ SIFT$ Spin$image$ HoG$ RIFT$ Textons$ GLOH$ Slide$Credit:$Honglak$Lee$ Extract features Use simple classifier e.g., logistic regression, SVMs Face
  36. 36. 37 Many hand crafted features exist… Computer$vision$features$ SIFT$ Spin$image$ HoG$ RIFT$ Textons$ GLOH$ Slide$Credit:$Honglak$Lee$ … but very painful to design
  37. 37. 38 Change image classification approach? Input Computer$vision$features$ SIFT$ Spin$image$ HoG$ RIFT$ Textons$ GLOH$ Slide$Credit:$Honglak$Lee$ Extract features Use simple classifier e.g., logistic regression, SVMs FaceCan we learn features from data?
  38. 38. 39 Use neural network to learn features Input Learned hierarchy Output Lee et al. ‘Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations’ ICML 2009
  39. 39. Sample results • Traffic sign recognition (GTSRB) - 99.2% accuracy • House number recognition (Google) - 94.3% accuracy 40
  40. 40. Krizhevsky et al. ’12: 60M parameters, won 2012 ImageNet competition 41
  41. 41. 42 ImageNet 2012 competition: 1.2M images, 1000 categories 42
  42. 42. 43 Application to scene parsing ©Carlos Guestrin 2005-2014 Y LeCun MA Ranzato Semantic Labeling: Labeling every pixel with the object it belongs to [ Farabet et al. ICML 2012, PAMI 2013] Would help identify obstacles, targets, landing sites, dangerous areas Would help line up depth map with edge maps
  43. 43. Challenges of deep learning
  44. 44. Deep learning score card Pros • Enables learning of features rather than hand tuning • Impressive performance gains on - Computer vision - Speech recognition - Some text analysis • Potential for much more impact Cons
  45. 45. Deep learning workflow Lots of labeled data Training set Validation set 80% 20% Learn deep neural net model Validate
  46. 46. Deep learning score card Pros • Enables learning of features rather than hand tuning • Impressive performance gains on - Computer vision - Speech recognition - Some text analysis • Potential for much more impact Cons • Computationally really expensive • Requires a lot of data for high accuracy • Extremely hard to tune - Choice of architecture - Parameter types - Hyperparameters - Learning algorithm - … • Computational + so many choices = incredibly hard to tune
  47. 47. 48 Can we do better? Input Learned hierarchy Output Lee et al. ‘Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations’ ICML 2009
  48. 48. Deep features: Deep learning + Transfer learning
  49. 49. 50 Transfer learning: Use data from one domain to help learn on another Lots of data: Learn neural net Great accuracy Some data: Neural net as feature extractor + Simple classifier Great accuracy on new problem Old idea, explored for deep learning by Donahue et al. ’14
  50. 50. 51 What’s learned in a neural net Neural net trained for Task 1 Very specific to Task 1More generic Can be used as feature extractor vs.
  51. 51. 52 Transfer learning in more detail… Neural net trained for Task 1 Very specific to Task 1More generic Can be used as feature extractor Keep weights fixed! For Task 2, learn only end part Use simple classifier e.g., logistic regression, SVMs Class?
  52. 52. 53 Using ImageNet-trained network as extractor for general features • Using classic AlexNet architechture pioneered by Alex Krizhevsky et. al in ImageNet Classification with Deep Convolutional Neural Networks • It turns out that a neural network trained on ~1 million images of about 1000 classes makes a surprisingly general feature extractor • First illustrated by Donahue et al in DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition 53
  53. 53. Transfer learning with deep features Training set Validation set 80% 20% Learn simple model Some labeled data Extract features with neural net trained on different task Validate Deploy in production
  54. 54. In real life
  55. 55. What else can we do with Deep Features? 56
  56. 56. 57 Clustering images Goldberger et al. Set of Images
  57. 57. Finding similar images 58
  58. 58. 59 Finding similar images A B C A B C
  59. 59. Deploying ML in production
  60. 60. 67 Deployment system Model Historical Data Predictions Batch training Real-time predictions REST Client Model Mgmt Dato Predictive Services Robust, Elastic Direct Train deep model REST API for predictions for new images, text,…
  61. 61. Summary
  62. 62. Deep learning made easy with deep features Deep learning: exciting ML development Slow, lots of tuning, needs lots of data Can still achieve excellent performance Deep features: reuse deep models for new domains Needs less data Faster training times Much simpler tuning
  63. 63. 70 Next… Build a Deep Learning model with GLC Creating an Image Similarity Service Deploying with Predictive Services
  64. 64. 71 First things first, installation http://bit.ly/dato_pydata import graphlab sf = graphlab.Sframe() sa = graphlab.Sarray(range(100)).apply(lambda x: x)

×