Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Machine Learning Tokyo - Deep Neural Networks for Video - NumberBoost
Similar to Machine Learning Tokyo - Deep Neural Networks for Video - NumberBoost (20)
Recently uploaded
Recently uploaded (20)
Machine Learning Tokyo - Deep Neural Networks for Video - NumberBoost
- 1. Deep Neural Networks for VIDEO Neither Proprietary nor Confidential – Please Distribute ;) Alex Conway alex @ numberboost.com @ALXCNWY MACHINE LEARNING TOKYO MEETUP 2019.03.18 Neitherconfidential norproprietary- pleasedistribute;)
- 2. 2016 MultiChoice Innovation Competition 1st PrizeWinners 2017 Mercedes-Benz Innovation Competition 1st PrizeWinners 2018 Lloyd’s Register Innovation Competition 1st PrizeWinners 2019 NTT & Dimension Data Innovation Competition 1st PrizeWinners
- 4. Hands up!
- 5. Deep learning is MagicDeep learning is MagicDeep learning is EASY!
- 13. This image is 3.8 kb Super-Resolution
- 14. 14 Original input Rear Window (1954) Pix2pix output Fully Automated Remastered Painstakingly by Hand https://hackernoon.com /remastering-classic- films-in-tensorflow-with- pix2pix-f4d551fa0503
- 26. Object detection
- 42. 42 … Video Applications • Video Generation • Clip Classification • Frame Classification • Object Detection etc. • Caption Generation • Video Q & A
- 43. 43 … SIZE DOES MATTER w h (w,h,3,t) e.g. 500x500x3x4x60 = 180 million
- 44. 44 1. Neural Network Crash Course 2. Convolutional Neural Networks 3. Recurrent Neural Networks 4. Object Detection 5. Show and tell 6. Edge Computing & Federated ML 7. SEBENZ.AI
- 45. Jeremy Howard & Rachel Thomas http://course.fast.ai Richard Socher’s Class on NLP (great RNN resource) http://web.stanford.edu/class/cs224n/ Andrej Karpathy’s Class on Computer Vision (great CNN resource) http://cs231n.github.io Keras docs https://keras.io/ Great Free Resources
- 47. What is a neuron? 47 • 3 inputs [x1,x2,x3] • 3 weights [w1,w2,w3] • 1 bias term b • Element-wise multiply and sum: inputs & weights • Add bias term • Apply activation function f • Output of neuron is just a number (like theinputs!)
- 48. 48 linear
- 49. 49 non-linear
- 50. What is an Activation Function? 50 Sigmoid Tanh ReLU Nonlinearities … “squashing functions” … transformneuron’s output NB: sigmoid output in[0,1]
- 51. So what is a Neural Network?
- 52. What is a 52 Inputs outputs hidden layer 1 hidden layer 2 hidden layer 3 Note: Outputs of one layer are inputs into the next layer This (non-convolutional) architecture is called a “multi-layered perceptron” Neural Network?(Deep)
- 53. 53
- 54. 54
- 55. 2 Layer Neural Network 55https://ml4a.github.io/ml4a/looking_inside_neural_nets/ 0 1 2 3 4 5 6 7 8 9
- 59. “How much error increases when we increase this weight” How does a neural network learn? 59 New weight = Old weight Learning rate- Gradient of Error with respect to Weight ( )x
- 62. For much more detail, see: 1. Michael Nielson’s Neural Networks & Deep Learning free online book http://neuralnetworksanddeeplearning.com/chap1.html 2. Anrej Karpathy’s CS231n Notes http://neuralnetworksanddeeplearning.com/chap1.html 62 Neural Networks
- 63. Why is ML Succeeding? 1. Algorithms 2. Compute 3. Data
- 68. Image Classification 68 http://yann.lecun.com/exdb/mnist/ https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py (99.25% test accuracy in 192 seconds and 46 lines of code)
- 70. Image Classification 70 ImageNet Classification withDeepConvolutional Neural Networks, Krizhevsky et. Al. Advances in Neural Information Processing Systems 25 (NIPS2012)
- 71. Image & Video Moderation TODO 71 Large internationalgay datingapp with tens of millions of users uploadinghundreds-of-thousandsofphotosperday
- 74. 74 This is VGGNet – don’t panic, we’ll break it down piece by piece Example Architecture
- 75. 75 This is VGGNet – don’t panic, we’ll break it down piece by piece Example Architecture
- 78. Many Kernels = Many Activation Maps = Volume 78http://cs231n.github.io/convolutional-networks/
- 81. 81
- 82. 82
- 83. 83
- 85. Max Pooling 85
- 86. Max Pooling 86
- 87. RINSE WASH REPEAT … DEEEEEP! 87 Convolution+ max pooling + fully connected+ softmax
- 88. 88 Convolution+ max pooling + fully connected+ softmax RINSE WASH REPEAT … DEEEEEP!
- 89. 89 RINSE WASH REPEAT … DEEEEEP!
- 90. 90 RINSE WASH REPEAT … DEEEEEP!
- 91. 91 Flatten Convolution+ max pooling + fully connected+ softmax Flatten the final7 x 7 x 512 max poolinglayer Add fully-connecteddense layerontop
- 92. 92 Bringing it all together Convolution+ max pooling + fully connected+ softmax
- 93. Softmax Convert scores ∈ ℝ to probabilities ∈ [0,1] Final output prediction = highest probability class 93
- 94. 94
- 95. We need labelled training data!
- 96. ImageNet 96 http://image-net.org/explore 1000 object categories 1.2 million training images
- 98. ImageNet 98
- 99. ImageNet 99
- 100. ImageNet Top 5 Error Rate 100 Traditional Image Processing Methods AlexNet 8 Layers ZFNet 8 Layers GoogLeNet 22 Layers ResNet 152 Layers SENet Ensamble TSNet Ensamble
- 101. Using a CNN
- 102. Example: Classifying Product Images 102 https://github.com/alexcnwy/DeepLearning4ComputerVision Classifying products into 8 categories
- 104. 104 https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html Start with Pre-Trained ImageNet Model Consumers vs Producers of Machine Learning
- 105. 105
- 106. …but we don’t want to ImageNet categories… we want to predict something new!
- 109. 109 After Fine-Tuning Replace Final layers of Pre-trained Model
- 110. Fine-tuning A CNN To Solve A New Problem • Load pre-trained VGG network • Remove final dense & softmax layers that predict 1000 ImageNet classes • Replace with new dense & softmax layer to predict 8 categories 110 96.3% accuracy in under 2 minutes for classifying products into categories (WITH ONLY 3467TRAINING IMAGES!!1!) Fine-tuning is awesome! Insert obligatory brain analogy
- 113. Visual Similarity 113 • Chop off VGG dense layers and keep bottleneck features • Add single dense layer with 256 activations • “Predict” to get activation for each image • Compute nearest neighbours in the space of these activations https://memeburn.com/2017/06/spree-image-search/
- 114. 114
- 116. 116 Input Image not seen by model Results Top 10 most “visually similar”
- 117. 117 Input Image not seen by model Results Top 10 most “visually similar”
- 120. Pro-tip… 120
- 121. Use a Better Architecture (or all of them!) 121 “Ensambles win” learn a weighted average of many models’ predictions
- 124. (2009)
- 128. 128
- 131. http://colah.github.io/posts/2015-08-Understanding-LSTMs/ Long Short Term Memory Networks
- 133. “Encoding” 133 ) =f(
- 134. KERAS = CHEAT CODEZ • https://github.com/keras-team/keras/blob/master/examples/imdb_lstm.py 134
- 138. Penguin vid
- 142. 142
- 144. Video Q&A XXX 144
- 146. Object Detection
- 152. Show And tell
- 167. Deep Neural Network … P(A) = 0.005 P(B) = 0.002 P(C) = 0.98 P(9) = 0.001 P(0) = 0.03
- 177. OFF-THE-SHELF ML MODELS DO NOT WORK IN AFRICA
- 179. Taxi vid data: 1 week, 1 cam = ~250GB
- 180. 180 … SIZE DOES MATTER w h (w,h,3,t) e.g. 500x500x3x4x60 = 180 million
- 190. MOBILE GAME A MOBILE GAME THAT CREATES JOBS IN AFRICA MOBILE GAMEMOBILE GAMEMOBILE GAMEMOBILE GAMEMOBILE GAMEMOBILE GAMEMOBILE GAME
- 191. 27.5% UNEMPLOYMENT!
- 194. let’s build the future together alex@numberboost.com THANK YOUTHANK YOUTHANK YOU https://twitter.com/alxcnwy