Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to PyConZA 2019 Keynote - Deep Neural Networks for Video Applications
Similar to PyConZA 2019 Keynote - Deep Neural Networks for Video Applications (20)
Recently uploaded
Recently uploaded (20)
PyConZA 2019 Keynote - Deep Neural Networks for Video Applications
- 1. DEEP NEURAL NETWORKS Alex Conway alex @ numberboost.com PYCONZA Keynote 2019 Neither confidential nor proprietary - please distribute ;) for Video Applications
- 2. 2016 MultiChoice Innovation Competition 1st PrizeWinners 2017 Mercedes-Benz Innovation Competition 1st PrizeWinners 2018 Lloyd’s Register Innovation Competition 1st PrizeWinners 2019 NTT & Dimension Data Innovation Competition 1st PrizeWinners
- 3. HANDS UP! 🙌
- 11. ORIGINAL FILM Rear Window (1954) PIX2PIX MODEL OUTPUT Fully Automated RE-MASTERED BY HAND Painstakingly https://hackernoon.com/remastering-classic-films-in- tensorflow-with-pix2pix-f4d551fa0503
- 52. NEURAL NETWORKS Set of connected Neurons with randomly initialized weights and non-linear activation functions connected in a Network that are optimized (learned) using training data to minimize prediction error
- 54. WHAT IS A NEURON?
- 55. LINEAR
- 56. NON-LINEAR
- 58. Inputs outputs hidden layer 1 hidden layer 2 hidden layer 3 Note: Outputsof one layer are inputsinto the next layer This (non-convolutional)architecture is called a “multi-layered perceptron” (DEEP) NEURAL NETWORKS
- 59. HOW DOES A NEURAL NETWORK LEARN? New weight = Old weight Learning rate- ( )x “How much error increases whenwe increase this weight”
- 61. 1 1, 3, 3, 7, … [[1, 2, 3 ] [3, 2, 1] [3, 4, 5] [7, 8, 9] …] [[1, 2, 3 ] [3, 2, 1] [3, 4, 5] [7, 8, 9] …] [[1, 2, 3 ] [3, 2, 1] [3, 4, 5] [7, 8, 9] …]
- 63. image tensor 500 x 500 x 3 = 750’000 60 second video at 10 FPS tensor 500 x 500 x 3 x 10 x 60 = 450’000’000
- 67. INPUT 28 x 28 pixel grayscale images = 784 numbers
- 68. 2 LAYER NEURAL NETWORK 0 1 2 3 4 5 6 7 8 9
- 72. https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py (99.25% test accuracy in 192 seconds and 46 lines of code)
- 75. 3 KEY CONVOLUTIONAL NETWORK ARCHITECTURE IDEAS: 1. Local receptive fields 2. Shared weights 3. Subsampling
- 76. 76 VGGNet
- 79. 79
- 80. 80 Zeiler, M.D. and Fergus, R., 2014, September. Visualizing and understanding convolutional networks. In European conference on computer vision (pp. 818-833).
- 81. 81 Zeiler, M.D. and Fergus, R., 2014, September. Visualizing and understanding convolutional networks. In European conference on computer vision (pp. 818-833).
- 82. Convolutional Nets Learn Hierarchical Features 82
- 84. 84 VGGNet
- 87. 14,197,122 images, 21841 synsets indexed ILSVRC: 1‘200‘000 images, 1000 categories ImageNet
- 89. 89 ImageNet
- 90. 90 ImageNet
- 91. IMAGENET TOP-5 ERROR RATE Traditional Image Processing Methods AlexNet 8 Layers ZFNet 8 Layers GoogLeNet 22 Layers ResNet 152 Layers SENet Ensamble TSNet Ensamble
- 95. 95https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html
- 96. Example: Use CNN to Classify Product Images 96 https://github.com/alexcnwy/ DeepLearning4ComputerVision
- 97. 97
- 99. 99 USING A CNN AS A FEATURE EXTRACTOR Feature Extractor (“ENCODER”) Classifier
- 100. Extracting Features from an Image
- 102. Adding a New Classifier
- 103. Fine-tuning A CNN To Solve A New Problem 96.3% accuracy in under 2 minutes for classifying products into categories (WITH ONLY 3467 TRAINING IMAGES!!1!)
- 106. IMAGE & VIDEO MODERATION TODO 106
- 107. Object Detection
- 112. 1.5 million object instances 80 object categories http://cocodataset.org
- 114. DEMO (HOLD THUMBS) 😅
- 123. CNN … P(A) = 0.005 P(B) = 0.002 P(C) = 0.98 P(9) = 0.001 P(0) = 0.03
- 136. https://www.pyimagesearch.com/2018/07/23/simple-object-tracking-with-opencv/ ID #2 ID #1 “CENTROID TRACKING”
- 137. https://www.pyimagesearch.com/2018/07/23/simple-object-tracking-with-opencv/ “CENTROID TRACKING”
- 138. https://www.pyimagesearch.com/2018/07/23/simple-object-tracking-with-opencv/ “CENTROID TRACKING” For each object with ID in frame t, compute distance to centroid of every object in frame t + 1 and assign same ID provided distance less than threshold, else assign new ID
- 139. https://www.pyimagesearch.com/2018/07/23/simple-object-tracking-with-opencv/ “CENTROID TRACKING”
- 140. https://www.pyimagesearch.com/2018/07/23/simple-object-tracking-with-opencv/ ID #1 ID #2 “CENTROID TRACKING”
- 144. 144
- 146. SPATIO-TEMPORAL
- 148. SPORTS 1-M
- 149. SPATIAL … THEN TEMPORAL
- 153. Frame model accuracy <<< Video model accuracy
- 159. Frame model accuracy <<< Video model accuracy
- 168. FACE SWAP https://www.youtube.com/watch?v=7XchCsYtYMQ Detect face & crop then run “face swap” model… 2 networks: - same CNN encoder, - different decoder Feed image to encoder to create vector of input face then feed to decoder B to produce output face
- 171. Few-Shot Adversarial Learning of Realistic Neural Talking Head Models https://www.youtube.com/watch?v=p1b5aiTrGzY
- 172. Few-Shot Adversarial Learning of Realistic Neural Talking Head Models https://www.youtube.com/watch?v=p1b5aiTrGzY
- 173. Few-Shot Adversarial Learning of Realistic Neural Talking Head Models Network 1: CNN embedder compresses faces & landmarks to vector
- 174. Few-Shot Adversarial Learning of Realistic Neural Talking Head Models Network 2: Generator takes landmarks and synthesizes photo
- 175. Few-Shot Adversarial Learning of Realistic Neural Talking Head Models Network 3: Discriminator learns to tell apart real and synthesized photos
- 185. REMEMBER 💡
- 187. Don’t be scared to git clone functions and use deep learning!
- 188. Deep Learning Indaba http://www.deeplearningindaba.com Jeremy Howard & Rachel Thomas http://course.fast.ai Andrej Karpathy’s Class on Computer Vision http://cs231n.github.io Richard Socher’s Class on NLP (great RNN resource) http://web.stanford.edu/class/cs224n/ Keras docs https://keras.io/ GREAT FREE RESOURCES
- 189. THANK YOU! @alxcnwy alex @ numberboost.com