Download as PDF, PPTX
![What is a neuron?
21
• 3 inputs[x1,x2,x3]
• 3 weights[w1,w2,w3]
• Element-wise multiply and sum
• Apply activationfunction f
• Often add a bias too (weightof 1) – not shown](https://image.slidesharecdn.com/dl4cvexemlslides-170921121930/85/Deep-Learning-for-Computer-Vision-ExecutiveML-21-320.jpg)
![What is an Activation Function?
22
Sigmoid Tanh ReLU
Nonlinearities … “squashing functions” … transformneuron’s output
NB: sigmoid output in[0,1]](https://image.slidesharecdn.com/dl4cvexemlslides-170921121930/85/Deep-Learning-for-Computer-Vision-ExecutiveML-22-320.jpg)
![What is a Convolutional Neural Network?
30
“like a ordinary neural network but with special
types of layers that work well on images”
(math works on numbers)
• Pixel = 3 colour channels (R, G, B)
• Pixel intensity ∈[0,255]
• Image has width w and height h
• Therefore image is w x h x 3 numbers](https://image.slidesharecdn.com/dl4cvexemlslides-170921121930/85/Deep-Learning-for-Computer-Vision-ExecutiveML-30-320.jpg)
![Softmax
• Convert scores ∈ ℝ to probabilities∈ [0,1]
• Then predict the class with highest probability
47](https://image.slidesharecdn.com/dl4cvexemlslides-170921121930/85/Deep-Learning-for-Computer-Vision-ExecutiveML-47-320.jpg)
More Related Content
Similar to Deep Learning for Computer Vision - ExecutiveML
![[DSC Europe 25] Stefan Brankovic - #ResumeIsDead. AI-Powered Interviews and C...](https://cdn.slidesharecdn.com/ss_thumbnails/qnmbsv0xq3uysdrq3sev-2-stefan-brankovic-job-bolt-260114111931-a065aa3d-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Ivica Milaric - The Future of Gaming and AI Tools.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/tijgzsmgse2kj2y5pzzp-5-ivica-milaric-the-future-of-gaming-x-ai-tools-260114111931-87c2b3ac-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Elena Menshikova - AI-Powered Operational Excellence: Revolut...](https://cdn.slidesharecdn.com/ss_thumbnails/es6nholbqy3zaao2c2yd-2-elena-menshikova-data-ai-in-decision-making-260115093812-4fba8b38-thumbnail.jpg?width=640&height=640&fit=bounds)
Deep Learning for Computer Vision - ExecutiveML
- 1. Deep Learning for ComputerVision Executive-ML 2017/09/21 Neither Proprietary nor Confidential – Please Distribute ;) Alex Conway alex @ numberboost.com @alxcnwy
- 2.
- 3. Check out the DeepLearning Indaba videos & practicals! http://www.deeplearningindaba.com/videos.html http://www.deeplearningindaba.com/practicals.html
- 4. Deep Learning isSexy (for a reason!) 4
- 5. Image Classification 5 http://yann.lecun.com/exdb/mnist/ https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py (99.25% testaccuracy in 192 seconds and 70 lines of code)
- 6.
- 7.
- 8.
- 9.
- 10. Image Captioning &Visual Attention XXX 10 https://einstein.ai/research/knowing-when-to-look-adaptive-attention- via-a-visual-sentinel-for-image-captioning
- 11.
- 12.
- 13.
- 14.
- 15. 15 Original input Rear Window(1954) Pix2pix output Fully Automated Remastered Painstakingly by Hand https://hackernoon.com /remastering-classic- films-in-tensorflow-with- pix2pix-f4d551fa0503
- 16.
- 17.
- 18. 1. What isa neural network? 2. What is a convolutionalneural network? 3. How to use a convolutionalneural network 4. More advanced Methods 5. Case studies & applications 18
- 19. Big Shout Outs JeremyHoward & Rachel Thomas http://course.fast.ai Andrej Karpathy http://cs231n.github.io François Chollet (Keras lead dev) https://keras.io/ 19
- 20. 1.What is aneural network?
- 21. What is aneuron? 21 • 3 inputs[x1,x2,x3] • 3 weights[w1,w2,w3] • Element-wise multiply and sum • Apply activationfunction f • Often add a bias too (weightof 1) – not shown
- 22. What is anActivation Function? 22 Sigmoid Tanh ReLU Nonlinearities … “squashing functions” … transformneuron’s output NB: sigmoid output in[0,1]
- 23. What is a(Deep) Neural Network? 23 Inputs outputs hidden layer 1 hidden layer 2 hidden layer 3 Outputs of one layer are inputs into the next layer
- 24. How does aneural network learn? 24 • You need labelled examples “trainingdata” • Initially, the network makes random predictions (weights initializedrandomly) • For each training data point, we calculate the error between the network’s predictions and the ground-truth labels (aka “loss function”) • Use ‘backpropagation’ (really just the chain rule), to update the network parameters (weights) in the opposite direction to the error
- 25. How does aneural network learn? 25 New weight = Old weight Learning rate- Gradient of weight with respect to Error( )x “How much error increases when we increase this weight”
- 26.
- 27.
- 28. What is aNeural Network? For much more detail, see: 1. Michael Nielson’s Neural Networks & Deep Learning free online book http://neuralnetworksanddeeplearning.com/chap1.html 2. Anrej Karpathy’s CS231n Notes http://neuralnetworksanddeeplearning.com/chap1.html 28
- 29. 2. What isa convolutional neural network?
- 30. What is aConvolutional Neural Network? 30 “like a ordinary neural network but with special types of layers that work well on images” (math works on numbers) • Pixel = 3 colour channels (R, G, B) • Pixel intensity ∈[0,255] • Image has width w and height h • Therefore image is w x h x 3 numbers
- 31. 31 This is VGGNet– don’t panic, we’ll break it down piece by piece Example Architecture
- 32. 32 This is VGGNet– don’t panic, we’ll break it down piece by piece Example Architecture
- 33.
- 34.
- 35. New Layer Type:ConvolutionalLayer 35 • 2-d weighted average when multiply kernel over pixel patches • We slide the kernel over all pixels of the image (handle borders) • Kernel starts off with “random” values and network updates (learns) the kernel values (using backpropagation) to try minimize loss • Kernels shared across the whole image (parameter sharing)
- 36. Many Kernels =Many “Activation Maps” = Volume 36http://cs231n.github.io/convolutional-networks/
- 37. New Layer Type:ConvolutionalLayer 37
- 38.
- 39.
- 40.
- 41.
- 42. Convolution Learn HeirarchialFeatures 42
- 43.
- 44. New Layer Type:Max Pooling 44
- 45. New Layer Type:Max Pooling • Reduces dimensionality from one layer to next • …by replacing NxN sub-area with max value • Makes network “look” at larger areas of the image at a time • e.g. Instead of identifying fur, identify cat • Reduces overfittingsince losing information helps the network generalize 45
- 46. New Layer Type:Max Pooling 46
- 47. Softmax • Convert scores∈ ℝ to probabilities∈ [0,1] • Then predict the class with highest probability 47
- 48. Bringing it alltogether 48 Convolution+ max pooling+ fully connected+ softmax
- 49. Bringing it alltogether 49 Convolution+ max pooling+ fully connected+ softmax
- 50. Bringing it alltogether 50 Convolution+ max pooling+ fully connected+ softmax
- 51. Bringing it alltogether 51 Convolution+ max pooling+ fully connected+ softmax
- 52. Bringing it alltogether 52 Convolution+ max pooling+ fully connected+ softmax
- 53. Bringing it alltogether 53 Convolution+ max pooling+ fully connected+ softmax
- 54. We need labelledtraining data!
- 55. ImageNet 55 http://image-net.org/explore 1000 object categories 1.2million training images
- 56.
- 57.
- 58. ImageNet Top 5Error Rate 58 Traditional Image Processing Methods AlexNet 8 Layers ZFNet 8 Layers GoogLeNet 22 Layers ResNet 152 Layers SENet Ensamble TSNet Ensamble
- 59. 3. How touse a convolutional neural network
- 60. Using a Pre-TrainedImageNet-Winning CNN 60 • We’ve been looking at “VGGNet” • Oxford Visual Geometry Group (VGG) • ImageNet 2014 Runner-up • Network is 16 layers (deep!) • Easy to fine-tune https://blog.keras.io/building-powerful-image-classification-models-using- very-little-data.html
- 61. Example: Classifying ProductImages 61 https://github.com/alexcnwy/CTDL_CNN_TALK_20170620 Classifying products into 9 categories
- 62.
- 63. “Transfer Learning” is agame changer
- 64. Fine-tuning A CNNTo Solve A New Problem • Cut off last layer of pre-trained Imagenet winning CNN • Keep learned network (convolutions) but replace final layer • Can learn to predict new (completely different) classes • Fine-tuning is re-training new final layer - learn for new task 64
- 65. Fine-tuning A CNNTo Solve A New Problem 65
- 66.
- 67.
- 68. Fine-tuning A CNNTo Solve A New Problem • Fix weights in convolutional layers (set trainable=False) • Remove final dense layer that predicts 1000 ImageNet classes • Replace with new dense layer to predict 9 categories 68 88% accuracy in under 2 minutes for classifying products into categories Fine-tuning is awesome! Insert obligatory brain analogy
- 69. Visual Similarity 69 • Chopoff last 2 VGG layers • Use dense layer with 4096 activations • Compute nearest neighbours in the space of these activations https://memeburn.com/2017/06/spree-image-search/
- 70.
- 71. 71 Input Image not seenby model Results Top 10 most “visually similar”
- 72. 4. More AdvancedMethods
- 73. Use a BetterArchitecture (or all of them!) 73 “Ensambles win” learn a weighted average of many models’ predictions
- 74. cs231n.stanford.edu/slides/2017/cs231n_2017_lecture11.pdf There are MANYComputer Vision Tasks
- 75. > Long,Shelhamer,and Darrell,“FullyConvolutional Networks for SemanticSegmentation”CVPR2015 > Noh et al, “LearningDeconvolution Networkfor SemanticSegmentation”CVPR2015 Semantic Segmentation
- 76.
- 77.
- 78.
- 79. http://blog.romanofoti.com/style_transfer/ https://github.com/fchollet/keras/blob/master/examples/neural_style_transfer.py Style Transfer Loss =content_loss + style_loss Content loss = convolutions from pre-trained network Style loss = gram matrix from style image convolutions
- 80.
- 81.
- 82.
- 83. Image & VideoModeration TODO 83 Large internationalgay datingapp with tens of millions of users uploadinghundreds-of-thousandsofphotosperday
- 84. Estimating Accident RepairCost from Photos TODO 84 Prototype for large SA insurer Detect car make & model from registrationdisk Predict repair cost using learnedmodel
- 85.
- 86.
- 87. m Counting People TODO Countshoppers,segment onage & gender facial recognition loyalty is next
- 88.
- 89.
- 90.
- 91. GET IN TOUCH! AlexConway alex @ numberboost.com @alxcnwy
- 92.
- 93.
- 94. Practical Tips • usea GPU – AWS p2 instances(use spot!) • when overfitting (validation_accuracy<<<training_accuracy) – Increase dropout – Early stopping • when underfitting (low training_accuracy) 1. Add moredata 2. Use data augmentation – Flipping/stretching / rotating – Slightly change hues / brightness 3. Use more complex model – Resnet / Inception /Densenet – Ensamble (average<<< weightedaveragewith learnedweights) 94