Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Learning - Overview of my work II

8,437 views

Published on

Deep Learning Machine Learning MNIST CIFAR 10 Residual Network AlexNet VGGNet GoogleNet Nvidia Deep learning (DL) is a hierarchical structure network which through simulates the human brain’s structure to extract the internal and external input data’s features

Published in: Education
  • If you’re struggling with your assignments like me, check out ⇒ www.WritePaper.info ⇐. My friend sent me a link to to tis site. This awesome company. After I was continuously complaining to my family and friends about the ordeals of student life. They wrote my entire research paper for me, and it turned out brilliantly. I highly recommend this service to anyone in my shoes. ⇒ www.WritePaper.info ⇐.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Earn Up To $600/Month or $50/Survey ➣➣➣ https://dwz1.cc/EWG1lhe4
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Gym guru exposes fat burning secret! I'm going to reveal to you the secret method that allows you to get the equivalent exercise of 45 minutes at the gym, in just a few minutes per day! Just watch this short video explaining how it works, JUST CLICK HERE ◆◆◆ http://t.cn/A6PnIVD3
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • MADE $30 ON MY FIRST DAY! Being a fresh graduate and having lots of free time, I stumbled upon your site when I was searching for work at home opportunities, good thing I did! Just on my first day of joining I already made $30! Now I'm averaging close to $80 a day just for filling out surveys! ♥♥♥ https://tinyurl.com/vd3y33w
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Easy and hassle free way to make money online! I have just registered with this site and straight away I was making money! It doesn't get any better than this. Thank you for taking out all the hassle and making money answering surveys as easy as possible even for non-techie guys like me! ●●● https://tinyurl.com/vd3y33w
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Deep Learning - Overview of my work II

  1. 1. Deep Learning Deep Learning II
  2. 2. Deep Learning 0 10 20 30 40 50 60 70 80 90 100 2004-01 2004-03 2004-05 2004-07 2004-09 2004-11 2005-01 2005-03 2005-05 2005-07 2005-09 2005-11 2006-01 2006-03 2006-05 2006-07 2006-09 2006-11 2007-01 2007-03 2007-05 2007-07 2007-09 2007-11 2008-01 2008-03 2008-05 2008-07 2008-09 2008-11 2009-01 2009-03 2009-05 2009-07 2009-09 2009-11 2010-01 2010-03 2010-05 2010-07 2010-09 2010-11 2011-01 2011-03 2011-05 2011-07 2011-09 2011-11 2012-01 2012-03 2012-05 2012-07 2012-09 2012-11 2013-01 2013-03 2013-05 2013-07 2013-09 2013-11 2014-01 2014-03 2014-05 2014-07 2014-09 2014-11 2015-01 2015-03 2015-05 2015-07 2015-09 2015-11 2016-01 2016-03 2016-05 2016-07 2016-09 2016-11 2017-01 2017-03 Google Trends Deep learning Machine learning neural network
  3. 3. Deep Learning
  4. 4. Deep Learning
  5. 5. Deep Learning
  6. 6. Deep Learning  Artificial Narrow Intelligence (ANI): Machine intelligence that equals or exceeds human intelligence or efficiency at a specific task.  Artificial General Intelligence (AGI): A machine with the ability to apply intelligence to any problem, rather than just one specific problem (human-level intelligence).  Artificial Super Intelligence (ASI): An intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills
  7. 7. Deep Learning Machine Learning is a type of Artificial Intelligence that provides computers with the ability to learn Machine Learning Supervised learning Unsupervised learning
  8. 8. Deep Learning  Part of the machine learning field of learning representations of data.  hierarchy of multiple layers that mimic the neural networks of our brain  If you provide the system tons of information, it begins to understand it and respond in useful ways.
  9. 9. Deep Learning  SuperIntelligent Devices  Best Solution for image recognition speech recognition natural language processing Big Data
  10. 10. Deep Learning
  11. 11. Deep Learning Geoffrey Hinton: University of Toronto & Google Yann LeCun: New York University & Facebook Andrew Ng: Stanford & Baidu Yoshua Bengio: University of Montreal
  12. 12. Deep Learning
  13. 13. Deep Learning Today NVidia Support my work with NVIDIA TITAN X THE MOST ADVANCED GPU EVER BUILT
  14. 14. Deep Learning TITAN X Specifications GPU Architecture Pascal Standard Memory Config 12 GB GDDR5X Memory Speed 10 Gbps Boost Clock 1531 MHz NVIDIA CUDA® Cores 3584 Transistors 12,000 million
  15. 15. Deep Learning TITAN X In Research Deep Learning Augmented Reality Machine Learning Image Recognition Computer Vision Data Science
  16. 16. Deep Learning  Deep learning (DL) is a hierarchical structure network which through simulates the human brain’s structure to extract the internal and external input data’s features
  17. 17. Deep Learning Large data set with good quality Measurable and describable goals Enough computing power Neural Network (Brain of Human)
  18. 18. Deep Learning Deep neural networks Deep belief networks Convolutional neural networks Deep Boltzmann machines Deep stacking networks
  19. 19. Deep Learning Axon Terminal Branches of Axon Dendrites S x1 x2 w1 w2 wn xn x3 w3
  20. 20. Deep Learning
  21. 21. Deep Learning  The advantages of using Rectified Linear Units in neural networks are: ReLU doesn't face gradient vanishing problem as with sigmoid and tanh function. It has been shown that deep networks can be trained efficiently using ReLU even without pre-training.
  22. 22. Deep Learning  Convolution Neural Networks (CNN) is supervised learning and a family of multi-layer neural networks particularly designed for use on two dimensional data, such as images and videos.  A CNN consists of a number of layers:  Convolutional layers.  Pooling Layers.  Fully-Connected Layers.
  23. 23. Deep Learning
  24. 24. Deep Learning
  25. 25. Deep Learning  Convolutional layer acts as a feature extractor that extracts features of the inputs such as edges, corners , endpoints.
  26. 26. Deep Learning
  27. 27. Deep Learning
  28. 28. Deep Learning  The pooling layer reduces the resolution of the image that reduce the precision of the translation (shift and distortion) effect.
  29. 29. Deep Learning
  30. 30. Deep Learning
  31. 31. Deep Learning  fully connected layer have full connections to all activations in the previous layer.  Fully connect layer act as classifier.
  32. 32. Deep Learning LeNet :The first successful applications of CNN AlexNet: The first work that popularized CNN in Computer Vision ZF Net: The ILSVRC 2013 winner GoogLeNet: The ILSVRC 2014 winner VGGNet: The runner-up in ILSVRC 2014 ResNet: The winner of ILSVRC 2015
  33. 33. Deep Learning
  34. 34. Deep Learning
  35. 35. Deep Learning
  36. 36. Deep Learning The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) evaluates algorithms for object detection and image classification at large scale.
  37. 37. Deep Learning
  38. 38. Deep Learning MNIST Handwritten digits – 60000 Training + 10000 Test Data Google House Numbers from street view - 600,000 digit images CIFAR-10 60000 32x32 colour images in 10 classes IMAGENET >150 GB Tiny Images 80 Million tiny images Flickr Data 100 Million Yahoo dataset
  39. 39. Deep Learning  MNIST is a large database of handwritten digits.  MNIST contains 60,000 training images and 10,000 testing images
  40. 40. Deep Learning  CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes  CIFAR-10 contains 50000 training images and 10000 test images
  41. 41. Deep Learning  Overfitting Problem  Larger network have a lots of weights this lead to high model complexity  Network do excellent on training data but very bad on validation data
  42. 42. Deep Learning  CNN Optimization used to reduce the overfitting problem in CNN by: 1) Dropout 2) L2 Regularization 3) Mini-batch 4) Gradient descent algorithm 5) Early stopping 6) Data augmentation
  43. 43. Deep Learning  Dropout is a technique of reducing overfitting in CNN.
  44. 44. Deep Learning  L2 Regularization: Adding a regularization term for the weights to the loss function is a way to reduce overfitting.  where w is the weight vector, λ is the regularization factor (coefficient), and the regularization function, Ω(w) is:
  45. 45. Deep Learning  Mini-batch is to divide the dataset into small batches of examples, compute the gradient using a single batch, make an update, then move to the next batch.
  46. 46. Deep Learning  The gradient descent algorithm updates the coefficients (weights and biases) so as to minimize the error function by taking small steps in the direction of the negative gradient of the loss function  where i stands for the iteration number, α > 0 is the learning rate, P is the parameter vector, and E(Pi) is the loss function.
  47. 47. Deep Learning  Early stopping monitoring the deep learning process of the network from overfitting.  If there is no more improvement, or worse, the performance on the test set degrades, then the learning process is aborted
  48. 48. Deep Learning  Data augmentation means increasing the number of dataset.
  49. 49. Deep Learning  MADBase is Arabic Handwritten Digit Dataset composed of 70,000 digits written by 700 writers.  MADBase is partitioned into two data sets:  60,000 Training Data  10,000 Testing Data
  50. 50. Deep Learning  We built a new CNN architecture:
  51. 51. Deep Learning  Confusion Matrix
  52. 52. Deep Learning  We collect a dataset that composed of 16,800 characters written by 60 participants, the age range is between 19 to 40 years.  The forms were scanned at the resolution of 300 dpi. Each block is segmented automatically using Matlab 2016a to determining the coordinates for each block.  The database is partitioned into two sets: a training set (13,440 characters to 480 images per class) and a test set (3,360 characters to 120 images per class).
  53. 53. Deep Learning  Each participant wrote each character (from ’alef’ to ’yeh’) ten times on two forms
  54. 54. Deep Learning  We built a new CNN architecture:
  55. 55. Deep Learning  Confusion Matrix  Error Rate= 5.15% Class 1 2 3 4 5 6 7 Arabic Character alef beh teh theh jeem hah khah Correct Classification 120 116 110 110 115 117 112 Wrong Classification 0 4 10 10 5 3 8 Classification Accuracy 100% 96.70% 91.70% 91.70% 95.80% 97.50% 93.30% Miss-Classification 0.00% 3.30% 8.30% 8.30% 4.20% 2.50% 6.70% Class 8 9 10 11 12 13 14 Arabic Character dal thal reh zain seen sheen sad Correct Classification 114 110 120 105 117 115 118 Wrong Classification 6 10 0 15 3 5 2 Classification Accuracy 95.00% 91.70% 100%% 87.50% 79.50% 95.80% 98.70% Miss-Classification 5.00% 8.30% 0.00% 12.50% 2.50% 4.20% 1.70% Class 15 16 17 18 19 20 21 Arabic Character dad tah zah ain ghain feh qaf Correct Classification 109 116 110 113 112 114 111 Wrong Classification 11 4 10 7 8 6 9 Classification Accuracy 90.80% 96.70% 91.70% 94.20% 93.30% 95.00% 92.50% Miss-Classification 9.20% 3.30% 8.30% 5.80% 6.70% 5.00% 7.50% Class 22 23 24 25 26 27 28 Arabic Character kaf lam meem noon heh waw yeh Correct Classification 114 119 119 106 114 115 116 Wrong Classification 6 1 1 14 6 5 4 Classification Accuracy 95.00% 99.20% 99.20% 88.30% 95.00% 95.80% 96.70% Miss-Classification 5.00% 0.80% 0.80% 11.70% 5.00% 4.20% 3.30%
  56. 56. Deep Learning  The total of wrong classification is 173 from 3187.
  57. 57. Deep Learning  Deep learning is a class of machine learning algorithms.  Harder problems such as video understanding, image understanding , natural language processing and Big data will be successfully tackled by deep learning algorithms.
  58. 58. Deep Learning facebook.com/mloey mohamedloey@gmail.com twitter.com/mloey linkedin.com/in/mloey mloey@fci.bu.edu.eg mloey.github.io
  59. 59. Deep Learning www.YourCompany.com © 2020 Companyname PowerPoint Business Theme. All Rights Reserved. THANKS FOR YOUR TIME

×