Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Deep learning

181 views

Published on

My slides on 'Introduction to Deep learning' presented for the first
meeting of deep learning @ Leeds community.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Introduction to Deep learning

  1. 1. Introduction to Deep Learning Leo Pauly 1st year PhD Researcher in Computer vision & Deep Learning University of Leeds
  2. 2. Contents • What is deep learning ? • Convolutional neural networks : explained • My research : Deep learning based Image segmentation • Applications in different domains • Deep learning with HPC Leeds • Future activities of Deep Learning @ Leeds
  3. 3. TLU : threshold logic unit 1943: Warren McCulloch and Walter Pitts create a computational model for neural networks based on mathematics and algorithms called threshold logic.
  4. 4. Perceptron 1958: Frank Rosenblatt creates the perceptron, an algorithm for pattern recognition based on a two-layer computer neural network using simple addition and subtraction Activation function
  5. 5. Similarity with biological neurons
  6. 6. A simple neural network
  7. 7. How neural networks work Optimisation algorithm : Mini batch stochastic gradient descendant algorithm
  8. 8. AI winter (90’s to early 2000) Computational power Difficulty to train larger network : Vanishing gradient problem Lack of large dataset
  9. 9. Resurgence as Deep learning (Mid 2000-Present)
  10. 10. Resurgence as Deep learning (Mid 2000-Present) Vanishing gradient problem : ReLUs [1] [1] Nair, Vinod, and Geoffrey E. Hinton. "Rectified linear units improve restricted boltzmann machines." Proceedings of the 27th international conference on machine learning (ICML-10). 2010
  11. 11. Resurgence as Deep learning (Mid 2000-Present) Computational powerVanishing gradient problem : ReLUs
  12. 12. Resurgence as Deep learning (Mid 2000-Present) Computational powerVanishing gradient problem : ReLUs Lack of large dataset
  13. 13. Resurgence as Deep learning (Mid 2000-Present) Articles to read: - A brief historyof neural nets and deep learning - Welcome to the AI Conspiracy: The 'Canadian Mafia' Yann LeCun, New York University & Facebook Yoshua Bengio, Universite de Montreal Geoffrey Hinton, Google & University of Toronto Jurgen Schmidhuber, Dalle Molle Institutefor ArtificialIntelligence Research
  14. 14. Deeper neural networks rebranded as deep learning
  15. 15. Deeper neural networks rebranded as deep learning
  16. 16. Deeper neural networks rebranded as deep learning
  17. 17. Revolution of depth
  18. 18. Amount of data vs performance
  19. 19. Types of networks used for deep learning • Convolutional neural networks
  20. 20. Types of networks used for deep learning • Convolutional neural networks • Recurrent neural networks
  21. 21. Types of networks used for deep learning • Convolutional neural networks • Recurrent neural networks • Long Short term memory (LSTM) networks
  22. 22. Types of networks used for deep learning • Convolutional neural networks • Recurrent neural networks • Long Short term memory (LSTM) networks • Deep Boltzmann machines
  23. 23. Types of networks used for deep learning • Convolutional neural networks • Recurrent neural networks • Long Short term memory (LSTM) networks • Deep Boltzmann machines • Deep Q-networks
  24. 24. Types of networks used for deep learning • Convolutional neural networks • Recurrent neural networks • Long Short term memory (LSTM) networks • Deep Boltzmann machines • Deep Q-networks • Deep belief networks
  25. 25. Types of networks used for deep learning • Convolutional neural networks • Recurrent neural networks • Long Short term memory (LSTM) networks • Deep Boltzmann machines • Deep Q-networks • Deep belief networks • Deep stacking networks
  26. 26. Convolutional Neural Networks
  27. 27. 1. Convolutional operation in convolutional layer
  28. 28. Link: chttps://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/ 3.Pooling operation in pooling layer2. ReLU operating in activation layer
  29. 29. My research My research interests: Intersection of Computer vision, Machine Learning & Robotics My current research focus for PhD: Image segmentation & Deep learning Theoretical level: - Conditional/ Parametric CNNs for segmentation - Transfer learning for segmentation Application level: - Pavement crack segmentation
  30. 30. Parametric / Conditional CNN Exploring the possibility of using extra parameter to steer the output of the network in one direction Additional control parameter Input image Output image
  31. 31. Transfer learning Training machine learning models in one domain and deploying it in another domain: Training: Deploying: Deep learning model Deep learning model
  32. 32. Applications of Deep learning
  33. 33. Applications of Deep learning
  34. 34. Applications of Deep learning
  35. 35. Applications of Deep learning
  36. 36. Applications of Deep learning
  37. 37. Applications of Deep learning
  38. 38. Applications of Deep learning
  39. 39. Computer Vision Object detection [2] Image classification [1] Image segmentation [3] Edge detection [4] [1] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012. [2] Girshick, Ross. "Fast r-cnn." Proceedings of the IEEE International Conference on Computer Vision. 2015. [3] Zheng, Shuai, et al. "Conditional random fields as recurrent neural networks." Proceedings of the IEEE International Conference on Computer Vision. 2015. [4] Xie, Saining, and Zhuowen Tu. "Holistically-nested edge detection." Proceedings of the IEEE International Conference on Computer Vision. 2015.
  40. 40. [1]Kafle, Kushal, and Christopher Kanan. "Visual Question Answering: Datasets, Algorithms, and Future Challenges." arXiv preprint arXiv:1610.01465 (2016). Natural Language processing Visual question answering [1]
  41. 41. [1] Levine, Sergey, et al. "Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection." arXiv preprint arXiv:1603.02199 (2016). [2] Chen, Chenyi, et al. "Deepdriving: Learning affordance for direct perception in autonomous driving." Proceedings of the IEEE International Conference on Computer Vision. 2015. Robotics Grasping objects[1] Autonomous driving[2]
  42. 42. [1] Esteva, Andre, et al. "Dermatologist-level classification of skin cancer with deep neural networks." Nature 542.7639 (2017): 115-118. [2] Maninis, Kevis-Kokitsi, et al. "Deep retinal image understanding." International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer International Publishing, 2016. [3] Ramsundar, Bharath, et al. "Massively multitask networks for drug discovery." arXiv preprint arXiv:1502.02072 (2015). Medicine Skin cancer classification[1] Retinal vessel segmentation[2] Drug Discovery[3]
  43. 43. Several other applications - Agriculture - Game playing systems : AlphaGo - Language-Language translation - Synthetic sound generation - Deep reinforcement learning in robotics - So on……..
  44. 44. Deep learning using containers at HPC, Leeds What & Why containers ? + + + OpenCv + NLTK+ Numpy + Anaconda = DS1 Container in HPC What it means to user ? - Load container engine : Singularity / Docker - Load container image - Start writing code Courtesy : Martin Callaghan, HPC, University of Leeds
  45. 45. Deep learning using containers at HPC, Leeds Courtesy : Martin Callaghan, HPC, University of Leeds - Demo…!! - Guide - Course on June 26th : Dockers & Containers - P100s coming up…..!!!!!
  46. 46. Discussion : Deep learning @ Leeds • Regular meetings : - Start with monthly meetings ( July- Oct) - Followed by biweekly meetings (1 research group meeting + 1 talk by a speaker) • Mailing list • Lightening talks @ Departments • Robotics away day • General discussion…!!
  47. 47. cnlp@leeds.ac.uk

×