Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)

3,790 views

Published on

http://imatge-upc.github.io/telecombcn-2016-dlcv/

Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.

Published in: Data & Analytics
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The video stops at 8:32. Maybe it is corrupted. Could you please correct it?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)

  1. 1. Day 2 Lecture 6 Recurrent Neural Networks Xavier Giró-i-Nieto [course site]
  2. 2. 2 Acknowledgments Santi Pascual
  3. 3. 3 General idea ConvNet (or CNN)
  4. 4. 4 General idea ConvNet (or CNN)
  5. 5. 5 Multilayer Perceptron Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” The output depends ONLY on the current input.
  6. 6. 6 Recurrent Neural Network (RNN) Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” The hidden layers and the output depend from previous states of the hidden layers
  7. 7. 7 Recurrent Neural Network (RNN) Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” The hidden layers and the output depend from previous states of the hidden layers
  8. 8. 8 Recurrent Neural Network (RNN) time time Rotation 90o Front View Side View Rotation 90o
  9. 9. 9 Recurrent Neural Networks (RNN) Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” Each node represents a layer of neurons at a single timestep. tt-1 t+1
  10. 10. 10 Recurrent Neural Networks (RNN) Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” tt-1 t+1 The input is a SEQUENCE x(t) of any length.
  11. 11. 11 Recurrent Neural Networks (RNN) Common visual sequences: Still image Spatial scan (zigzag, snake) The input is a SEQUENCE x(t) of any length.
  12. 12. 12 Recurrent Neural Networks (RNN) Common visual sequences: Video Temporal sampling The input is a SEQUENCE x(t) of any length. ... t
  13. 13. 13 Recurrent Neural Networks (RNN) Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” Must learn temporally shared weights w2; in addition to w1 & w3. tt-1 t+1
  14. 14. 14 Bidirectional RNN (BRNN) Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” Must learn weights w2, w3, w4 & w5; in addition to w1 & w6.
  15. 15. 15 Alex Graves, “Supervised Sequence Labelling with Recurrent Neural Networks” Bidirectional RNN (BRNN)
  16. 16. 16Slide: Santi Pascual Formulation: One hidden layer Delay unit (z-1 )
  17. 17. 17Slide: Santi Pascual Formulation: Single recurrence One-time Recurrence
  18. 18. 18Slide: Santi Pascual Formulation: Multiple recurrences Recurrence One time-step recurrence T time steps recurrences
  19. 19. 19Slide: Santi Pascual RNN problems Long term memory vanishes because of the T nested multiplications by U. ...
  20. 20. 20Slide: Santi Pascual RNN problems During training, gradients may explode or vanish because of temporal depth. Example: Back-propagation in time with 3 steps.
  21. 21. 21 Long Short-Term Memory (LSTM)
  22. 22. 22 Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9, no. 8 (1997): 1735-1780. Long Short-Term Memory (LSTM)
  23. 23. 23Figure: Cristopher Olah, “Understanding LSTM Networks” (2015) Long Short-Term Memory (LSTM) Based on a standard RNN whose neuron activates with tanh...
  24. 24. 24 Long Short-Term Memory (LSTM) Ct is the cell state, which flows through the entire chain... Figure: Cristopher Olah, “Understanding LSTM Networks” (2015)
  25. 25. 25 Long Short-Term Memory (LSTM) ...and is updated with a sum instead of a product. This avoid memory vanishing and exploding/vanishing backprop gradients. Figure: Cristopher Olah, “Understanding LSTM Networks” (2015)
  26. 26. 26 Long Short-Term Memory (LSTM) Three gates are governed by sigmoid units (btw [0,1]) define the control of in & out information.. Figure: Cristopher Olah, “Understanding LSTM Networks” (2015)
  27. 27. 27 Long Short-Term Memory (LSTM) Forget Gate: Concatenate Figure: Cristopher Olah, “Understanding LSTM Networks” (2015) / Slide: Alberto Montes
  28. 28. 28 Long Short-Term Memory (LSTM) Input Gate Layer New contribution to cell state Classic neuron Figure: Cristopher Olah, “Understanding LSTM Networks” (2015) / Slide: Alberto Montes
  29. 29. 29 Long Short-Term Memory (LSTM) Update Cell State (memory): Figure: Cristopher Olah, “Understanding LSTM Networks” (2015) / Slide: Alberto Montes
  30. 30. 30 Long Short-Term Memory (LSTM) Output Gate Layer Output to next layer Figure: Cristopher Olah, “Understanding LSTM Networks” (2015) / Slide: Alberto Montes
  31. 31. 31 Long Short-Term Memory (LSTM) Figure: Cristopher Olah, “Understanding LSTM Networks” (2015) / Slide: Alberto Montes
  32. 32. 32 Gated Recurrent Unit (GRU) Cho, Kyunghyun, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." AMNLP 2014. Similar performance as LSTM with less computation.
  33. 33. 33Figure: Cristopher Olah, “Understanding LSTM Networks” (2015) / Slide: Alberto Montes Gated Recurrent Unit (GRU)
  34. 34. 34 Applications: Machine Translation Cho, Kyunghyun, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." arXiv preprint arXiv:1406.1078 (2014). Language IN Language OUT
  35. 35. 35 Applications: Image Classification van den Oord, Aaron, Nal Kalchbrenner, and Koray Kavukcuoglu. "Pixel Recurrent Neural Networks." arXiv preprint arXiv:1601.06759 (2016). RowLSTM Diagonal BiLSTM Classification MNIST
  36. 36. 36 Applications: Segmentation Francesco Visin, Marco Ciccone, Adriana Romero, Kyle Kastner, Kyunghyun Cho, Yoshua Bengio, Matteo Matteucci, Aaron Courville, “ReSeg: A Recurrent Neural Network-Based Model for Semantic Segmentation”. DeepVision CVPRW 2016.
  37. 37. 37 Learn more Nando de Freitas (University of Oxford)
  38. 38. 38 Thanks ! Q&A ? Follow me at https://imatge.upc.edu/web/people/xavier-giro @DocXavi /ProfessorXavi

×