Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Learning for Video: Motion Estimation (UPC 2018)

1,348 views

Published on

https://mcv-m6-video.github.io/deepvideo-2018/

Overview of deep learning solutions for video processing. Part of a series of slides covering topics like action recognition, action detection, object tracking, object detection, scene segmentation, language and learning from videos.

Prepared for the Master in Computer Vision Barcelona:
http://pagines.uab.cat/mcv/

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Deep Learning for Video: Motion Estimation (UPC 2018)

  1. 1. @DocXavi Xavier Giró-i-Nieto [http://pagines.uab.cat/mcv/] Module 6 Deep Learning for Video: Motion Estimation 22nd March 2018
  2. 2. ● MSc course (2017) ● BSc course (2018) 2 Deep Learning online courses by UPC: ● 1st edition (2016) ● 2nd edition (2017) ● 3rd edition (2018) ● 1st edition (2017) ● 2nd edition (2018) Next edition Autumn 2018 Next edition Winter/Spring 2019Summer School (late June 2018)
  3. 3. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D. and Brox, T., FlowNet: Learning Optical Flow With Convolutional Networks. ICCV 2015 3 End to end supervised learning of optical flow. Motion: Optical Flow: FlowNet
  4. 4. 4 Option A: Stack both input images together and feed them through a generic network. Motion: Optical Flow: FlowNet (encoder) Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D. and Brox, T., FlowNet: Learning Optical Flow With Convolutional Networks. ICCV 2015
  5. 5. 5 Option B: Create two separate, yet identical processing streams for the two images and combine them at a later stage. Motion: Optical Flow: FlowNet (encoder) Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D. and Brox, T., FlowNet: Learning Optical Flow With Convolutional Networks. ICCV 2015
  6. 6. 6 Option B: Create two separate, yet identical processing streams for the two images and combine them at a later stage. Correlation layer: Convolution of data patches from the layers to combine. Motion: Optical Flow: FlowNet (encoder) Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D. and Brox, T., FlowNet: Learning Optical Flow With Convolutional Networks. ICCV 2015
  7. 7. 7 Upconvolutional layers: Unpooling features maps + convolution. Upconvolutioned feature maps are concatenated with the corresponding map from the contractive part. Motion: Optical Flow: FlowNet (decoder) Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D. and Brox, T., FlowNet: Learning Optical Flow With Convolutional Networks. ICCV 2015
  8. 8. 8 Convnets trained on these unrealistic data generalize well to existing datasets such as Sintel and KITTI. Data augmentation Motion: Optical Flow: FlowNet (synthetic) Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D. and Brox, T., FlowNet: Learning Optical Flow With Convolutional Networks. ICCV 2015
  9. 9. 9 Ilg, Eddy, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. "Flownet 2.0: Evolution of optical flow estimation with deep networks." CVPR 2017. [code]
  10. 10. 10 Motion: Optical Flow: FlowNet 2.0 Ilg, Eddy, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. "Flownet 2.0: Evolution of optical flow estimation with deep networks." CVPR 2017. [code]
  11. 11. 11 Motion: MP-Net Tokmakov, Pavel, Karteek Alahari, and Cordelia Schmid. "Learning motion patterns in videos." CVPR 2017 Predict directly if the object is in motion, instead of the optical flow..

×