SlideShare a Scribd company logo
1 of 29
Download to read offline
Optical flow estimation
with Blueoil
K-Inoue @ki42 & Oscar @wang
Blueoil Division
1
What is optical flow estimation?
● Optical flow...
represents the apparent motion of objects.
● Optical flow estimation...
can predict the movement of objects in a video.
Miloud, Hadj achour. (2017). Fragmentation de métal liquide dans l’eau.
https://www.codeproject.com/Articles/1192205/Capturing-motion-from-video-using-the-Emgu-CV-libr
2
Optical flow estimation is important
● Widely used by insects and birds
● Practical usage
○ Analyze motion
○ Avoid collision
○ Assist in navigation
● Real-world Applications
○ Video/Motion classification
○ Navigation assistance
■ Self driving cars
■ Drones https://nanonets.com/blog/optical-flow/
3
DL approaches are increasing
4
● 2015. FlowNet S (Simple)
● 2015. FlowNet C (Correlated)
● 2016. FlowNet 2
● 2018. LiteFlowNet
● ...
Color coding
https://www.youtube.com/watch?v=k_wkDLJ8lJE https://www.youtube.com/watch?v=pfQ0zFwv-hM
However...
● Existing DL approaches require GPU to execute
👎 High power consumption
👎 Low runtime speed on CPU environment
● We propose LmFlowNet S 👍
○ Modification of FlowNet S [P. Fischer+ 2015]
○ Goals:
■ Edge Computing
■ Run on FPGA-based accelerator
■ Use quantization to reduce inference time
while achieving good prediction performance
5
FlowNet S vs LmFlowNet S
6
Network of FlowNet S
7
[N, 384, 512, 6]
[N, 384, 512, 2]
Detailed ops inside each color block is shown in appendix
Network of FlowNet S
8
[N, 384, 512, 6]
[N, 384, 512, 2]
Encoder
Decoder Detailed ops inside each color block is shown in appendix
Network of FlowNet S
9
[N, 384, 512, 6]
[N, 384, 512, 2]
Detailed ops inside each color block is shown in appendix
Not supported
by Blueoil DLK
10
Network of LmFlowNet S (DLK supported)
[N, 384, 512, 6]
[N, 384, 512, 2]
Detailed ops inside each color block is shown in appendix
11
Network of LmFlowNet S
[N, 384, 512, 6]
[N, 384, 512, 2]
Quantized
Detailed quantization inside each color block is shown in appendix
Loss function: End Point Error (EPE)
12
(x1
,y1
)
(x2
,y2
)
EPEflow2
EPEflow3
EPEflow4
EPEflow5
EPEflow6
Weighted EPE =
0.32 * EPEflow2
+
0.08 * EPEflow3
+
0.02 * EPEflow4
+
0.01 * EPEflow5
+
0.005 * EPEflow6
Down-
sampled
Ground
Truth
Training hyper-parameters are shown in appendix
24 x 32
48 x 6496 x 128
192 x 256
12 x 16
Artificial dataset: Flying Chairs
13
● Dataset
Name Frame pairs Train validation ratio size
Flying Chairs 22,872 9:1 30GB
● Data Augmentation
○ Crop, Rotate, Translate, FlipLeftRight, FlipTopBottom
○ Gaussian noise, Brightness, Contrast, Gamma, and Color
Parameters used in data augmentation are shown in appendix
https://arxiv.org/pdf/1504.06852.pdf
14
Results
15
Results - Avg. EPE & Inference time
Method
Avg. EPE (pixel)
(Flying Chairs)
Inference time per frame (ms) [1]
CPU (dlk-convert) GPU (tensorflow)
FlowNet S 2.94 - 11.65
LmFlowNet S 5.33 1360.49 13.81
LmFlowNet S
Quantized
9.01 637.467 17.60
[1] CPU and GPU specs available in appedix
FlowNet S
Trained for 1.2M
LmFlowNet S
Trained for 400K
LmFlowNet S Quantized
Trained for 400K
16
Live Demo
17
Live demonstration
● Three demonstration
○ FlowNet S
○ LmFlowNet S
○ LmFlowNet S Quantized
● NOTE: Running on GPU (not on CPU / FPGA)
○ Failed to run on CPU/FPGA due to several problems 😢
■ etc. segmentation fault, memory error...
○ Fixing and debugging them in the future 👊
18
Challenges
19
Challenges
● Training takes a very long time ( > 2 weeks...😢)
○ Heavy data augmentation & pre-processing
■ Pre-processing on GPU is not supported now.
● Unique network structure, not compatible with Blueoil
○ Input is a stack of 2 images (6 channels)
○ Multiple and branched outputs
● DLK Limitation. No documentation. 🤯
○ No support for kernel size 7x7, 5x5
○ No support for Conv2dTranspose
○ Cannot concat quantized value and float together
○ Requires the depth of Space2Depth to be 32 * N 20
Thank you for your
attention!
our source code:
https://github.com/ki-lm/blueoil/tree/lmflownets
21
22
Appendix
23
24
LmFlowNet S | Training & Hyper-params
● Optimizer: Adam
● Max steps: 1200k
● Fixed parameters in Adam: β1
=0.9 and β2
= 0.999.
● Batch size: 8
● Learning rate
○ values: [1e-4
, 5e-5
, 2.5e-5
,1.25e-5
,6.25e-6
]
○ boundaries: [400K, 600K, 800K, 1000K]
● Learning rate for quantization
○ values: [1.25e-5
, 1e-4
, 5e-5
, 2.5e-5
,1.25e-5
,6.25e-6
]
○ boundaries: [50K, 400K, 600K, 800K, 1000K]
25
Tokunaga Scheduling 😆
LmFlowNet S | Data Augmentation
● Translation: [20%, 20%] of the image width for x and y
● Rotation: [17o
, 17o
]
● Scaling: [0.9, 2.0]
● Gaussian noise: sigma uniformly sampled from [0, 0.04]
● Contrast: [0.8, 0.4]
● Multiplicative color changes to the RGB channels per
image: [0.5, 2]
● Gamma values: [0.7, 1.5]
● Additive brightness changes: Gaussian with a sigma of
0.2
26
27
CPU & GPU specs
● CPU
○ Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz
○ 4 cores, 8 threads
○ Inference time (ms)
■ LmFlowNet S: 1358.39, 1358.39, 1293.26, 1307.28, 1485.11
■ LmFlowNet S Quant: 640.541, 640.732, 635.354, 635.354, 635.354
● GPU
○ NVIDIA Tesla V100 on DGX-1
○ Inference time (ms)
■ FlowNet S: 0.0111, 0.0105, 0.0116, 0.0117, 0.0119, 0.0118, 0.0121, 0.0118, 0.0116,
0.0124
■ LmFlowNet S: 0.0137, 0.0140, 0.0137, 0.0136, 0.0141, 0.0135, 0.0139, 0.0138, 0.0136,
0.0142
■ LmFlowNet S Quant: 0.0168, 0.0161, 0.0165, 0.0177, 0.0173, 0.0166, 0.0186, 0.0195,
0.0183, 0.0187
Our FlowNet S versions
28
Version Architecture DLK support
V1
(FlowNet S)
Same as the paper X
V2
7x7, 5x5 => 3x3
Striding 2 => SpaceToDepth
X
V3
Conv2dTranspose =>
ResizeNearestNeighbor + Conv2d
ResizeBilinear =>
ResizeNearestNeighbor
X
V3 Quant.
(LmFlowNet S)
Quantize except first, last layer,
and activation before last layer
△
V4 Quant.
Change all output depths from
SpaceToDepth to 32 * N
O
List of source code links
● FlowNet S/C, and 2 (TensorFlow):
https://github.com/sampepose/flownet2-tf/
● FlowNet S/C (Original paper, Caffe):
https://lmb.informatik.uni-freiburg.de/Publications/2015/DF
IB15/
● FlowNet 2 (Original paper, Caffe):
https://github.com/lmb-freiburg/flownet2
29

More Related Content

What's hot

Pixel Recurrent Neural Networks
Pixel Recurrent Neural NetworksPixel Recurrent Neural Networks
Pixel Recurrent Neural Networksneouyghur
 
Optic flow estimation with deep learning
Optic flow estimation with deep learningOptic flow estimation with deep learning
Optic flow estimation with deep learningYu Huang
 
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Universitat Politècnica de Catalunya
 
Visual odometry & slam utilizing indoor structured environments
Visual odometry & slam utilizing indoor structured environmentsVisual odometry & slam utilizing indoor structured environments
Visual odometry & slam utilizing indoor structured environmentsNAVER Engineering
 
BRDFモデルの変遷
BRDFモデルの変遷BRDFモデルの変遷
BRDFモデルの変遷Teppei Kurita
 
[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the Wild
[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the Wild[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the Wild
[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the WildDeep Learning JP
 
Deep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentationDeep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentation경훈 김
 
[DL輪読会]In Search of Lost Domain Generalization
[DL輪読会]In Search of Lost Domain Generalization[DL輪読会]In Search of Lost Domain Generalization
[DL輪読会]In Search of Lost Domain GeneralizationDeep Learning JP
 
Applying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksApplying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksDatabricks
 
오토인코더의 모든 것
오토인코더의 모든 것오토인코더의 모든 것
오토인코더의 모든 것NAVER Engineering
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networksYunjey Choi
 
AlexNet, VGG, GoogleNet, Resnet
AlexNet, VGG, GoogleNet, ResnetAlexNet, VGG, GoogleNet, Resnet
AlexNet, VGG, GoogleNet, ResnetJungwon Kim
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep LearningSebastian Ruder
 
20211019 When does label smoothing help_shared ver
20211019 When does label smoothing help_shared ver20211019 When does label smoothing help_shared ver
20211019 When does label smoothing help_shared verHsing-chuan Hsieh
 
Generating Diverse High-Fidelity Images with VQ-VAE-2
Generating Diverse High-Fidelity Images with VQ-VAE-2Generating Diverse High-Fidelity Images with VQ-VAE-2
Generating Diverse High-Fidelity Images with VQ-VAE-2harmonylab
 
[DL輪読会]Generative Models of Visually Grounded Imagination
[DL輪読会]Generative Models of Visually Grounded Imagination[DL輪読会]Generative Models of Visually Grounded Imagination
[DL輪読会]Generative Models of Visually Grounded ImaginationDeep Learning JP
 
Variational Autoencoder
Variational AutoencoderVariational Autoencoder
Variational AutoencoderMark Chang
 
Object Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksObject Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksUsman Qayyum
 

What's hot (20)

Pixel Recurrent Neural Networks
Pixel Recurrent Neural NetworksPixel Recurrent Neural Networks
Pixel Recurrent Neural Networks
 
Optic flow estimation with deep learning
Optic flow estimation with deep learningOptic flow estimation with deep learning
Optic flow estimation with deep learning
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
 
Visual odometry & slam utilizing indoor structured environments
Visual odometry & slam utilizing indoor structured environmentsVisual odometry & slam utilizing indoor structured environments
Visual odometry & slam utilizing indoor structured environments
 
BRDFモデルの変遷
BRDFモデルの変遷BRDFモデルの変遷
BRDFモデルの変遷
 
[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the Wild
[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the Wild[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the Wild
[DL輪読会]Domain Adaptive Faster R-CNN for Object Detection in the Wild
 
Deep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentationDeep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentation
 
[DL輪読会]In Search of Lost Domain Generalization
[DL輪読会]In Search of Lost Domain Generalization[DL輪読会]In Search of Lost Domain Generalization
[DL輪読会]In Search of Lost Domain Generalization
 
Wasserstein GAN
Wasserstein GANWasserstein GAN
Wasserstein GAN
 
Applying your Convolutional Neural Networks
Applying your Convolutional Neural NetworksApplying your Convolutional Neural Networks
Applying your Convolutional Neural Networks
 
오토인코더의 모든 것
오토인코더의 모든 것오토인코더의 모든 것
오토인코더의 모든 것
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
AlexNet, VGG, GoogleNet, Resnet
AlexNet, VGG, GoogleNet, ResnetAlexNet, VGG, GoogleNet, Resnet
AlexNet, VGG, GoogleNet, Resnet
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep Learning
 
20211019 When does label smoothing help_shared ver
20211019 When does label smoothing help_shared ver20211019 When does label smoothing help_shared ver
20211019 When does label smoothing help_shared ver
 
Generating Diverse High-Fidelity Images with VQ-VAE-2
Generating Diverse High-Fidelity Images with VQ-VAE-2Generating Diverse High-Fidelity Images with VQ-VAE-2
Generating Diverse High-Fidelity Images with VQ-VAE-2
 
[DL輪読会]Generative Models of Visually Grounded Imagination
[DL輪読会]Generative Models of Visually Grounded Imagination[DL輪読会]Generative Models of Visually Grounded Imagination
[DL輪読会]Generative Models of Visually Grounded Imagination
 
Variational Autoencoder
Variational AutoencoderVariational Autoencoder
Variational Autoencoder
 
Object Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksObject Detection using Deep Neural Networks
Object Detection using Deep Neural Networks
 

Similar to Final presentation optical flow estimation with DL

DALL-E.pdf
DALL-E.pdfDALL-E.pdf
DALL-E.pdfdsfajkh
 
IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」Preferred Networks
 
VRP2013 - Comp Aspects VRP
VRP2013 - Comp Aspects VRPVRP2013 - Comp Aspects VRP
VRP2013 - Comp Aspects VRPVictor Pillac
 
High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...
High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...
High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...IOSR Journals
 
Landmark Retrieval & Recognition
Landmark Retrieval & RecognitionLandmark Retrieval & Recognition
Landmark Retrieval & Recognitionkenluck2001
 
rit seminars-privacy assured outsourcing of image reconstruction services in ...
rit seminars-privacy assured outsourcing of image reconstruction services in ...rit seminars-privacy assured outsourcing of image reconstruction services in ...
rit seminars-privacy assured outsourcing of image reconstruction services in ...thahirakabeer
 
Online advertising and large scale model fitting
Online advertising and large scale model fittingOnline advertising and large scale model fitting
Online advertising and large scale model fittingWush Wu
 
Fast Fingerprint Classification with Deep Neural Networks
Fast Fingerprint Classification with Deep Neural NetworksFast Fingerprint Classification with Deep Neural Networks
Fast Fingerprint Classification with Deep Neural NetworksDaniel Michelsanti
 
Semantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImagerySemantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImageryRAHUL BHOJWANI
 
B Eng Final Year Project Presentation
B Eng Final Year Project PresentationB Eng Final Year Project Presentation
B Eng Final Year Project Presentationjesujoseph
 
Toronto meetup 20190917
Toronto meetup 20190917Toronto meetup 20190917
Toronto meetup 20190917Bill Liu
 
One-Pass Clustering Superpixels
One-Pass Clustering SuperpixelsOne-Pass Clustering Superpixels
One-Pass Clustering SuperpixelsKesavan Yogarajah
 
AI optimizing HPC simulations (presentation from 6th EULAG Workshop)
AI optimizing HPC simulations (presentation from  6th EULAG Workshop)AI optimizing HPC simulations (presentation from  6th EULAG Workshop)
AI optimizing HPC simulations (presentation from 6th EULAG Workshop)byteLAKE
 
Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759
Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759
Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759Vandna Sambyal
 
Runtime Performance Optimizations for an OpenFOAM Simulation
Runtime Performance Optimizations for an OpenFOAM SimulationRuntime Performance Optimizations for an OpenFOAM Simulation
Runtime Performance Optimizations for an OpenFOAM SimulationFisnik Kraja
 
1c03projectlinkedin
1c03projectlinkedin1c03projectlinkedin
1c03projectlinkedinKeyur Patel
 
The Principle Of Ultrasound Imaging System
The Principle Of Ultrasound Imaging SystemThe Principle Of Ultrasound Imaging System
The Principle Of Ultrasound Imaging SystemMelissa Luster
 
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...Mohamed Elawady
 

Similar to Final presentation optical flow estimation with DL (20)

DALL-E.pdf
DALL-E.pdfDALL-E.pdf
DALL-E.pdf
 
IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」IIBMP2019 講演資料「オープンソースで始める深層学習」
IIBMP2019 講演資料「オープンソースで始める深層学習」
 
VRP2013 - Comp Aspects VRP
VRP2013 - Comp Aspects VRPVRP2013 - Comp Aspects VRP
VRP2013 - Comp Aspects VRP
 
High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...
High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...
High Speed and Time Efficient 1-D DWT on Xilinx Virtex4 DWT Using 9/7 Filter ...
 
Landmark Retrieval & Recognition
Landmark Retrieval & RecognitionLandmark Retrieval & Recognition
Landmark Retrieval & Recognition
 
rit seminars-privacy assured outsourcing of image reconstruction services in ...
rit seminars-privacy assured outsourcing of image reconstruction services in ...rit seminars-privacy assured outsourcing of image reconstruction services in ...
rit seminars-privacy assured outsourcing of image reconstruction services in ...
 
Online advertising and large scale model fitting
Online advertising and large scale model fittingOnline advertising and large scale model fitting
Online advertising and large scale model fitting
 
Fast Fingerprint Classification with Deep Neural Networks
Fast Fingerprint Classification with Deep Neural NetworksFast Fingerprint Classification with Deep Neural Networks
Fast Fingerprint Classification with Deep Neural Networks
 
Semantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImagerySemantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite Imagery
 
B Eng Final Year Project Presentation
B Eng Final Year Project PresentationB Eng Final Year Project Presentation
B Eng Final Year Project Presentation
 
Electrolux meetup
Electrolux meetupElectrolux meetup
Electrolux meetup
 
Toronto meetup 20190917
Toronto meetup 20190917Toronto meetup 20190917
Toronto meetup 20190917
 
One-Pass Clustering Superpixels
One-Pass Clustering SuperpixelsOne-Pass Clustering Superpixels
One-Pass Clustering Superpixels
 
AI optimizing HPC simulations (presentation from 6th EULAG Workshop)
AI optimizing HPC simulations (presentation from  6th EULAG Workshop)AI optimizing HPC simulations (presentation from  6th EULAG Workshop)
AI optimizing HPC simulations (presentation from 6th EULAG Workshop)
 
Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759
Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759
Optimizedfeedforwardnetworkofcnnwithxnorv5 180321130759
 
Optimized feedforward network of cnn with xnor v5
Optimized feedforward network of cnn with xnor v5Optimized feedforward network of cnn with xnor v5
Optimized feedforward network of cnn with xnor v5
 
Runtime Performance Optimizations for an OpenFOAM Simulation
Runtime Performance Optimizations for an OpenFOAM SimulationRuntime Performance Optimizations for an OpenFOAM Simulation
Runtime Performance Optimizations for an OpenFOAM Simulation
 
1c03projectlinkedin
1c03projectlinkedin1c03projectlinkedin
1c03projectlinkedin
 
The Principle Of Ultrasound Imaging System
The Principle Of Ultrasound Imaging SystemThe Principle Of Ultrasound Imaging System
The Principle Of Ultrasound Imaging System
 
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
 

More from LeapMind Inc

[Icml2019] mix hop higher-order graph convolutional architectures via spars...
[Icml2019]  mix hop  higher-order graph convolutional architectures via spars...[Icml2019]  mix hop  higher-order graph convolutional architectures via spars...
[Icml2019] mix hop higher-order graph convolutional architectures via spars...LeapMind Inc
 
[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...
[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...
[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...LeapMind Inc
 
[Icml2019] parameter efficient training of deep convolutional neural network...
[Icml2019] parameter efficient training of  deep convolutional neural network...[Icml2019] parameter efficient training of  deep convolutional neural network...
[Icml2019] parameter efficient training of deep convolutional neural network...LeapMind Inc
 
エッジ向けDeepLearningプロジェクトで必要なこと
エッジ向けDeepLearningプロジェクトで必要なことエッジ向けDeepLearningプロジェクトで必要なこと
エッジ向けDeepLearningプロジェクトで必要なことLeapMind Inc
 
20190227[EDLS]JAL's INNOVATION エアラインのAI活用
20190227[EDLS]JAL's INNOVATION エアラインのAI活用20190227[EDLS]JAL's INNOVATION エアラインのAI活用
20190227[EDLS]JAL's INNOVATION エアラインのAI活用LeapMind Inc
 
E20190227[EDLS]インテル®︎FPGAによるエッジAI
E20190227[EDLS]インテル®︎FPGAによるエッジAIE20190227[EDLS]インテル®︎FPGAによるエッジAI
E20190227[EDLS]インテル®︎FPGAによるエッジAILeapMind Inc
 
20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係
20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係
20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係LeapMind Inc
 
20180831 [DeLTA TECH] 深く青い脂
20180831 [DeLTA TECH] 深く青い脂20180831 [DeLTA TECH] 深く青い脂
20180831 [DeLTA TECH] 深く青い脂LeapMind Inc
 
20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜
20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜
20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜LeapMind Inc
 
20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)
20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)
20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)LeapMind Inc
 
20180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.1
20180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.120180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.1
20180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.1LeapMind Inc
 
20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説
20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説
20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説LeapMind Inc
 
An Introduction of DNN Compression Technology and Hardware Acceleration on FPGA
An Introduction of DNN Compression Technology and Hardware Acceleration on FPGAAn Introduction of DNN Compression Technology and Hardware Acceleration on FPGA
An Introduction of DNN Compression Technology and Hardware Acceleration on FPGALeapMind Inc
 
2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会
2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会
2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会LeapMind Inc
 
JUIZ DLK 組込み向けDeep Learningコンパイラ
JUIZ DLK 組込み向けDeep LearningコンパイラJUIZ DLK 組込み向けDeep Learningコンパイラ
JUIZ DLK 組込み向けDeep LearningコンパイラLeapMind Inc
 

More from LeapMind Inc (16)

[Icml2019] mix hop higher-order graph convolutional architectures via spars...
[Icml2019]  mix hop  higher-order graph convolutional architectures via spars...[Icml2019]  mix hop  higher-order graph convolutional architectures via spars...
[Icml2019] mix hop higher-order graph convolutional architectures via spars...
 
[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...
[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...
[Icml2019]LIT: Learned Intermediate Representation Training for Model Compres...
 
[Icml2019] parameter efficient training of deep convolutional neural network...
[Icml2019] parameter efficient training of  deep convolutional neural network...[Icml2019] parameter efficient training of  deep convolutional neural network...
[Icml2019] parameter efficient training of deep convolutional neural network...
 
エッジ向けDeepLearningプロジェクトで必要なこと
エッジ向けDeepLearningプロジェクトで必要なことエッジ向けDeepLearningプロジェクトで必要なこと
エッジ向けDeepLearningプロジェクトで必要なこと
 
20190227[EDLS]JAL's INNOVATION エアラインのAI活用
20190227[EDLS]JAL's INNOVATION エアラインのAI活用20190227[EDLS]JAL's INNOVATION エアラインのAI活用
20190227[EDLS]JAL's INNOVATION エアラインのAI活用
 
E20190227[EDLS]インテル®︎FPGAによるエッジAI
E20190227[EDLS]インテル®︎FPGAによるエッジAIE20190227[EDLS]インテル®︎FPGAによるエッジAI
E20190227[EDLS]インテル®︎FPGAによるエッジAI
 
20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係
20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係
20190227[EDLS]進化するAI on Edge 〜 CloudとEdgeの最適な関係
 
20180831 [DeLTA TECH] 深く青い脂
20180831 [DeLTA TECH] 深く青い脂20180831 [DeLTA TECH] 深く青い脂
20180831 [DeLTA TECH] 深く青い脂
 
20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜
20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜
20180831 [DeLTA TECH] 新・深層の世紀 〜第3集 ディープラーニング・時代はAIを求めた 〜
 
20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)
20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)
20180831 [DeLTA TECH] DeLTA-Liteを支える技術(システム構成編)
 
20180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.1
20180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.120180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.1
20180831 [DeLTA TECH] DeLTA-FamilyによるIndustry4.1
 
20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説
20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説
20180613 [TensorFlow分散学習] Horovodによる分散学習の実装方法と解説
 
An Introduction of DNN Compression Technology and Hardware Acceleration on FPGA
An Introduction of DNN Compression Technology and Hardware Acceleration on FPGAAn Introduction of DNN Compression Technology and Hardware Acceleration on FPGA
An Introduction of DNN Compression Technology and Hardware Acceleration on FPGA
 
2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会
2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会
2018年1月19日開催 IoTビジネス共創ラボ 第6回勉強会
 
Pitch v2.2
Pitch v2.2Pitch v2.2
Pitch v2.2
 
JUIZ DLK 組込み向けDeep Learningコンパイラ
JUIZ DLK 組込み向けDeep LearningコンパイラJUIZ DLK 組込み向けDeep Learningコンパイラ
JUIZ DLK 組込み向けDeep Learningコンパイラ
 

Recently uploaded

Virtual memory management in Operating System
Virtual memory management in Operating SystemVirtual memory management in Operating System
Virtual memory management in Operating SystemRashmi Bhat
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdfCaalaaAbdulkerim
 
Robotics Group 10 (Control Schemes) cse.pdf
Robotics Group 10  (Control Schemes) cse.pdfRobotics Group 10  (Control Schemes) cse.pdf
Robotics Group 10 (Control Schemes) cse.pdfsahilsajad201
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxRomil Mishra
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Sumanth A
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSneha Padhiar
 
Turn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptxTurn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptxStephen Sitton
 
Novel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsNovel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsResearcher Researcher
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionSneha Padhiar
 
The Satellite applications in telecommunication
The Satellite applications in telecommunicationThe Satellite applications in telecommunication
The Satellite applications in telecommunicationnovrain7111
 
CS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfCS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfBalamuruganV28
 
Secure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech LabsSecure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech Labsamber724300
 
Theory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfTheory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfShreyas Pandit
 
FUNCTIONAL AND NON FUNCTIONAL REQUIREMENT
FUNCTIONAL AND NON FUNCTIONAL REQUIREMENTFUNCTIONAL AND NON FUNCTIONAL REQUIREMENT
FUNCTIONAL AND NON FUNCTIONAL REQUIREMENTSneha Padhiar
 
A brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision ProA brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision ProRay Yuan Liu
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionMebane Rash
 
Stork Webinar | APM Transformational planning, Tool Selection & Performance T...
Stork Webinar | APM Transformational planning, Tool Selection & Performance T...Stork Webinar | APM Transformational planning, Tool Selection & Performance T...
Stork Webinar | APM Transformational planning, Tool Selection & Performance T...Stork
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfalene1
 

Recently uploaded (20)

Virtual memory management in Operating System
Virtual memory management in Operating SystemVirtual memory management in Operating System
Virtual memory management in Operating System
 
Versatile Engineering Construction Firms
Versatile Engineering Construction FirmsVersatile Engineering Construction Firms
Versatile Engineering Construction Firms
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdf
 
Robotics Group 10 (Control Schemes) cse.pdf
Robotics Group 10  (Control Schemes) cse.pdfRobotics Group 10  (Control Schemes) cse.pdf
Robotics Group 10 (Control Schemes) cse.pdf
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
 
ASME-B31.4-2019-estandar para diseño de ductos
ASME-B31.4-2019-estandar para diseño de ductosASME-B31.4-2019-estandar para diseño de ductos
ASME-B31.4-2019-estandar para diseño de ductos
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
 
Turn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptxTurn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptx
 
Novel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsNovel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending Actuators
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based question
 
The Satellite applications in telecommunication
The Satellite applications in telecommunicationThe Satellite applications in telecommunication
The Satellite applications in telecommunication
 
CS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfCS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdf
 
Secure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech LabsSecure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech Labs
 
Theory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfTheory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdf
 
FUNCTIONAL AND NON FUNCTIONAL REQUIREMENT
FUNCTIONAL AND NON FUNCTIONAL REQUIREMENTFUNCTIONAL AND NON FUNCTIONAL REQUIREMENT
FUNCTIONAL AND NON FUNCTIONAL REQUIREMENT
 
A brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision ProA brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision Pro
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of Action
 
Stork Webinar | APM Transformational planning, Tool Selection & Performance T...
Stork Webinar | APM Transformational planning, Tool Selection & Performance T...Stork Webinar | APM Transformational planning, Tool Selection & Performance T...
Stork Webinar | APM Transformational planning, Tool Selection & Performance T...
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
 

Final presentation optical flow estimation with DL

  • 1. Optical flow estimation with Blueoil K-Inoue @ki42 & Oscar @wang Blueoil Division 1
  • 2. What is optical flow estimation? ● Optical flow... represents the apparent motion of objects. ● Optical flow estimation... can predict the movement of objects in a video. Miloud, Hadj achour. (2017). Fragmentation de métal liquide dans l’eau. https://www.codeproject.com/Articles/1192205/Capturing-motion-from-video-using-the-Emgu-CV-libr 2
  • 3. Optical flow estimation is important ● Widely used by insects and birds ● Practical usage ○ Analyze motion ○ Avoid collision ○ Assist in navigation ● Real-world Applications ○ Video/Motion classification ○ Navigation assistance ■ Self driving cars ■ Drones https://nanonets.com/blog/optical-flow/ 3
  • 4. DL approaches are increasing 4 ● 2015. FlowNet S (Simple) ● 2015. FlowNet C (Correlated) ● 2016. FlowNet 2 ● 2018. LiteFlowNet ● ... Color coding https://www.youtube.com/watch?v=k_wkDLJ8lJE https://www.youtube.com/watch?v=pfQ0zFwv-hM
  • 5. However... ● Existing DL approaches require GPU to execute 👎 High power consumption 👎 Low runtime speed on CPU environment ● We propose LmFlowNet S 👍 ○ Modification of FlowNet S [P. Fischer+ 2015] ○ Goals: ■ Edge Computing ■ Run on FPGA-based accelerator ■ Use quantization to reduce inference time while achieving good prediction performance 5
  • 6. FlowNet S vs LmFlowNet S 6
  • 7. Network of FlowNet S 7 [N, 384, 512, 6] [N, 384, 512, 2] Detailed ops inside each color block is shown in appendix
  • 8. Network of FlowNet S 8 [N, 384, 512, 6] [N, 384, 512, 2] Encoder Decoder Detailed ops inside each color block is shown in appendix
  • 9. Network of FlowNet S 9 [N, 384, 512, 6] [N, 384, 512, 2] Detailed ops inside each color block is shown in appendix Not supported by Blueoil DLK
  • 10. 10 Network of LmFlowNet S (DLK supported) [N, 384, 512, 6] [N, 384, 512, 2] Detailed ops inside each color block is shown in appendix
  • 11. 11 Network of LmFlowNet S [N, 384, 512, 6] [N, 384, 512, 2] Quantized Detailed quantization inside each color block is shown in appendix
  • 12. Loss function: End Point Error (EPE) 12 (x1 ,y1 ) (x2 ,y2 ) EPEflow2 EPEflow3 EPEflow4 EPEflow5 EPEflow6 Weighted EPE = 0.32 * EPEflow2 + 0.08 * EPEflow3 + 0.02 * EPEflow4 + 0.01 * EPEflow5 + 0.005 * EPEflow6 Down- sampled Ground Truth Training hyper-parameters are shown in appendix 24 x 32 48 x 6496 x 128 192 x 256 12 x 16
  • 13. Artificial dataset: Flying Chairs 13 ● Dataset Name Frame pairs Train validation ratio size Flying Chairs 22,872 9:1 30GB ● Data Augmentation ○ Crop, Rotate, Translate, FlipLeftRight, FlipTopBottom ○ Gaussian noise, Brightness, Contrast, Gamma, and Color Parameters used in data augmentation are shown in appendix https://arxiv.org/pdf/1504.06852.pdf
  • 15. 15 Results - Avg. EPE & Inference time Method Avg. EPE (pixel) (Flying Chairs) Inference time per frame (ms) [1] CPU (dlk-convert) GPU (tensorflow) FlowNet S 2.94 - 11.65 LmFlowNet S 5.33 1360.49 13.81 LmFlowNet S Quantized 9.01 637.467 17.60 [1] CPU and GPU specs available in appedix FlowNet S Trained for 1.2M LmFlowNet S Trained for 400K LmFlowNet S Quantized Trained for 400K
  • 17. 17 Live demonstration ● Three demonstration ○ FlowNet S ○ LmFlowNet S ○ LmFlowNet S Quantized ● NOTE: Running on GPU (not on CPU / FPGA) ○ Failed to run on CPU/FPGA due to several problems 😢 ■ etc. segmentation fault, memory error... ○ Fixing and debugging them in the future 👊
  • 18. 18
  • 20. Challenges ● Training takes a very long time ( > 2 weeks...😢) ○ Heavy data augmentation & pre-processing ■ Pre-processing on GPU is not supported now. ● Unique network structure, not compatible with Blueoil ○ Input is a stack of 2 images (6 channels) ○ Multiple and branched outputs ● DLK Limitation. No documentation. 🤯 ○ No support for kernel size 7x7, 5x5 ○ No support for Conv2dTranspose ○ Cannot concat quantized value and float together ○ Requires the depth of Space2Depth to be 32 * N 20
  • 21. Thank you for your attention! our source code: https://github.com/ki-lm/blueoil/tree/lmflownets 21
  • 22. 22
  • 24. 24
  • 25. LmFlowNet S | Training & Hyper-params ● Optimizer: Adam ● Max steps: 1200k ● Fixed parameters in Adam: β1 =0.9 and β2 = 0.999. ● Batch size: 8 ● Learning rate ○ values: [1e-4 , 5e-5 , 2.5e-5 ,1.25e-5 ,6.25e-6 ] ○ boundaries: [400K, 600K, 800K, 1000K] ● Learning rate for quantization ○ values: [1.25e-5 , 1e-4 , 5e-5 , 2.5e-5 ,1.25e-5 ,6.25e-6 ] ○ boundaries: [50K, 400K, 600K, 800K, 1000K] 25 Tokunaga Scheduling 😆
  • 26. LmFlowNet S | Data Augmentation ● Translation: [20%, 20%] of the image width for x and y ● Rotation: [17o , 17o ] ● Scaling: [0.9, 2.0] ● Gaussian noise: sigma uniformly sampled from [0, 0.04] ● Contrast: [0.8, 0.4] ● Multiplicative color changes to the RGB channels per image: [0.5, 2] ● Gamma values: [0.7, 1.5] ● Additive brightness changes: Gaussian with a sigma of 0.2 26
  • 27. 27 CPU & GPU specs ● CPU ○ Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz ○ 4 cores, 8 threads ○ Inference time (ms) ■ LmFlowNet S: 1358.39, 1358.39, 1293.26, 1307.28, 1485.11 ■ LmFlowNet S Quant: 640.541, 640.732, 635.354, 635.354, 635.354 ● GPU ○ NVIDIA Tesla V100 on DGX-1 ○ Inference time (ms) ■ FlowNet S: 0.0111, 0.0105, 0.0116, 0.0117, 0.0119, 0.0118, 0.0121, 0.0118, 0.0116, 0.0124 ■ LmFlowNet S: 0.0137, 0.0140, 0.0137, 0.0136, 0.0141, 0.0135, 0.0139, 0.0138, 0.0136, 0.0142 ■ LmFlowNet S Quant: 0.0168, 0.0161, 0.0165, 0.0177, 0.0173, 0.0166, 0.0186, 0.0195, 0.0183, 0.0187
  • 28. Our FlowNet S versions 28 Version Architecture DLK support V1 (FlowNet S) Same as the paper X V2 7x7, 5x5 => 3x3 Striding 2 => SpaceToDepth X V3 Conv2dTranspose => ResizeNearestNeighbor + Conv2d ResizeBilinear => ResizeNearestNeighbor X V3 Quant. (LmFlowNet S) Quantize except first, last layer, and activation before last layer △ V4 Quant. Change all output depths from SpaceToDepth to 32 * N O
  • 29. List of source code links ● FlowNet S/C, and 2 (TensorFlow): https://github.com/sampepose/flownet2-tf/ ● FlowNet S/C (Original paper, Caffe): https://lmb.informatik.uni-freiburg.de/Publications/2015/DF IB15/ ● FlowNet 2 (Original paper, Caffe): https://github.com/lmb-freiburg/flownet2 29