Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Video Stabilization with L1-L2
Optimization
Hui Qu, Li Song
Institute of Image Communication and Network
Engineering
Shang...
Outline
 Introduction
 Benchmark work
 Our algorithm
L1-L2 mixed optimization model
Online video stabilization scheme
...
Introduction
Video is often shaky
Introduction
Video stabilization steps
 Original camera path estimation
shaky
Introduction
Video stabilization steps
 Smooth camera path computation
smooth
Introduction
Video stabilization steps
 Synthesizing the stabilized video
shaky stabilized
Introduction
Video Stabilization methods
 Original camera path estimation
2D camera path——2D linear model
3D camera path—...
 Grundmann et al. 2011
L1 camera path optimization
Integrated into Google ‘s YouTube Editor
Benchmark work
L1 Camera Path...
 Objective function:
D means derivative operator
𝜔1, 𝜔2, 𝜔3 are empirical weights
2 3
1 2 31 1 1
( ) ( ) ( ) ( )P D P D P...
 Objective function:
2 3
1 2 31 1 1
( ) ( ) ( ) ( )P D P D P D P
Benchmark work
L1 Camera Path Optimization method
path w...
 Objective function:
2 3
1 2 31 1 1
( ) ( ) ( ) ( )P D P D P D P
Benchmark work
L1 Camera Path Optimization method
path w...
Benchmark work
L1 Camera Path Optimization method
 Inclusion constraint:
stabilized frame
Benchmark work
Problem of L1 Path Method
 The method discards information due to cropping
not suitable for videos with im...
Benchmark work
Problem of L1 Path Method
 𝜔1, 𝜔2, 𝜔3 are empirically set
hard to be adaptable to different kinds of video...
Our algorithm
L1-L2 mixed optimization model
 Objective function:
2 3
1 1 1 2
( ) ( ) ( ) ( )P D P D P D P P C
L1 part, e...
Our algorithm
L1-L2 mixed optimization model
 Objective function:
2 3
1 1 1 2
( ) ( ) ( ) ( )P D P D P D P P C
Our algorithm
L1-L2 mixed optimization model
 Objective function:
2 3
1 1 1 2
( ) ( ) ( ) ( )P D P D P D P P C
Our algorithm
L1-L2 mixed optimization model
𝜆 = 0.1 𝜆 = 0.5
𝜆 = 1.0 𝜆 = 2.0
Our algorithm
L1-L2 mixed optimization model
𝜆 can be relatively small 𝜆 should be relatively large
Our algorithm
Online processing scheme
…… … ……
time
segment 1:N frames
……
segment 2:N frames
K overlapped frames
Our algorithm
Online processing scheme
segment 1:
N=150 frames
segment 2:
N=150 frames
K=30 overlapped
frames
Our algorithm
Online processing scheme
 Optimal path for the overlapped frames
𝑃𝑡
1
: optimal path of previous segment
𝑃𝑡...
Experimental results
L1 path vs. L1-L2 path
Input shaky video
Experimental results
L1 path vs. L1-L2 path
Grundmann et al’s result our result
Experimental results
L1 path vs. L1-L2 path
Grundmann et al’s result our result
Experimental results
Different values of 𝜆
Input shaky video
Experimental results
Different values of 𝜆
𝜆 = 0.1 𝜆 = 0.5
𝜆 = 1.0 𝜆 = 2.0
Experimental results
Different values of 𝜆
𝜆 = 0.1
80% crop
𝜆 = 0.5
85% crop
𝜆 = 1.0
90% crop
𝜆 = 2.0
95% crop
Experimental results
Speed
 Platform
OS: Windows 7
CPU: Inter Core i5 & 3.1 GHz
 Execution time
About 20 fps ( resolutio...
Conclusion
 Video stabilization method by mixed L1-
L2 optimization
Stabilize & preserve video content
Adjust the degree ...
Thank you!
Q&A
Insert inconsecutive frames of another video
original
stabilized
Transition between different scene
original
stabilized
Insert some consecutive frames of few features
original
stabilized
Upcoming SlideShare
Loading in …5
×

ICIP2013-video stabilization with l1 l2 optimization

0 views

Published on

This is our oral presentation for ICIP2013 Paper.

Published in: Technology, Business
  • Be the first to comment

ICIP2013-video stabilization with l1 l2 optimization

  1. 1. Video Stabilization with L1-L2 Optimization Hui Qu, Li Song Institute of Image Communication and Network Engineering Shanghai Jiao Tong University 17 September 2013
  2. 2. Outline  Introduction  Benchmark work  Our algorithm L1-L2 mixed optimization model Online video stabilization scheme  Experimental results  Conclusion
  3. 3. Introduction Video is often shaky
  4. 4. Introduction Video stabilization steps  Original camera path estimation shaky
  5. 5. Introduction Video stabilization steps  Smooth camera path computation smooth
  6. 6. Introduction Video stabilization steps  Synthesizing the stabilized video shaky stabilized
  7. 7. Introduction Video Stabilization methods  Original camera path estimation 2D camera path——2D linear model 3D camera path——Structure from Motion (SfM) ……  Smooth camera path computation Filtering Optimization ……  Synthesizing the stabilized video Cropping——keep the central part Inpainting——full frame ……
  8. 8.  Grundmann et al. 2011 L1 camera path optimization Integrated into Google ‘s YouTube Editor Benchmark work L1 Camera Path Optimization method Original camera path: 𝐶 Optimal camera path: 𝑃
  9. 9.  Objective function: D means derivative operator 𝜔1, 𝜔2, 𝜔3 are empirical weights 2 3 1 2 31 1 1 ( ) ( ) ( ) ( )P D P D P D P Benchmark work L1 Camera Path Optimization method constant path static camera
  10. 10.  Objective function: 2 3 1 2 31 1 1 ( ) ( ) ( ) ( )P D P D P D P Benchmark work L1 Camera Path Optimization method path with constant velocity panning or dolly shot
  11. 11.  Objective function: 2 3 1 2 31 1 1 ( ) ( ) ( ) ( )P D P D P D P Benchmark work L1 Camera Path Optimization method path with constant acceleration ease in and out transition
  12. 12. Benchmark work L1 Camera Path Optimization method  Inclusion constraint: stabilized frame
  13. 13. Benchmark work Problem of L1 Path Method  The method discards information due to cropping not suitable for videos with important information near the boundary.
  14. 14. Benchmark work Problem of L1 Path Method  𝜔1, 𝜔2, 𝜔3 are empirically set hard to be adaptable to different kinds of videos sequence 1 sequence 2
  15. 15. Our algorithm L1-L2 mixed optimization model  Objective function: 2 3 1 1 1 2 ( ) ( ) ( ) ( )P D P D P D P P C L1 part, ensure smoothness L2 part, ensure proximity to original path weight, adjust the degree of smoothness and fidelity
  16. 16. Our algorithm L1-L2 mixed optimization model  Objective function: 2 3 1 1 1 2 ( ) ( ) ( ) ( )P D P D P D P P C
  17. 17. Our algorithm L1-L2 mixed optimization model  Objective function: 2 3 1 1 1 2 ( ) ( ) ( ) ( )P D P D P D P P C
  18. 18. Our algorithm L1-L2 mixed optimization model 𝜆 = 0.1 𝜆 = 0.5 𝜆 = 1.0 𝜆 = 2.0
  19. 19. Our algorithm L1-L2 mixed optimization model 𝜆 can be relatively small 𝜆 should be relatively large
  20. 20. Our algorithm Online processing scheme …… … …… time segment 1:N frames …… segment 2:N frames K overlapped frames
  21. 21. Our algorithm Online processing scheme segment 1: N=150 frames segment 2: N=150 frames K=30 overlapped frames
  22. 22. Our algorithm Online processing scheme  Optimal path for the overlapped frames 𝑃𝑡 1 : optimal path of previous segment 𝑃𝑡 2 : optimal path of current segment 𝜐𝑖: weights      1 2 1t i t i tP P P    0.5 cos 1 0.5i i K              
  23. 23. Experimental results L1 path vs. L1-L2 path Input shaky video
  24. 24. Experimental results L1 path vs. L1-L2 path Grundmann et al’s result our result
  25. 25. Experimental results L1 path vs. L1-L2 path Grundmann et al’s result our result
  26. 26. Experimental results Different values of 𝜆 Input shaky video
  27. 27. Experimental results Different values of 𝜆 𝜆 = 0.1 𝜆 = 0.5 𝜆 = 1.0 𝜆 = 2.0
  28. 28. Experimental results Different values of 𝜆 𝜆 = 0.1 80% crop 𝜆 = 0.5 85% crop 𝜆 = 1.0 90% crop 𝜆 = 2.0 95% crop
  29. 29. Experimental results Speed  Platform OS: Windows 7 CPU: Inter Core i5 & 3.1 GHz  Execution time About 20 fps ( resolution: 640*360)
  30. 30. Conclusion  Video stabilization method by mixed L1- L2 optimization Stabilize & preserve video content Adjust the degree of stabilization according to different demands Able to handle online stabilization and unlimited length videos
  31. 31. Thank you! Q&A
  32. 32. Insert inconsecutive frames of another video original stabilized
  33. 33. Transition between different scene original stabilized
  34. 34. Insert some consecutive frames of few features original stabilized

×