ICIP2013-video stabilization with l1 l2 optimization

1,173 views
990 views

Published on

This is our oral presentation for ICIP2013 Paper.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,173
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ICIP2013-video stabilization with l1 l2 optimization

  1. 1. Video Stabilization with L1-L2 Optimization Hui Qu, Li Song Institute of Image Communication and Network Engineering Shanghai Jiao Tong University 17 September 2013
  2. 2. Outline  Introduction  Benchmark work  Our algorithm L1-L2 mixed optimization model Online video stabilization scheme  Experimental results  Conclusion
  3. 3. Introduction Video is often shaky
  4. 4. Introduction Video stabilization steps  Original camera path estimation shaky
  5. 5. Introduction Video stabilization steps  Smooth camera path computation smooth
  6. 6. Introduction Video stabilization steps  Synthesizing the stabilized video shaky stabilized
  7. 7. Introduction Video Stabilization methods  Original camera path estimation 2D camera path——2D linear model 3D camera path——Structure from Motion (SfM) ……  Smooth camera path computation Filtering Optimization ……  Synthesizing the stabilized video Cropping——keep the central part Inpainting——full frame ……
  8. 8.  Grundmann et al. 2011 L1 camera path optimization Integrated into Google ‘s YouTube Editor Benchmark work L1 Camera Path Optimization method Original camera path: 𝐶 Optimal camera path: 𝑃
  9. 9.  Objective function: D means derivative operator 𝜔1, 𝜔2, 𝜔3 are empirical weights 2 3 1 2 31 1 1 ( ) ( ) ( ) ( )P D P D P D P Benchmark work L1 Camera Path Optimization method constant path static camera
  10. 10.  Objective function: 2 3 1 2 31 1 1 ( ) ( ) ( ) ( )P D P D P D P Benchmark work L1 Camera Path Optimization method path with constant velocity panning or dolly shot
  11. 11.  Objective function: 2 3 1 2 31 1 1 ( ) ( ) ( ) ( )P D P D P D P Benchmark work L1 Camera Path Optimization method path with constant acceleration ease in and out transition
  12. 12. Benchmark work L1 Camera Path Optimization method  Inclusion constraint: stabilized frame
  13. 13. Benchmark work Problem of L1 Path Method  The method discards information due to cropping not suitable for videos with important information near the boundary.
  14. 14. Benchmark work Problem of L1 Path Method  𝜔1, 𝜔2, 𝜔3 are empirically set hard to be adaptable to different kinds of videos sequence 1 sequence 2
  15. 15. Our algorithm L1-L2 mixed optimization model  Objective function: 2 3 1 1 1 2 ( ) ( ) ( ) ( )P D P D P D P P C L1 part, ensure smoothness L2 part, ensure proximity to original path weight, adjust the degree of smoothness and fidelity
  16. 16. Our algorithm L1-L2 mixed optimization model  Objective function: 2 3 1 1 1 2 ( ) ( ) ( ) ( )P D P D P D P P C
  17. 17. Our algorithm L1-L2 mixed optimization model  Objective function: 2 3 1 1 1 2 ( ) ( ) ( ) ( )P D P D P D P P C
  18. 18. Our algorithm L1-L2 mixed optimization model 𝜆 = 0.1 𝜆 = 0.5 𝜆 = 1.0 𝜆 = 2.0
  19. 19. Our algorithm L1-L2 mixed optimization model 𝜆 can be relatively small 𝜆 should be relatively large
  20. 20. Our algorithm Online processing scheme …… … …… time segment 1:N frames …… segment 2:N frames K overlapped frames
  21. 21. Our algorithm Online processing scheme segment 1: N=150 frames segment 2: N=150 frames K=30 overlapped frames
  22. 22. Our algorithm Online processing scheme  Optimal path for the overlapped frames 𝑃𝑡 1 : optimal path of previous segment 𝑃𝑡 2 : optimal path of current segment 𝜐𝑖: weights      1 2 1t i t i tP P P    0.5 cos 1 0.5i i K              
  23. 23. Experimental results L1 path vs. L1-L2 path Input shaky video
  24. 24. Experimental results L1 path vs. L1-L2 path Grundmann et al’s result our result
  25. 25. Experimental results L1 path vs. L1-L2 path Grundmann et al’s result our result
  26. 26. Experimental results Different values of 𝜆 Input shaky video
  27. 27. Experimental results Different values of 𝜆 𝜆 = 0.1 𝜆 = 0.5 𝜆 = 1.0 𝜆 = 2.0
  28. 28. Experimental results Different values of 𝜆 𝜆 = 0.1 80% crop 𝜆 = 0.5 85% crop 𝜆 = 1.0 90% crop 𝜆 = 2.0 95% crop
  29. 29. Experimental results Speed  Platform OS: Windows 7 CPU: Inter Core i5 & 3.1 GHz  Execution time About 20 fps ( resolution: 640*360)
  30. 30. Conclusion  Video stabilization method by mixed L1- L2 optimization Stabilize & preserve video content Adjust the degree of stabilization according to different demands Able to handle online stabilization and unlimited length videos
  31. 31. Thank you! Q&A
  32. 32. Insert inconsecutive frames of another video original stabilized
  33. 33. Transition between different scene original stabilized
  34. 34. Insert some consecutive frames of few features original stabilized

×