By Guillaume Emont. The topic for this talk was encountered when trying to make a time lapse film with a camera attached to a tethered helium balloon high above. This talk will introduce the issues of stabilising a sequence of images. It will mainly focus on "discreet" sequences of images (time lapses, stop motion) but will have many bits relevant to more typical continous-ish videos. It will introduce some methods to solve that problem, and various benefits of having these methods in GStreamer, so that they are available to solve other related problems (such as following an object, movement detection, automated compositing or 3D reconstruction). Of course, there will be cool demos, hopefully featuring stunning aerial pictures getting transformed into a smooth video. More information at http://emont.org/blog/2011/11/10/gstreamer-and-opencv-for-image-stabilisation/.

1. Image Stabilisation
Guillaume Emont
Igalia

2. Image Stabilisation
Holiday pictures

3. that picture is not by me
(it is © 2011 Ugo Riboni)

4. ...but I was there!

5. with a plan!

7. What the balloon sees

8. Let's go higher

9. Higher!

10. HIGHER!

11. I said: HIIIGHEEEER!

12. Ok, that will do

13. We wanted to take pictures for 7 days

14. The balloon agreed to 4 hours and a half

15. There was an intimate ceremony
with family and friends

16. Nice pics! ...now what?

17. Time lapse!

18. -- [timelapse-theory.png] [fit] [text-align=center]
<span font="20">(image from wikipedia under GFDL)</span>
-- [timelapse-theory.png] [fit] [text-align=center]
Crazier: one frame per minute!

19. Removing the training wheels:
No tripod.
(luckily, we have OpenCV)

20. Stabilisation in post-processing:
1. detect the movement (optical flow)

21. Optical flow: position changes of features

22. Stabilisation in post-processing:
2. compensate

23. Compensation:
OpenCV provides nice tools
(but we won't talk about them here)

24. Optical flow: a few algorithms

25. Horn-Schunck

26. Lucas-Kanade
gst-launch-0.10 filesrc location=shaky-hippo.ogg
! decodebin ! ffmpegcolorspace
! opticalflowcorrector algorithm=1
! ffmpegcolorspace ! xvimagesink

27. Err, yeah but...
What about the balloon pictures?

28. Horn-Schunck

29. Lucas-Kanade

30. Speeded
Up
Robust
Feature

31. SURF

32. Woohoo!

33. Bonus track

34. TODO:
Check out the optical flow work by David Jordan (used for slow motion)
and try to apply it to stabilisation

35. TODO:
better compensation (tesselation?)

36. TODO:
try more algorithms (BRIEF and ORB)

37. TODO:
automatic optimal choice of algorithm and parameters

38. TODO:
reimplement in C and put in -bad?

39. TODO:
for videos: movement prediction/averaging
(follow the intended movement of the cameraman)

40. questions?
Image credits:
training wheels: CC BY-NC-SA 2.0 Dale Basler http://www.flickr.com/photos/baslercast/
surf: CC BY-NC-SA 2.0 Guilherme Meneghelli http://www.flickr.com/people/iberere/
last one: CC BY-NC-SA 2.0 Martin Ibert http://www.flickr.com/photos/mar_ibert/

41. Thanks!
http://www.gitorious.org/gststabilizer
http://balloonfreaks.mooo.com/
http://emont.org/blog/
http://www.igalia.com/
guijemont@igalia.com