1. Passive Sound Recovery
Using the Optical Flow of Objects
Recorded with a Video Camera
SUPERVISOR
EMMANOYIL PSARAKIS
STUDENT
PETROS BANTOLAS

2. Motivation
Sound hits objects and causes small vibrations
Using these minute motions, we extract the original signal
A high-speed camera is needed for sufficient sampling rate

3. Previous Research
Laser Doppler Vibrometer Phase-based video motion processing

4. The Complex Steerable Pyramid
Fourier Shift Theorem – Links phase
variations to translation
S.P.’s phase variations correspond to local
motion

5. Steerable Pyramid
Multiple scales and orientations

6. The Visual Microphone
Calculate phase variations
Local motion signal
Align local motions

7. Local Motions
Weigh phases with the squared amplitude, because of noise in low-texture regions

8. Local Motion Alignment
Avoid destructive interference for
negatively correlated phase
variations
Use cross-correlation with
reference signal

9. EXPERIMENTAL
DESIGN
240 FPS
Motions ~1/100 of a pixel
Natural Scenes
Everyday objects

10. ARTIFICIAL
SEQUENCES
3000 FPS
Motion ~1/1000 of a pixel
2D and 3D Scenes

11. Natural Experiments
SNR
-0.49 – 1.96
Similarity Coefficient
0.21 – 0.64

12. Artificial Experiments
SNR 0-11 dB
Similarity coefficient 0.53-0.99
2D

13. 3D scenes have larger motion, better reconstruction
3D
Artificial Experiments

14. STATISTICAL
ANALYSIS
Motion amplitude
Anti-aliasing level
Lossless compression
Similarity coefficient
SNR

15. CONCLUSIONS
IT IS POSSIBLE to recover sound information through
small vibrations
Framerate, motion granularity LIMITING factors
LIGHT and RIGID object are effective VISUAL
MICROPHONES