3. Imperceptible Changes and Motions in the World
Blood flow
Building and structure
movements
Camera moves due to motion
of shutter and mirror
Breathing
Eye movements (Microsaccades)Michael Rubinstein, MIT 2013
4. Imperceptible Changes and Motions in the World
Low frequency motions Mid-range frequency motionsMichael Rubinstein, MIT 2013
5. Talk Overview
• Recap: Eulerian Video Magnification (SIGGRAPH’12)
– With Hao-Yu Wu, Eugene Shih, John Guttag,
Fredo Durand, William T. Freeman
• EVM in the Wild
• Phase-based Video Motion Processing (SIGGRAPH’13)
– With Neal Wadhwa, Fredo Durand, William T. Freeman
Michael Rubinstein, MIT 2013
6. Talk Overview
• Recap: Eulerian Video Magnification (SIGGRAPH’12)
– With Hao-Yu Wu, Eugene Shih, John Guttag,
Fredo Durand, William T. Freeman
• EVM in the Wild
• Phase-based Video Motion Processing (SIGGRAPH’13)
– With Neal Wadhwa, Fredo Durand, William T. Freeman
Michael Rubinstein, MIT 2013
7. Eulerian Video Processing
• Each pixel is processed independently
• We treat each pixel as a time series and apply
signal processing to it
y
x
time
Michael Rubinstein, MIT 2013
16. Selective Motion Magnification in Natural Videos
Source
(600 fps)
72-92 Hz
Amplified
Low E (82.4 Hz)
A (110 Hz)
100-120 Hz
Amplified
Michael Rubinstein, MIT 2013
17. Related Work: Motion Magnification [Liu 2005]
Liu et al. Motion Magnification, SIGGRAPH 2005
Source Motion-magnified
Michael Rubinstein, MIT 2013
18. Related Work: Motion Magnification [Liu 2005]
+ +
++ +
Liu et al. Motion Magnification, 2005
Michael Rubinstein, MIT 2013
19. Talk Overview
• Recap: Eulerian Video Magnification (SIGGRAPH’12)
– With Hao-Yu Wu, Eugene Shih, John Guttag,
Fredo Durand, William T. Freeman
• EVM in the Wild
• Phase-based Video Motion Processing (SIGGRAPH’13)
– With Neal Wadhwa, Fredo Durand, William T. Freeman
Michael Rubinstein, MIT 2013
20. EVM in the Wild: Pregnancy
Original Processed
“Tomez85” https://www.youtube.com/watch?v=J1wvFmWv7zY
Michael Rubinstein, MIT 2013
21. EVM in the Wild: Pregnancy
“Tomez85” https://www.youtube.com/watch?v=gDpNN4g1klU
Michael Rubinstein, MIT 2013
22. EVM in the Wild: Blood flow Visualization
Institute for Biomedical Engineering, Dresden Germany
https://www.youtube.com/watch?v=Nb18CRVmXGY
Red = high blood volume
Blue = low blood volume
Michael Rubinstein, MIT 2013
23. EVM in the Wild: Guinea Pig!
“SuperCreaturefan”: “Guinea pig Tiffany is the first rodent
on Earth to undergo Eulerian Video Magnification.”
http://www.youtube.com/watch?v=uXOSJvNwtIk
Source Motion-magnified
Michael Rubinstein, MIT 2013
26. Talk Overview
• Recap: Eulerian Video Magnification (SIGGRAPH’12)
– With Hao-Yu Wu, Eugene Shih, John Guttag,
Fredo Durand, William T. Freeman
• EVM in the Wild
• Phase-based Video Motion Processing (SIGGRAPH’13)
– With Neal Wadhwa, Fredo Durand, William T. Freeman
Michael Rubinstein, MIT 2013
32. Improvement #1: More Amplification
Improves the bound
by a factor of 4!
(derivation in the paper)
Amplification factor Motion in the sequence
Range of linear method:
Range of Phase-based method:
Michael Rubinstein, MIT 2013
33. Improvement #2: Better Noise Performance
Noise amplified Noise translated
Michael Rubinstein, MIT 2013
34. Results: Phase-based vs. Linear
Linear (SIGGRAPH’12) Phase-based (SIGGRAPH’13)
Clipping artifacts near
Sharp edges and larger motions
Michael Rubinstein, MIT 2013
35. Results: Phase-based vs. Linear
Linear (SIGGRAPH’12) Phase-based (SIGGRAPH’13)Michael Rubinstein, MIT 2013
36. Phase-based Motion Attenuation
Source Linear Motion attenuation +
Color amplification
Amplifies color
And motion jointly
Amplifies color
Without amplifying
motionMichael Rubinstein, MIT 2013
37. Phase-based Motion Attenuation
Source Phase-based motion attenuation
Courtesy of YouTube user ComputerPhysicsLab
Similar to Motion Denoising
[Rubinstein et al. 2010]
[Bai et al. 2012]
Michael Rubinstein, MIT 2013
39. Revealing Invisible Changes in the World
• NSF International Science and Engineering Visualization Challenge (SciVis), 2012
• Science Vol. 339 No. 6119 Feb 1 2013
Michael Rubinstein, MIT 2013
40. Conclusions
• The world is full of small motions and changes we cannot normally see
• We develop algorithms to analyze and visualize them through videos
– Many potential uses in medical applications and scientific analysis
• Phase-based Video Motion Processing (to be presented at SIGGRAPH’13)
– More magnification, less noise
– Phase-based motion analysis is around for a while (Fleet and Jepson 1990)
but not commonly used for editing
• New paper and code available soon!
Michael Rubinstein, MIT 2013
