HR3D: Content Adaptive Parallax Barriers

Content-Adaptive Parallax Barriers for Automultiscopic 3D Display
Douglas Lanman Matthew Hirsch
Yunhee Kim Ramesh Raskar
MIT Media Lab
Imaging Hardware Session
Friday, 17 December | 4:15 PM – 6:00 PM | Room E1-E4

A Renewed Interest in 3D Displays
Theatrical and Home Movies
Consumer Electronics
A recent history of 3D displays
Avatar [released December 2009, has grossed >$2.7 billion so far]
RealD Cinema, Dolby 3D, XpanD 3D, MasterImage 3D, etc.
NVIDIA 3D Vision
Nintendo 3DS
Expansion of 3D to our living rooms, mobile phones, advertising, etc.

What is a 3D Display?
Monocular Depth Cues Supported by Conventional Photography
relative and familiar size
perspective and occlusion
texture gradient, shading and lighting, and atmospheric effects

Binocular Depth Cues
“It being thus established that the mind perceives an object of three dimensions by means of the two dissimilar pictures projected by it on the two retinae, the following question occurs: What would be the visual effect of simultaneously presenting to each eye, instead of the object itself, its projection on a plane surface as it appears to that eye?”
Binocular Depth Cues
retinal disparity [Charles Wheatstone, 1838]
convergence

What is missing?
Additional Monocular Depth Cues
motion parallax [Hermann von Helmholtz, 1866]
accommodation

Conventional vs. Light Field Displays
Lenslet Array
Pinhole Array
u
s
Virtual Scene
Viewers
Display
Conventional displays are view-independent
Consider two-plane light field parameterization [Levoy and Hanrahan, 1996]
Pixel intensity/color does not vary as a function of viewing angle
Light field displays are view-dependent (automultiscopic)
Achieved with pinhole/lenslet array placed close to the display
Trades decreased spatial resolution for increased angular resolution
Frederic Ives. Parallax Stereogram and Process of Making the Same. 1903.
Gabriel Lippmann. Épreuves Réversibles Donnant la Sensation du Relief. 1908.

Previous Approaches
barrier
lenslet
sensor/display
sensor/display
Achieving Light Field (Automultiscopic) Display
Two alternatives for spatially-multiplexed light field display:
Attenuating masks using parallax barriers [Ives, 1903]
Refracting lenslet arrays [Lippmann, 1908]
Barriers cause severe attenuation  dim displays
Lenslets impose fixed trade-off between spatial and angular resolution
Masks can be optimized to increase optical efficiency
Enables flexible 3D displays with increased brightness and refresh rate

Target 4D Light Field

viewer moves right
viewer moves up

Parallax Barrier Front Mask (1 of 9)

Parallax Barrier Rear Mask (1 of 9)

Outline
Introduction
Content-Adaptive Parallax Barriers
Implementation and Results
Discussion and Future Work
Conclusion

Light Field Analysis of Barriers
k
L[i,k]
i
`
k
g[k]
i
L[i,k]
f[i]
light box

Time-Multiplexing using Shifted Pinholes
L[i,k]
`
k
g[k]
i
f[i]
light box
Ken Perlinet al. An Autosteroscopic Display. 2000.
Yunhee Kim et al. Electrically Movable Pinhole Arrays. 2007.

L[i,k]
`
k
g[k]
i
f[i]
light box

`
=

viewer moves right
viewer moves up

Optimization: Iteration 1
rear mask: f1[i,j]
front mask: g1[k,l]
reconstruction (central view)
Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999.
Vincent Blondel et al. Weighted Non-negative Matrix Factorization. 2008.

rear mask: f1[i,j]
front mask: g1[k,l]

Content-Adaptive Front Mask (1 of 9)

Content-Adaptive Rear Mask (1 of 9)

Emitted 4D Light Field

Benefits of Content-Adaptation
1) Increasing brightness:
2) Increasing refresh rate:

Outline
Introduction
Conclusion

Implementation
Components
22 inch ViewSonic FuHzion VX2265wm LCD [1680×1050 @ 120 fps]

Motion Parallax
viewer moves right
viewer moves up
Light Field

Increasing Brightness and Refresh Rate
1) Increasing brightness:
2) Increasing refresh rate:

Increasing Brightness
Time-Shifted Barriers (central view)
Content-Adaptive Barriers (central view)

Increasing Refresh Rate
Time-Shifted Barriers (central view)
Content-Adaptive Barriers (central view)

Outline
Introduction
Conclusion

Future Work: Analytic Adaptation?
rear mask: f1[i,j]
front mask: g1[k,l]
What patterns does content-adaptation produce?
Optimization appears to produce local parallax barriers

Future Work: Analytic Adaptation?
viewer moves right (b)
viewer moves up (a)
Light Field
(step edge)
Rear mask
Front mask
What patterns does content-adaptation produce?
Optimization appears to produce local parallax barriers
Related to aperture problem(i.e., motion of windowed grating)
NMF converges to local stationary point, not global minimum
Emergent structure predicted by angular gradient:
David Marr and Shimon Ullman. Directional Selectivity and Its Use in Early Visual Processing. 1981.

Future Work: Local Parallax Barriers
viewer moves right (b)
viewer moves up (a)
Light Field
Streamlines of the Angular Gradient*
Rear Mask
Front Mask
*Brian Cabral and Leith Casey Leedom. Imaging Vector Fields using Line Integral Convolution. 1993.

Outline
Introduction
Conclusion

Conclusion
`
=
Described a rank constraint for all dual-layer displays
With a fixed pair of masks, emitted light field is rank-1
Achieved higher-rank approximation using temporal multiplexing
With T time-multiplexed masks, emitted light field is rank-T
Constructed a prototype using off-the-shelf panels
Demonstrated light field display is a matrix approximation problem
Introduced content-adaptive parallax barriers
Applied weighted NMF to optimize weighted Euclidean distance to target
Adaptation increases brightness and refresh rate of dual-stacked LCDs

Questions and Answers
“Can we request that Photography renders the full variety offered by the direct observation of objects? Is it possible to create a photographic print in such a manner that it represents the exterior world framed, in appearance, between the boundaries of the print, as if those boundaries were that of a window opened on reality? It appears that yes, we can request from Photography infinitely more than from the human hand.”
Gabriel Lippmann, 1908

http://web.media.mit.edu	11335
http://translate.googleusercontent.com	89
http://augmentedlifeinmotion.com	69
http://ideca.bellasartes.uclm.es	41
http://diyiat.blogspot.com	12
http://webcache.googleusercontent.com	7
http://www.slideshare.net	5
http://fanyi.youdao.com	4
http://sirius.dn.ua	4
http://forum.sirius.dn.ua	4
http://honyaku.yahoofs.jp	3
http://perevod.yandex.net	1
http://66.163.168.225	1
http://cameraculture.media.mit.edu	1

HR3D: Content Adaptive Parallax Barriers

by Matt Hirsch - MIT Media Lab on Dec 22, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

14 Embeds 11,576

Statistics

HR3D: Content Adaptive Parallax Barriers — Presentation Transcript