Your SlideShare is downloading. ×
0
Camera Culture Ramesh  Raskar Camera Culture MIT Media Lab Computational Displays  in  4D, 6D and 8D
Slow Glass: Time Shift http:// baens-universe.com/articles/otherdays Light of Other Days by Bob Shaw http://www.fantasticf...
Shift  Glass
Shift  Glass Space Shifting Angle Shifting Time Shifting Illumination Shifting 4D 4D t 4D 4D
Capture Analyze Display Shift Glass
Capture Analyze Display 5D: Looking around corners 4D: Plenoptic Camera 3D: Flutter Shutter Camera 6D: View and Lighting A...
Can you look around a corner ? Without any device  in the line of sight
Femto-Photography: Higher Dimensional LF FemtoFlash UltraFast Detector Computational Optics Serious Sync
Kirmani, Hutchinson, Davis, Raskar ICCV’2009,  Marr Prize Honorable Mention
Streak Camera  =  Inverse of CRT/CRO Femto-laser
Multi-Dimensional Light Transport 5-D Transport
 
Rescue and Planning
Robot, Car Path Planning
Endoscopy
Camera Culture Ramesh  Raskar Team Moungi G. Bawendi, Professor, Dept of Chemistry, MIT James Davis, UC Santa Cruz Andreas...
Capture Analyze Display 5D: Looking around corners 4D: Plenoptic Camera 3D: Flutter Shutter Camera 6D: View and Lighting A...
Capture Analyze Display 5D: Looking around corners 4D: Plenoptic Camera 3D: Flutter Shutter Camera 6D: View and Lighting A...
Slow Display
Light Reactive Monostable Materials 16 Megapixel, 2 Watt
Day/Night visible
g SlowDisplay.org Saakes, Chiu, Hutchison, .., Inami, Raskar,  Siggraph 2010 Etech Demo
6D Photo Frames One Pixel of a 6D Display = 4D Display Single  Pixel of  6D Frame Martin Fuchs, Ramesh Raskar, Hans-Peter ...
Respond  to Viewpoint  + Ambient Light
6D Display Light sensitive 4D display One Pixel of a 6D Display = 4D Display Raskar, Saakes, Fuchs, Siedel, Lensch, 2008
Beyond Multi-touch: Thin LCD for touch+hover Laptops Mobile
BiDi Screen: Multi-touch + Hover 3D interface
Converting LCD Screen = large Camera for 3D Interactive HCI and Video Conferencing Matthew Hirsch, Henry Holtzman Doug Lan...
Theory: Benefits of Tiled-broadband Masks 0 10 20 30 40 50 60 0 0.1 0.2 0.3 0.4 0.5 0.6 Angular Resolution Average Transmi...
Overview: Sensing Depth from    Array of Virtual Cameras in LCD
Bits Photons CV / Machine Learning Optics Sensors Computational  Displays Signal Processing Light Transport Displays HCI
 
View Dependent Appearance and Iridescent color  Cross section through a single M. rhetenor scale
Two Layer Displays PB = dim displays Lenslets = fixed spatial and angular resolution Dynamic Masks = Brighter, High spatia...
Limitations of 3D Display Lanman,  Hirsch, Kim, Raskar   Siggraph Asia 2010 Front Back Parallax   barrier LCD display
` Light Field Analysis of Barriers L[i,k] L[i,k] i k light box g[k] k f[i] i
f[i] g[k] L[i,k] light box ` Content-Adaptive Parallax Barriers k i
Implementation <ul><li>Components </li></ul><ul><li>22 inch ViewSonic FuHzion VX2265wm LCD [1680×1050 @ 120 fps] </li></ul>
f[i] g[k] L[i,k] light box ` Content-Adaptive Parallax Barriers k i
= Content-Adaptive Parallax Barriers `
Rank-Constrained Displays and LF Adaptation <ul><li>All dual layer display =  rank-1 constraint   </li></ul><ul><li>Light ...
rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Optimization: Iteration 1 Daniel Lee and Sebastia...
Optimization: Iteration 10 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 20 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 30 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 40 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 50 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 60 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 70 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 80 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Optimization: Iteration 90 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebasti...
Content-Adaptive Front Mask (1 of 9)
Content-Adaptive Rear Mask (1 of 9)
Emitted 4D Light Field
Conclusion <ul><li>Described a  rank constraint  for all dual-layer displays </li></ul><ul><ul><li>With a fixed pair of ma...
Lightfield  vs  Hologram  Displays
Is hologram just another ray-based light field? Can a hologram create any intensity distribution in 3D? Why hologram creat...
Parallax Barrier: N p =10 3  pix. Hologram: N H =10 5  pix. θ p =10 pix w θ H  =1000 pix ϕ P ∝w/d ϕ H ∝λ/t H Fourier Patch...
Augmenting Plenoptic Function Wigner Distribution Function Traditional Light Field WDF Traditional Light Field Augmented L...
Light Fields Goal: Representing propagation, interaction and image formation of light using  purely position and angle par...
Augmented Lightfield  for  Wave  Optics  Effects Wigner Distribution Function Light Field LF < WDF Lacks phase properties ...
<ul><ul><li>Free-space propagation </li></ul></ul><ul><ul><li>Light field transformer </li></ul></ul><ul><ul><li>Virtual l...
L(x, θ ) W(x,u) W m = sinc d = delta q  W m p q p  d(θ) p q q  d(θ) *  p  W m *  *  Rays: No Bending 1 Fresnel HG Patch θ ...
Algebraic Rank Constraint s 1 m 2 s 1 m 2 s 1 * s 1 s 1 s 1 * Rank-1 Rank-1 Rank-3
- Transform <t(x+x ʹ /2)t*(x-x ʹ /2)> Interference xʹ x (a) Two Slits, Coherent t(x+x ʹ /2)t*(x-x ʹ /2) W(x,u) Rank-1 t(x ...
L2 L1 L3 ϕ 1 ϕ 1 ϕ 1 ϕ 1 L 1 (x,θ) L 2 (x,θ) L 3 (x,θ) d z 1 h H r z 2 L 1 (x,θ) L 2 (x,θ) L 3 (x,θ) s 1 m 2 (a)
A B C Vary Illumination Direction: -5   ̊ ,  0   ̊,  5   ̊  A B C A  … -5  ̊  5  ̊  0  ̊  No Slits 24mm 36mm t H =25μm w=1...
M2 M1 M3 ϕ 1 ϕ 1 L 1 (x,θ) L 2 (x,θ) L 3 (x,θ) d z 1 r z 2 s 1 m 2 s 1 m 2 s 1 m 2 s 1 * s 1 s 1 s 1 * Rank-1 Rank-1 Rank-3
Is hologram just another ray-based light field? Can a hologram create any intensity distribution in 3D? Why hologram creat...
NETRA:  Interactive Display for Estimating  Refractive Errors and Focal Range Vitor Pamplona  Ankit Mohan  Manuel Oliveira...
Vitor Pamplona  Ankit Mohan  Manuel Oliveira  Ramesh Raskar NETRA:  N ear  E ye  T ool for  R efractive  A ssessment
0.6B  uncorrected  refractive errors NETRA at  LVP Eye Institute 6.5 Billion people 4.5B with Mobile phone 2B refractive e...
* Phoropter-based: $5,000.00 Needs expert,  Moving parts,  Shining lasers Retino scope w/ Lenses Auto-refracto-meter Chart...
Shack-Hartmann Wavefront Sensor Expensive; Bulky, Requires trained professionals Wavefront aberrometer
Shack-Hartmann Wavefront Sensor Laser Sensor Microlens Array Planar Wavefront Shack & Platt 1971 Liang et al 1994 David Wi...
Laser Sensor Displacement =  Local Slope  of the Wavefront Spot Diagram Shack-Hartmann Wavefront Sensor Shack-Hartmann  ~ ...
NETRA   =  Inverse  of Shack-Hartmann Spot Diagram on LCD Cell Phone Display Eye Piece
NETRA   =  Inverse  of Shack-Hartmann Spot Diagram on LCD Cell Phone Display Eye Piece
Spot Diagram on LCD Inverse  of Shack-Hartmann User interactively creates the Spot Diagram Displace 25 points
Spot Diagram on LCD Inverse  of Shack-Hartmann User interactively creates the Spot Diagram Displace 25 points but  3 param...
Limitations <ul><li>Children </li></ul><ul><li>Ability to align lines </li></ul><ul><ul><li>Retina, Animals </li></ul></ul...
Capture Analyze Display 5D: Looking  around corners 6D: View and Lighting Aware 4D: Rank Deficient, multilayer 4D: Netra f...
Upcoming SlideShare
Loading in...5
×

Raskar Keynote at Stereoscopic Display Jan 2011

1,721

Published on

Computational Displays in 4D, 6D, 8D

We have explored how light propagates from thin elements into a volume for viewing for both automultiscopic displays and holograms. In particular, devices that are typically connected with geometric optics, like parallax barriers, differ in treatment from those that obey physical optics, like holograms. However, the two concepts are often used to achieve the same effect of capturing or displaying a combination of spatial and angular information. Our work connects the two approaches under a general framework based in ray space, from which insights into applications and limitations of both parallax-based and holography-based systems are observed.

Both parallax barrier systems and the practical holographic displays are limited in that they only provide horizontal parallax. Mathematically, this is equivalent to saying that they can always be expressed as a rank-1 matrix (i.e, a matrix in which all the columns are linearly related). Knowledge of this mathematical limitation has helped us to explore the space of possibilities and extend the capabilities of current display types. In particular, we have designed a display that uses two LCD panels, and an optimisation algorithm, to produce a content-adaptive automultiscopic display (SIGGRAPH Asia 2010).

(Joint work with R Horstmeyer, Se Baek Oh, George Barbastathis, Doug Lanman, Matt Hirsch and Yunhee Kim) http://cameraculture.media.mit.edu

In other work we have developed a 6D optical system that responds to changes in viewpoint as well as changes in surrounding light. Our lenticular array alignment allows us to achieve such a system as a passive setup, omitting the need for electrical components. Unlike traditional 2D flat displays, our 6D displays discretize the incident light field and modulate 2D patterns in order to produce super-realistic (2D) images. By casting light at variable intensities and angles onto our 6D displays, we can produce multiple images as well as store greater information capacity on a single 2D film (SIGGRAPH 2008).




Ramesh Raskar joined the Media Lab from Mitsubishi Electric Research Laboratories in 2008 as head of the Lab’s Camera Culture research group. His research interests span the fields of computational photography, inverse problems in imaging and human-computer interaction. Recent inventions include transient imaging to look around a corner, next generation CAT-Scan machine, imperceptible markers for motion capture (Prakash), long distance barcodes (Bokode), touch+hover 3D interaction displays (BiDi screen), low-cost eye care devices (Netra) and new theoretical models to augment light fields (ALF) to represent wave phenomena.

In 2004, Raskar received the TR100 Award from Technology Review, which recognizes top young innovators under the age of 35, and in 2003, the Global Indus Technovator Award, instituted at MIT to recognize the top 20 Indian technology innovators worldwide. In 2009, he was awarded a Sloan Research Fellowship. In 2010, he received the Darpa Young Faculty award. He holds over 40 US patents and has received four Mitsubishi Electric Invention Awards. He is currently co-authoring a book on Computational Photography. http://raskar.info

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
1,721
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
84
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • This kind of a technique can be used in other scenarios as well such as Rescue and Planning
  • Robot and car navigation to avoid collisions by estimating position of objects around the bend
  • Martin Fuchs, Ramesh Raskar, Hans-Peter Seidel, Hendrik P. A. Lensch Siggraph 2008
  • This video is only for 4D display that responds to light Bonny’s lenticular prints outside
  • Since we are adapting LCD technology we can fit a BiDi screen into laptops and mobile devices.
  • Here’s a quick teaser to illustrate the capabilities I’m describing. &lt;wait for multi- to hover part to pass&gt; Here you see a user pulling her hands away to rotate and zoom a 3-D model. We also show a use of 3D gesture to navigate a 3D world. We support these modes by creating an array of virtual cameras on an LCD using a technique known as Spatial Heterodyning. Because we’re using an optical technique, we also enable dynamic relighting applications, where real-world lighting is transfered to a rendered scene.
  • So here is a preview of our quantitative results. I’ll explain this in more detail later on, but you can see we’re able to accurately distinguish the depth of a set of resolution targets. We show above a portion of portion of views form our virtual cameras, a synthetically refocused image, and the depth map derived from it.
  • A cross section through a single M. rhetenor scale. Light reflected off each level of the “Christmas tree structure” gives the butterfly its iridescent color. Credit: Pete Vukusic, University of Exeter
  • Augmented plenoptic function the motivation, to augment lf, model diffraction in light field formulation
  • In this paper, we show a self-optometry solution. You look at a cell phone display thru a clip-on eye piece, interactively align a few patterns, hit calculate and get data for your eye prescription.
  • We call our tool NETRA: near eye tool for refractive assessment such as nearsightedness/far/astigmatism Basic idea is to create a unique interactive lightfield display near the eye and is possible due to the highresolution of modern LCDs.
  • 2 billion people have refractive errors And half a billion in developing countries worldwide have uncorrected vision that affects their daily livelyhood. They don’t have access to an optometrist or it simply too expensive. While making and distributing of lenses has become quite easy now, surprisingly there is still no easy solution for measuring eyesight. Can we use a fraction of the 4.5B cellphone displays to address this problem?
  • For better precision, there are many kinds of solutions, some really clever. The beauty of netra is that it avoids moving parts or shining lasers, and all intelligence is in the software.
  • The most accurate method is based on a so called SH WS. It involves shining a laser at the back of the retina and observing the wavefront using a sophisticated sensor. We ask user to generate a spot diagram. But navigating in a high dimensional space is challenging so we come up with a strikingly simple approach to let the user interactively create the spot diagram. We are first to make connection between Shack Hartmann and Lightfields (and it goes well with recent work in computational photography about ALF and Zhang/Levoy). Connection to Adaptive optics/ Astronomy. The way that this device works is that, it shines a lasers in the eye, the laser is reflected in the retina and comes out of the eye being distorted by the cornea. These light rays reaches an array of lenses that focus them to dots in a sensor. The device measures how much this dots deviate from the ideal case. Since it uses lasers, the device is expensive and requires trained professionals
  • For a normal eye, the light coming out of the eye forms a parallel wavefront. The sensor has a lenslet array and we get a spot diagram of uniform dots. This lenslet should remind you of a lightfield camera, and in fact Levoy and others showed last year that there is a close relationship between the two. In addition, Zhang and Levoy, plus our grp has shown the relationship between wavefront sensing and lightfield sensing.
  • When the eye has a distortion, the spot diagram is not uniform. And the displacement of the spots from the center indicates the local slope of the wavefront. From the slope one can integrate and recover the wave shape.
  • NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user. And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
  • NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user. And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
  • For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently. But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
  • For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently. But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
  • Since we are relying on the user interaction, the subject has to be aware of the alignment tasks. So, very young Children may not be able to run the test. Instead of just one eye, one may use both eyes to exploit convergence. And of course, the resolution of NETRA itself is a function of the resolution of the display. With a 326 dpi display, resolution is 0.14 diopters and presciption glasses come in increments of 0.25 diopters. So our system is already sufficiently accurate.
  • Transcript of "Raskar Keynote at Stereoscopic Display Jan 2011"

    1. 1. Camera Culture Ramesh Raskar Camera Culture MIT Media Lab Computational Displays in 4D, 6D and 8D
    2. 2. Slow Glass: Time Shift http:// baens-universe.com/articles/otherdays Light of Other Days by Bob Shaw http://www.fantasticfiction.co.uk/s/bob-shaw/other-days-other-eyes.htm
    3. 3. Shift Glass
    4. 4. Shift Glass Space Shifting Angle Shifting Time Shifting Illumination Shifting 4D 4D t 4D 4D
    5. 5. Capture Analyze Display Shift Glass
    6. 6. Capture Analyze Display 5D: Looking around corners 4D: Plenoptic Camera 3D: Flutter Shutter Camera 6D: View and Lighting Aware 4D: Rank Deficient 4D: Netra for Optometry 4D, 6D, 8D: Augmented Light Field Shift Glass
    7. 7. Can you look around a corner ? Without any device in the line of sight
    8. 8. Femto-Photography: Higher Dimensional LF FemtoFlash UltraFast Detector Computational Optics Serious Sync
    9. 9. Kirmani, Hutchinson, Davis, Raskar ICCV’2009, Marr Prize Honorable Mention
    10. 10. Streak Camera = Inverse of CRT/CRO Femto-laser
    11. 11. Multi-Dimensional Light Transport 5-D Transport
    12. 13. Rescue and Planning
    13. 14. Robot, Car Path Planning
    14. 15. Endoscopy
    15. 16. Camera Culture Ramesh Raskar Team Moungi G. Bawendi, Professor, Dept of Chemistry, MIT James Davis, UC Santa Cruz Andreas Velten, Postdoctoral Associate, MIT Media Lab Ahmed Kirmani, RA, MIT Media Lab Tyler Hutchison, RA, MIT Media Lab Rohit Pandharkar, RA, MIT Media Lab Andrew Matthew Bardagjy, RA, MIT Media Lab Everett Lawson, MIT Media Lab Ramesh Raskar, MIT Media Lab
    16. 17. Capture Analyze Display 5D: Looking around corners 4D: Plenoptic Camera 3D: Flutter Shutter Camera 6D: View and Lighting Aware 4D: Rank Deficient 4D: Netra for Optometry 4D, 6D, 8D: Augmented Light Field Shift Glass
    17. 18. Capture Analyze Display 5D: Looking around corners 4D: Plenoptic Camera 3D: Flutter Shutter Camera 6D: View and Lighting Aware 4D: Rank Deficient 4D: Netra for Optometry 4D, 6D, 8D: Augmented Light Field
    18. 19. Slow Display
    19. 20. Light Reactive Monostable Materials 16 Megapixel, 2 Watt
    20. 21. Day/Night visible
    21. 22. g SlowDisplay.org Saakes, Chiu, Hutchison, .., Inami, Raskar, Siggraph 2010 Etech Demo
    22. 23. 6D Photo Frames One Pixel of a 6D Display = 4D Display Single Pixel of 6D Frame Martin Fuchs, Ramesh Raskar, Hans-Peter Seidel, Hendrik P. A. Lensch 1 2 1 1 2D 2D 2D
    23. 24. Respond to Viewpoint + Ambient Light
    24. 25. 6D Display Light sensitive 4D display One Pixel of a 6D Display = 4D Display Raskar, Saakes, Fuchs, Siedel, Lensch, 2008
    25. 26. Beyond Multi-touch: Thin LCD for touch+hover Laptops Mobile
    26. 27. BiDi Screen: Multi-touch + Hover 3D interface
    27. 28. Converting LCD Screen = large Camera for 3D Interactive HCI and Video Conferencing Matthew Hirsch, Henry Holtzman Doug Lanman, Ramesh Raskar Siggraph Asia 2009 BiDi Screen
    28. 29. Theory: Benefits of Tiled-broadband Masks 0 10 20 30 40 50 60 0 0.1 0.2 0.3 0.4 0.5 0.6 Angular Resolution Average Transmission (%) Pinholes Sum-of-Sinusoids MURA 11x11 23x23 43x43 Angular Resolution Tiled-Broadband Code MURA Sum-of-Sinusoids Pinholes
    29. 30. Overview: Sensing Depth from Array of Virtual Cameras in LCD
    30. 31. Bits Photons CV / Machine Learning Optics Sensors Computational Displays Signal Processing Light Transport Displays HCI
    31. 33. View Dependent Appearance and Iridescent color Cross section through a single M. rhetenor scale
    32. 34. Two Layer Displays PB = dim displays Lenslets = fixed spatial and angular resolution Dynamic Masks = Brighter, High spatial resolution barrier sensor/display lenslet sensor/display
    33. 35. Limitations of 3D Display Lanman, Hirsch, Kim, Raskar Siggraph Asia 2010 Front Back Parallax barrier LCD display
    34. 36. ` Light Field Analysis of Barriers L[i,k] L[i,k] i k light box g[k] k f[i] i
    35. 37. f[i] g[k] L[i,k] light box ` Content-Adaptive Parallax Barriers k i
    36. 38. Implementation <ul><li>Components </li></ul><ul><li>22 inch ViewSonic FuHzion VX2265wm LCD [1680×1050 @ 120 fps] </li></ul>
    37. 39. f[i] g[k] L[i,k] light box ` Content-Adaptive Parallax Barriers k i
    38. 40. = Content-Adaptive Parallax Barriers `
    39. 41. Rank-Constrained Displays and LF Adaptation <ul><li>All dual layer display = rank-1 constraint </li></ul><ul><li>Light field display is a matrix approximation problem </li></ul><ul><li>Exploit content-adaptive parallax barriers </li></ul>Lanman, Hirsch, Kim, Raskar Siggraph Asia 2010 ` = Content-Adaptive Parallax Barriers
    40. 42. rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Optimization: Iteration 1 Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    41. 43. Optimization: Iteration 10 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    42. 44. Optimization: Iteration 20 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    43. 45. Optimization: Iteration 30 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    44. 46. Optimization: Iteration 40 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    45. 47. Optimization: Iteration 50 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    46. 48. Optimization: Iteration 60 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    47. 49. Optimization: Iteration 70 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    48. 50. Optimization: Iteration 80 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    49. 51. Optimization: Iteration 90 rear mask: f 1 [i,j] front mask: g 1 [k,l] reconstruction (central view) Daniel Lee and Sebastian Seung. Non-negative Matrix Factorization. 1999. Vincent Blondel et al . Weighted Non-negative Matrix Factorization. 2008.
    50. 52. Content-Adaptive Front Mask (1 of 9)
    51. 53. Content-Adaptive Rear Mask (1 of 9)
    52. 54. Emitted 4D Light Field
    53. 55. Conclusion <ul><li>Described a rank constraint for all dual-layer displays </li></ul><ul><ul><li>With a fixed pair of masks, emitted light field is rank- 1 </li></ul></ul><ul><li>Achieved higher-rank approximation using temporal multiplexing </li></ul><ul><ul><li>With T time-multiplexed masks, emitted light field is rank- T </li></ul></ul><ul><ul><li>Constructed a prototype using off-the-shelf panels </li></ul></ul><ul><li>Demonstrated light field display is a matrix approximation problem </li></ul><ul><li>Introduced content-adaptive parallax barriers </li></ul><ul><ul><li>Applied weighted NMF to optimize weighted Euclidean distance to target </li></ul></ul><ul><li>Adaptation increases brightness and refresh rate of dual-stacked LCDs </li></ul>` = Content-Adaptive Parallax Barriers
    54. 56. Lightfield vs Hologram Displays
    55. 57. Is hologram just another ray-based light field? Can a hologram create any intensity distribution in 3D? Why hologram creates a ‘wavefront’ but PB does not? Why hologram creates automatic accommodation cues? What is the effective resolution of HG vs PB?
    56. 58. Parallax Barrier: N p =10 3 pix. Hologram: N H =10 5 pix. θ p =10 pix w θ H =1000 pix ϕ P ∝w/d ϕ H ∝λ/t H Fourier Patch Horstmeyer, Oh, Cuypers, Barbastathis, Raskar, 2009
    57. 59. Augmenting Plenoptic Function Wigner Distribution Function Traditional Light Field WDF Traditional Light Field Augmented LF Interference & Diffraction Interaction w/ optical elements ray optics based simple and powerful wave optics based rigorous but cumbersome Oh, Raskar, Barbastathis 2009: Augmented Light Field
    58. 60. Light Fields Goal: Representing propagation, interaction and image formation of light using purely position and angle parameters Reference plane position angle LF propagation (diffractive) optical element LF LF LF LF LF propagation light field transformer
    59. 61. Augmented Lightfield for Wave Optics Effects Wigner Distribution Function Light Field LF < WDF Lacks phase properties Ignores diffraction, interferrence Radiance = Positive ALF ~ WDF Supports coherent/incoherent Radiance = Positive/Negative Virtual light sources LF Augmented Light Field WDF
    60. 62. <ul><ul><li>Free-space propagation </li></ul></ul><ul><ul><li>Light field transformer </li></ul></ul><ul><ul><li>Virtual light projector </li></ul></ul><ul><ul><li>Possibly negative radiance </li></ul></ul>
    61. 63. L(x, θ ) W(x,u) W m = sinc d = delta q W m p q p d(θ) p q q d(θ) * p W m * * Rays: No Bending 1 Fresnel HG Patch θ u * Zooming into the Light Field
    62. 64. Algebraic Rank Constraint s 1 m 2 s 1 m 2 s 1 * s 1 s 1 s 1 * Rank-1 Rank-1 Rank-3
    63. 65. - Transform <t(x+x ʹ /2)t*(x-x ʹ /2)> Interference xʹ x (a) Two Slits, Coherent t(x+x ʹ /2)t*(x-x ʹ /2) W(x,u) Rank-1 t(x 1 )t*(x 2 ) R 45 , D Transform -1 u
    64. 66. L2 L1 L3 ϕ 1 ϕ 1 ϕ 1 ϕ 1 L 1 (x,θ) L 2 (x,θ) L 3 (x,θ) d z 1 h H r z 2 L 1 (x,θ) L 2 (x,θ) L 3 (x,θ) s 1 m 2 (a)
    65. 67. A B C Vary Illumination Direction: -5 ̊ , 0 ̊, 5 ̊ A B C A … -5 ̊ 5 ̊ 0 ̊ No Slits 24mm 36mm t H =25μm w=125μm z H =10cm (c)
    66. 68. M2 M1 M3 ϕ 1 ϕ 1 L 1 (x,θ) L 2 (x,θ) L 3 (x,θ) d z 1 r z 2 s 1 m 2 s 1 m 2 s 1 m 2 s 1 * s 1 s 1 s 1 * Rank-1 Rank-1 Rank-3
    67. 69. Is hologram just another ray-based light field? Can a hologram create any intensity distribution in 3D? Why hologram creates a ‘wavefront’ but PB does not? Why hologram creates automatic accommodation cues? What is the effective resolution of HG vs PB?
    68. 70. NETRA: Interactive Display for Estimating Refractive Errors and Focal Range Vitor Pamplona Ankit Mohan Manuel Oliveira Ramesh Raskar
    69. 71. Vitor Pamplona Ankit Mohan Manuel Oliveira Ramesh Raskar NETRA: N ear E ye T ool for R efractive A ssessment
    70. 72. 0.6B uncorrected refractive errors NETRA at LVP Eye Institute 6.5 Billion people 4.5B with Mobile phone 2B refractive errors
    71. 73. * Phoropter-based: $5,000.00 Needs expert, Moving parts, Shining lasers Retino scope w/ Lenses Auto-refracto-meter Chart with Lenses In-Focus: Focometer Optiopia Solo-health: EyeSite NETRA Technology Shining Light plus lenses Fundus Camera Moving lenses + target Moving lenses + target Reading chart on monitor Cellphone + eyepiece Cost to buy $2,000* ~$10,000 ~$100 ~$495 ~$200 -- $30 Cost per test ~$36 ~$36 ~$5 -- -- -- ~$1 Data capture No Comp. No No No Comp. Phone Mobility <500g >10Kg 2kg 1kg <5kg >10Kg <100g Speed Fast Fast Medium Medium -- Fast Fast Scalability No No No Yes Probably No Yes Accuracy 0.15 0.15 0.5 0.75 -- -- <0.5 Self evaluation No No Yes Yes Yes Yes Yes Electricity Req No Yes No No -- Yes No Astigmatism Yes Yes Yes/No No -- Yes Yes Network No Yes No No No Yes Yes Training High High High Medium Medium Low Low
    72. 74. Shack-Hartmann Wavefront Sensor Expensive; Bulky, Requires trained professionals Wavefront aberrometer
    73. 75. Shack-Hartmann Wavefront Sensor Laser Sensor Microlens Array Planar Wavefront Shack & Platt 1971 Liang et al 1994 David Williams et al, Rochester Spot Diagram
    74. 76. Laser Sensor Displacement = Local Slope of the Wavefront Spot Diagram Shack-Hartmann Wavefront Sensor Shack-Hartmann ~ Lightfields Levoy et al 2009 Zhang and Levoy 2009: Observable Light Field Oh, Raskar, Barbastathis 2009: Augmented Light Field
    75. 77. NETRA = Inverse of Shack-Hartmann Spot Diagram on LCD Cell Phone Display Eye Piece
    76. 78. NETRA = Inverse of Shack-Hartmann Spot Diagram on LCD Cell Phone Display Eye Piece
    77. 79. Spot Diagram on LCD Inverse of Shack-Hartmann User interactively creates the Spot Diagram Displace 25 points
    78. 80. Spot Diagram on LCD Inverse of Shack-Hartmann User interactively creates the Spot Diagram Displace 25 points but 3 parameters
    79. 81. Limitations <ul><li>Children </li></ul><ul><li>Ability to align lines </li></ul><ul><ul><li>Retina, Animals </li></ul></ul><ul><li>Single Eye test </li></ul><ul><ul><li>Other eye for convergence-forced accommodation </li></ul></ul><ul><li>Resolution is a function of the display DPI </li></ul><ul><ul><li>Samsung Behold II – 160 DPI – 0.35D </li></ul></ul><ul><ul><li>Google Nexus One – 250 DPI – 0.2D </li></ul></ul><ul><ul><li>Apple iPhone 4G – 326 DPI – 0.14D </li></ul></ul>
    80. 82. Capture Analyze Display 5D: Looking around corners 6D: View and Lighting Aware 4D: Rank Deficient, multilayer 4D: Netra for Optometry 4D, 6D, 8D: Augmented Light Field MIT Media Lab Ramesh Raskar http://raskar.info Shift Glass ` = WDF Light Field Augmented LF
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×