Downloaded 54 times
Uploaded byVitor Pamplona
Tailored Displays to Compensate for Visual Aberrations - SIGGRAPH Presentation
The document discusses tailored displays designed to compensate for various visual aberrations such as myopia, hyperopia, presbyopia, astigmatism, and cataracts, emphasizing their potential to provide customized viewing experiences through multi-depth and multi-focus technology. User evaluations indicate strong preferences for tailored solutions over conventional options, demonstrating effectiveness across different refractive conditions. The document also outlines future developments in addressing retinal diseases and enhancing display technology for various ocular requirements.
More Related Content
Similar to Tailored Displays to Compensate for Visual Aberrations - SIGGRAPH Presentation
Tailored Displays to Compensate for Visual Aberrations - SIGGRAPH Presentation
- 1. Vitor Pamplona TailoredDisplays.com Manuel M. Oliveira Daniel Aliaga Ramesh Raskar Tailored Displays to Compensate for Visual Aberrations
- 2.
- 3. Farsighted View (need ofreading glasses)
- 4. Tailored Dashboard (actual results)
- 5.
- 6.
- 7.
- 8. Computer Generated Glasses Perfect vision Focal Range
- 9. Computer Generated Glasses Focusing Here Myope’s Focal Range Subject’s Focal Point Does Not Change
- 10. Computer Generated Glasses Focusing Here Hyperope’s and Presbyope’s Focal Range Subject’s Focal Point Does Not Change
- 11. We can createa hologram for the visually impaired
- 12. because uncorrected individualsare already not focusing on the display
- 13. and there isno mismatch between convergence and accommodation
- 14. Tailoring Process Myopic View: -3D Focusing Here He can focus up to 33cm (12in) Blurred Image Distance Display-Eye: 50cm
- 15. Tailoring Process Myopic View: -3D Light-field Focusing Here He can focus up to 33cm (12in) Display Distance Display-Eye: 50cm
- 16. Tailoring Process Myopic View: -3D Light-field Focusing Here He can focus up to 33cm (12in) Display Distance Display-Eye: 50cm
- 17. Tailoring Process Myopic View: -3D Light-field Focusing Here He can focus up to 33cm (12in) Display Pixel Size of 96um at 33cm 1 arc-minute Distance Display-Eye: 50cm Resolution
- 18. Pixel Size andAlignment 1 arc minute Pixel on the Virtual image Zoomed Version
- 19. Pixel Size andAlignment 1 arc minute Pixel on the Virtual image Zoomed Version
- 20. Working Resolution: 1800DPI Channel Size 4.7um! $150 Vuzix HMD LCD
- 21.
- 22.
- 23.
- 24. Tailoring Process Subject’s prescription -2D -1D @ 90 Light-field Two Points in Focus He focus at 30cm to 50cm. Display 50cm 30cm Where the Subject’s Accommodate
- 25. Wavefront Maps 90 degrees Sphere: -2D Cylinder: -1D Axis: 90° 0 degrees k Lens focal length in k Zernike Polynomials
- 26. Inputs Tailoring Process Light-field f(k) Display LCD1 LCD2 LCD1
- 27. Single-Focus Multi-Depth Displays Onlyone object Eye
- 28. f(k) Display k Eye R S1 S2 t a æ -k k - S2 ö R(S2 , k) = a ç + ÷+ k è f (k) t ø
- 30. Cataract Maps k Nuclear Cataract Cataract density in k Sub-capsular Cataract
- 31.
- 32.
- 33. Projector Dual Stack ofLCDs Camera – the “eye”
- 34. Lens Array Vuzix HeadMounted Display Vuzix Head Mounted Display
- 35. 3.25D Myopic Eye 1mm using a display at 37cm from the eye Input Image As Seen on a Regular Display As Seen on a Tailored Display Resolution of 0.87 arc minutes
- 36. 3.5D Myopic Eyewith 0.9mm Cataract Nuclear Cataracts Display at 47cm from the eye 7.9mm Input p = 20mm No Cataracts No Cataract Tailoring Tailored for Cataracts Resolution of 0.87 arc minutes
- 37. Presbyopic +3D View 20cm / 7.8in Out-of-focus Blur Size
- 38. Presbyopic +3D View 20cm / 7.8in 8cmx8cm Size Out-of-focus Blur Size
- 39. 1x1cm Presbyopic +3D View Light-Field Display 20cm / 7.8in 50cm / 20in 8cmx8cm Size Out-of-focus Blur Size
- 40. Presbyopic +3D View 20cm / 7.8in Out-of-focus Blur Size
- 41. Presbyopic +3D View 20cm / 7.8in 8cmx8cm Size Out-of-focus Blur Size
- 42. Presbyopic +3D View 20cm / 7.8in 50cm / 20in 8cmx8cm Size Out-of-focus Blur Size
- 43. Presbyopic +3D View 20cm / 7.8in Out-of-focus Blur Size
- 44. Presbyopic +3D View 20cm / 7.8in 8cmx8cm Size Out-of-focus Blur Size
- 45. Presbyopic +3D View 20cm / 7.8in 50cm / 20in 8cmx8cm Size Out-of-focus Blur Size
- 46. User Evaluations: StrongMyopia Perfect vision Focal Range 0D 1D 2D 3D 4D 5D
- 47. User Evaluations: StrongMyopia 5D 0D 1D 2D 3D 4D 5D
- 48. User Evaluations: StrongMyopia 0D 1D 2D 3D 4D 5D
- 49. User Evaluations: StrongMyopia 5D 0D 1D 2D 3D 4D 5D
- 50. User Evaluations: StrongMyopia 100 100% 100% 98% 80 Preferability ( %) 97% 60 +5D Lens 40 54% No Lens 46% 3% 2% 20 0 1D (1m ) 2D (50cm ) 3D (33cm ) 4D (25cm ) 5D (20cm ) Optical Power of the Projected Image (D) 13 Volunteers, 16 votes each.
- 51. User Evaluations: Astigmatism 100 46 44 98% 80 42 90% 60 +4D Ast. Lens No Lens 40 10% 66% 34% 2% 20 0 0D (Inf) 2D (50cm ) 4D (25cm ) Optical Power of the Projected Image (D) 10 Volunteers, 16 votes each.
- 52. User Evaluations: Keratoconus 48 47 A AA A A 46 45 44 43 42 41 Regular Tailored Diopters display display 1 Volunteer, 80 votes.
- 53. User Evaluations: Keratoconus 100 100% User Perspective (%) 48 80 47 92% 46 60 45 8% 44 40 43 How subject saw it 0% Corrected 42 20 Diopters 41 Duplicated 0 0D (Inf) Keratoconus Correction of the Projected Image 1 Volunteer, 80 votes.
- 54. Multi-Focus Multi-Depth Display Input Image Input Depth +0.5D from the Image Plane -0.5D from the Image Plane
- 55. Limitations • Eyes fixedrelative to the display – Similar to glasses-free 3D Displays – Higher-order requires more fixation • High-resolution LCD panels (PPI) – Giga-pixel displays for monitors • Other ocular diseases may not be corrected for 61
- 56. Tailored Displays toCompensate for Visual Aberrations • The First Multi-depth Multi-focus Display – Time-varying Optical Corrections – Resolution around 1 arc minute • Correcting for a Wide Range of Conditions: – Myopia, Hyperopia and Presbyopia – Astigmatism and High-order aberrations – Cataracts www.TailoredDisplays.com
- 57. Future Works • RetinalDiseases on the Tailoring Pipeline – Retinal Displacements (Image Warping) – Color Blindness (Re-coloring) • Light-Field to Hardware Mapping – Temporal and Context-Based Optimizations for Brightness • Stereo Multi-focus Multi-depth Displays
- 58. Vitor Pamplona TailoredDisplays.com Manuel M. Oliveira Daniel Aliaga Ramesh Raskar Tailored Displays to Compensate for Visual Aberrations
- 59. Tailored Displays toCompensate for Visual Aberrations • The First Multi-depth Multi-focus Display – Time-varying Optical Corrections – Resolution around 1 arc minute • Correcting for a Wide Range of Conditions: – Myopia, Hyperopia and Presbyopia – Astigmatism and High-order aberrations – Cataracts www.TailoredDisplays.com
- 60. Cell Phone-based Refractive Measurements The Inverse of Shack-Hartmann Vitor Pamplona, Manuel Oliveira, Ankit Mohan, Ramesh Raskar
- 61. CATRA: Quantitative Cataract Maps Unique, low-cost quantitative lens mapping Vitor Pamplona, Erick Passos, Jan Zizka, Everett Lawson, Esteban Clua, Manuel Oliveira, Ramesh Raskar
- 62.
- 63. Tailored Displays toCompensate for Visual Aberrations • The First Multi-depth Multi-focus Display – Time-varying Optical Corrections – Resolution around 1 arc minute • Correcting for a Wide Range of Conditions: – Myopia, Hyperopia and Presbyopia – Astigmatism and High-order aberrations – Cataracts www.TailoredDisplays.com
- 64.
- 65. Challenge Billions of People with Uncorrected Refractive Error, by Region 1.70 1.80 1.60 1.40 1.20 1.00 4 billion people estimated 0.80 to need eyeglasses 0.60 0.50 worldwide 0.40 0.20 0.13 0.10 0.02 0.00 Emerging Africa & Latin Europe North Asia Middle America America East Source: Essilor, Infomarket 2009, CPB Research,
- 66. Challenge Due to: 1. Heavy “near work” 2. Intensity of education. 90% of high-school kids are myopic in 3. Games, computers and TV. major Asian cities 4. Insufficient time spent outdoors. Morgan et al. 2012. Science 2012.
- 67. User Experience DesignChallenges
- 68. Needs expert, Annoying Experience, Limited Eye Glasses Contact Multi- Surgery Adaptive Adaptive Tailoring Lenses focals optics spectacles Technology Coated Hard Flexible/Hard Hard Lenses LASIK and Dynamic Liquid / Parallax Lenses+Frame Lenses with many Cataract micro mirror Electronic / Barrier focal Surgery array on Mechanic powers contacts Lenses Cost in the US <$100 <$50 ~$200 ~$2,000 Expensive $30 Feature Quality 1 arc-min 1 arc-min 1 arc-min 1 arc-min 1 arc-min 1 arc-min 1 arc-min Astigmatism Yes Yes Yes Yes Yes No Yes Cataracts No No No Yes Yes No Yes Keratoconus No Yes No No No No Yes Adaptive No No No No Yes Yes Yes Carry on Yes No Yes No No Yes No Need doctor Yes Yes Yes Yes ? No No Maintenance Easy Medium Easy Nothing Hard Medium Easy Invasive No Yes No Yes Yes No No
- 69. User Evaluations: StrongMyopia 0D 1D 2D 3D 4D 5D
- 70. User Evaluations: StrongMyopia 5D 0D 1D 2D 3D 4D 5D
- 71. User Evaluations: StrongMyopia 0D 1D 2D 3D 4D 5D
Editor's Notes
- #2 Imagine a special kind of display that adjusts itself to compensate for vision problems in such a way that it avoids the use of eyeglasses when looking at these displays. We call our solution the Tailored Display. The basic idea is to create a light-field rendering technique that projects partial holograms in space for the visually impared using a low-cost 3D hardware, such as simple stack of LCDs.
- #3 Driving may be a problem for farsighted individuals that usually rely on reading glasses or bi-focals to see both the streets and the dashboard of the car at, kind of, the same time.
- #4 If they don’t wear them, the dashboard, of course, gets blurry.
- #5 A Tailored display allow a safe drive by compensating for whatever refractive condition the driver has in software in real time.
- #6 Although wearing eye glasses appear to be an easy solution, they are annoying in many ways, Specially if you have to put them on and off all the time, such as when you need them just to check you phone.
- #7 OR when you are running and you don’ t want to carry your glasses but you still want to read to your newest gadget to check for you performance.
- #8 The beauty of tailoring is that it avoids extra carry-ons for certain activities. And since all the intelligence is in the software, you can share the same hardware with many people and all of them will be corrected for their problems.
- #9 So, how does it work? We have low order aberratons and high order aberrations. Let’ s start easy. Low-order refractive aberrations are Myopia, Hyperopia, And presbiopia. And they essentially shift the focal range or in a better word the accoomodation range. MyopiaHyperopya shifts it farther from the viewreAnd Presbyopia decreases the range.
- #10 Usually a person that does not need eyeglasses has the display inside his focal range.He can accommodate anywhere in this range to see the display in focus.
- #11 Myopia, on the other way, shifts the focal range closer to the viewer The device is placed outside his accommodation range, so he cannot focus there But his desire to see the display will make his eyes accommodate to the closest point in focus. The tailoring process creates a hologram image at the plane the subject is accommodating. And this is one of the big points of the paper, the subject’s point of focus or accommodation actually does not change.
- #12 For hyperopes/nearsighted the accommodation range is shifted against the viwers. These individuals usually have the display before their closest point in focus and thus they see a blurred image. The tailoring process then moves the image away from the eye, inside their focal range. And this intuition of working where the subject is focusing instead of trying to chang his accommodation state led us to our first lesson we learned during this work
- #13 That we can easily create a hologram for the visually impaired.
- #14 Mainly because we do not need to create any accommodation clues to allow them to focus in mid air. They are already focusing there. The fact that they are not focusing in the display plane, makes things a lot easier. // We know how tricky is to drive accommodation to create a multi-focus display. Specially using low-cost hardware such as stack of LCDs.
- #15 And there is no clear mismatch between convergence and accommodation, as standard 3D displays have. The viewer is already used to the situation they have. We are not doing anything unnatural for him.
- #16 So, an emmetrope can see the image clearly because his accommodation moves to conjugate the real object with the retina.
- #17 But say a 3D myope don’t have enough accommodation to get to focus at 50cm-away display. They light coming from this device converges before the retina and he only sees a blur. In order to minimize the blur, this subject will accommodate at 33cm from his eye.
- #18 3 dioptermyope doesn’t have enough accommodation to get to focus at 50cm-away display. In order to minimize the blur, this subject will accommodate at 33cm from his eye. Trying to focus in in mid air. To create the tailored display, we need to replace the standard display by a stack of LCDs
- #19 that allows us to cast directed light. And Then recreate the image, pixel by pixel at 33cm from that eye. Every pixel on this image, will be composed by a set of pixels from both LCD that converge at 33cm, and spread again simulating this virtual point in space.
- #20 We just repeat the process for all the pixels and we have a hologram right there.
- #21 Another important fact we learned while doing this work is that we have the resolution required to create that. In a ophthalmology perspective, a corrected human eye can distinguish features on a resolution of about 1 arc minute, which would mean in this case that we need 96 microns pixels at 33cm. A pretty reasonable resolution,
- #22 Except for the fact that we are playing on a discrete domain and pixels on LCD1 and 2 have a given size, And we probably won’ t be able to align them all into a single spot. If the misalignment is bigger than 1-arc minute the individual will end up seeing a blured pixelAnd building a tailored display would not make any sense.
- #23 We want this misalignment to be smaller than the human resolution. Meaning the person cannot see this blur. And for that we need high-resolution displays.
- #24 And we ended using a resolution that is around 1800 dpi, in which color chaneels are down to 4 microns.These displays you can find in many nonexpensive equipment such as on head mounted displays (in which for 150 dollars you get two) and LCD projectorsOn that resolution we can pretty much correct for a large set of conditions and optical powers.
- #25 Things get more exciting when you have more interesting aberrations. Astigmatism, for instance, create two main focal points for each accommodative state.
- #26 In this one we have a zero power at horizontal
- #27 And positive power at 90 degrees. The intermediate meridians between these two are an non-linear interpolation between these powers.
- #28 So, let say we have a 2D myope individual with a 1D astigmatism at 90 degrees, looking to our display. This means that the horizontal meridian he will focus at 30cm while the vertical at 50cm. The subject will usually accommodate in between them, to try to get both meridians with the minimum blur . What we need to do is to create an image at 50cm and another one at 30. That would be enoug, but you can also go beyond and compensate for all other meridians.
- #29 To make that happen we need to usea good interpolation function that will create this wavefront map. Essentially every prescription can be translated to these map of focal lengths and we can use them as an input for the tailoring procedure.
- #30 The procedure takes a wavefront map and the image that will be displayedEssentially, We raytrace for each pair of pixels from both lcds, we check where the light ray is falling on the retina and we sample the input image. to set the color of the pixel on LCD1Doing this for all the pixels we end up creating these holograms in space according to the subjects wavefront map. This one in particular has a 2D optical coma. In the end, we have a distorted light-field to be rendered.
- #31 AT these stage,A single pixel in the retina can come from many virtual depths , each depth for each part of the corneaYou can create as much as you want to increase brightness. `This is why we call it a multi-depth display, because for a given pixel in the retina and a accommodative state we may have objects in many depths
- #32 And of course, by distributing the images in space we also have to scale them accodingly.
- #33 scale them accordingly.
- #34 So, everything I explained since the beginning reduces to this very simple equation based on the parameters of the system.Which is a mapping between the light-field and the image in the retina. The equation can be easily converted to a matrix representation and parallelized in the gpu.
- #35 And what about cataracts? Can we do something about it? Cataracts are these clouds you may see in someone’ s eye. They scatter light, decreasing contrast and increasing glare.
- #36 They can be measured in terms of density maps, just like the wavefront maps. Each point there represents an attenuation coefficient. And if we use them as input
- #37 the procedure can detect where the rays are going throught the blob and remove them.
- #38 WE then normalize the image by the number of rays each pixel in the retina is receiving. And we have the image ready to be displayed.
- #39 We tested the system in two prototypes, one uses a dual stack of LCDs from a low-cost LCD Projector. These lenses you see are optometric lenses to simulate aberrations we want to correct.
- #40 And the second prototype comes from a Vizux Head Mounted display with a tiny Lens Array on top of the display. Both are 1800 DPI displays.
- #41 So here is a tiny snoopy as an input image for a 3.25 diopter myopic subjectAnd here is how he see it on a regular display at a 37cm away display. And on a Tailored display
- #42 The tinysmile is corrected for a 3.5 Myopic subject with Nuclear Cataracts. Here you see a picture without the cataract simulation, With the cataract on the lens of the cameraAnd tailored to avoid scattering.
- #44 We also experimented with color images with the size of a tablet for a presbyopic which needs +3D on reading glasses. The display is at 20cm from the eye, but the subject can only focus at 50cm.
- #45 So, here is how he would see the image in a regular display without his glasses
- #46 And by looking to a tailored display. This tiling effect you see comes from our array of light-field displays. Since our testing prototypes are about 1x1cm, we tiled them up to generate a complete image
- #53 For the user evaluations, the idea is to test the method, not the individual. So we prepared 5 holograms from 1 to 5 diopters to be seen by testersWe asked individuals to look at them, one at a time, and vote for the best view,
- #54 with a 5D lens, which creates myopia
- #55 Or without the lens, in which the subject is supposed to be able to sharply focus on holograms from 1D to 3D (meaning from 1meter to 33cm). The idea is that, if the displayed hologram is inside their focal range, they would prefer seeing it without the lens.
- #56 And for the 5 diopter hologram, the subject is supposed to choose the lens all the time.
- #57 And the experiment confirmed the expectations we hadAs you can see, from 1 to 3 diopters, the volunteers preffered no lens. For 4 diopters we have a tie. And for 5 diopters, our testers choose the lens 98% of the time.
- #58 A similar test for astigmatism indicated that 66% of the testers choose a 4D astigmatic lens when the hologram was corrected for 4D of astigmatism.
- #59 For high-order aberrations we asked a individual to vote whether he could see the multiple copies effect in a corrected and non-corrected set of holograms.
- #60 And 92% of the corrected holograms he could not see the multiple copies.
- #61 We also made a simple test with a Multi-focus, Multi-depth display in which The three first letters of SIGGRAPH are placed in front of the display and the five least letters behind it. And for each depth we have a multi-depths to compensate for a miopic individual. So the viewer will be able to see only part of the word according to his accommodation state.
- #62 We have 3 main limitations: The eyes of the viewer must be fixed relative to the display, similar to glasses-free 3D displays. However, real-time eyetracking systems can eliminate this limitation. We need high-resolution LCD panels. But we hope the display industry follows the path of the camera industry, and start using a exceeding resolution to exploit refocusing features.And other ocular diseases may influence the results.
- #63 So, in a summany,We have the first multi-focus, multi-depth display which generates images with the resolution around 1 arc minute. We have shown evaluations with cameras and users for myopia, hyperopia, presbyopia, astigmatism, keratoconus and cataracts.,
- #64 For the future works we can easily see: Retinal diseases being integrated into the systemBetter light-field to hardware mappings exploring optimizations algorithms. And the evolution to Stereo Multi-focus, Multi-depths Displays.,
- #65 Thanks,
- #67 So joining low-cost techniques to measure refractive error.
- #68 And cataracts
- #69 We can think on a new trend in which consumers would be able to measure and get corrected instantly at their homes.
- #71 Before I finish, I would like to leave you with a thought to think about. The standard human acuity of an emmetrope - the person that does not need glasses - is about 1 arc-minute. We know that the theoretical resolution of our retina is about 0.3 arc-minute, which means that the average human being is under corrected. And his visual experience could be 3 times superior than what he sees right now. That’s like moving from a you tube video to a blue ray. So, who will be the first to make us free?
- #72 Since we have a dual stack of LCDs, we have to deal with Cross-talk.
- #73 And the way to do that is to keep the front-lcd apertures at a certain distance form each other. In our prototypes, they are about 0.3mm apart. Pretty much a 90 DPI setup.
- #74 Using a regular grid of apertures, you will find that you can actually have multiple points of view, because it does not really matter if the aberration designed for one aperture is going through another.
- #75 Two thirds of the world’s population, 4 billion people, have refractive errors. 2 billion are corrected and 2 billion have no idea they need glasses. Majority of the uncorrected cases are in the Emerging Asia mostly due to the lack of eye care but Also to an increasing prevalence of myopia.
- #76 Which Today reaches about 90% of the high-school dropouts in major Asian cities This increase of prevalence of the past years has been associated with education, reading, gaming, computers and TVs, an scope we pretty much own these days, specially with the new ebook readers. So, can we help mitigate visual problems by transforming the way we display information and accepting this kind of variability among the population?
- #77 In Computer Graphics, we try to understand how perception works to come up with nice graphics, overcome the computing limitations and extend the human abilities to play with the computer. Although we usually design for an average individual, it is clear that we all have distinct abilities to process the sensing data we capture. Can we build a common framework to deal with this intrinsic variability, at least in the perception space, that allows our techniques behave better even for outliers, say un uncorrected myope?
- #78 Although wearing eye glasses appear to be an easy solution, they are annoying in many ways. And we can find several reasons just googling. They require periodic visits to a doctor, if you have access to one in your country. The good ones are expensive, having the price of a mobile phone these days. They change your visual, your style.
- #79 Although wearing eye glasses appear to be an easy solution, they are annoying in many ways, Specially if you have to put them on and off all the time, such as when you need them just to check you phone. OR when you are running and you don’ t want to carry your glasses but you still want to look to your newst gadget. Good eyeglasses are expensive, having the price of a mobile phone these days. The beauty of tailoring is that it avoids extra carry-ons for certain activities. And since all the intelligence is in the software, you can share the same hardware with many people and all of them will be corrected for their problems.
- #80 For the user evaluations, the idea is to test the method, not the individual. So we prepare 5 holograms from 1 to 5 diopters. We asked individuals to look at them, one at a time and vote for the best view,
- #81 with a 5D lens, which creates myopia
- #82 Or without the lens that only holograms from 1D to 3D (meaning from 1meter to 33cm) are sharp to the tester. The idea was that, if the displayed hologram is inside their focal range, they would prefer seeing it without the lens because they can focus on any of them.
- #83 Two thirds of the world’s population, 4 billion people, have refractive errors.
- #84 2 billion are corrected and 2 billion have no idea they need glasses. So, can we help mitigate visual problems by transforming the way we display information Accepting this variability among the population making our techniques behave even for outliers, say un uncorrected myope?