Interactive Measurements and  Tailored Displays for OpticalAberrations of the Human Eye        Vitor Pamplona   Advisor: M...
Thesis Contributions                           TailoredNETRA          CATRA                           Displays            ...
Cell Phone-basedRefractive Measurements        The Inverse of       Shack-Hartmannwith: Manuel Oliveira, Ankit Mohan, Rame...
CATRA: Quantitative                                               Cataract Maps                                           ...
5with: Manuel Oliveira, Daniel Aliaga, Ramesh Raskar
Thesis Contributions                           TailoredNETRA          CATRA                           Displays            ...
NETRA: Measuring Refractive  Errors and Focal Range
Thermometer for visionVitor F. Pamplona   Ankit Mohan   Manuel M. Oliveira   Ramesh Raskar
Millions have poor vision, but are not getting corrected…                                                                 ...
2.4 Billion People w/out Glasses                   who need them around the world                                         ...
Shack-Hartmann Wavefront Sensor   Wavefront aberrometerExpensive; Bulky, Requires trained professionals                   ...
Shack-Hartmann Wavefront SensorSpot Diagram   Laser                              Planar         Sensor    Microlens Wavefr...
Shack-Hartmann Wavefront Sensor    Spot Diagram   Laser              SensorDisplacement =   Local Slopeof the Wavefront   ...
NETRA = Inverse of Shack-HartmannSpot Diagram on LCD              Cell      Eye             Phone     Piece             Di...
Inverse of Shack-Hartmann       User interactively creates the Spot DiagramSpot Diagram on LCD                            ...
Inverse of Shack-Hartmann       User interactively creates the Spot DiagramSpot Diagram on LCD                            ...
Relaxed Eye with Myopia                                                  Eye  Point                                       ...
Relaxed Eye with Myopia                                                    Eye                                         Pin...
Relaxed Eye with Myopia                                              Eye                                    Display       ...
Relaxed Eye with Myopia                                               Eye                                     Display Move...
Relaxed Eye with Myopia                                               Eye                                     Display Move...
Relaxed Eye with Myopia                                                   Eye                                        Displ...
Relaxed Eye with Myopia                                                    Eye                                            ...
Relaxed Eye with Myopia                                                           Eye                                     ...
Interactive Method    Farthest Focal Point      (myopia, hyperopia)   26
Interactive Method    Farthest Focal Point      (myopia, hyperopia)   27
Interactive Method    Farthest Focal Point      (myopia, hyperopia)   28
Astigmatism: angle-dependent refractive error                    http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html
Astigmatism: angle-dependent refractive error                    http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html
Astigmatism: angle-dependent refractive error                    http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html
AstigmatismCross or points may never meet with a 1d search !                                               32
AstigmatismLines reduce the problem to a 1d search   33
Interactive Method      Farthest Focal Point   (myopia, hyperopia, astigmatism)   34
Interactive Method      Farthest Focal Point   (myopia, hyperopia, astigmatism)   35
Interactive Method      Farthest Focal Point   (myopia, hyperopia, astigmatism)   36
Interactive Method      Farthest Focal Point   (myopia, hyperopia, astigmatism)   37
Best fitting on a Astigmatic Curve                          2      P(            C sin (           ) S   Unknowns:   Cylin...
Interactive Method      Farthest Focal Point   (myopia, hyperopia, astigmatism)   39
Measuring Accommodation Range                       Perfect vision                                Myopia              Hype...
Measuring Accommodation Range                       Perfect vision                                Myopia              Hype...
Measuring Accommodation Range                       Perfect vision                                Myopia              Hype...
Relaxed Eye                   Display                    AVirtual Point at                       Points the far limit     ...
Accommodated Eye                      DisplayMove points towards    each other                       A                    ...
Accommodated Eye                      DisplayMove points towards    each other                       A                    ...
Accommodated Eye                      DisplayMove points towards    each other                       A                    ...
Patterns for Alignment Task               A    B    A    B       A    B          A   B   A   B DisplayedSubject view      ...
Patterns for Alignment Task               A    B    A    B       A    B          A   B   A   B DisplayedSubject view      ...
Patterns for Alignment Task               A    B    A    B       A    B          A   B   A   B DisplayedSubject view      ...
Summary of Interaction                                   Accommodation Range              Farthest Point                  ...
Device Resolution                Channel Size                   25umResolution is a function of the display DPI   Samsung ...
Limitations• Children• Ability to align lines• Resolution is a function of the display DPI   – Samsung Behold II – 160 DPI...
Evaluation PrototypeTrial lenses simulate          Camera simulates   lens aberration              the perfect eye        ...
ACM SIGGRAPH 2010                    57
US + International Patent                            58
Chad FowlerFounder and CEO - YouTube   60
Early Awards + Recognition                   NETRA: $50K Innovation Grant                                                 ...
62
NETRA Prototypes Worldwide       29 partners in 14 countries.                                      63
Mumbai Slum Outreach: Lotus   Mumbai Optical Shop        Hyderabad Eye Clinic: LV Prasad Eye    Eye Hospital in Dharavi   ...
13 adults – 0.3D Average Difference                              5.00                              4.00                   ...
Frontiers In Optics & American Academy of Optometry                                      2010            66
mHealth Summit 2010                      67
NETRA Team at LVPEI, India                          29 adults – 0.5D Average Difference                                   ...
Association for Research on Vision and Ophthalmology                                                       69
OneSight, Kenya                  70
NETRA Team at Conceição, Brazil                    29 adults – 0.64D Average Difference            -4.00   -3.00    -2.00 ...
Pan-American and Brazilian Ophthalmology Congress                                                    72
Association for Research on Vision and Ophthalmology                                                       73
NETRA team at NECO             11 adults – 0.34D Average Difference         from Subjective Evaluation with no cycloplegia...
American Optometry Academy 2012   2012                           75
$300K Vodafone Award                       78
79
Ron GaranISS Astronaut   80
81
83  83
http://eyenetra.comPicture: Anderson Maciel.
Thesis Contributions                           TailoredNETRA          CATRA                           Displays            ...
CATRA: Interactive Measuring                        and Modeling of Cataracts                Vitor F. Pamplona Erick B. Pa...
CATRA: Quantitative Maps forSelf-assessment of Early Cataracts                                     89
Main Cause of Preventable Blindness                                      90
Slit-Lamp Microscope              92
93
CATRA        94
Four Resulting Maps                   Occlusion                                        Scattering Opacity Map             ...
Four Stages of Interaction                   Occlusion                                        Scattering Opacity Map      ...
Forward Scattering Sensed on Fovea                                Testing                               Sections   Project...
Forward Scattering Sensed on Fovea                                Testing                               Sections   Project...
Testing Sections of the Eye Lens  LCD1   LCD2   Lens                                   102
Trading Resolution for Brightness                        Testing                        Section   LCD1   LCD2   Lens      ...
Cataract Screening  Presence of    Cataracts(Binary Answer)                                       108
Detecting the Presence of CataractsMoving patterns   on LCD1     LCD1    LCD2                                      109
Detecting the Presence of Cataracts LCD1        LCD2                                      110
Detecting the Presence of Cataracts LCD1        LCD2    Eye lens                                      111
Detecting the Presence of Cataracts LCD1        LCD2    Eye lens   Perceived Image                                        ...
Cataract Screening  Presence of    Cataracts(Binary Answer)                                       113
Interactive Techniques and Maps  Presence of    Cataracts(Binary Answer)                   Position, Size                 ...
Estimating an Opacity MapMoving patterns   on LCD1  LCD1     LCD2                              115
Interactive Techniques and Maps  Presence of    Cataracts(Binary Answer)                   Position, Size                 ...
Interactive Techniques and Maps  Presence of                                      Brightness Test    Cataracts            ...
Estimating Attenuation Maps  Decreasing   Brightnesson the clear path     LCD1      LCD2                               118
Estimating Attenuation Maps  Decreasing   Brightnesson the clear path                           Same Perceived            ...
Estimating Attenuation MapsDecreasing Intensity                To Match Brightness of the Clear Path      LCD1            ...
Estimating Attenuation MapsDecreasing Intensity                To Match Brightness of the Clear Path      LCD1            ...
Interactive Techniques and Maps  Presence of                                      Brightness Test    Cataracts            ...
Interactive Techniques and Maps  Presence of                                      Brightness Test    Cataracts            ...
Contrast Test        Increasing ContrastLCD1          LCD2            Eye   Perceived Image       Rotated Low Contrast    ...
Contrast Test        Increasing ContrastLCD1          LCD2            Eye   Perceived Image       Rotated Low Contrast    ...
Contrast Test        Increasing ContrastLCD1          LCD2            Eye   Perceived Image       Rotated Low Contrast    ...
Contrast Test        Increasing ContrastLCD1          LCD2            Eye   Perceived Image       Rotated Low Contrast    ...
Interactive Techniques and Maps  Presence of                                      Brightness Test    Cataracts            ...
Interactive Techniques and Maps  Presence of                                                            Sub-aperture      ...
Point Spread Function Matching LCD1      LCD2    Eye   Perceived Image                                      132
Point Spread Function Matching LCD1      LCD2    Eye   Perceived Image                                      133
Point Spread Function Matching LCD1      LCD2    Eye   Perceived Image                                      134
Point Spread Function Matching LCD1          LCD2             Eye   Perceived Image            Sub-aperture        Point S...
Interactive Techniques and Maps  Presence of                                                            Sub-aperture      ...
Reducing Search Space for PSF  Presence of                                                            Sub-aperture        ...
140
Dual Monitor Stack                     141
Dual Monitor Stack                     142
Dual Monitor Stack                         Lens  MedicalMonochrome Monitors LCD Stack                                  143
DLP-Mask Prototype                           High Contrast                           DLP Projector                        ...
Cell Phone Prototype   LensPinhole Mask                            Stephen Wolfram                         CEO – Wolfram R...
Validation using Cameras  Thin Diffuser toSimulate Cataracts                        Estimated                     Attenuat...
Validation using Cameras                      Estimated     Measured        Estimated                     Opacity Map   At...
Scratching Contact LensesAdvanced        Mild             EarlyCataracts     Cataracts        Cataracts                   ...
Elderly Evaluation              18 volunteers              - 6 with early cataracts              - 12 with no cataracts   ...
Limitations• Active user participation• One clear light path• Retinal diseases                              Patience and c...
ACM SIGGRAPH 2011                    158
US + International Patent                            159
LVPEI CATRA Early Tests.                           160
Lotus University Trial                         161
Lotus University Trial                         162
Mexico VOSH/Rotary Trip.            0.00    5.00   10.00   15.00    20.00       25.00                                     ...
mHealth Summit 2011                      164
Awards: MIT Global Challenge & MIT Ideas CompetitionsEyeCatra: $5K Winner AwardMIT Ideas Competition 2011.EyeCatra: $5K Pu...
166
167
eyecatra.com                                             CATRA: Quantitative                                              ...
Thesis Contributions                           TailoredNETRA          CATRA                           Displays            ...
171with: Manuel M. Oliveira, Daniel Aliaga, Ramesh Raskar
172
173
174
175
176
177
Computer Generated GlassesFocusing Here            Focusing Here                Perfect vision Myopia                Focal...
Computer Generated Glasses                          Focusing Here          Focusing Here         Perfect vision Hyperopia ...
Tailoring is the easiest way   to create a hologram
Because uncorrected individuals are already focusing where you want.
Tailoring Process                             Myopic View: -3DFocusing Here           He can focus up to 33cm (12in)Distan...
Tailoring Process                                           Myopic View: -3DLight-field   Focusing Here           He can f...
Tailoring Process                                           Myopic View: -3DLight-field   Focusing Here           He can f...
Tailoring Process                                           Myopic View: -3DLight-field   Focusing Here           He can f...
Working Resolution: 1800 DPI                     Channel Size                       4.7um!$150 Vuzix HMD LCD              ...
Astigmatism Correction                         187
Tailoring for Astigmatism                                             Subject’s prescription                              ...
Single-Focus Multi-Depth Displays       For a given depth in focus           (accommodation),   a single object may be spl...
Wavefront Maps                    90 degrees                                             Sphere:           -2D            ...
InputsTailoring Process        f(k)           Light-field             Display           LCD1 LCD2 LCD1                    ...
Single-Focus Multi-Depth Displays                           Eye                                    197
Builds the Image Pixel by Pixel          Display                             Eye          A          B      d             ...
Scaling Pixels by Depth                DisplayScale                                 Eye                A                B ...
Mapping Light-Field Display <-> Retina                               f(k)            Display                              ...
201
Cataract Density Maps                   kNuclear Cataract       Cataract density in k   Sub-capsular Cataract             ...
Avoiding Cataracts   Display                     Eye                           203
Normalization Step   Display                     Eye                           204
Projector                     Camera – the “eye”Dual Stack of LCDs                        205
Lens ArrayVuzix Head Mounted       Display                     Vuzix Head Mounted                            Display      ...
3.25D Myopic                            1mm     Eye                                  Input Image     As Seen on a Regular ...
3.5D Myopic                                      CataractEye with Nuclear    Cataracts                                    ...
1mm1D Astigmatic  Eye at 180                         No Correction    Corrected                                           ...
210
211
212
213
214
215
216
217
218
Users Prefer the Tailored Image                 100                                            100%                       ...
Users Prefer the Tailored Image100                                              46                                        ...
Users Prefer the Tailored Image                                                                                           ...
Multi-Focus Multi-Depth Display                    Input Image   Input Depth+0.5D from the Image Plane          -0.5D from...
ACM SIGGRAPH 2012                    223
INPI Patent              224
225
Tailored Display Limitations• Eyes fixed relative to the display  – Similar to 3D Displays  – Depends on the eye aberratio...
Need       227
Thesis Conclusions• NETRA: Optics and UI for Refraction   – The Inverse of Shack-Hartmann Aberrometer   – Myopia, Hyperopi...
Defesa de Tese
Defesa de Tese
Upcoming SlideShare
Loading in …5
×

Defesa de Tese

1,404 views

Published on

Published in: Technology, Health & Medicine
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,404
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
55
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Everybody knows this machine, right? They call it thermometer. For me, this is the most amazing device medicine has ever used. And you know, nobody teaches on how to use a thermometer. We just somehow learned when we were kids. We started using and seeing that after the red mark something bad is going on and we need to see a doctor. It’s cheap, simple to use, no language barriers, no versions for rich and poor, it has a global spread and provides the first screen for a lot of diseases. I would guess that this guy has saved more life than anything else by just telling people when they should see a doctor.
  • Thanks, xxx
  • Everybody knows this machine, right? They call it thermometer. For me, this is the most amazing device medicine has ever used. And you know, nobody teaches on how to use a thermometer. We just somehow learned when we were kids. We started using and seeing that after the red mark something bad is going on and we need to see a doctor. It’s cheap, simple to use, no language barriers, no versions for rich and poor, it has a global spread and provides the first screen for a lot of diseases. I would guess that this guy has saved more life than anything else by just telling people when they should see a doctor.
  • It is a clip-on for phones that you put it on, run the app, look close, align those red and green lines and the result is your refractive error, or cataract condition. You can test your eyesight how many times you want, by yourself, anywhere, with the same accuracy an optometric tool would do today.
  • Now Imagine if you have a 2 dollar plastic device that you put close to your eye, press a button that says compute, and it gives you numbers that represent your nearsightedness, farsightedness and astigmatism, presbyopia and cataracts. Sort of a thermometer for vision. In the sense that since you have your number you know when you need to see a doctor. Well, what we have been working on is called NETRA, and it does exactly what I just said
  • 2 billion people have refractive errorsAnd half a billion in developing countries worldwide have uncorrected vision that affects their daily livelihood. They don’t have access to an optometrist or it simply too expensive. While making and distributing of lenses has become quite easy now, surprisingly there isstill no easy solution for measuring eyesight.Can we use a fraction of the 4.5B cellphone displays to address this problem?
  • Why? Lack of felt need for eye careWhile eyeglass manufacturers/retailers can produce $0.25 eyeglasses, they have minimal cost effective, quality, or remote eye diagnostics to build demand + efficiently service those in need.  Current eye testing tools are expensive, bulky, require significant training, and don’t allow for data digitization nor remote linkages to products/services.
  • The most accurate method is based on a so called SH WS. It involves shining a laser at the back of the retina and observing the wavefront using a sophisticated sensor.We ask user to generate a spot diagram. But navigating in a high dimensional space ischallenging so we come up with a strikingly simple approach to let the user interactively create the spotdiagram.We are first to make connection between Shack Hartmann and Lightfields (and it goes well with recentwork in computational photography about ALF and Zhang/Levoy). Connection to Adaptive optics/Astronomy. The way that this device works is that, it shines a lasers in the eye, the laser is reflected in the retina and comes out of the eye being distorted by the cornea. These light rays reaches an array of lenses that focus them to dots in a sensor. The device measures how much this dots deviate from the ideal case. Since it uses lasers, the device is expensive and requires trained professionals
  • For a normal eye, the light coming out of the eye forms a parallel wavefront. The sensor has a lenslet array and we get a spot diagram of uniform dots.This lenslet should remind you of a lightfield camera, and in fact Levoy and others showed last year that there is a close relationship between the two.In addition, Zhang and Levoy, plus our grp has shown the relationship between wavefront sensing and lightfield sensing.
  • When the eye has a distortion, the spot diagram is not uniform.And the displacement of the spots from the center indicates the local slope of the wavefront. From the slope one can integrate and recover the wave shape.
  • NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user.And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
  • For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently.But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
  • For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently.But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
  • We need to measure the difference between the subject’s farthest focal point wrt infinity.
  • So, lets start with an eye with myopia. Remember, they cannot see far, so a red point at infinity for them will look like a red blur.
  • Using Shceiner’s principle, if we put two pinholes in the field, this will instead create two distinct dots.
  • Instead of a distant point source, we put an LCD display behind the pinholes. If we draw two spots exactly under these pin-holes, we create a virtual point at infinity.
  • So, as we move the two red circles toward each other, the virtual point gets closer to the subject and he sees the two red dots getting closer.
  • When this two red circles overlaps for the subject, we can compute d based on the spot displacements
  • Which is the distance between the eye and this virtual point.
  • Turns out that the inverse of D is the refractive power required for this person to see clearly objects at infinity. In other words, the lens that will shift the accommodation range of this subject back to the regular one.
  • Which is the distance between the eye and this virtual point.
  • And number of clicks required for alignment indicates the refractive error
  • In practice we display lines on the screen and the subject overlaps these lines by pressing the buttons of the cell phone or in the computer.
  • which is anangle-dependent refractive error. An astigmatic subject has two main focal lengths in perpendicular meridians. One …
  • Stronger and one weaker
  • Think of a cornea with the shape of an american football creating a cylindrical aberration with unknown focal length and axis.
  • As you can see in this video, the astigmatic lenses create a deviation on the path of the pattern, and they may never overlap, turning the alignment task into a 2D search for some angles.
  • However, if we drawn lines perpendicular to the measured angle, the alignment task is again an 1D search. The deviation still exists, but the pattern makes the task easier.
  • So, we do the alignment task for a few meridians
  • By showing oriented lines on the display.
  • In the end, we best fit the sinusoidal curve over the four measured values to estimate the astigmatic parameters.
  • The required correction is now a function of measured angle. In order to measure the farthest point for these guys, we need to evaluate Cylindrical component, the Spherical component, and the angle theta on the equation. However, the interpolation of refractive powers between C and S leads to a situation where the pattern drawn on the screen matters.
  • In the end, we best fit the sinusoidal curve over the four measured values to estimate the astigmatic parameters.
  • Ours is the only system where one can estimate not only the farthest point
  • one can focus but also
  • the nearest point without any mechanically moving parts. So, in order to measure the closest reading point
  • We draw a pattern on the screen that induces accommodation. In this way, when we move A and B closer on the screen,
  • the user will try to focus on a closer object. We can move this virtual point all the way to the nearest discernable point.
  • When the user is not able to focus anymore, the visual system give up and the user start seeing more than one pattern.
  • As I sad before, this is possible because we can draw whatever we want in the display. We tested many patterns, static and dynamic, including visual cryptography.
  • As I sad before, this is possible because we can draw whatever we want in the display. We tested many patterns, static and dynamic, including visual cryptography.
  • Turns out that the best pattern to induce accommodation is the sinosoidal curves aligned perpendicular to the measurement angle.
  • We have complete freedom for pattern G on display and the filter pattern h, which has been pin hole grid so far. But observe that subjects view is just a convolution of the pattern g and the filter h. So here is a very interesting effect. If we show this convoluted pattern with same filter, we get double convolution. If h is a broadband random dot pattern, the double convolution is a delta function, which means user will again see the pattern g.
  • We exploited this trick to build a viewmaster system. In this case, instead of moving lines closer we scale the pattern. The amount of scale give us the refractive power needed.
  • As a summary, our method has two steps. First measures the farthest point in focus in many angles using lines and the second step measures the nearest point using sinusoidals oriented on the angle of astigmatism.
  • Reading charts appear to be an easy solution, this method has too many problems. Sharpness of legible text is very subjective. The brightness of the chart has to be very carefully chosen otherwise the pupil size will change, increasing depth of field, and allowing user to recognize even lower rows.The trial lenses + the lens frame the doctor will use also cost over $150% Reading chart tests involve using a frame or a phoropter. The doctor will swing a sequence of lenses in front of your eye and ask for which lens allows you to see the lower rows on the reading chart.
  • Since we are relying on the user interaction, the subject has to be aware of the alignment tasks. So, very young Children may not be able to run the test. Instead of just one eye, one may use both eyes to exploit convergence. And of course, the resolution of NETRA itself is a function of the resolution of the display. With a 326 dpi display, resolution is 0.14 diopters and presciption glasses come in increments of 0.25 diopters. So our system is already sufficiently accurate.
  • Using a minification system, we performed user study with a high resolution display. Using a a camera to simulate perfect eye and a trial set of lenses to simulate lens aberration, the average spherical error was under 0.09 diopter and astigmatism axis error of 8 degrees.
  • We started winning awards at MIT, including MIT Ideas Award. Almost at the same time, we went to the finals of MIT 100K, business plan competition, by proposing a model to take the eye care center home. And subsequently we won a Google Grant and a Deshpande Grant.Desphande Center grant process was rigorous and competitive, forcing the early team to clearly communicate and convince technology and business leaders of applicability of the device in a real world, scaleable business. Google grant essentially came after Larry Page asked Ramesh if we could do our testing on Android – of course we can, and Larry was excited! (true story)
  • NETRA-G has been tried out by people from around India and the world, ranging from optical shops to hospitals.
  • We validate this extension by measuring the closest sharp point in cameras, and comparing with physical measurements.
  • The second round of validation included 6 humans. Both cases we could get pretty close to the actual closest sharp point.
  • We were accepted into the Launch program, an incubation program sponsored by NASA, USAID, and Nike to honor the top 5 health innovations in America.
  • Morgethaler Ventures, a prominent bay area VC firm held a yearly competition looking for the best Health IT companies in the world. Through hundreds of applicants, EyeNetra was chosen to be the most promising Health IT start-up in the world.
  • USA = $8 market eyeglass marketPoC diagnostics: $18.7 billion market by 2014Homes: Device Sales (to consumer via best buy)Royalty on eyeglasses soldPharmacies:Royalty on eyeglasses sold at the storesOpthalmologists/Optometrists:Device sales and pay per use -- Lower cost autoref
  • Thanks XXXNETRA is a clip-on device that you attach to your cell phone. You look close, press some buttons, you hit calculate and it gives you the prescription for glasses. It’s a 2-dollar device that measures nearsightedness, farsightedness and astigmatism with the same accuracy that doctors have in their clinic.To understand what happened here, let’s think about the evolution of photography.
  • Everybody knows this machine, right? They call it thermometer. For me, this is the most amazing device medicine has ever used. And you know, nobody teaches on how to use a thermometer. We just somehow learned when we were kids. We started using and seeing that after the red mark something bad is going on and we need to see a doctor. It’s cheap, simple to use, no language barriers, no versions for rich and poor, it has a global spread and provides the first screen for a lot of diseases. I would guess that this guy has saved more life than anything else by just telling people when they should see a doctor.
  • CATRA is a Snap-On eyepiece for mobile phones that measures and quantifies cataracts in the human eye.The patient looks tru it, respond to few patterns that are drawn on the screen, and the app generates, for the first time, maps to show occluders and their scatering profile.
  • We’ve been working with NGO’s on NETRA, and they reminded us that although refractive error is the second leading cause of preventable blindness, cataract is the first one. We ended up realizing that we actually were targeting one of the most prevalent diseases on this planet. In fact, allof us will have cataracts if we live long enough.
  • Well, cataracts are these clouds you may see in someone&apos;s eye which reflect and scatter light as it goes through on of these white blobs.For the subject&apos;s view, they create glare and blurriness.
  • Cataracts are detected, measured and diagnosed through this device.It is called slit-lamp microscope and is essentially a searching platform for doctors. Clinicians will change the several degrees of freedom this device has to manually search for cataracts in one&apos;s eye.This device has really not changed for several decades…
  • Conceptually speaking, this device is very simple. It shines a slit of light into the eye, which gets reflected in the cataract and goes back to the viewer.Clinician will se a white blob and will subjectively rate from 1 to 4 according to his notion of severity. 3 and 4 are advanced cases of the disease and suggest surgery. As you may see on this image, this method works on what we call back scattering analysis. Clinician relies on the reflex of the scattering spot which may not represent the actual effects cataracts are creating on the subject&apos;s view.
  • Instead of relying on someone else’s judgement of severity, CATRA works with forward scattering analysis.This snap-on can be seen as a light-field display, which when placed up close, scans the lens of the eye section by section. So, by relying on the ability to show the scattering profile of a section of the lens, we built interactive techniques to transform the visual information the subject is seen into quantifiable data.
  • We propose 4 maps to model occluders and replace the currently used subjective evaluation, that one from 1 to 4. The first map is what we call an opacity map. It consists of a binary information (has or has not) cataracts per section of the lens. It tell us position, size and shape of the occluders.The attenuation map is a density test per section of the eye. It tell us, how reflexive and transmissive an occluded section is. The third map is what we call contrast map. A contrast test is made per section of the eye, and tell us how big is the spreading of light from each section. The fourth map holds the point spread function per section of the eye. This four maps are divided into occlusion and scattering analisis
  • Based on these maps, we can simulate an individual cataract-affected vision and the progress of the disease
  • Based on these maps, we can simulate an individual cataract-affected vision and the progress of the disease
  • Based on these maps, we can simulate an individual cataract-affected vision and the progress of the disease
  • Now, notice that these maps are measuring a region that has about 3mm in diameter. Each section has only 6mmAnd thus any small variation on the position of the device, face or gazing will make the software miss a cataract spot.
  • We though a lot about it, and after several iterations, we came up with a design that relies on forward scattering and always projects patterns on fovea, so the subject will not gaze Our design is essentially a modified 3D display. We have two LCDs and a light box behind them, LCDs work as programmable masks, and an additional lens, placed one focal length from the parallax barrier. This setup allows us, for instance, to open a pinhole in LCD2 and 3 pixels on LCD1, and light rays coming out of the device will pass trough 3 testing regions on the crystallin lens and converge to a single point in the fovea. With this setup, we can alternate among testing sections without breaking the subject&apos;s visual point of reference. It does not matter which position of the lens we are testing, the subject will always see a green steady dot.
  • The intuition of this design relies on the role of each LCD. Each pixel on LCD1 represents a position on the crystalline, or a testing site. Each pixel on LCD2 is mapped to a position onto de fovea. So, if we want to draw visual stimulus, we draw it on LCD2, if we want to test different positions, we change LCD1.
  • Here is an example on a subject with cataracts. With our setup it is possible to shine a light ray that hits the cataract spot. Cataract will reflect and scatter light and a small amount of the scattered energy will reach the retina.
  • If the scattering is too big or the cataract is too reflexive, we can trade resolution for brightness and open neighbor pixels on LCD1 to create collimated beams of light, increasing the testing site and also the amount of energy been focused into a single point in the retina. This tool allows us to play with the scattering element and identify its properties without changing the users point of reference.
  • Well, the eye as any other imaging device has a point spread function. If the eye does not have cataracts, it PSF is a peak.
  • In case of mild cataracts, this peak decreases and the PSF assumes a gaussian profile.
  • In advanced cases of the disease, you cannot even see a peak al all. At this stage, the subject will not be able to notice objects in front of him.
  • In this work, we want to map what is on the aperture of the eye, in order to estimate the point spread function and thus compute a visual representation for an individual cataract affected eye. In order to do so, we have to figure out values for sigma and the peak of the point spread function. This values are estimated through 5 interactive techniques that run in sequence.
  • The first technique is a binary test for the presence of cataracts. So, yes or no if the subject has cataracts.
  • Although we’ve shown the single-LCD mobile phone based solution, we’re now going to move on to the general design. In our optical design, we open a pinhole in the center of LCD2 and we keep moving a pixel on LCD1. When this scanning procedure hits a cataract spot, the dot disappears and the subject realizes he has something blocking his view.
  • In a 2D example, we have a moving dot on screen, a pinhole open on LCD2, this will scan the lens of the eye and the subject will notice a difference between occluded and clear paths.
  • In a 2D example, we have a moving dot on screen, a pinhole open on LCD2, this will scan the lens of the eye and the subject will notice a difference between occluded and clear paths.
  • In a 2D example, we have a moving dot on screen, a pinhole open on LCD2, this will scan the lens of the eye and the subject will notice a difference between occluded and clear paths.
  • Let&apos;s say the subject has cataracts and thus we move forward to our maps.
  • The first one is the opacity map. Tell us the position, size and density of the cataract. GROUP AGAIN
  • On the optical scheme, it is exact the same procedure as before, a pinhole on LCD2 and a moving pattern on LCD1. However, the pattern on LCD1 moves slower and when the scanning process hit a cataract, the dot fades away and subject presses a button. By marking all regions the dot faded, app computes an opacity map.
  • Since now we know where cataracts are, we can now compute what are their densities.
  • Since now we know where cataracts are, we can now compute what are their densities.This will tell us how reflexive they are.
  • Now we alternate pixels on LCD1 in such a way that one point will hit a cataract spot and the second is a clear path. Subject see both alternating on his view and will decrease the intensity of the clear path, by pressing buttons on the phone, in order to match the occluded one.
  • In essence, subject will decrease brightness up to the point he does not notice any difference between patterns.
  • Again on 2D, we have alternating patterns on LCD1, still a pinhole on LCD2 and the subject will change the brightness of the clear path in order to math the intensity of the occluded path on his retina. By executing this procedure for all sections, we built an attenuation map.
  • Again on 2D, we have alternating patterns on LCD1, still a pinhole on LCD2 and the subject will change the brightness of the clear path in order to math the intensity of the occluded path on his retina. By executing this procedure for all sections, we built an attenuation map.
  • Each value on this map is an estimation for the peak of the gaussian PSF function.
  • Each value on this map is an estimation for the peak of the gaussian PSF function.
  • The fourth map is what we call a contrast map.We will conduct contrasts sensitivity tests per section of the eye. We will show a low contrast letter C, which may be rotated, and the subject will answer where C is pointing to when he notice it.
  • C is drawn on LCD2 and a pixel is opened on LCD1 which will make C go thought the cataract spot. Subject increases the contrast of C up to the point he notice where C is pointing to, in this case, he presses the right key.
  • Subject increases the contrast of C up to the point he notice where C is pointing to, in this case, he presses the right key.
  • Subject increases the contrast of C up to the point he notice where C is pointing to, in this case, he presses the right key.
  • Subject increases the contrast of C up to the point he notice where C is pointing to, in this case, he presses the right key.
  • The contrast map tell us sigma for the gaussian PSF function.
  • The contrast map tell us sigma for the gaussian PSF function.
  • The fifth test is computes a point spread function per section of the eye.
  • Just like the attenuation map, we have alternating points on LCD1, one for an occluded path and the other for a clear one. The pinhole on LCD2 for the clear path is changed for a gaussian,
  • which peak is read from the attenuation map. Subject will only increase sigma to match the point spread function that is been created by the occluded path. When he finishes, the drawing on LCD2 is the actual point spread function of the sub-aperture.
  • which peak is read from the attenuation map. Subject will only increase sigma to match the point spread function that is been created by the occluded path. When he finishes, the drawing on LCD2 is the actual point spread function of the sub-aperture.
  • which peak is read from the attenuation map. Subject will only increase sigma to match the point spread function that is been created by the occluded path. When he finishes, the drawing on LCD2 is the actual point spread function of the sub-aperture.
  • PSF maps also estimate sigma, but there is a difference between the sigma from a PSF map and from the contrast map.
  • PSF maps also estimate sigma, but there is a difference between the sigma from a PSF map and from the contrast map.
  • If respective attenuation value is high subject may not be able to match the point spread function accurately. So, the contrast map replaces the PSF matching.
  • And thus our algorithm follows these steps, one after the other, with a decision point after the brightness test. In the remaining of the talk, Erick will show our prototypes, validations and how to compute an individual cataract affected view.
  • ThanksVitor.We’ve built several prototypes…
  • This one is made of a pair of stacked LCDs. We disassembled and re-builtthese high-contrast monochromaticmedical displays.Interaction is made though a keyboard.
  • This one is made of a pair of stacked LCDs. We disassembled and re-builtthese high-contrast monochromaticmedical displays.Interaction is made though a keyboard.
  • This one is made of a pair of stacked LCDs. We disassembled and re-builtthese high-contrast monochromaticmedical displays.Interaction is made though a keyboard.
  • Thanks Vitor…Here is the first prototype. It is composed of a DLP projector with a diffuser, a pinhole mask, and an eyepiece where the subject should look into. Interaction is made though the keys of a laptop.
  • The simplest possible setup, you already saw it, a clip-on for mobile phones with a pinhole mask on top of the display. This mobile phone prototype can only generate opacity and attenuation maps. With the stacked LCDs one, we implemented the full interactive procedure.
  • We validated our method in 3 steps.Firstly we validated the technique itself, and how accurate our interactive method can be under a highly controlled environment.We added diffusers to camera lenses to simulate cataracts and computed, in this example, attenuation and point spread function maps.
  • Here is another example: a picture of the simulated cataracts, opacity map, and the measured attenuation map, which was created by taking per-section pictures and summing up pixels on the resulting image. The estimated attenuation map matches the actual value.
  • In the second step, we validated alignment and gaze control.So, if you’re a young graduate student without cataracts, how would you do the experimentation? I, for instance, don’t have cataracts…Guess what, we SCRATCHED these contact lenses to create simulations for advanced, mild and early cataracts, and we wore them, er… Vitor did, since I wasn’t brave enough.For instance, we could successfully measure the size of a simulated cataract of about 0.5mm^2 as 0.45mm^2.WHOLE FACE PICTURES INSTEAD
  • In the last step, we tested how elderly subjects interacted with our device. 18 people took the test: 5 of them had early cataracts, all confirmed through our method; and 1 individual discovered he had cataracts with our device, which was confirmed afterwards by an ophthalmologist.
  • Well, now that we know how to measure approximate point spread functions per sub-aperture, we can build a single point spread function for the eye by summing up all of them. However, because of cataracts, eye’s PSFs are depth dependent
  • , so for scenes with objects out of focus, there is a shift to be applied for each sub-aperture PSF that is proportional to the distance from the focal plane.
  • So, here is a scenario for our rendering.
  • A photography taken with a cataract-affected lens
  • And a simulation using our estimated PSFs.
  • In some out of focus objects, you may even see the cataract shape inside the bokeh effect.
  • Our technique has a few limitations.It requires active user participation, so if the user cannot understand our procedures, he may not get reliable results.We need a clear path in the lens in order to estimate the attenuation, contrast and PSF of occluded paths.Retinal diseases may change the results as well.
  • In summary, we have introduced a co-design of optics and interactive techniquesfor measuring cataracts which works with forward scattering analysis and holds gaze through foveal projection…We proposed 4 quantitative maps to replace the subjective evaluation doctors currently use...And we developed the first simulation for an individual cataract-affected vision.
  • Everybody knows this machine, right? They call it thermometer. For me, this is the most amazing device medicine has ever used. And you know, nobody teaches on how to use a thermometer. We just somehow learned when we were kids. We started using and seeing that after the red mark something bad is going on and we need to see a doctor. It’s cheap, simple to use, no language barriers, no versions for rich and poor, it has a global spread and provides the first screen for a lot of diseases. I would guess that this guy has saved more life than anything else by just telling people when they should see a doctor.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • Which is the distance between the eye and this virtual point.
  • E estesistemafecha a minhatesequedescreve o ciclo com 3 tecnologias de medi’c~ao e correcao. Nao e dificilimaginarque no futuro as pessoaspossammedirseussistemasvisuaisemqualquerlugar, postar num facebookdavida, e obter displays corrigidospara voce emtodososlugares. Entao 4 anos de doutorado e 300 mil reais de investimento, euesperancosamentereceberei um novo papel, quedizque agora emposso ser chamado de doutor.
  • Defesa de Tese

    1. 1. Interactive Measurements and Tailored Displays for OpticalAberrations of the Human Eye Vitor Pamplona Advisor: Manuel M. Oliveira Neto
    2. 2. Thesis Contributions TailoredNETRA CATRA Displays 2
    3. 3. Cell Phone-basedRefractive Measurements The Inverse of Shack-Hartmannwith: Manuel Oliveira, Ankit Mohan, Ramesh Raskar
    4. 4. CATRA: Quantitative Cataract Maps Unique, low-cost quantitative lens mappingwith: Erick Passos, Jan Zizka, Everett Lawson, Esteban Clua , Manuel M. Oliveira, Ramesh Raskar
    5. 5. 5with: Manuel Oliveira, Daniel Aliaga, Ramesh Raskar
    6. 6. Thesis Contributions TailoredNETRA CATRA Displays 6
    7. 7. NETRA: Measuring Refractive Errors and Focal Range
    8. 8. Thermometer for visionVitor F. Pamplona Ankit Mohan Manuel M. Oliveira Ramesh Raskar
    9. 9. Millions have poor vision, but are not getting corrected… Kenya 2B have refractive errors 0.6B have 5B have a URE cell phone India 7 Billion peopleSource: World Heath Organisation, Vision 2020 Tech Report. 9
    10. 10. 2.4 Billion People w/out Glasses who need them around the world Billions of People with Uncorrected Refractive Error, by 1.70 Region 1.80 1.6 Billion 2.4 Billion 1.60 1.40 1.20 1.00 0.80 0.60 0.50 0.40 0.20 0.13 0.10 0.02 0.00 Emerging Africa & Latin Europe North Asia Middle America America EastSource: Essilor, Infomarket 2009, CPB Research, numbers may not add due to rounding 10
    11. 11. Shack-Hartmann Wavefront Sensor Wavefront aberrometerExpensive; Bulky, Requires trained professionals 11
    12. 12. Shack-Hartmann Wavefront SensorSpot Diagram Laser Planar Sensor Microlens Wavefront Array 12
    13. 13. Shack-Hartmann Wavefront Sensor Spot Diagram Laser SensorDisplacement = Local Slopeof the Wavefront 13
    14. 14. NETRA = Inverse of Shack-HartmannSpot Diagram on LCD Cell Eye Phone Piece Display 14
    15. 15. Inverse of Shack-Hartmann User interactively creates the Spot DiagramSpot Diagram on LCD 15
    16. 16. Inverse of Shack-Hartmann User interactively creates the Spot DiagramSpot Diagram on LCD 16
    17. 17. Relaxed Eye with Myopia Eye Point Blurredat infinity point Focusing Range perfect vision myopia hyperopiainfinity ~10cm 18
    18. 18. Relaxed Eye with Myopia Eye Pinholes Point Distinctat infinity image points Focusing Range perfect vision myopia Scheiner’s Principle hyperopia infinity ~10cm 19
    19. 19. Relaxed Eye with Myopia Eye Display A Virtual point Distinct at infinity image B points Focusing Range perfect vision myopia hyperopiainfinity ~10cm 20
    20. 20. Relaxed Eye with Myopia Eye Display Move spotstowards each other A Distinct image Virtual point B pointsat finite distance Focusing Range perfect vision myopia hyperopia infinity ~10cm 21
    21. 21. Relaxed Eye with Myopia Eye Display Move spotstowards each other A Points overlap Virtual point Bat finite distance Focusing Range perfect vision myopia hyperopia infinity ~10cm 22
    22. 22. Relaxed Eye with Myopia Eye Display Move spotstowards each other A Points overlap Virtual point Bat finite distance d Focusing Range perfect vision myopia hyperopia infinity ~10cm 23
    23. 23. Relaxed Eye with Myopia Eye 1 dPoint at Pointsinfinity overlap d Focusing Range perfect vision myopia hyperopia infinity ~10cm 24
    24. 24. Relaxed Eye with Myopia Eye Display Move spotstowards each other c A a Points overlapVirtual red point Bat finite distance f t d Focusing Range perfect vision fa hyperopia myopia d = +t infinity ~10cm 2c 25
    25. 25. Interactive Method Farthest Focal Point (myopia, hyperopia) 26
    26. 26. Interactive Method Farthest Focal Point (myopia, hyperopia) 27
    27. 27. Interactive Method Farthest Focal Point (myopia, hyperopia) 28
    28. 28. Astigmatism: angle-dependent refractive error http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html
    29. 29. Astigmatism: angle-dependent refractive error http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html
    30. 30. Astigmatism: angle-dependent refractive error http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html
    31. 31. AstigmatismCross or points may never meet with a 1d search ! 32
    32. 32. AstigmatismLines reduce the problem to a 1d search 33
    33. 33. Interactive Method Farthest Focal Point (myopia, hyperopia, astigmatism) 34
    34. 34. Interactive Method Farthest Focal Point (myopia, hyperopia, astigmatism) 35
    35. 35. Interactive Method Farthest Focal Point (myopia, hyperopia, astigmatism) 36
    36. 36. Interactive Method Farthest Focal Point (myopia, hyperopia, astigmatism) 37
    37. 37. Best fitting on a Astigmatic Curve 2 P( C sin ( ) S Unknowns: Cylinder Axis Cyl. Sphere 38
    38. 38. Interactive Method Farthest Focal Point (myopia, hyperopia, astigmatism) 39
    39. 39. Measuring Accommodation Range Perfect vision Myopia Hyperopia Infinity ~10cm Step 1: Far limit Step 2: Near limit 40
    40. 40. Measuring Accommodation Range Perfect vision Myopia Hyperopia Infinity ~10cm Step 1: Far limit Step 2: Near limit 41
    41. 41. Measuring Accommodation Range Perfect vision Myopia Hyperopia Infinity ~10cm Step 1: Far limit Step 2: Near limit 42
    42. 42. Relaxed Eye Display AVirtual Point at Points the far limit B overlap 43
    43. 43. Accommodated Eye DisplayMove points towards each other A Points B overlap Virtual point getting closer Subject Accommodates to fix the “blur” 44
    44. 44. Accommodated Eye DisplayMove points towards each other A Points B overlap Virtual point getting closer Subject Accommodates to fix the “blur” 45
    45. 45. Accommodated Eye DisplayMove points towards each other A Points B overlap Virtual point getting closer Subject cannot accommodate more than the previous point 46
    46. 46. Patterns for Alignment Task A B A B A B A B A B DisplayedSubject view A B A B A B A B A B DisplayedSubject view Visual 47 Cryptography [NaorShamir94]
    47. 47. Patterns for Alignment Task A B A B A B A B A B DisplayedSubject view A B A B A B A B A B DisplayedSubject view Visual 48 Cryptography [NaorShamir94]
    48. 48. Patterns for Alignment Task A B A B A B A B A B DisplayedSubject view A B A B A B A B A B DisplayedSubject view Visual 49 Cryptography [NaorShamir94]
    49. 49. Summary of Interaction Accommodation Range Farthest Point Nearest Point(myopia, hyperopia, astigmatism) (presbyopia) 52
    50. 50. Device Resolution Channel Size 25umResolution is a function of the display DPI Samsung Behold II – 160 DPI = 0.35D Google Nexus One – 250 DPI = 0.20D Apple iPhone 4 – 326 DPI = 0.14D 53
    51. 51. Limitations• Children• Ability to align lines• Resolution is a function of the display DPI – Samsung Behold II – 160 DPI – 0.35D – Google Nexus One – 250 DPI – 0.2D – Apple iPhone 4G – 326 DPI – 0.14D 55
    52. 52. Evaluation PrototypeTrial lenses simulate Camera simulates lens aberration the perfect eye MinificationLCD Display 56
    53. 53. ACM SIGGRAPH 2010 57
    54. 54. US + International Patent 58
    55. 55. Chad FowlerFounder and CEO - YouTube 60
    56. 56. Early Awards + Recognition NETRA: $50K Innovation Grant 61 NETRA: $50K Research Grant
    57. 57. 62
    58. 58. NETRA Prototypes Worldwide 29 partners in 14 countries. 63
    59. 59. Mumbai Slum Outreach: Lotus Mumbai Optical Shop Hyderabad Eye Clinic: LV Prasad Eye Eye Hospital in Dharavi Institute Primary vision center Chennai Optical Shop School, Nairobi, KenyaHyderabad Eye Hospital: LV Prasad Eye Institute Patient’s home, Teresina, PI, Brazil Mumbai Optical Shop 64
    60. 60. 13 adults – 0.3D Average Difference 5.00 4.00 3.00 2.00 1.00Netra (D) 0.00 -5.00 -3.00 -1.00 1.00 3.00 5.00 -1.00 -2.00 -3.00 -4.00 -5.00 Subjective Refraction (D)NETRA vs Manifest Refraction 65
    61. 61. Frontiers In Optics & American Academy of Optometry 2010 66
    62. 62. mHealth Summit 2010 67
    63. 63. NETRA Team at LVPEI, India 29 adults – 0.5D Average Difference 7.00 5.00 3.00 1.00 Netra (D) -7.00 -1.00 -2.00 3.00 -3.00 -5.00 -7.00 Retinoscopy (D) NETRA vs Retinoscopy 68
    64. 64. Association for Research on Vision and Ophthalmology 69
    65. 65. OneSight, Kenya 70
    66. 66. NETRA Team at Conceição, Brazil 29 adults – 0.64D Average Difference -4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 3.00 2.00 1.00 0.00 NETRA -1.00 -2.00 -3.00 -4.00 -5.00 AR1 AR2 SR Reference NETRA vs Auto-Refractor 71
    67. 67. Pan-American and Brazilian Ophthalmology Congress 72
    68. 68. Association for Research on Vision and Ophthalmology 73
    69. 69. NETRA team at NECO 11 adults – 0.34D Average Difference from Subjective Evaluation with no cycloplegia -8.00 -7.00 -6.00 -5.00 -4.00 -3.00 -2.00 -1.00 0.00 1.00 1.00 0.00 -1.00 -2.00NETRA -3.00 -4.00 -5.00 -6.00 -7.00 -8.00 AR Subj Reference 71% of the measurements have a max error of 0.5D 74
    70. 70. American Optometry Academy 2012 2012 75
    71. 71. $300K Vodafone Award 78
    72. 72. 79
    73. 73. Ron GaranISS Astronaut 80
    74. 74. 81
    75. 75. 83 83
    76. 76. http://eyenetra.comPicture: Anderson Maciel.
    77. 77. Thesis Contributions TailoredNETRA CATRA Displays 87
    78. 78. CATRA: Interactive Measuring and Modeling of Cataracts Vitor F. Pamplona Erick B. Passos Jan Zizka Manuel M. Oliveira Everett Lawson Esteban Clua Ramesh RaskarMIT Media Lab – Camera Culture
    79. 79. CATRA: Quantitative Maps forSelf-assessment of Early Cataracts 89
    80. 80. Main Cause of Preventable Blindness 90
    81. 81. Slit-Lamp Microscope 92
    82. 82. 93
    83. 83. CATRA 94
    84. 84. Four Resulting Maps Occlusion Scattering Opacity Map Attenuation Map Contrast Map PSF Map(position, size) (brightness) (contrast) C C C C C C C C C C C C C C C C C C C C 95
    85. 85. Four Stages of Interaction Occlusion Scattering Opacity Map Attenuation Map Contrast Map PSF Map(position, size) (brightness) (contrast) C C C C C C C C C C C C C C C C C C C C 0.6mm 3mm 99
    86. 86. Forward Scattering Sensed on Fovea Testing Sections Projection on the FoveaLight Box LCD1 LCD2 Lens 100
    87. 87. Forward Scattering Sensed on Fovea Testing Sections Projection on the FoveaLight Box LCD1 LCD2 Lens 101
    88. 88. Testing Sections of the Eye Lens LCD1 LCD2 Lens 102
    89. 89. Trading Resolution for Brightness Testing Section LCD1 LCD2 Lens 103
    90. 90. Cataract Screening Presence of Cataracts(Binary Answer) 108
    91. 91. Detecting the Presence of CataractsMoving patterns on LCD1 LCD1 LCD2 109
    92. 92. Detecting the Presence of Cataracts LCD1 LCD2 110
    93. 93. Detecting the Presence of Cataracts LCD1 LCD2 Eye lens 111
    94. 94. Detecting the Presence of Cataracts LCD1 LCD2 Eye lens Perceived Image 112
    95. 95. Cataract Screening Presence of Cataracts(Binary Answer) 113
    96. 96. Interactive Techniques and Maps Presence of Cataracts(Binary Answer) Position, Size and Shape (Opacity Map) 114
    97. 97. Estimating an Opacity MapMoving patterns on LCD1 LCD1 LCD2 115
    98. 98. Interactive Techniques and Maps Presence of Cataracts(Binary Answer) Position, Size and Shape (Opacity Map) 116
    99. 99. Interactive Techniques and Maps Presence of Brightness Test Cataracts (Attenuation Map)(Binary Answer) Position, Size and Shape (Opacity Map) 117
    100. 100. Estimating Attenuation Maps Decreasing Brightnesson the clear path LCD1 LCD2 118
    101. 101. Estimating Attenuation Maps Decreasing Brightnesson the clear path Same Perceived Brightness LCD1 LCD2 119
    102. 102. Estimating Attenuation MapsDecreasing Intensity To Match Brightness of the Clear Path LCD1 LCD2 Eye Perceived Image 120
    103. 103. Estimating Attenuation MapsDecreasing Intensity To Match Brightness of the Clear Path LCD1 LCD2 Eye Perceived Image 121
    104. 104. Interactive Techniques and Maps Presence of Brightness Test Cataracts (Attenuation Map)(Binary Answer) Position, Size and Shape (Opacity Map) 122
    105. 105. Interactive Techniques and Maps Presence of Brightness Test Cataracts (Attenuation Map)(Binary Answer) Position, Size Sub-aperture and Shape Contrast Test (Opacity Map) (Contrast Map) C C C C C C C C C C C C C C C C C C C C 124
    106. 106. Contrast Test Increasing ContrastLCD1 LCD2 Eye Perceived Image Rotated Low Contrast Letter C 125
    107. 107. Contrast Test Increasing ContrastLCD1 LCD2 Eye Perceived Image Rotated Low Contrast Letter C 126
    108. 108. Contrast Test Increasing ContrastLCD1 LCD2 Eye Perceived Image Rotated Low Contrast Letter C 127
    109. 109. Contrast Test Increasing ContrastLCD1 LCD2 Eye Perceived Image Rotated Low Contrast Press the right key Letter C 128
    110. 110. Interactive Techniques and Maps Presence of Brightness Test Cataracts (Attenuation Map)(Binary Answer) Position, Size Sub-aperture and Shape Contrast Test (Opacity Map) (Contrast Map) C C C C C C C C C C C C C C C C C C C C 129
    111. 111. Interactive Techniques and Maps Presence of Sub-aperture Brightness Test Cataracts PSF Match (Attenuation Map)(Binary Answer) (PSF Map) Position, Size Sub-aperture and Shape Contrast Test (Opacity Map) (Contrast Map) C C C C C C C C C C C C C C C C C C C C 131
    112. 112. Point Spread Function Matching LCD1 LCD2 Eye Perceived Image 132
    113. 113. Point Spread Function Matching LCD1 LCD2 Eye Perceived Image 133
    114. 114. Point Spread Function Matching LCD1 LCD2 Eye Perceived Image 134
    115. 115. Point Spread Function Matching LCD1 LCD2 Eye Perceived Image Sub-aperture Point Spread Function 135
    116. 116. Interactive Techniques and Maps Presence of Sub-aperture Brightness Test Cataracts PSF Match (Attenuation Map)(Binary Answer) (PSF Map) Position, Size Sub-aperture and Shape Contrast Test (Opacity Map) (Contrast Map) C C C C C C C C C C C C C C C C C C C C 136
    117. 117. Reducing Search Space for PSF Presence of Sub-aperture Brightness Test Cataracts PSF Match (Attenuation Map)(Binary Answer) (PSF Map) Low Attenuation Position, Size Sub-aperture and Shape Contrast Test (Opacity Map) (Contrast Map) C C C C C C C C High C C C C Attenuation C C C C C C C C 139
    118. 118. 140
    119. 119. Dual Monitor Stack 141
    120. 120. Dual Monitor Stack 142
    121. 121. Dual Monitor Stack Lens MedicalMonochrome Monitors LCD Stack 143
    122. 122. DLP-Mask Prototype High Contrast DLP Projector Mask Lens Eye 144
    123. 123. Cell Phone Prototype LensPinhole Mask Stephen Wolfram CEO – Wolfram Research 145
    124. 124. Validation using Cameras Thin Diffuser toSimulate Cataracts Estimated Attenuation Map Estimated PSF Map 146
    125. 125. Validation using Cameras Estimated Measured Estimated Opacity Map Attenuation AttenuationSimulated Cataract (Pictures) (User Interaction) 147
    126. 126. Scratching Contact LensesAdvanced Mild EarlyCataracts Cataracts Cataracts Measured: 0.5mm2 Estimated: 0.45mm2 148
    127. 127. Elderly Evaluation 18 volunteers - 6 with early cataracts - 12 with no cataracts 149
    128. 128. Limitations• Active user participation• One clear light path• Retinal diseases Patience and concentration are required to follow steps 157
    129. 129. ACM SIGGRAPH 2011 158
    130. 130. US + International Patent 159
    131. 131. LVPEI CATRA Early Tests. 160
    132. 132. Lotus University Trial 161
    133. 133. Lotus University Trial 162
    134. 134. Mexico VOSH/Rotary Trip. 0.00 5.00 10.00 15.00 20.00 25.00 25.00 20.00 CATRA 15.00 10.00 5.00 0.00 LOCS III Reference 84% Correlation for 37 eyes. 163
    135. 135. mHealth Summit 2011 164
    136. 136. Awards: MIT Global Challenge & MIT Ideas CompetitionsEyeCatra: $5K Winner AwardMIT Ideas Competition 2011.EyeCatra: $5K Public Choice AwardMIT Global Challenge 2011. 165
    137. 137. 166
    138. 138. 167
    139. 139. eyecatra.com CATRA: Quantitative Cataract Maps Unique, low-cost quantitative lens mappingwith: Erick Passos, Jan Zizka, Everett Lawson, Esteban Clua , Manuel M. Oliveira, Ramesh Raskar
    140. 140. Thesis Contributions TailoredNETRA CATRA Displays 170
    141. 141. 171with: Manuel M. Oliveira, Daniel Aliaga, Ramesh Raskar
    142. 142. 172
    143. 143. 173
    144. 144. 174
    145. 145. 175
    146. 146. 176
    147. 147. 177
    148. 148. Computer Generated GlassesFocusing Here Focusing Here Perfect vision Myopia Focal Range Focal Range Subject’s Focal Point Does Not Change 178
    149. 149. Computer Generated Glasses Focusing Here Focusing Here Perfect vision Hyperopia Focal Range PresbyopiaFocal Range Subject’s Focal Point Does Not Change 179
    150. 150. Tailoring is the easiest way to create a hologram
    151. 151. Because uncorrected individuals are already focusing where you want.
    152. 152. Tailoring Process Myopic View: -3DFocusing Here He can focus up to 33cm (12in)Distance Display-Eye: 50cm 182
    153. 153. Tailoring Process Myopic View: -3DLight-field Focusing Here He can focus up to 33cm (12in) Display Distance Display-Eye: 50cm 183
    154. 154. Tailoring Process Myopic View: -3DLight-field Focusing Here He can focus up to 33cm (12in) Display Distance Display-Eye: 50cm 184
    155. 155. Tailoring Process Myopic View: -3DLight-field Focusing Here He can focus up to 33cm (12in) Display Pixel Size of 96um at 33cm 1-arc minute Distance Display-Eye: 50cm Resolution 185
    156. 156. Working Resolution: 1800 DPI Channel Size 4.7um!$150 Vuzix HMD LCD 186
    157. 157. Astigmatism Correction 187
    158. 158. Tailoring for Astigmatism Subject’s prescription -2D -1D @ 90Light-field Two Points in Focus He focus at 30cm to 50cm. Display 50cm 30cm Where the Subject’s Accommodate 188
    159. 159. Single-Focus Multi-Depth Displays For a given depth in focus (accommodation), a single object may be splitted into anisotropic instances that are placed at distinct depths 189
    160. 160. Wavefront Maps 90 degrees Sphere: -2D Cylinder: -1D Axis: 90° 0 degrees k Lens focal length in kZernike Functions 190
    161. 161. InputsTailoring Process f(k) Light-field Display LCD1 LCD2 LCD1 191
    162. 162. Single-Focus Multi-Depth Displays Eye 197
    163. 163. Builds the Image Pixel by Pixel Display Eye A B d t d 198
    164. 164. Scaling Pixels by Depth DisplayScale Eye A B d t d 199
    165. 165. Mapping Light-Field Display <-> Retina f(k) Display k Eye R S1 S2 t a æ -k k - S2 ö R(S2 , k) = a ç + ÷+ k è f (k) t ø 200
    166. 166. 201
    167. 167. Cataract Density Maps kNuclear Cataract Cataract density in k Sub-capsular Cataract 202
    168. 168. Avoiding Cataracts Display Eye 203
    169. 169. Normalization Step Display Eye 204
    170. 170. Projector Camera – the “eye”Dual Stack of LCDs 205
    171. 171. Lens ArrayVuzix Head Mounted Display Vuzix Head Mounted Display 206
    172. 172. 3.25D Myopic 1mm Eye Input Image As Seen on a Regular Display As Seen on a Tailored Display 207
    173. 173. 3.5D Myopic CataractEye with Nuclear Cataracts 7.9mm 0.9mm p = 20mm Input No Cataracts No Cataract Tailoring Tailored for Cataracts 208
    174. 174. 1mm1D Astigmatic Eye at 180 No Correction Corrected 23.5 23.2 23.0 22.7 Normal Light Field Corrected Light Field 209
    175. 175. 210
    176. 176. 211
    177. 177. 212
    178. 178. 213
    179. 179. 214
    180. 180. 215
    181. 181. 216
    182. 182. 217
    183. 183. 218
    184. 184. Users Prefer the Tailored Image 100 100% 100% 98% 80Preferability ( %) 97% 60 +5D Lens 40 54% No Lens 46% 3% 2% 20 0 1D (1m ) 2D (50cm ) 3D (33cm ) 4D (25cm ) 5D (20cm ) Optical Power of the Projected Image (D) 13 Volunteers, 16 votes each. 219
    185. 185. Users Prefer the Tailored Image100 46 44 98% 80 42 90% 60 +4D Ast. Lens No Lens 40 10% 66% 34% 2% 20 0 0D (Inf) 2D (50cm ) 4D (25cm ) Optical Power of the Projected Image (D) 10 Volunteers, 16 votes each. 220
    186. 186. Users Prefer the Tailored Image 48 47 46 100 45 100% User Perspective (%) 44 80 43 42 92% Diopters 41 60 8% 40 How subject saw it 0% 20 Corrected Duplicated 0 0D (Inf) Keratoconus Correction of the Projected Image 1 Volunteer, 80 votes. 221
    187. 187. Multi-Focus Multi-Depth Display Input Image Input Depth+0.5D from the Image Plane -0.5D from the Image Plane 222
    188. 188. ACM SIGGRAPH 2012 223
    189. 189. INPI Patent 224
    190. 190. 225
    191. 191. Tailored Display Limitations• Eyes fixed relative to the display – Similar to 3D Displays – Depends on the eye aberrations• High-resolution LCD panels (PPI) – Giga-pixel displays for monitors• Other ocular diseases may affect our results. 226
    192. 192. Need 227
    193. 193. Thesis Conclusions• NETRA: Optics and UI for Refraction – The Inverse of Shack-Hartmann Aberrometer – Myopia, Hyperopia, Astigmatism, Focal range – Accuracy and Resolution close to Standard Practice• CATRA: Optics and UI for Cataracts – Forward Scattering and Foveal Projection – Four brand new Maps• Tailored Displays: Compensate for Aberrations – First-of-its-kind Multi-Depth Display – High-order Aberrations and Cataracts 228

    ×