Lytro Light Field Camera: from scientific research to a $50-million business
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Lytro Light Field Camera: from scientific research to a $50-million business

  • 1,078 views
Uploaded on

I prepared these slides while I had somehow lost myself. Lytro and its story make one willing to believe again, those brave crazy ones who would like to change the world.

I prepared these slides while I had somehow lost myself. Lytro and its story make one willing to believe again, those brave crazy ones who would like to change the world.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,078
On Slideshare
1,078
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
28
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Light Field Camera from scientific research to a $50-million business SHI Weili, Tsinghua UniversityPhoto from Lytro.com
  • 2. With big aperture, DCLRs can achieve shallow focus.
  • 3. Consumer DCs produce photos with deep focus.
  • 4. What we’d prefer...(post-processed with Adobe Photoshop)
  • 5. But how about loss of focus?...(previously irreparable)
  • 6. Introducing Lytro: refocus afterwards!Photos from Lytro.com
  • 7. Introducing Lytro: refocus afterwards!Photos from Lytro.com
  • 8. Technological features + Refocusing+ Extending depth of field + 3D images + Speed + Low-light sensitivity + Easy sharing - Low resolution
  • 9. "Our vision is a product that allows people to shoot and share very simply." Ren Ng, founder and CEO of LytroPhoto and quote from dpreview.com
  • 10. Industrial designPhoto from Lytro.com
  • 11. Industrial designPhoto from Lytro.com
  • 12. Interaction designPhoto from Lytro.com
  • 13. Interaction designImage from NewDealDesign
  • 14. "We worked really hard to create an iconic design that really conveys the idea that this is ‘camera 3.0’." Ren Ng, founder and CEO of LytroPhoto and quote from dpreview.com
  • 15. Depth of field: source of blurrinessFigure from Wikipedia
  • 16. Depth of field: source of blurrinessSome information is missing: If we know the direction of each ray of light, we can trace the rays back to their source, and don’t have to mix them up.
  • 17. 4D light fieldFigures from Ng, R. 2005. Fourier slice photography.
  • 18.  2D light field study with a linear camerachapter . light fields and photographs Figure .: Parameterization for the light field flowing into the camera. parameterization for the light field flowing into the camera the photosensor. Let us refer to u as the directional axis, because the u intercept on the lens determines the direction at which the ray strikes the sensor. In addition, let us refer to x asFigure from Ng, R. 2006. spatial axis. Of course in general the ray exists in d and we would consider intersections the Digital light field photography.
  • 19. 2D light field study with a linear camera .. photograph formation  Figure .: The set of all rays flowing into the camera. the set of all rays flowing into the camera of rays in a sampled light field, and it has become very common in the light field literature.Figure from Ng, R. 2006. Digital light field photography. 2.3 Photograph Formation
  • 20.  2D light field study with a linear camera chapter . light fields and photographs Figure .: The cone of rays summed to produce one pixel in a photograph. the cone of rays summed to produce one pixel in a photograph which arrive at the sensor from more oblique angles, contribute less energy to the value of the pixel. Another example is that the photosensitive portion of a pixel in a cmos sensor isFigure from Ng, R. 2006. Digital light field an overlay of metal wires [Catrysse and Wandell ], so rays from typically obscured by photography.
  • 21. A photograph is an integral projection of the canonical light field.Quote from Ng, R. 2006. Digital light field photography.
  • 22. RefocusingFigure from Ng, R. 2006. Digital light field photography. and Wikipedia (a) (b)
  • 23. A photograph is an integral projection of the canonical light field, where the trajectory of the projection depends on the depth at which the photograph is focused.Quote from Ng, R. 2006. Digital light field photography.
  • 24. Recording the light field .. a plenoptic camera records the light field  Figure .: Sampling of a photograph’s light field provided by a plenoptic camera. sampling of a photograph’s light field provided by a plenoptic camera camera the width of a grid column is the width of a photosensor pixel. In the plenoptic camera, on the other hand, the grid cells are shorter and wider. The column width is theFigure from Ng, R. 2006. Digital microlens, and the column is vertically divided into the number of pixels across width of a light field photography.
  • 25.  Recording the light field chapter . recording a photograph’s light field (z) (z Figure .: Raw light field photograph read off the photo array. The figure shows a crop of approximately one qua crolenses are clearly visible in print. (z) (z) Figure .: Raw light field photograph read off the photosensor underneath thFigure from Ng, R. 2006. Digital light field photography. a crop of approximately one quarter the full image so array. The figure shows
  • 26. Recording the light field (z) (z Figure .: Raw light field photograph read off the photo array. The figure shows a crop of approximately one qua crolenses are clearly visible in print. (z) (z) Integrating on each microlen images gives the conventional photography. Figure .: Raw light field photograph read off the photosensor underneath thFigure from Ng, R. 2006. Digital light field photography. a crop of approximately one quarter the full image so array. The figure shows
  • 27. Refocusing .. image synthesis algorithms  Two sub-aperture photographs obtained from a light field by extracting the shown pixel under each microlens (depicted on left). Note that the images are not the same, but represents different viewpoints. (a): No refocus (b): Refocus closer (c): Refocus further Figure .: Shift-and-add refocus algorithm, illustrated with just two sub-aperture imagesFigure from Ng, R. 2006. Digital light field photography. for didactic purposes.
  • 28. . three views of the recorded light field  Refocusing  chapter . light fields and photographs (b) (b) (a) (b) (a) (b) Figure .: Overview of processing the recorded light field. Figure .: The projection of the light field corresponding to focusing further and closer than the chosen x parameterization plane for the light field. Figure from Ng, R. 2006. Digital light field photography.
  • 29. .. image synthesis algorithms Refocusing  (a): No refocus (b): Refocus closer (c): Refocus further Figure .: Shift-and-add refocus algorithm, illustratedillustratedsub-aperture images Shift-and-add refocus algorithm, with just two with just for didactic purposes. two sub-aperture images for didactic purposes.Figure from Ng,with bothDigital light field photography. from the center of the lens (u, v), and the relative R. 2006. the distance of the sub-aperture
  • 30.  Refocusing chapter . digital refocusing (a) (a) (a) (a) (a) (b) Examples of refocusing (a1–a5) and extended depth of field (b). Figure .: Examples of refocusing (a–a) and extended depth of field (b).Figure from Ng, R. 2006. Digital light field photography.
  • 31. Extending the depth of field .. image synthesis algorithms  The sub-aperture photographs themselves have infinite DOF, resulted from the minor aperture. (a): No refocus (b): Refocus closer (c): Refocus furtherFigure from Ng, R. 2006. Digital light field photography. Figure .: Shift-and-add refocus algorithm, illustrated with just two sub-aperture images
  • 32.  Extending thedigital refocusing field chapter . depth of (a): Unrefocused (b): Sub-aperture image (c): Extended dof Refocusing each pixel gives extended an image with much digitally ex- Figure .: Comparison of a sub-aperture image and DOF computed with higher SNR. tended depth of field.Figure from Ng, R. 2006. integration of Equation ., we obtain high snr by combining the contributions numerical Digital light field photography.
  • 33. In the spatial domain, photographs are integral projections of the light field. In the Fourier domain, photographs are just 2D slice in the 4D light field.Quote from Ng, R. 2005. Fourier slice photography.
  • 34. Classical Fourier Slice Theorem projection Fourier slice vs. integral Integral Projection 2D Fourier 1D Fourier Transform Transform SlicingPage from Ng, R. 2006. Digital light field photography.
  • 35. Classical Fourier Slice Theorem projection Fourier slice vs. integral Integral Projection 2D Fourier 1D Fourier Transform Transform SlicingPage from Ng, R. 2006. Digital light field photography.
  • 36. Fourier slice vs. integral projection Integral Projection 4D Fourier Transform Spatial Domain Inverse Fourier Domain 2D Fourier Transform SlicingPage from Ng, R. 2006. Digital light field photography.
  • 37. Fourier slice vs. integral projectionPage from Ng, R. 2006. Digital light field photography.
  • 38. In the Fourier domain, photographs are just 2D slice in the 4D light field. That’s much simpler than in the spatial domain, where photographs are integral projections of the light field.Quote from Ng, R. 2005. Fourier slice photography.
  • 39. Fourier-domain slicing algorithm Pre-process: O(N4 log N) Refocusing: O(N2 log N) Spatial-domain integration algorithm Refocusing: O(N 4)Quote from Ng, R. 2005. Fourier slice photography.
  • 40. Resolution of the light-field sample .. a plenoptic camera records the light field  Figure .: Sampling of a photograph’s light field provided by a plenoptic camera. sampling of a photograph’s light field provided by a plenoptic camera camera the width of a grid column is the width of a photosensor pixel. In the plenoptic camera, on the other hand, the grid cells are shorter and wider. The column width is theFigure from Ng, R. 2006. Digital microlens, and the column is vertically divided into the number of pixels across width of a light field photography.
  • 41. Band-Limited Analysis Band-limited analysis Band-width of measured light field Light field shot with cameraPage from Ng, R. 2006. Digital light field photography.
  • 42. Band-Limited Analysis Band-limited analysisPage from Ng, R. 2006. Digital light field photography.
  • 43. Band-Limited Analysis Band-limited analysisPage from Ng, R. 2006. Digital light field photography.
  • 44. Band-Limited Analysis Band-limited analysisPage from Ng, R. 2006. Digital light field photography.
  • 45. Ren Ng improved the theory and technology of light field camera during his doctoral program.
  • 46. "It was a scientific breakthrough we were working towards. The next step weve been working on has been making a commercial breakthrough." Ren Ng, founder and CEO of LytroPhoto and quote from dpreview.com
  • 47. Technological evolution: from research to consumer product early 2000sPhoto from Lytro.com
  • 48. Technological evolution: rd Tech Report CTSR 2005-02 from research to consumer product Stanford Tech Report CTSR 2005-02he pinhole g becauseminate the ssing raysays do not ive imageeriphery. Figure 8: Top: Exploded view of assembly for attaching the microlens array to the digital back. Bottom: Cross-section through assembled parts. Figure 7: Technique for ameliorating vignetting. Top: Moving the pinhole observer beyond the bounds shown in Figure 6 results in vignetting because Our mi- art 0125- some required rays are unavailable (shaded gray). Bottom: To eliminate the e, square we use the closest available rays, by clamping the missing rays vignetting, ctor. The to the bounds of the aperture (shaded region). Note that these rays do notnumber is pass through the original pinhole, so the resulting multi-perspective image and used has a different center of projection for each ray in the corrected periphery. Figure 8: Top: Exploded view of assembly for attaching the microlens array se lenses to the digital back. Bottom: Cross-section through assembled parts. on tubes 4 image- approximately 4000×4000 pixels that are 9 microns wide. Our mi-ns holder, array was made by Adaptive Optics Associates (part 0125- crolens mid 2000sphotosen-It has 296×296 lenslets that are 125 field camera in use. 0.5-S). Figure 9: Our light microns wide, square Figures from Ng,with very close to 100% fill-factor. Thewith three and square packed shaped, R. 2006. Digital light field photography. focal length of the microlenses is 500 microns, so their f -number is
  • 49. Technological evolution: from research to consumer product 2011: LytroImages from Lytro.com
  • 50. Technological evolution: from research to consumer product 2011: LytroFigure from Lytro.com
  • 51. The company was founded in 2006.It has raised approximately $50 million of venture capital. Its first camera went on sale October 19, 2011, and began shipping on February 29, 2012, starting with a very affordable price of $399.
  • 52. "At first well be making those decisions for the user - so that we can make the process as simple as possible but, further down the line, well provide tools to give more control over the final output. Its important to understand that Lytros camera will record full light fields at day one." Ren Ng, founder and CEO of LytroPhoto and quote from dpreview.com
  • 53. The Lytro Desktop application is required to interact with the light field data format (.lfp). The Lytro Desktop software comes on the Lytro camera. The install window will pop up the first time you plug in your camera into your computer. You can then start the install process. If it doesn’t, find the disk image on your desktop to start the install. 2. Unplug and re-plug in the camera. After you install the software, you must plug the camera back in to prompt an import of your first light field pictures. 3. Back up process begins. A back up process will start after the first time the Lytro Desktop software runs. This happens only the first time you plug your camera in and takes about 4-5 minutes. Reminder: The minimum spec is Mac OS X 10.6.6 or higher.Images from Lytro User Manual and Lytro.com
  • 54. "Were very keen to see light field images develop through an ecosystem of software." Ren Ng, founder and CEO of LytroPhoto and quote from dpreview.com
  • 55. Back to its limitation... + Refocusing+ Extending depth of field + 3D images + Speed + Low-light sensitivity + Easy sharing - Low resolution Solution: much higher density of microlenses and sensors
  • 56. "Its not technological limitations that are defining that figure, its a marketing-driven progression. If you applied the technology being developed for mobile phone cameras and applied it to an APS-C sensor, you could in theory make a sensor with hundreds of millions of pixels." Ren Ng, founder and CEO of LytroPhoto and quote from dpreview.com
  • 57. Or think about Nokia’s 40-megapixel 808 PureView!Pictures from Nokia
  • 58. Revolution led by the crazy one “Lytro is developing a new type of camera that dramatically changes photography for the first time since the 1800s.” –TechCrunchPhoto from NewDealDesign
  • 59. A tribute to Apple?Photo by Annie Tao
  • 60. Light Field Camera from scientific research to a $50-million business SHI Weili, Tsinghua UniversityPhoto from Lytro.com