Ren Ng improved light field camera technology during his doctoral research. This led to the founding of Lytro in 2006 to develop a consumer light field camera. Lytro launched its first camera in 2011, priced at $399, capturing full light field data while providing an easy user experience of refocusing photos after capture. The Lytro camera required its desktop software to interact with the proprietary light field data format and refocus images on a computer.

1. Light Field Camera
from scientific research to a $50-million business
SHI Weili, Tsinghua University
Photo from Lytro.com

2. With big aperture, DCLRs can achieve shallow focus.

3. Consumer DCs produce photos with deep focus.

4. What we’d prefer...
(post-processed with Adobe Photoshop)

5. But how about
loss of focus?...
(previously irreparable)

6. Introducing Lytro: refocus afterwards!
Photos from Lytro.com

7. Introducing Lytro: refocus afterwards!
Photos from Lytro.com

8. Technological features
+ Refocusing
+ Extending depth of field
+ 3D images
+ Speed
+ Low-light sensitivity
+ Easy sharing
- Low resolution

9. "Our vision is a product
that allows people to
shoot and share very simply."
Ren Ng, founder and CEO of Lytro
Photo and quote from dpreview.com

10. Industrial design
Photo from Lytro.com

11. Industrial design
Photo from Lytro.com

12. Interaction design
Photo from Lytro.com

13. Interaction design
Image from NewDealDesign

14. "We worked really hard to
create an iconic design that
really conveys the idea that
this is ‘camera 3.0’."
Ren Ng, founder and CEO of Lytro
Photo and quote from dpreview.com

16. Depth of field: source of blurriness
Figure from Wikipedia

17. Depth of field: source of blurriness
Some information is missing:
If we know the direction of each ray of light,
we can trace the rays back to their source,
and don’t have to mix them up.

18. 4D light field
Figures from Ng, R. 2005. Fourier slice photography.

19.  2D light field study with a linear camerachapter . light fields and photographs
Figure .: Parameterization for the light field flowing into the camera.
parameterization for the light field flowing into the camera
the photosensor. Let us refer to u as the directional axis, because the u intercept on the lens
determines the direction at which the ray strikes the sensor. In addition, let us refer to x as
Figure from Ng, R. 2006. spatial axis. Of course in general the ray exists in d and we would consider intersections
the Digital light field photography.

20. 2D light field study with a linear camera
.. photograph formation 
Figure .: The set of all rays flowing into the camera.
the set of all rays flowing into the camera
of rays in a sampled light field, and it has become very common in the light field literature.
Figure from Ng, R. 2006. Digital light field photography.
2.3 Photograph Formation

21.  2D light field study with a linear camera
chapter . light fields and photographs
Figure .: The cone of rays summed to produce one pixel in a photograph.
the cone of rays summed to produce one pixel in a photograph
which arrive at the sensor from more oblique angles, contribute less energy to the value of
the pixel. Another example is that the photosensitive portion of a pixel in a cmos sensor is
Figure from Ng, R. 2006. Digital light field an overlay of metal wires [Catrysse and Wandell ], so rays from
typically obscured by photography.

22. A photograph is an integral projection
of the canonical light field.
Quote from Ng, R. 2006. Digital light field photography.

23. Refocusing
Figure from Ng, R. 2006. Digital light field photography. and Wikipedia
(a) (b)

24. A photograph is an integral projection
of the canonical light field, where the
trajectory of the projection depends on
the depth at which the photograph is
focused.
Quote from Ng, R. 2006. Digital light field photography.

25. Recording the light field
.. a plenoptic camera records the light field 
Figure .: Sampling of a photograph’s light field provided by a plenoptic camera.
sampling of a photograph’s light field provided by a plenoptic camera
camera the width of a grid column is the width of a photosensor pixel. In the plenoptic
camera, on the other hand, the grid cells are shorter and wider. The column width is the
Figure from Ng, R. 2006. Digital microlens, and the column is vertically divided into the number of pixels across
width of a light field photography.

26.  Recording the light field
chapter . recording a photograph’s light field
(z) (z
Figure .: Raw light field photograph read off the photo
array. The figure shows a crop of approximately one qua
crolenses are clearly visible in print.
(z) (z)
Figure .: Raw light field photograph read off the photosensor underneath th
Figure from Ng, R. 2006. Digital light field photography. a crop of approximately one quarter the full image so
array. The figure shows

27. Recording the light field
(z) (z
Figure .: Raw light field photograph read off the photo
array. The figure shows a crop of approximately one qua
crolenses are clearly visible in print.
(z) (z)
Integrating on each microlen images gives the conventional photography.
Figure .: Raw light field photograph read off the photosensor underneath th
Figure from Ng, R. 2006. Digital light field photography. a crop of approximately one quarter the full image so
array. The figure shows

28. Refocusing
.. image synthesis algorithms 
Two sub-aperture photographs obtained from a light field by
extracting the shown pixel under each microlens (depicted on left).
Note that the images are not the same, but represents different
viewpoints.
(a): No refocus (b): Refocus closer (c): Refocus further
Figure .: Shift-and-add refocus algorithm, illustrated with just two sub-aperture images
Figure from Ng, R. 2006. Digital light field photography.
for didactic purposes.

29. . three views of the recorded light field 
Refocusing
 chapter . light fields and photographs
(b)
(b)
(a) (b)
(a) (b)
Figure .: Overview of processing the recorded light field. Figure .: The projection of the light field corresponding to focusing further and closer
than the chosen x parameterization plane for the light field.
Figure from Ng, R. 2006. Digital light field photography.

30. .. image synthesis algorithms
Refocusing

(a): No refocus (b): Refocus closer (c): Refocus further
Figure .: Shift-and-add refocus algorithm, illustratedillustratedsub-aperture images
Shift-and-add refocus algorithm, with just two with just
for didactic purposes.
two sub-aperture images for didactic purposes.
Figure from Ng,with bothDigital light field photography. from the center of the lens (u, v), and the relative
R. 2006. the distance of the sub-aperture

31. 
Refocusing
chapter . digital refocusing
(a) (a) (a)
(a) (a) (b)
Examples of refocusing (a1–a5) and extended depth of field (b).
Figure .: Examples of refocusing (a–a) and extended depth of field (b).
Figure from Ng, R. 2006. Digital light field photography.

32. Extending the depth of field
.. image synthesis algorithms 
The sub-aperture photographs themselves have infinite DOF,
resulted from the minor aperture.
(a): No refocus (b): Refocus closer (c): Refocus further
Figure from Ng, R. 2006. Digital light field photography.
Figure .: Shift-and-add refocus algorithm, illustrated with just two sub-aperture images

33. 
Extending thedigital refocusing field
chapter .
depth of
(a): Unrefocused (b): Sub-aperture image (c): Extended dof
Refocusing each pixel gives extended an image with much digitally ex-
Figure .: Comparison of a sub-aperture image and DOF computed with higher SNR.
tended depth of field.
Figure from Ng, R. 2006. integration of Equation ., we obtain high snr by combining the contributions
numerical Digital light field photography.

34. In the spatial domain, photographs are
integral projections of the light field.
In the Fourier domain, photographs are
just 2D slice in the 4D light field.
Quote from Ng, R. 2005. Fourier slice photography.

35. Classical Fourier Slice Theorem projection
Fourier slice vs. integral
Integral
Projection
2D Fourier 1D Fourier
Transform Transform
Slicing
Page from Ng, R. 2006. Digital light field photography.

36. Classical Fourier Slice Theorem projection
Fourier slice vs. integral
Integral
Projection
2D Fourier 1D Fourier
Transform Transform
Slicing
Page from Ng, R. 2006. Digital light field photography.

37. Fourier slice vs. integral projection
Integral
Projection
4D Fourier
Transform Spatial Domain
Inverse
Fourier Domain 2D Fourier
Transform
Slicing
Page from Ng, R. 2006. Digital light field photography.

38. Fourier slice vs. integral projection
Page from Ng, R. 2006. Digital light field photography.

39. In the Fourier domain, photographs are
just 2D slice in the 4D light field.
That’s much simpler than in the spatial
domain, where photographs are integral
projections of the light field.
Quote from Ng, R. 2005. Fourier slice photography.

40. Fourier-domain slicing algorithm
Pre-process: O(N4 log N)
Refocusing: O(N2 log N)
Spatial-domain integration algorithm
Refocusing: O(N 4)
Quote from Ng, R. 2005. Fourier slice photography.

41. Resolution of the light-field sample
.. a plenoptic camera records the light field 
Figure .: Sampling of a photograph’s light field provided by a plenoptic camera.
sampling of a photograph’s light field provided by a plenoptic camera
camera the width of a grid column is the width of a photosensor pixel. In the plenoptic
camera, on the other hand, the grid cells are shorter and wider. The column width is the
Figure from Ng, R. 2006. Digital microlens, and the column is vertically divided into the number of pixels across
width of a light field photography.

42. Band-Limited Analysis Band-limited analysis
Band-width of
measured light field
Light field shot
with camera
Page from Ng, R. 2006. Digital light field photography.

43. Band-Limited Analysis Band-limited analysis
Page from Ng, R. 2006. Digital light field photography.

44. Band-Limited Analysis Band-limited analysis
Page from Ng, R. 2006. Digital light field photography.

45. Band-Limited Analysis Band-limited analysis
Page from Ng, R. 2006. Digital light field photography.

46. Ren Ng improved the theory and technology of light field camera
during his doctoral program.

47. "It was a scientific breakthrough
we were working towards.
The next step we've been working
on has been making a commercial
breakthrough."
Ren Ng, founder and CEO of Lytro
Photo and quote from dpreview.com

48. Technological evolution:
from research to consumer product
early 2000s
Photo from Lytro.com

49. Technological evolution:
rd Tech Report CTSR 2005-02
from research to consumer product
Stanford Tech Report CTSR 2005-02
he pinhole
g because
minate the
ssing rays
ays do not
ive image
eriphery. Figure 8: Top: Exploded view of assembly for attaching the microlens array
to the digital back. Bottom: Cross-section through assembled parts.
Figure 7: Technique for ameliorating vignetting. Top: Moving the pinhole
observer beyond the bounds shown in Figure 6 results in vignetting because
Our mi-
art 0125-
some required rays are unavailable (shaded gray). Bottom: To eliminate the
e, square we use the closest available rays, by clamping the missing rays
vignetting,
ctor. The
to the bounds of the aperture (shaded region). Note that these rays do not
number is
pass through the original pinhole, so the resulting multi-perspective image
and used
has a different center of projection for each ray in the corrected periphery. Figure 8: Top: Exploded view of assembly for attaching the microlens array
se lenses to the digital back. Bottom: Cross-section through assembled parts.
on tubes
4 image-
approximately 4000×4000 pixels that are 9 microns wide. Our mi-
ns holder, array was made by Adaptive Optics Associates (part 0125-
crolens mid 2000s
photosen-It has 296×296 lenslets that are 125 field camera in use.
0.5-S). Figure 9: Our light microns wide, square
Figures from Ng,with very close to 100% fill-factor. The
with three and square packed
shaped, R. 2006. Digital light field photography.
focal length of the microlenses is 500 microns, so their f -number is

50. Technological evolution:
from research to consumer product
2011: Lytro
Images from Lytro.com

51. Technological evolution:
from research to consumer product
2011: Lytro
Figure from Lytro.com

52. The company was founded in 2006.
It has raised approximately $50 million of venture capital.
Its first camera went on sale October 19, 2011,
and began shipping on February 29, 2012,
starting with a very affordable price of $399.

53. "At first we'll be making those decisions for
the user - so that we can make the process as
simple as possible but, further down the line,
we'll provide tools to give more control over
the final output.
It's important to understand that Lytro's
camera will record full light fields at day one."
Ren Ng, founder and CEO of Lytro
Photo and quote from dpreview.com

54. The Lytro Desktop application is required
to interact with the light field data format
(.lfp). The Lytro Desktop software comes
on the Lytro camera. The install window
will pop up the first time you plug in
your camera into your computer. You can
then start the install process. If it doesn’t,
find the disk image on your desktop to
start the install.
2. Unplug and re-plug in the camera.
After you install the software, you must plug the camera
back in to prompt an import of your first light field pictures.
3. Back up process begins.
A back up process will start after the first time the Lytro Desktop software runs.
This happens only the first time you plug your camera in and takes about 4-5 minutes.
Reminder: The minimum spec is Mac OS X 10.6.6 or higher.
Images from Lytro User Manual and Lytro.com

55. "We're very keen to see light
field images develop through
an ecosystem of software."
Ren Ng, founder and CEO of Lytro
Photo and quote from dpreview.com

56. Back to its limitation...
+ Refocusing
+ Extending depth of field
+ 3D images
+ Speed
+ Low-light sensitivity
+ Easy sharing
- Low resolution Solution:
much higher density of
microlenses and sensors

57. "It's not technological limitations that
are defining that figure, it's a
marketing-driven progression.
If you applied the technology being
developed for mobile phone cameras
and applied it to an APS-C sensor, you
could in theory make a sensor with
hundreds of millions of pixels."
Ren Ng, founder and CEO of Lytro
Photo and quote from dpreview.com

58. Or think about Nokia’s 40-megapixel 808 PureView!
Pictures from Nokia

59. Revolution led by the crazy one
“Lytro is developing a new type of camera
that dramatically changes photography
for the first time since the 1800s.”
–TechCrunch
Photo from NewDealDesign

60. A tribute to Apple?
Photo by Annie Tao

61. Light Field Camera
from scientific research to a $50-million business
SHI Weili, Tsinghua University
Photo from Lytro.com