This document describes a computational endomicroscopy platform that uses compressed sensing to achieve higher resolution images than the physical sensor resolution allows. It uses a digital micromirror device as a spatial light modulator to modulate scenes at a conjugate image plane. A camera then collects multiple coded measurements to reconstruct higher resolution images through compressed sensing algorithms. Experiments demonstrate reconstructing higher resolution images than the individual fiber spacing of fiber optic bundles used in endomicroscopy. Future work aims to further reduce measurements needed and apply the techniques to fiber bundle platforms.

1. Design and Characterization of a Computational Endomicroscopy
Platform for Optical Biopsy
John Paul Dumas1, Mark C. Pierce1, Muhammad Lodhi2, Waheed U. Bajwa2
1Department of Biomedical Engineering, Rutgers, The State University of New Jersey
2Department of Electrical and Computer Engineering, Rutgers, The State University of New Jersey
Introduction Methods Experiments & Results
Fiber Optic Bundles:
1) Coherent fiber-optic bundles are small in diameter
and highly flexible, allowing for excellent tissue
access in-vivo for optical biopsy.
2) Manufacturing constraints limit individual fiber
spacing to ~2 μm, resulting in relatively few
resolvable points per unit area [1].
3) New signal processing concepts can potentially
surpass the spatial resolution limit associated with
fiber-optic bundles.
Compressed Sensing:
1) Compressed sensing (CS) techniques can be
implemented in applications where signal acquisition
is sparse in some known domain (e.g. Fourier
transform of a checkerboard scene).
2) The CS framework states that high-fidelity images
can be constructed with a larger number of pixels
than are physically present in the optical sensor.
3) The instrumentation function (Φ) accounts for system
specific functions imposed on the scene as it is
imaged through the imaging hardware to the sensor.
Highly Parallel Single Pixel Camera:
1) The single-pixel camera (SPC) generates images with
many pixels using many sequential coded measurements
at a single point detector.
2) In the SPC framework, spatial light modulators (SLMs)
are used to generate mask patterns that modulate a
scene at a conjugate image plane [2].
3) Using a sensor array, rather than a single sensor, in a
parallel SPC architecture reduces the number of mask
patterns needed to reconstruct a scene, thereby
improving light collection efficiency and imaging speed.
Experimental CS Platform:
1) Imaging optics relay the scene onto a Digital Micromirror
Device (DMD), which is used as an SLM.
2) Projection optics image the modulated scene onto a CCD
camera such that multiple mask elements map to each
CCD pixel. The number of mask elements per CCD pixel
is termed the “undersampling factor.”
CS Platform Results (Top) [3]:
1) The scene (1951 USAF Resolution Target) was imaged
with a microscope at 4x for ground truth comparison.
2) Imaging the target with the CS platform and no
conjugate image plane mask shows the loss of detail
due to CCD pixel size. Bicubic interpolation on this
image only slightly recovers the lost detail.
3) The undersampling factor (4x or 16x) determines how
many mask elements project onto a single CCD pixel.
4) 50 sequential measurements were used to reconstruct
images based on Nesterov’s proximal gradient method
for CS reconstruction [4].
Fiber Bundle Platform Results (Bottom):
1) Imaging the grayscale cameraman scene with the fiber
bundle platform shows detail loss due to individual fibers.
2) The random arrangement and circular shape of fibers
makes exact geometric undersampling unachievable.
3) Mask-to-bundle calibration is achieved by sequentially
turning each DMD element on and recording the
respective response at the fiber bundle plane.
4) To reconstruct images of the scene, each element of the
scene is sequentially turned on and the response is
compared to the calibration record to perform element-
by-element image reconstruction.
Conclusions References
Beating The Sensor Size Limit:
1) CS based on a highly parallel SPC architecture is able to generate images with
resolution higher than that imposed by the pixel count of the physical sensor.
2) Increasing the undersampling factor results in finer detail recovery, albeit at the cost
of temporal resolution because more measurements are needed for accurate image
reconstruction.
3) When applied to endomicroscopy, image plane masks can be used to reconstruct
images with higher resolution than the individual fiber spacing allows. This element-
by-element reconstruction requires many measurements to form a single image.
Future Work:
1) An alternate CS architecture with masks placed at an aperture plane instead of a
conjugate image plane may further reduce the number of measurements required for
image reconstruction.
2) Applying CS-based, rather then element-by-element-based, reconstruction to
masked measurements taken with the fiber bundle platform will require adding a term
to the CS algorithm that quantifies the non-geometric mapping of square mask
elements to circular fibers.
Flusberg, Benjamin A., et al.; Nature
Methods 2 (2005): 941-950.
Duarte, M. F., et al.; IEEE Sig. Proc.
Mag. 25(2), 83-91. (2008).
Dumas, John P., et al.; Optics
Express 24.6 (2016): 6145-6155.
R. Gu and A. Dogandzic, in
Proceedings of Asilomar Conference
on Signals, Systems, and
Computers, pp.1662–1667, (2014).
Object
DMD
Camera
Imaging optics
Projection Optics
Squamous Tissue
Imaged Directly
Squamous Tissue Imaged Through
Fiber-Optic Bundle
Measurements
Vector (y(m))
= *
Instrumentation
Function (Φ)
Vectorized
Scene (x)
θΨy(m)
=
Φ
θ
**
y(m)
= *
Φ*Ψ
θ
*
Φ*Ψ
=
Scene Mask
Single
Sensor
Sensor
Array
2f 2f 2f 2f
...
ϕ(m) y(m)x0 * =
Scene (x0) Mask ϕ(1) Measurement y(1)
Scene (x0) Mask ϕ(2) Measurement y(2)
SPC
Architecture
Parallel SPC
Architecture
Imaging
Optics
Projection
Optics
Both Architectures
Collect Multiple
Measurements By
Changing Masks
DMD Projection Optics Fiber Bundle Relay Optics Camera
Experimental Fiber Bundle Platform:
1) MATLAB is used to generate simulated object/mask
modulations, which are then displayed on the DMD.
2) The front of a large fiber bundle (3 mm diameter, 50 μm
fiber spacing) is placed at a conjugate image plane to
relay the image from the DMD to the CCD.
=
x θΨ
Scene (x)
Sparse Representation
Of Scene In The
Transform Domain (θ)
*
Many Zero-
Valued
Elements
Transformation (Ψ)
Microscope Image Of
The Scene
CS Platform Measurement
With No Mask
Bicubic Interpolation
CS Reconstruction With
4x Undersampling
CS Reconstruction With
16x Undersampling
Cameraman Scene
Single Measurement
(No Mask)
Element-By-Element
Reconstruction
Tissue Scene
Single Measurement
(No Mask)
Element-By-Element
Reconstruction
[1]
Acknowledgements: Funding from the
National Science Foundation (NSF)
(CCF-1453073, ECCS-1509260),
and Army Research Office (ARO)
(W911NF-14-1-0295).
[2]
[3]
[4]