The document summarizes research related to computational imaging being conducted by the Imaging Concepts group led by Prof. Andy Harvey at the University of Glasgow. Specific areas discussed include: - Developing low-cost infrared camera array systems using computational techniques as an alternative to more complex optical setups. - Applying techniques like synthetic aperture, light field imaging, temporal interpolation and pixel super-resolution to camera arrays. - Demonstrating snapshot pixel super-resolution and integral imaging using a 5x5 camera array to produce higher resolution images. - Developing a low-cost multispectral LWIR system using 6 FLIR LEPTON cameras and filters to enable applications like gas detection and classification

1. Computational imaging.
Low-cost infrared camera array systems.
Imaging concepts group, lead by Prof. Andy Harvey
Dr. Miguel A. Preciado
ma.preciado@upm.es
@miguelSciLight
https://sites.google.com/view/miguelscilight/

2. Computational imaging
• Computational imaging refers to the enhancement of the
capabilities of an imaging system by the application of
computational techniques, as an alternative to increasing the optical
setup complexity.
• One of the aims of computational imaging is the development of
cost-effective imaging systems, supported by the continuous price
reduction of computation power predicted by Moore law.

3. Camera array systems
http://graphics.stanford.edu/p
rojects/array/
• Synthetic aperture
• Light field / Integral
imaging.
• Temporal interpolation
• Pixel super-resolution.
• Multi-spectral imaging.

4. • Bullet time systems
Computational camera array imaging
Bullet Time systems
– also known as time slice, stop time, or time freeze –
implement a camera array that ranges anywhere from 24
to 150+ cameras to create an effect of stopped time or
extremely slowed time. Although the process was
pioneered decades prior, it was widely popularized by the
Matrix films directed by the Wachowski Brothers and has
since been used more regularly in film production.
http://mitchmartinez.com/bullettime/

5. Dealing with multiple cameras…
-Image Registration
- Feature-based
- Calibration-based
(For finite conjugate distances, parallax, 3D effects need to be considered.)
-Synchronization (for non-static video-rate)

6. Registration by feature matching
• Automatic (if images have enough
features)
• Homography found by feature
extraction
• SIFT
• SURF
• …

7. Registration by camera calibration
• Camera calibration
• Camera intrinsic
• Distortion, optical centre, focal length (x and y), and skew.
• Camera extrinsic
• Relative rotation and position of the cameras
Calibration pattern
Hartley, Richard, and Andrew
Zisserman. Multiple view geometry
in computer vision. Cambridge
university press, 2003.
https://docs.opencv.org/3.4.1/d9/da
b/tutorial_homography.html

8. Registration by camera calibration
How to get started:
Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003.
https://docs.opencv.org/3.4.1/d9/dab/tutorial_homography.html
Planar homography relates the transformation between
two planes.
A “world plane” can be related with “images” planes,
finding the relation between two images planes for
registration.
The world plane can be define at a distance to one camera
(e.g. reference camera). The Registration depends on the
distance.

9. Software
• OpenCV (Python, or C/C++)
• Features based registration
• Calibration based registration
• Matlab Computer Vision System Toolbox
• Features based registration
• Calibration based registration
• Specific software (Feature matching based)
• VisualSFM
• http://ccwu.me/vsfm/index.html

10. Camera array systems work in the
Imaging Concepts group

11. Pixel super-resolution

12. Snapshot pixel super-resolution
G. Carles, J. Downing, and A. R. Harvey,
"Super-resolution imaging using a camera array," Opt.
Lett. 39, 1889-1892 (2014)
5 x 5 camera array
Inversion problem, solved by using Lucy-Richardson
algorithm.
Feature based registration (SIFT)
Green: features used
Red: Outliers

13. 13
• 600 Mvoxel Per-band
• Independent integration time per channel
• No achromatisation necessary
• ~10mm
• Arbitrary (optimal, sparse) bands
• Low Cost
• High optical throughput
Snapshot spectral imager
Multiaperture imaging system in the visible band.
25 Bands
Plants classification
J. Downing and A. R. Harvey, "Multi-aperture hyperspectral imaging," in Imaging and Applied Optics,
OSA Technical Digest (online) (Optical Society of America, 2013), paper JW2B.2.

14. 14
This project has received funding from the European Union’s Horizon 2020 research and innovation
programme under grant agreement No 645114.
The dissemination of results herein reflects only the author’s view and the European Commission is not responsible
for any use that may be made of the information it contains.
Consortium
VIS/SWIR/MWIR/LWIR
Multiaperture array
This research has been carried out as a part of SEERS project www.seersproject.eu
The major goal of SEERS is to develop a modular, compact and cost effective snapshot
spectral imager in the Visible and IR range based on low cost uncooled FPAs, with
embedded processing capabilities.

15. • Robust surveillance
• Gas detection and
classification
• Oil Spill detection
• Robust temperature
measurement
Multispectral (course bands, VIS + multiple IR)
Deep learning (convolutional neural network)
robust surveillance.
Maritime scenario.
Car traffic scenario
Multispectral detection of gases
(filtered narrow bands in LWIR/MWIR

16. ≃2 k£ system vs ≃100k£ systems!!
Multispectral LWIR system
• 6x multiaperture multispectral system.
(1) FLIR LEPTON® based 6x LWIR detectors and filters
(2) 6x Raspberry-pi ® based CPU rack.
(3) Ethernet switch-hub for networking
4) USB hub for power supply.
6 cameras
6 infrared interference filters
6 channels of LWIR
FLIR LEPTON LWIR uncool detector
Dimension: 10.6 x 11.7 x 5.9 mm
HFOV: 25 deg (50 deg also available)
LWIR: 8 to 14 µm
80 x 60 pixels
Thermal sensitivity <50 mK
Frame Rate 9 Hz (actual Frame rate 27Hz
limited by firmware due to US export reg.)
~150 pounds
Based on low-cost FLIR
LEPTON LWIR uncool detector

17. Image reconstruction
Extrinsic parameters (relative position and orientation)
Intrinsic parameters (magnification and distortion)
Calibration of a pair of cameras using
Matlab Vision toolbox

18. Image reconstruction
Extrinsic parameters (relative position and orientation)
Intrinsic parameters (magnification and distortion)
Object
Multispectral camera array
Image detection
Multispectral images
Multispectral registration
(numerical, using camera parameters)
Multispectral datacube

19. 19
Multispectral LWIR proposed system
CH1 CH3 CH4 CH5 CH6CH2
6 Spectral bands
adapted to this
specific problem
CH1: 7.2 µm
CH2: 7.6 µm
CH3: 8.6 µm
CH4: 9.4 µm
CH5: 10.5 µm
CH6: 12.2 µm
Application in gas detection and classification.

20. Gas imaging results

21. Gas imaging results
Detection of propane in 7.2 µm, overlapped with visible channel.

22. Gas imaging results
22
Representation of 6-bands gas absorption multispectral images in 2xRGB images
RGB-1
RGB-2
RGB-1
RGB-2
RGB-1
RGB-2
RGB-1
RGB-2
RGB-1
RGB-2
RGB-1
RGB-2
Methanol Ethanol Isopropanol
Butane Propane Methane
CH1-CH6
CH1-CH3
CH4-CH6

23. Gas imaging results
• Hydrocarbon video examples
23
Propane (~ 100 %)Methane (10 %) Butane (~100 %)

24. Gas imaging results
• Alcohols video examples
24
Methanol Ethanol IPA
Methanol Ethanol IPA
Alcohol gas released by alcohol
liquid solution in flask

25. 25
PEPPPVC
Thin plastics classification
PP: Polypropylene
PE: Polyethylene
PVC: Polyvinyl chloride
Clear potential in recycling processes!

26. Snapshot pixel super-resolution in LWIR
Visible Single camera 6x camera SR
Snapshot (6 cameras, 6 simultaneous acquisitions)
M. A. Preciado, G. Carles, and A. R. Harvey,
"Video-rate computational super-resolution and
integral imaging at longwave-infrared wavelengths,"
OSA Continuum1, 170-180 (2018)
Panchromatic!
(without filters)

27. Computational super-resolution
• Panchromatic super-resolution (SR) based on integral imaging
reconstruction
Digital focus
Integral imaging
reconstruction
M. A. Preciado, G. Carles, and A. R. Harvey,
"Video-rate computational super-resolution and integral imaging at longwave-infrared wavelengths," OSA Continuum1, 170-180 (2018)
Digital focusing, increased resolution
Visible
Static 3D Integral Imaging
super-resolution reconstruction

28. • Panchromatic super-resolution (SR) based on integral imaging
reconstruction
Computational Super-resolution
Fixed digital focus
Digital focus
Integral imaging
reconstruction
Fixed digital focus Fixed digital focus
Variable digital focus
M. A. Preciado, G. Carles, and A. R. Harvey,
"Video-rate computational super-resolution and integral imaging at longwave-infrared wavelengths," OSA Continuum1, 170-180 (2018)

29. Other configurations
• 360 degrees
Work on progress currently developed by Laura Cowan
In partnership with

30. Thanks for your attention!
Check other research done in the ICG and myself in the following extra slides, and:
https://www.gla.ac.uk/schools/physics/research/groups/imagingconcepts/
https://sites.google.com/view/miguelscilight/

31. Compact low-cost wide field of view retinal
imager
Acknowledgements: Andrew Harvey, Guillem Carles, Paul Zammit, Ross Drysdale
31
The illumination beam is separated
from the observation beam
Narrow field of view Wide field of view
For wide field of view, there
are illumination and
observation can mix,
resulting in glare.
(Gullstrand principle)

32. Compact low-cost wide field of view retinal
imager
Acknowledgements: Andrew Harvey, Guillem Carles, Paul Zammit, Ross Drysdale
32
Extension of Gullstrand principle to wide field of view (~ 80 deg.)

33. Compensating waveforms.
Generalization of Airy pulses under lossy medium.
M. A. Preciado and K. Sugden, "Proposal and design of Airy-based rocket pulses for
invariant propagation in lossy dispersive media," Opt. Lett. 37, 4970-4972 (2012)
Optical pulse in lossy dispersive media Optical beam in lossy diffractive media
Miguel A. Preciado, Kishan Dholakia, and Michael Mazilu, "Generation of
attenuation-compensating Airy beams," Opt. Lett. 39, 4950-4953 (2014)
Ultrafast Optics Spatial Optics
Total energy
decreases
Shape and
peak remains
Intensity profile
carries high
frequency
content
https://gist.github.com/mpreciado/115637b0d557a4eb8b7a1a0723308119#file-compairysim-m

34. Compensating light sheet microscopy
Biological Sample
Fluorescently labelled Kisspeptins
neurons in the hypothalamic arcuate
nucleus of mouse brain sections,
attenuation estimated 100 cm-1
Estimated attenuation 100 cm-1,…
…partially compensated to an
equivalent attenuation of 35
cm-1
"Light-sheet microscopy with attenuation-compensated propagation-invariant beams" J. Nylk, K. McCluskey, M.
A. Preciado, M. Mazilu, F. J. Gunn-Moore, S. Aggarwal, J. A Tello, D. E. K. Ferrier, K. Dholakia Science Advances
06 Apr 2018: Vol. 4, no. 4, eaar4817
Patent “Airy beam light sheet and airy beam light sheet microscope” M Mazilu, T
Vettenburg, K Dholakia, M Preciado, Y Zhengyi. US Patent App. 15/326,073
Currently licenced for exploitation to
M squared lasers LTD

35. Bragg-based Mach-Zehnder interferometer
"Bragg-based Mach-Zehnder interferometer" Miguel A.
Preciado, Atalla El-Taher, Jiabi Xiong, Zuowei Xu, Yu Yu, Kate
Sugden, And Xuewen Shu. Optica (currently under review)
Applied in digital DPSK signal demodulation
Fabricated Bragg grating
Patent:
Single fiber bragg grating as delay line interferometer
MA Preciado, X Shu, US Patent 9,709,745 (2017)
@miguelSciLight
https://sites.google.com/view/miguelscilight/

36. Full phase-polarization control in a multimode fiber
Cleaved fibre immersed
in sample
Microscope objective
50/50 beam
splitter
CCD camera
Photo detector
Fibre
couplerDMD
Laser
source
Beam
Expander
Set-up for trapping in liquid
DMD TI DLP lightcrafter
4500
Multimode output beam
Corrected
2 um Polystyrene particle in water
Particle
Corrected
Beam
Particle trapped in corrected beam
Wavelength 532 nm
Acknolwedgment: Kishan Dholakia, Michael Mazilu (University of St Andrews)
Fiber output beam corrected at different
points of the facet, with polarization control
@miguelSciLight
https://sites.google.com/view/miguelscilight/