This presentation deals with the reconstruction of a 3-D spatio-spectral object observed by a multispectral imaging system, where the original object is blurred with a spectral-variant PSF (Point Spread Function) and integrated over few broad spectral bands. In order to tackle this ill-posed problem, we propose a linear forward model that accounts for direct (or auto) channels and between (or cross) channels degradation, by modeling the imaging system response and the spectral distribution of the object with a piecewise linear function. Reconstruction based on regularization method is proposed, by enforcing spatial and spectral smoothness of the object. We test our approach on simulated data of the Mid-InfraRed Instrument (MIRI) Imager of the James Webb Space Telescope (JWST). Results on simulated multispectral data show a significant improvement over the conventional multichannel method.

1. Spatio-Spectral Multichannel Reconstruction from few
Low-Resolution Multispectral Data
Mohamed Elamine HADJ-YOUCEF 1,2
François ORIEUX 1,2 Aurélia FRAYSSE 1 Alain ABERGEL 2
1Laboratoire des Signaux et Systèmes (L2S)
CNRS, CentraleSupélec, Université Paris-Saclay, France
2Institut d’Astrophysique Spatiale (IAS)
CNRS, Univ. Paris-Sud, Université Paris-Saclay, France
September 06, 2018
M.A HADJ-YOUCEF (L2S & IAS) 26th EUSIPCO September 06, 2018 1 / 27

2. Context
The James Webb Space Telescope (JWST) :
Organization NASA (ESA & CSA)
Expected Launch March 2021
Duration 5.5 - 10 years
Segmented Mirror 25 m2 (> 3× Hubble)
18 hexagonal segments
Wavelength Range 5 - 28 µm (factor of 5)
www.jwst.nasa.gov
Main objectives of the JWST mission :
Studying the formation and evolution of galaxies
Understanding formation of stars and exoplanetary system
M.A HADJ-YOUCEF (L2S & IAS) 26th EUSIPCO September 06, 2018 2 / 27

3. Instruments on board the JWST
Imager Spectrometer
Spatial Res.
Spectral Res.
Instruments :
Wavelength Range
NIRISS 0.6 − 5 µm
NIRCam 0.6 − 5 µm
NIRSpec 1 − 5 µm
MIRI 5 − 28 µm
The Mid-IR Instrument (MIRI)
Imager [Bouchet et al. 2015]
− 9 spectral bands 5 − 28 µm
− λ/ λ ∼ 5
− 2 dimensional detector matrix
− Field of View 74 × 113 (or
672 × 1024 pixels)
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 3 / 27

4. Outline
1 Introduction
Objective and Problems
Related Works
2 Proposed Methodology
Instrument Model
Forward model
Reconstruction
3 Results
4 Conclusion
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 4 / 27

5. Input and Output of the MIRI Imager
Discrete Output :
Set of 2D
Multispectral data
Conitnuous Input :
2D+λ Object
λ (µm)5 28 λ (µm)5 28
Imaging
System
MIRI Imager
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 5 / 27

6. Introduction Objective and Problems
Outline
1 Introduction
Objective and Problems
Related Works
2 Proposed Methodology
Instrument Model
Forward model
Reconstruction
3 Results
4 Conclusion
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 6 / 27

7. Introduction Objective and Problems
Objective and Problems
Main Objective
Reconstruction of a high-resolution spatio-spectral object from exploiting a
small number of degraded multispectral data
Problems
Diffraction of light on the focal plane
→ Dependence of the optical system response on the wavelength
→ Important variation of the optical system response over [5 − 28 µm]
→ Limitation of spatial resolution of the multispectral data
Very low spectral resolution of the multispectral data
Small number of multispectral data (e.g. only nine for the MIRI Imager)
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 7 / 27

8. Introduction Related Works
Outline
1 Introduction
Objective and Problems
Related Works
2 Proposed Methodology
Instrument Model
Forward model
Reconstruction
3 Results
4 Conclusion
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 8 / 27

9. Introduction Related Works
Related Works
Methods based on a stationary/non-stationary PSF :
− Measured PSF [Guillard et al. 2010], Broadband PSF
[Geis Lutz 2010]
− PSF linear interpolation [Soulez et al. 2013], PSF approximation
[Villeneuve Carfantan 2014]
→ Neglect the spectral variation of the PSF = inaccurate response
→ Use of a predefined spectrum
Processing of the data :
− Separately band per band [Aniano et al. 2011]
→ Neglect the cross-correlation between the multispectral data
Follow up of our work in [Hadj-Youcef et al. 2017] (EUSIPCO)
→ Limitation in the reconstruction of the spectral distribution
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 9 / 27

10. Proposed Methodology
Proposed Methodology
Propositions :
Modeling the instrument response by taking into account the detector
spectral integration and considering the spectral variation of the optical
response
Reconstruction of a discrete 2D+λ object from a set of multispectral dataset
Joint processing of all the multispectral data
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 10 / 27

11. Proposed Methodology Instrument Model
Outline
1 Introduction
Objective and Problems
Related Works
2 Proposed Methodology
Instrument Model
Forward model
Reconstruction
3 Results
4 Conclusion
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 11 / 27

12. Proposed Methodology Instrument Model
Modeling the Instrument Response
Block diagram of the instrument model :
Optics Filter Detector
MIRI Imager ωpJWST Mirror h
2D+λ Object
φ
2D Data
y(p)
Spectral Variations of the PSF : h
−10 −3 3 10
arcsec
−10
−3
3
10
arcsec
λ = 6 µm λ = 12 µm λ = 21 µm
10−7 10−6 10−5 10−4 10−3 10−2 10−1
Simulated PSF using WebbPSF [Perrin et al. 2014]
Spectral Responses : ωp
5 10 15 20 25
Wavelength λ (micrometer)
0.0
0.5
p=1
p=2
p=3
p=4
p=5
p=6
p=7
p=8
p=9
Nine spectral bands of the MIRI Imager
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 12 / 27

13. Proposed Methodology Instrument Model
Complete Equation of the Model
For the p-th spectral band and a (i, j)-th pixel :
y
(p)
i,j =
R+
ωp(λ)
Ωpix R2
φ(α , β , λ)h(α − α , β − β , λ)dα dβ
2D Convolution with PSF
bsamp(α − αi, β − βj )dαdβ dλ+ n
(p)
i,j
p = 1, . . . , P, i = 1, . . . , Ni, j = 1, . . . , Nj
Ni and Nj are numbers of rows and columns of the detector matrix
P is the number of bands
h is the PSF (Point Spread Function).
ωp is the spectral response of the instrument (filter + detector).
bsamp is a spatial sampling function over the pixel area Ωpix.
n
(p)
i,j is an additive noise.
Hypothesis : The non-ideal characteristics of the detector are assumed to be corrected upstream.
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 13 / 27

14. Proposed Methodology Forward model
Outline
1 Introduction
Objective and Problems
Related Works
2 Proposed Methodology
Instrument Model
Forward model
Reconstruction
3 Results
4 Conclusion
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 14 / 27

15. Proposed Methodology Forward model
Object Model
Spectral distribution at pixel (k, l) and P = 5
λ
φ(αk, βl, λ)
Original
Multispectral
data
Spectral
Response
Representation with a piecewise linear function :
φ(α, β, λ) =
Nλ
m=1
Nk
k=1
Nl
l=1
x
(m)
k,l bspat(α − αk, β − βl)bspec(λ)
bspat is a uniform
discretization function
and bspec a first-order
B-spline. Nλ number of
wavelengths
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 15 / 27

16. Proposed Methodology Forward model
Object Model
Spectral distribution at pixel (k, l) and P = 5
λ
φ(αk, βl, λ)
Broadband
Original
Multispectral
data
Spectral
Response
Representation with a piecewise linear function :
φ(α, β, λ) =
Nλ
m=1
Nk
k=1
Nl
l=1
x
(m)
k,l bspat(α − αk, β − βl)bspec(λ)
bspat is a uniform
discretization function
and bspec a first-order
B-spline. Nλ number of
wavelengths
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 15 / 27

17. Proposed Methodology Forward model
Object Model
Spectral distribution at pixel (k, l) and P = 5
λ1 λ3λ2 λλNλ
P...
x
(3)
k,l
x
(1)
k,l
x
(2)
k,l
x
(Nλ)
k,l
φ(αk, βl, λ)
Proposed
Broadband
Original
Multispectral
data
Spectral
Response
Representation with a piecewise linear function :
φ(α, β, λ) =
Nλ
m=1
Nk
k=1
Nl
l=1
x
(m)
k,l bspat(α − αk, β − βl)bspec(λ)
bspat is a uniform
discretization function
and bspec a first-order
B-spline. Nλ number of
wavelengths
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 15 / 27

18. Proposed Methodology Forward model
Forward Model
By substituting the object model in the instrument model we obtain :
y
(p)
i,j =
Nλ
m=1


Nk
k=1
Nl
l=1
Hp,m
i,j;k,l x
(m)
k,l

 + n
(p)
i,j , p = 1, . . . , P
Hp,m
i,j;k,l =
Ωpix
R+
ωp(λ)h(α, β, λ)bspec(λ) dλ ∗
α,β
bspat(α − αk, β − βl)
bsamp(α − αi, β − βj) dαdβ
Or in a vector-matrix representation :
y(p)
=
Nλ
m=1
Hp,m
x(m)
+ n(p)
, p = 1, . . . , P
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 16 / 27

19. Proposed Methodology Forward model
Forward Model
By joint processing of all multispectral data :






y(1)
y(2)
...
y(P )






y
=





H1,1 H1,2 · · · H1,Nλ
H2,1 H2,2 · · · H2,Nλ
...
...
...
...
HP,1 HP,2 · · · HP,Nλ





H






x(1)
x(2)
...
x(Nλ)






x
+






n(1)
n(2)
...
n(P )






n
y = Hx + n
Ill-posed problem :
The block-matrix H ∈ RP NkNl × NλNkNl
is a non-Toeplitz form and
ill-conditioned → unstable solution.
Correction of the ill-posedness :
Enforcing prior knowledge about the solution (e.g. smoothness).
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 17 / 27

20. Proposed Methodology Reconstruction
Outline
1 Introduction
Objective and Problems
Related Works
2 Proposed Methodology
Instrument Model
Forward model
Reconstruction
3 Results
4 Conclusion
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 18 / 27

21. Proposed Methodology Reconstruction
Regularized Least-Squares
Convex Minimization :
ˆx = argmin
x
J (x)
where the objective function is
J (x) = y − Hx
2
2
Q(x,y)
Data fidelity
+



µspat Dspatx
2
2
Rspat(x)
+µspec Dspecx
2
2
Rspec(x)



Quadratic Regularizations
Dspat and Dspec are 2D and 1D finite difference operator along the spatial and spectral
dimensions to enforce smoothness. µspat and µspec are regularization parameters.
Solution of the Problem : J is linear and differentiable
ˆx = HT
H + µspatDT
spatDspat + µspecDT
specDspec
−1
HT
y.
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 19 / 27

22. Proposed Methodology Reconstruction
Computation of the Solution
Computation of the solution requires the inversion of the Hessian matrix :
Q−1
= HT
H + µspatDT
spatDspat + µspecDT
specDspec
−1
Computation limit ! Q ∈ RNλNkNl × NλNkNl
is a high-dimensional block-matrix,
e.g. 3932160 × 3932160 (Nk = Nl = 256, Nλ = 60)
→ heavy to inverse and requires a very large memory.
Proposition :
Computation of the solution iteratively without matrix inversion using an
optimization algorithm such as the Conjugated gradient algorithm.
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 20 / 27

23. Proposed Methodology Reconstruction
Conjugate Gradient Algorithm [Shewchuk 1994]
Input H, y, Q
Initialization : ˆx0 = 0, r0 = d0 = HT
y − Qˆx0
for n = 0 : Niter do
Convergence parameter
an ←
rT
n rn
dT
n Qdn
Computation of the next iteration
ˆxn+1 ← ˆxn − an dn
Conjugate directions
rn+1 ← rn + anQdn
bn ←
rT
n+1rn+1
rT
n rn
dn+1 ← rn+1 + bn+1dn
return ˆx
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 21 / 27

24. Results
Setup of the Experiment
The HorseHead nebula [Abergel et al. 2003] :
(a) Visible
(b) Near-Infrared
Size of the cube :


Nk = 256
Nl = 256
Nλ = 60
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 22 / 27

25. Results
Simulation Results
Multispectral data of the JWST/MIRI Imager
Additive white Gaussian
noise (σn) of SNR=30 dB
SNR = 10 log10
1
N
y 2
2
σ2
n
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 23 / 27

26. Results
Reconstruction Results
Spatial distribution at λ = 7.8, 16, 21 µm with Nλ = 60
Reconstruction error :
Error =
xorig − xrec 2
xorig 2
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 24 / 27

27. Results
Reconstruction Results
Spectral Distribution at pixel (127, 100)
Proposed
Broadband
Original
Pixel =(127,100)
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 25 / 27

28. Conclusion
Conclusion
− Increasing the spatial and spectral resolutions of the reconstructed object compared to
conventional approaches.
− Proposition of a model for the JWST/MIRI Imager with a non-stationary response
(spectral variant PSF, spectral integration over broad bands).
− Representing the object with a piecewise linear function
− Processing jointly the multispectral data from different bands
− Reconstruction of a spatio-spectral object from a small number of low resolution
multispectral data.
Perspectives
− Compute the solution in the Fourier space for faster calculation
− Explore other object representation (e.g. linear mixing model)
− Enforce prior information for objects with different spatial and spectral distributions (e.g.
sparse, piecewise constant, . . .)
− Test on a real spatio-spectral object
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 26 / 27

29. Conclusion
Thank you for your attention.
M.A HADJ-YOUCEF (L2S IAS) 26th EUSIPCO September 06, 2018 27 / 27

30. References I
[Abergel et al. 2003] A Abergel, D Teyssier, JP Bernard, F Boulanger, A Coulais, D Fosse, E Falgarone, M Gerin,
M Perault, J-L Pugetet al.
ISOCAM and molecular observations of the edge of the Horsehead nebula.
Astronomy Astrophysics, vol. 410, no. 2, pages 577–585, 2003.
[Aniano et al. 2011] G Aniano, BT Draine, KD Gordon and K Sandstrom.
Common-resolution convolution kernels for space-and ground-based telescopes.
Publications of the Astronomical Society of the Pacific, vol. 123, no. 908, page 1218, 2011.
[Bouchet et al. 2015] Patrice Bouchet, Macarena García-Marín, P-O Lagage, Jérome Amiaux, J-L Auguéres, Eva
Bauwens, JADL Blommaert, CH Chen, ÖH Detre, Dan Dickenet al.
The Mid-Infrared Instrument for the James Webb Space Telescope, III : MIRIM, The MIRI Imager.
Publications of the Astronomical Society of the Pacific, vol. 127, no. 953, page 612, 2015.
[Geis Lutz 2010] N Geis and D Lutz.
Herschel/PACS modelled point-spread functions, 2010.
[Guillard et al. 2010] Pierre Guillard, Thomas Rodet, S Ronayette, J Amiaux, Alain Abergel, V Moreau,
JL Augueres, A Bensalem, T Orduna, C Nehméet al.
Optical performance of the JWST/MIRI flight model : characterization of the point spread function at high
resolution.
In SPIE Astronomical Telescopes+ Instrumentation, pages 77310J–77310J. International Society for Optics
and Photonics, 2010.

31. References II
[Hadj-Youcef et al. 2017] MA Hadj-Youcef, François Orieux, Aurélia Fraysse and Alain Abergel.
Restoration from multispectral blurred data with non-stationary instrument response.
In Signal Processing Conference (EUSIPCO), 2017 25th European, pages 503–507. IEEE, 2017.
[Perrin et al. 2014] Marshall D Perrin, Anand Sivaramakrishnan, Charles-Philippe Lajoie, Erin Elliott, Laurent
Pueyo, Swara Ravindranath and Loïc Albert.
Updated point spread function simulations for JWST with WebbPSF.
In Space Telescopes and Instrumentation 2014 : Optical, Infrared, and Millimeter Wave, volume 9143, page
91433X. International Society for Optics and Photonics, 2014.
[Shewchuk 1994] Jonathan Richard Shewchuk.
An introduction to the conjugate gradient method without the agonizing pain, 1994.
[Soulez et al. 2013] Ferréol Soulez, Eric Thiébaut and Loıc Denis.
Restoration of hyperspectral astronomical data with spectrally varying blur.
EAS Publications Series, vol. 59, pages 403–416, 2013.
[Villeneuve Carfantan 2014] E. Villeneuve and H. Carfantan.
Nonlinear Deconvolution of Hyperspectral Data With MCMC for Studying the Kinematics of Galaxies.
IEEE Transactions on Image Processing, vol. 23, no. 10, pages 4322–4335, Oct 2014.