Professor Gonzalo R. Arce gave a lecture on "Compressed sensing in spectral imaging" in the Distinguished Lecturer Series - Leon The Mathematician.
...

Professor Gonzalo R. Arce gave a lecture on "Compressed sensing in spectral imaging" in the Distinguished Lecturer Series - Leon The Mathematician.

More Information available at:
http://goo.gl/satkf

Compressed Sensing In Spectral Imaging Presentation Transcript

Introduction to Compressive Sensing Compressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI) Compressive Spectral Imaging Gonzalo R. Arce Department of Electrical and Computer Engineering University of Delaware Email:arce@ece.udel.edu Distinguished Lecture Series Aristotle University of Thessaloniki October 19th - 2010 Gonzalo R. Arce Compressive Spectral Imaging -1

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Outline Introduction to Compressive Sensing Sparsity and ℓ1 Norm Incoherent Sampling Sparse Signal Recovery Compressive Spectral Imaging Single Shot CASSI System Spectral Selectivity in (CASSI) Random Convolution SSI (RCSSI) Low-rank Anomaly Recovery in (CASSI) Gonzalo R. Arce Compressive Spectral Imaging -2

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryTraditional signal sampling and signal compression.Nyquist sampling rate gives exact reconstruction. Pessimistic for some types of signals! Gonzalo R. Arce Compressive Spectral Imaging -3

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoverySampling and Compression Transform data and keep important coefﬁcients. Lots of work to then throw away majority of data!. e.g. JPEG 2000 Lossy Compression: A digital camera can take millions of pixels but the picture is encoded on a few hundred of kilobytes. Gonzalo R. Arce Compressive Spectral Imaging -4

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryProblem: Recent applications require a very large number ofsamples: Higher resolution in medical imaging devices, cameras, etc. Spectral imaging, confocal microscopy, radar arrays, etc. y λ x Spectral Imaging Medical Imaging Gonzalo R. Arce Compressive Spectral Imaging -5

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryFundamentals of Compressive Sensing Donoho † , Candès ‡ , Romberg and Tao, discovered important results on the minimum number of data needed to reconstruct a signal Compressive Sensing (CS) uniﬁes sensing and compression into a single task Minimum number of samples to reconstruct a signal depends on its sparsity rather than its bandwidth. † D. Donoho. "Compressive Sensing". IEEE Trans. on Information Theory. Vol.52(2), pp.5406-5425, Dec.2006. ‡ E. Candès, J. Romberg and T. Tao. "Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information". IEEE Trans. on Information Theory. Vol.52(4), pp.1289-1306, Apr.2006. Gonzalo R. Arce Compressive Spectral Imaging -6

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoverySparsity Signal sparsity critical to CS Plays roughly the same role in CS that bandwidth plays in Shannon-Nyquist theory A signal x ∈ RN is S-sparse on the basis Ψ if x can be represented by a linear combination of S vectors of Ψ as x = Ψα with S ≪ N At most S non-zero components x Ψ α Gonzalo R. Arce Compressive Spectral Imaging -7

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryThe ℓ1 Norm and Sparsity Sparsity of x is measured by its number of non-zero elements, the ℓ0 norm x 0 = #{i : x(i) = 0} The ℓ1 norm can be used to measure sparsity of x x 1 = |x(i)| i The ℓ2 norm is not effective in measuring sparsity of x x 2 =( |x(i)|2 )1/2 i The ℓ0 and ℓ1 norms promote sparsity Gonzalo R. Arce Compressive Spectral Imaging -8

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryWhy ℓ1 Norm Promotes Sparsity? Given two N -dimensional signals: x1 = (1, 0, ..., 0) → "Spike" signal √ √ √ x2 = (1/ N , 1/ N , ..., 1/ N ) → "Comb" signal x 2 x1 and x2 have the same ℓ2 norm: x1 2 = 1 and x2 2 = 1. x 1 However, x1 √ 1 = 1 and x2 1 = N . Gonzalo R. Arce Compressive Spectral Imaging -9

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryCompressive Measurements Measurements in CS are different than samples taken in traditional A/D converters. The signal x is acquired as a series of non-adaptive inner products of different waveforms {φ1 , φ2 , ..., φM } yk =< φk , x >; k = 1, ..., M ; with M ≪ N y Φ x Mx1 MxN Measurements Sampling Operator Nx1 Sparse Signal Gonzalo R. Arce Compressive Spectral Imaging -10

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryRecoverability yk =< φk , x >; k = 1, ..., M ; with M ≪ N Recovering x from yk is an inverse problem. Need to solve an under determined system of equations y = Φx. Inﬁnitely solutions for the system since M ≪ N . Amplitude Amplitude Original sparse signal Compressed measurements Reconstructed signal using least-squares. Solution not sparse Gonzalo R. Arce Compressive Spectral Imaging -11

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryRecoverability: Incoherent Sampling The number of samples required to recover x from M samples depends on the mutual coherence between Φ and Ψ Mutual Coherence √ µ(Φ, Ψ) = N max{| < φk , ψ j > | : φk ∈ Rows(Φ), ψ j ∈ Columns(Ψ)}; where, ψj 2 = φk 2 =1 The coherence µ(Φ, Ψ) satisﬁes: √ 1 ≤ µ(Φ, Ψ) ≤ N Gonzalo R. Arce Compressive Spectral Imaging -12

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryRecoverability: Incoherent Sampling The random measurement matrix Φ has to be incoherent to the dictionary Ψ and x can be recovered from M samples exactly when M satisﬁes: M ≥ C · µ2 · S · log(N ), C ≥ 1 (a) (b) (a) Very sparse vector. (b) Examples of pseudorandom, incoherent test vectors φk † .† J. Romberg. "Imaging Via Compressive Sampling". IEEE Signal Processing Magazine. March,2008. Gonzalo R. Arce Compressive Spectral Imaging -13

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryCompressive Sensing Signal Reconstruction Goal: Recover signal x from measurements y Problem: Random projection Φ not full rank (ill-posed inverse problem) Solution: Exploit the sparse/compressible geometry of acquired signal x y Φ x Gonzalo R. Arce Compressive Spectral Imaging -14

Introduction to Compressive Sensing Sparsity and ℓ1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryReconstruction Algorithms Different formulations and implementations have been proposed to ﬁnd the sparsest x subject to y = Φx Those are broadly classiﬁed in: Regularization formulations (Replace combinatorial problem with convex optimization) Greedy algorithms (Iterative reﬁnement of a sparse solution) Bayesian framework (Assume prior distribution of sparse coefﬁcients) Gonzalo R. Arce Compressive Spectral Imaging -15

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Compressive Spectral Imaging Collects spatial information from across the electromagnetic spectrum. Applications, include wide-area airborne surveillance, remote sensing, and tissue spectroscopy in medicine. Gonzalo R. Arce Compressive Spectral Imaging -16

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Compressive Spectral Imaging Spectral Imaging System - Duke University†† A. Wagadarikar, R. John, R. Willett, D. Brady. "Single Disperser Design for Coded Aperture Snapshot Spectral Imaging." Applied Optics, vol.47, No.10, 2008.A. Wagadarikar and N. P. Pitsianis and X. Sun and D. J. Brady. "Video rate spectral imaging using a coded aperture snapshot spectral imager." Opt. Express, 2009. Gonzalo R. Arce Compressive Spectral Imaging -17

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot Compressive Spectral Imaging System design With linear dispersion: f1 (x, y; λ) = f0 (x, y; λ)T (x, y) f2 (x, y; λ) = δ(x′ − [x + α(λ − λc )]δ(y ′ − y)f1 (x′ , y ′ ; λ))dx′ dy ′ = δ(x′ − [x + α(λ − λc )]δ(y ′ − y)f0 (x′ , y ′ ; λ)T (x, y))dx′ dy ′ = f0 (x + α(λ − λc ), y; λ)T (x + α(λ − λc ), y) Gonzalo R. Arce Compressive Spectral Imaging -18

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot Compressive Spectral Imaging Experimental results from Duke University Original Image Reconstructed image cube of size:128x128x128. Measurements Spatial content of the scene in each of 28 spectral channels between 540 and 640nm.† A. Wagadarikar, R. John, R. Willett, D. Brady. "Single Disperser Design for Coded Aperture Snapshot Spectral Imaging." Applied Optics, vol.47, No.10, 2008. Gonzalo R. Arce Compressive Spectral Imaging -19

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot Compressive Spectral Imaging Simulation results in RGB Original Image Measurements R Reconstructed ¡ Image Gonzalo R. Arce Compressive Spectral Imaging -20

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System Object with spectral information only in (xo , yo ) Only two spectral component are present in the object Gonzalo R. Arce Compressive Spectral Imaging -21

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System Object with spectral information only in (xo , yo ) Gonzalo R. Arce Compressive Spectral Imaging -22

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System One pixel in the detector has information from different spectral bands and different spatial locations Gonzalo R. Arce Compressive Spectral Imaging -23

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System Each pixel in the detector has different amount of spectral information. The more compressed information, the more difﬁcult it is to reconstruct the original data cube. Gonzalo R. Arce Compressive Spectral Imaging -24

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System Each row in the data cube produces a compressed measurement totally independent in the detector. Gonzalo R. Arce Compressive Spectral Imaging -25

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System Undetermined equation system: Unknowns = N × N × M and Equations: N × (N + M − 1) Gonzalo R. Arce Compressive Spectral Imaging -26

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System Complete data cube 6 bands The dispersive element shifts each spectral band in one spatial unit In the detector appear the compressed and modulated spectral component of the object At most each pixel detector has information of six spectral components Gonzalo R. Arce Compressive Spectral Imaging -27

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot CASSI System We used the ℓ1 − ℓs reconstruction algorithm † .† S. J. Kim, K. Koh, M. Lustig, S. Boyd and D. Gorinevsky. "An interior-point method for large scale L1 regularized least squares." IEEE Journal of Selected Topics in Signal Processing, vol.1, pp. 606-617, 2007. Gonzalo R. Arce Compressive Spectral Imaging -28

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Coded Aperture Snapshot Spectral Image System(CASSI)(a) Advantages: Enables compressive spectral imag- ing Simple Low cost and complexity Limitations: Excessive compression Does not permit a controllable SNR May suffer low SNR gmn = f(m+k)nk P(m+k)n + wnm Does not permit to extract a speciﬁc k subset of spectral bands = (Hf )nm + wnm = (HW θ)nm + wnmA. Wagadarikar, R. John, R. Willett, and D. Brady. "Single disperser design for coded aperture snapshot spectral imaging." Appl. Opt., Vol.47, No.10, 2008. Gonzalo R. Arce Compressive Spectral Imaging -29

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Bands Recovery Typical example of a measurement of CASSI system. A set of bands constant spaced between them are summed to form a measurement Gonzalo R. Arce Compressive Spectral Imaging -30

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Multi-Shot CASSI System Multi-shot compressive spectral imaging system Advantages: Multi-Shot CASSI allows controllable SNR Permits to extract a hand- picked subset of bands Extend Compressive Sens- ing spectral imaging capabil- ities L gmni = fk (m, n + k − 1)Pi (m, n + k − 1) k=1 L i = fk (m, n + k − 1)Pr (m, n + k − 1)Pg (m, n + k − 1) k=1 Ye, P. et al. "Spectral Aperture Code Design for Multi-Shot Compressive Spectral Imaging". Dig. Holography and Three-Dimensional Imaging, OSA. Apr.2010. Gonzalo R. Arce Compressive Spectral Imaging -31

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Mathematical Model of CASSI System L gmni = fk (m, n + k − 1)Pi (m, n + k − 1) k=1 L i = fk (m, n + k − 1)Pr (m, n + k − 1)Pg (m, n + k − 1) k=1 where i expresses ith shot Each pattern Pi is given by, i Pi (m, n) = Pg (m, n)xPr (m, n) i 1 mod(n, R) = mod(i, R)Pg (m, n) = 0 otherwise One different code aperture is used for each shot of CASSI system Gonzalo R. Arce Compressive Spectral Imaging -32

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Code Apertures Code patterns used in multishot CASSI system Code patterns used in multishot CASSI system Gonzalo R. Arce Compressive Spectral Imaging -33

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Cube Information and Subsets of Spectral Bands Spectral axis, Spatial L bands axis, N Spectral data cube → L bands pixels R subsets of M bands each one Complete Spectral (L = RM ) Each component Data Cube of the subset is spaced by R Spatial bands of each other axis, N pixels Subset 1 M bands R R Subset 1 Subset 2 Subset 3 ... Subset R M=bands M=bands M=bands M bands Gonzalo R. Arce Compressive Spectral Imaging -34

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Cube Information and Subsets of Spectral Bands Spectral axis, Spatial L bands axis, N Spectral data cube → L bands pixels R subsets of M bands each one Complete (L = RM ) Each component Spectral of the subset is spaced by R Data Cube Spatial bands of each other axis, N R R pixels Subset 2 M bands Subset 1 Subset £ Subset ¢ ... Subset R M=bands M=bands M=bands M=bands Gonzalo R. Arce Compressive Spectral Imaging -35

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Multi-Shot CASSI System First shot and Second shot and R shot and measurement measurement measurement Gonzalo R. Arce Compressive Spectral Imaging -36

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot Multi-Shot One shot of CASSI Information of all band exists in all shots system. One high compressing measurement. First shot Second shot Third shot Reconstruction Algorithm Re-organization algorithm Reconstructed spectral data cube. Bands 1,4,7 Bands 2,5,8 Bands 3,6,9 Gonzalo R. Arce Compressive Spectral Imaging -37

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI) Multi-Shot Reorder Process R R R ′ Lgmnk = j=1 fj (m, n + j − 1)Pi (m, n + j − 1) L i First shot Second shot Third shot = j=1 fj (m, n + j − 1)Pr (m, n + j − 1)Pg (m, n + j − 1) Re-organization algorithm = mod(n+j−1,R)=mod(i,R) fk (m, n + k − 1)Pr (m, n + j − 1) = (Hk Fk )mn Bands 1,4,7 Bands 2,5,8 Bands 3,6,9 Gonzalo R. Arce Compressive Spectral Imaging -38

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI) Multi-Shot Reorder Process R R R ′ Lgmnk = j=1 fj (m, n + j − 1)Pi (m, n + j − 1) L i First shot Second shot Third shot = j=1 fj (m, n + j − 1)Pr (m, n + j − 1)Pg (m, n + j − 1) Re-organization algorithm = mod(n+j−1,R)=mod(i,R) fk (m, n + k − 1)Pr (m, n + j − 1) = (Hk Fk )mn Bands 1,4,7 Bands 2,5,8 Bands 3,6,9 Gonzalo R. Arce Compressive Spectral Imaging -39

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI) Multi-ShotRecover any of the subsetsindependentlyRecover of complete spec-tral data cube is not neces-sary Gonzalo R. Arce Compressive Spectral Imaging -40

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI) Multi-ShotHigh SNR in each re-constructionEnable to use paral-lel processingTo use one proces-sor for each indepen-dent reconstruction Gonzalo R. Arce Compressive Spectral Imaging -41

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI) Multi-ShotSingle Shot One shot of CASSI system. One high compressing measurement. Reconstruction Algorithm Reconstructed spectral data cube. Gonzalo R. Arce Compressive Spectral Imaging -42

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Multi-Shot Reconstruction Reconstructed image of one spec- tral channel in 256x256x24 data cube from multiple shot measure- ments. (a) One shot result,PSNR (a) One shot (b) 2 shots P SN R = 17.6dB (b) Two shots result,PSNR P SN R = 25.7dB (c) Eight shots result,PSNR P SN R = 29.4 (d) Original image (c) 8 shots (d) Original Gonzalo R. Arce Compressive Spectral Imaging -43

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Multi-Shot ReconstructionReconstructed image for dif-ferent spectral channels in the256x256x24 data cube fromsix shot measurements. (a) Band 1 (b) Band 13 (c) Band 8 (d) Band 20 (a) and (b) are recon- structed from the ﬁrst group of measurements (c) and (d) are recon- structed from the second group of measurements Gonzalo R. Arce Compressive Spectral Imaging -44

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution Spectral Imaging Gonzalo R. Arce Compressive Spectral Imaging -45

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution ImagingJ. Romberg. "Compressive Sensing by Random Convolution." SIAM Journal on Imaging Science, July,2008. Gonzalo R. Arce Compressive Spectral Imaging -46

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution Imaging Random Convolution Circularly convolve signal x ∈ Rn with a pulse h ∈ Rn , then subsample. The pulse is random, global, and broadband in that its energy is distributed uniformly across the discrete spectrum. x ∗ h = Hx where H = n−1/2 F ∗ ΣF Ft,ω = e−j2π(t−1)(ω−1)/n , 1 ≤ t, ω ≤ n Σ as a diagonal matrix whose non-zero entries are the Fourier transform of h. Gonzalo R. Arce Compressive Spectral Imaging -47

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution σ1 0 · · · 0 σ2 · · · Σ= . . .. . . σn ω=1 : σ1 ∼ ±1 with equal probability, 2 ≤ ω < n/2 + 1 : σω = ejθω , where θω ∼ Uniform([0, 2π]), ω = n/2 + 1 : σn/2+1 ∼ ±1 with equal probability, n/2 + 2 ≤ ω ≤ n : ∗ σω = σn−ω+2 , the conjugate of σn−ω+2 . Gonzalo R. Arce Compressive Spectral Imaging -48

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution H The effect of H on a signal x can be broken down into a discrete Fourier transform, followed by a randomization of the phase (with constraints that keep the entries of H real), followed by an inverse discrete Fourier transform. Since F F ∗ = F ∗ F = nI and ΣΣ∗ = I, H ∗ H = n−1 F ∗ Σ∗ F F ∗ ΣF = nI So convolution with h as a transformation into a random orthobasis. Gonzalo R. Arce Compressive Spectral Imaging -49

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Main Result H will not change the magnitude of the Fourier transform, so signals which are concentrated in frequency will remain concentrated and signals which are spread out will stay spread out. The randomness of Σ will make it highly probable that a signal which is concentrated in time will not remain so after H is applied. Gonzalo R. Arce Compressive Spectral Imaging -51

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Main Result (a) A signal x consisting of a single Daubechies-8 wavelet. (b) Magnitude of the Fourier transform F x. (c) Inverse Fourier transform after the phase has been randomized. Although the magnitude of the Fourier transform is the same as in (b), the signal is now evenly spread out in time. J. Romberg. "Compressive Sensing by Random Convolution." SIAM Journal on Imaging Science, July,2008. Gonzalo R. Arce Compressive Spectral Imaging -52

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Fourier Optics Fourier optics imaging experiment. (a) The 256 × 256 image x. (b) The 256 × 256 image Hx. (c) The 64 × 64 image P θHx. Gonzalo R. Arce Compressive Spectral Imaging -53

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)(a) The 256 × 256 image we wish to acquire.(b) High-resolution image pixellated by averaging over 4 × 4 blocks.(c) The image restored from the pixellated version in (b), plus a set ofincoherent measurements. The incoherent measurements allow us toeffectively super-resolve the image in (b). Gonzalo R. Arce Compressive Spectral Imaging -54

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Fourier Optics a) b) c) d) e) f) Pixellated images: (a) 2 × 2. (b) 4 × 4. (c) 8 × 8. Restored from: (d) 2 × 2 pixellated version. (e) 4 × 4 pixellated version. (f) 8 × 8 pixellated version. Gonzalo R. Arce Compressive Spectral Imaging -55

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution Spectral Imaging Gonzalo R. Arce Compressive Spectral Imaging -56

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI) 20 40 60 80 100 120 20 40 60 80 100 120 Gonzalo R. Arce Compressive Spectral Imaging -57

Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI) 20 40 60 80 100 120 20 40 60 80 100 120 Gonzalo R. Arce Compressive Spectral Imaging -58

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Low-rank Anomaly Recovery in (CASSI) Spectral video analysis Video surveillance: Anomaly detection Stationary background corresponds to low-rank contribution and the moving objects corresponds to sparse data. Gonzalo R. Arce Compressive Spectral Imaging -59

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Connection Between Low-Rank Matrix Recovery andCompressed Sensing Low-rank Rank miniz. Convex Relax. Recovery min rank(X) L min L s.t. M=S+L s.t. M=S+L Compressed Rank miniz. Convex Relax. Sensing B. Recht, M. Fazel and P. Parrilo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization," SIAM Review, Aug. 2010. Gonzalo R. Arce Compressive Spectral Imaging -60

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI) Problem Description (i) Consider the video surveillance of Fk,n1,n2 ∈ RN1 ×N2 ×K , i = 1, ..., N frames. The ith scene is assumed to be composed by a stationary background L(i) and an event changing in time S(i) , (i) (i) (i) Fk,n1,n2 = Lk,n1,n2 + Sk,n1 ,n2 CASSI encodes both 2D spatial information and spectral information in a 2Dsingle measurement G(i) for i = 1, ..., N . GOAL: recover anomalies occurring in both time and spectra from a sequence of spectrally compressed video frames G(1) , G(2) , ..., G(N ) . Gonzalo R. Arce Compressive Spectral Imaging -61

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI) Recovering anomalies: Form G as the large data matrix G = [g(1) , g(2) , . . . , g(N ) ], where g(i) is the column representation of G(i) . G = L + S where L is the stationary background and S is sparse capturing the anomalies in the foreground Gonzalo R. Arce Compressive Spectral Imaging -62

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Principal Component Pursuit The matrix G is decomposed into a low-rank matrix L and a sparse matrix S, such that G =L+S (1) Using Principal Component Pursuit. Principal Component Pursuit min L ∗ +λ S 1 n L ∗ = i=1 σi (L), is the nuclear norm of L. S 1 = ij Sij is the ℓ1 -norm of the matrix S E. J. Candès, X. Li, Y. Ma, and J. Wright. "Robust Principal Component Analysis?," Submitted to Journal of the ACM. 2009. Gonzalo R. Arce Compressive Spectral Imaging -63

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI) Spectral recovery of anomalies. Coded measurements in S have been biased by the background reconstruction ˆ Identify spatial location of the anomalies in S by: ˆ Filter |S| with a Weighted Median (WM) ﬁlter as (i) ˆ (i) Mn1 ,n2 = MEDIAN{Tv,w ⋄ |Sn1 +v,n2 +w | : (v, w) ∈ [−3, 3]} where T is a WM ﬁlter of size (L × L) with centered weight (L + 1)/2, and linearly decreasing weights Spectrally coded measurements of anomalies denoted by ˜ G(i) are estimated as G(i) = G(i) ⊙ U(M(i) − Th ) ˜ Th is a thresholding parameter that extracts the pixels that are most likely to be in the region of interest Gonzalo R. Arce Compressive Spectral Imaging -64

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI) ˆ ˜ Recover S(i) from G(i) by ˆ(i) = Ψ min( g(i) − HΨθ (i) s ˜ 2 2 + τ θ (i) 1 ) (2) θ s ˜ ˜ where ˆ(i) and g(i) are the column representation of S(i) ˜ and G (i) , respectively. Gonzalo R. Arce Compressive Spectral Imaging -65

Introduction to Compressive Sensing Compressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI) (video) (video) Gonzalo R. Arce Compressive Spectral Imaging -66

Introduction to Compressive Sensing Compressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI) (video) (video) Gonzalo R. Arce Compressive Spectral Imaging -67

Introduction to Compressive Sensing Compressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI) (video) (video) Gonzalo R. Arce Compressive Spectral Imaging -68

Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Summary Compressive Sensing Spectral Imaging Low-Rank Recovery ´ Euχαριστ ω ! Gonzalo R. Arce Compressive Spectral Imaging -69

1 comment11–1 of 1 previous next

Full NameComment goes here.Distinguished Lecturer Series - Leon The Mathematicianat Distinguished Lecturer Series - Leon The Mathematician More information for the lecture and the organization 'Distinguished Lecturer Series - Leon The Mathematician' available at: http://goo.gl/satkf