3. Exploiting Signals
• Not all signals are equal! Find a weakness then exploit.
• Shannon-Nyquist Bandlimited signals can be
sampled/reconstructed
4. Exploiting Signals
• Not all signals are equal! Find a weakness then exploit.
• Shannon-Nyquist Bandlimited signals can be
sampled/reconstructed
• Rank-constrained Optimization Low Rank signals can be
interpolated (Netflix Problem)
5. Exploiting Signals
• Not all signals are equal! Find a weakness then exploit.
• Shannon-Nyquist Bandlimited signals can be
sampled/reconstructed
• Rank-constrained Optimization Low Rank signals can be
interpolated. (Netflix Problem)
• Compressed Sensing Sparse signals can be undersampled and
recovered.
6. Outline of this talk.
• Compressed Sensing overview.
• Very brief explanation on the why and how of Compressed Sensing.
• ‘Apps’ that use compressed sensing.
• Practical strategies for implementation (e.g. pseudocode, libraries).
7. Motivation: JPEG Compression
Our visual system is less sensitive to high
(spatial) frequency detail. Can we throw away
these frequencies and retain a similar image?
This is the intuition behind JPEG.
Spatial Frequency E.g.:
High Hair, Blades of Grass, etc.
Low Sky, Skin, etc.
Compressed Sensing: If we are going to throw
away stuff … why spend time acquiring it?
10. 1D Implementation in L1Magic
Step 1: The original signal and its Fourier Transform.
Original Signal (N = 256)
Spectrum
11. Implementation in L1Magic
Step 2: The subsampled signal
Red Entries (80 samples) are observed.
Blue Entries (176 samples) must be
recovered.
That means we observe only 30% of the
original signal.
20. Inverse Problem
Example 1: Sinc Interpolation. Given the Data (a sufficiently sampled
signal), how can we obtain the original signal?
Example 2: Blurry Photos. Given a Blurry Photo, from a Camera, how can
we go back to the original, sharp image?
Example 3: Given a discrete time signal, how can we obtain its discrete
spectrum? **DFT problem is a Linear Inverse Problem
22. Solving the DFT Problem via
Optimization
Done?
Loss Function
23. Solving the DFT Problem via
Optimization
Done?
PseudoInverse:
Minimize MSE
24. Constraining our Solution via
Regularization
We can go beyond loss function,
e.g., Tikhonov Regularization
Additional Term allows for some prior on original signal. For
instance if Tikhonov Matrix is a first order difference, then you
are biasing x toward smooth solutions.
Linked to the Lagrange problem, as well as Maximum A
Posteriori from probability, and Weiner filter from Sig proc.
26. Compressed Sensing Structure
• Underdetermined system. y=Ax.
•
•
•
•
Y is m-dimensional sampled vector
A is mxn matrix
X is n-dimensional original vector.
And m << n
y
A
x
27. Simply Solving y=Ax not good enough
• This gives you an affine space with many solutions to y=Ax.
• So we must constrain our problem to look for the sparse solution to
y=Ax.
29. Occams Razor
Occam's Razor: among otherwise equal explanations,
the simplest is best
CS Occam's Razor: among otherwise equal solutions, the sparsest is best
Unfortunately, this optimization is not tractable
33. RIP/Spark/Coherence
• The sensing matrix A must be carefully chosen.
• For compressed sensing to work, the matrix A must satisfy the
Restricted Isometry Property (RIP):
• Calculating RIP is NP-hard. We can work with easier quantities than
the RIP, such as spark and mutual incoherence.
36. App2: Single-Pixel THz imaging.
Design Advantage: CS allows for
single-pixel THz sensors, which are
much easier to fabricate than pixel
array.
In general, you can buy amazing
things at single-pixel level, e.g.,
picosecond detectors, thermal IR
sensor, etc.
Chan et al. Applied Physics
2008
37. App2: Monitoring Breathing via
Smartphone
Very similar to
OMP, DFT
formulation.
Basically finding
sparse spectral
components that
characterize the
audio signal of
breathing.
Design Advantage: Compressed sensing
allows for low power acquisition and
reduced streaming.
Oletic, Skrapec, and Bilas MobiHealth 2012
38. App4: Biometrics … Face Recognition
Design Advantage: Using
compressed sensing to handle
the small sample size problem.
Before, the number of samples in
the database Is less than the
degrees of freedom of each
sample.
39. App5: Fast MRI
Design Advantage:
Less samples means
less time for an MRI
Scan, which means
less time a sick or
disabled patient lies
in the scanner.
41. App6: Compressive Sensing of High
Speed Periodic Videos
Design Advantage: Exploit
sparsity of Periodic Videos to
obtain a high speed video
without using a high speed
camera.
Veeraraghavan A, Reddy
D, Raskar R. IEEE PAMI
42. App7: Compressive Light Field
Photography
Marwah, Wetzstein, Bando, Ra
skar. ACM SIGGRAPH 2013.
Design Advantage:
Obtain High-Resolution
Light Field photos by
placing a coded mask in
front of the sensor.
43. App8: Sparsity-Induced Time of Flight
Cameras
Kadambi et al. ACM SIGGRAPH Asia
E.g., Light Sweep Movies from Refael’s talk last week. Goal is to
obtain a well-conditioned deconvolution problem.
Design Advantage: Deconvolve to obtain
bounces of light and construct a light
sweep video.
44. App9 YOUR App!
Design Advantage: <insert here>
45. Practical Strategies
Many libraries are available for C++/Matlab/etc.
• Recommended: L1magic (http://users.ece.gatech.edu/~justin/l1magic/)
• SPGL1 (http://www.cs.ubc.ca/~mpf/spgl1/)
• CVX (http://cvxr.com/cvx/)
• On phone, nothing exists yet, but you can use Efficient Java Matrix Library
(EJML) to implement solely in linear algebra.
46. L1Magic Pseudocode
For 1D signal.
x = original_signal;
R = randn(m,n);
A = orth(R’)’;
// read in your original signal.
// create a random matrix of dimension mxn
// sensing matrix with orthogonal columns.
y = A*x;
// create subsampled signal y of only m entries.
X0 = A’*y;
// Initial guess by taking matrix inverse.
x_hat = l1eq_pd(x0, A, [], y, 1e-3); // run l1 solver.
norm(x_hat – x);
// error; should be zero in ideal case
47. Take-home Messages
• Opportunity to integrate cutting-edge mathematical techniques into
your camera apps.
• Compressed Sensing is lightweight in terms of coding. The key is
correctly identifying the sparsity in your engineering problem.
• Not all signals are equal… find a weakness, e.g., sparsity/rank, and
exploit it.
• Exploit in Hardware, Exploit in Software…