This document discusses near-optimal sensor placement for linear inverse problems. It introduces the concept of using sensors to measure physical fields and describes how inverse problems aim to estimate parameters of interest from sensor measurements. It presents the FrameSense algorithm, which uses a greedy approach to minimize frame potential as a proxy for minimizing mean squared error in sensor placement. FrameSense provides near-optimal sensor placement for linear inverse problems in polynomial time. As an example application, the document describes how FrameSense can be used for optimal placement of temperature sensors on a microprocessor to reconstruct thermal maps from sparse measurements.

1. Juri Ranieri
Joint work with: Dr. Amina Chebira, Prof. Martin Vetterli, Prof. D. Atienza,
Dr. Z. Chen, I. Dokmanic, A. Vincenzi, R. Zhang
Near-optimal sensor placement for linear
inverse problems

2. EPFL, March 25th 2015
Sensing the real world
2
We experience the surrounding environment through sensors.
We have a set of natural sensors, i.e., eyes, ears, nose...

3. EPFL, March 25th 2015
Sensing the real world
2
We experience the surrounding environment through sensors.
We have a set of natural sensors, i.e., eyes, ears, nose...
Tech devices are equipped with many sensors, providing
an incredible amount of information about the real world.

4. EPFL, March 25th 2015
Inverse problems
3
We have access to an incredible amount of data.
How can we use it?
• Provide it to the end-user as measured,
• Store it on a server for future use,
• Use it to estimate other parameters of interest.

5. EPFL, March 25th 2015
Inverse problems
3
We have access to an incredible amount of data.
How can we use it?
• Provide it to the end-user as measured,
• Store it on a server for future use,
• Use it to estimate other parameters of interest.
Parameters
Measurements
Physical
model

6. EPFL, March 25th 2015
Inverse problems
3
We have access to an incredible amount of data.
How can we use it?
• Provide it to the end-user as measured,
• Store it on a server for future use,
• Use it to estimate other parameters of interest.
Inverse
problem
Parameters
Measurements
Physical
model

7. EPFL, March 25th 2015
Inverse problems
3
We have access to an incredible amount of data.
How can we use it?
• Provide it to the end-user as measured,
• Store it on a server for future use,
• Use it to estimate other parameters of interest.
Inverse
problem
Inverse problem are variegated.
Signal processing problems are inverse problems.
Parameters
Measurements
Physical
model

8. EPFL, March 25th 2015
A discrete model for physical fields
4
We consider a discretization of the physical field:
Environmental sensing
Pollution
IC temperature

9. EPFL, March 25th 2015
A discrete model for physical fields
4
We consider a discretization of the physical field:
f
Environmental sensing
Pollution
IC temperature

10. EPFL, March 25th 2015
A discrete model for physical fields
4
We consider a discretization of the physical field:
f
Environmental sensing
Pollution
IC temperature

11. EPFL, March 25th 2015
Solving a linear inverse problem
5
• Source localization.
• Data interpolation (low-dim representation).
• Boundary estimation.
• Parameter estimation.

12. EPFL, March 25th 2015
Solving a linear inverse problem
5
We aim at precisely estimating .↵
• Source localization.
• Data interpolation (low-dim representation).
• Boundary estimation.
• Parameter estimation.

13. EPFL, March 25th 2015
Sensing is expensive
6
Sensing is generally expensive and maybe technically difficult.
Where do we place the sensors to get the maximum information?

14. EPFL, March 25th 2015
Where do we place the sensors?
7

15. EPFL, March 25th 2015
Where do we place the sensors?
7

16. EPFL, March 25th 2015
Where do we place the sensors?
7
↵
Sensor placement finds .
pick rows!

17. EPFL, March 25th 2015
Where do we place the sensors?
7
↵
Sensor placement finds .
LProblem: choose the placement to minimize the MSE of .
pick rows!

18. EPFL, March 25th 2015
Where do we place the sensors?
7
↵
Sensor placement finds .
Subset selection: NP-hard! [Das 2008]
LProblem: choose the placement to minimize the MSE of .
pick rows!

19. EPFL, March 25th 2015
Classic approximation strategy: go greedy!
8
Iteration 0
Iteration 1
Iteration 2
Optimal
Greedy
Greedy algorithms:
at each iteration, pick the best local choice.
• No guarantees about the distance
between the greedy and the global
optimal solution
• Polynomial time (if the cost function
can be efficiently computed)

20. EPFL, March 25th 2015
Classic approximation strategy: go greedy!
8
Can we optimize directly the MSE?
MSE greedy minimization is usually inefficient [Das 2008] and slow.
Iteration 0
Iteration 1
Iteration 2
Optimal
Greedy
Greedy algorithms:
at each iteration, pick the best local choice.
• No guarantees about the distance
between the greedy and the global
optimal solution
• Polynomial time (if the cost function
can be efficiently computed)

21. EPFL, March 25th 2015
Frame Potential as a proxy of the MSE
9
FP( L) =
X
i,j2L
|h i, ji|2
Frame Potential (FP) as a cost function: .

22. EPFL, March 25th 2015
Frame Potential as a proxy of the MSE
9
FP( L) =
X
i,j2L
|h i, ji|2
Frame Potential (FP) as a cost function: .
• FP is a measure of the closeness to orthogonality,
• The minimizers of the FP are UN tight frames [Casazza 2009],
• Minimizing the FP induces a minimization of the MSE.

23. EPFL, March 25th 2015
Frame Potential as a proxy of the MSE
9
FP( L) =
X
i,j2L
|h i, ji|2
Frame Potential (FP) as a cost function: .
Physical interpretation [Fickus 2006]:
vectors repulse each other like
electrons under the Coulomb force.
• FP is a measure of the closeness to orthogonality,
• The minimizers of the FP are UN tight frames [Casazza 2009],
• Minimizing the FP induces a minimization of the MSE.

24. EPFL, March 25th 2015
Frame Potential as a proxy of the MSE
9
FP( L) =
X
i,j2L
|h i, ji|2
Frame Potential (FP) as a cost function: .
Physical interpretation [Fickus 2006]:
vectors repulse each other like
electrons under the Coulomb force.
• FP is a measure of the closeness to orthogonality,
• The minimizers of the FP are UN tight frames [Casazza 2009],
• Minimizing the FP induces a minimization of the MSE.

25. EPFL, March 25th 2015 10
Theorem [Nemhauser 1978]: Consider a greedy
algorithm maximizing a submodular, normalized,
monotonically increasing set function .
Then, the greedy solution is near-optimal:
g(·)
g(Sopt) g(Sgreedy)
✓
1
1
e
◆
g(Sopt)
• Submodularity ~ concept of diminishing returns.
Greedy algorithms and submodularity

26. EPFL, March 25th 2015 10
Theorem [Nemhauser 1978]: Consider a greedy
algorithm maximizing a submodular, normalized,
monotonically increasing set function .
Then, the greedy solution is near-optimal:
g(·)
g(Sopt) g(Sgreedy)
✓
1
1
e
◆
g(Sopt)
• Submodularity ~ concept of diminishing returns.
• Frame Potential is submodular
Greedy algorithms and submodularity

27. EPFL, March 25th 2015
FrameSense
11
Greedy worst-out sensor selection:
• At the k-th iteration, we remove the row maximizing the FP of ,
• After iterations, the sensor placement is .L = SN L

28. EPFL, March 25th 2015
FrameSense is near-optimal w.r.t. MSE
12
Our strategy:
• FP is submodular,
• FrameSense is near-optimal w.r.t. FP,
• We derive LB and UB of the MSE w.r.t. FP,
• FrameSense is near-optimal w.r.t. MSE.

29. EPFL, March 25th 2015
FrameSense is near-optimal w.r.t. MSE
12
Our strategy:
• FP is submodular,
• FrameSense is near-optimal w.r.t. FP,
• We derive LB and UB of the MSE w.r.t. FP,
• FrameSense is near-optimal w.r.t. MSE.
and are the greedy and the optimal solution, respectively.
Theorem [Ranieri C. V. 2013]:
Given and sensors, FrameSense is
near-optimal w.r.t. MSE (under certain conditions on ):
,
where the approximation factor depends on the spectrum and
the norms of the rows of .

30. EPFL, March 25th 2015
FrameSense is near-optimal w.r.t. MSE
12
Our strategy:
• FP is submodular,
• FrameSense is near-optimal w.r.t. FP,
• We derive LB and UB of the MSE w.r.t. FP,
• FrameSense is near-optimal w.r.t. MSE.
and are the greedy and the optimal solution, respectively.
Theorem [Ranieri C. V. 2013]:
Given and sensors, FrameSense is
near-optimal w.r.t. MSE (under certain conditions on ):
,
where the approximation factor depends on the spectrum and
the norms of the rows of .

31. −5 −4 −3 −2 −1 0 1
15
20
25
30
35
40
Computational time [log s]
NormalizedMSE[dB]
Random tight frames
FrameSense
Determinant
Mutual Information
MSE
Random
Convex Opt. Method [6]
EPFL, March 25th 2015
Numerical experiment on synthetic data
13

32. EPFL, March 25th 2015
Proposed algorithm
14
FrameSense (polytime, no heuristics, guarantees):
• Greedy algorithm optimizing the Frame Potential (FP),
• Near-optimal w.r.t. the MSE,
• State-of-the-art performance w.r.t. the MSE,
• Low computational complexity.

33. One of the applications where FrameSense shines...
Sensing the temperature of a processor

34. EPFL, March 25th 2015
Application: temperature sensing
16
A modern 8 core microprocessor

35. EPFL, March 25th 2015
Application: temperature sensing
16
A modern 8 core microprocessor

36. EPFL, March 25th 2015
Application: temperature sensing
16
A modern 8 core microprocessor
• Thermal stress: failures, reduced performance, increased power
consumption, mechanical stress.
• Temperature information is desirable to optimize workload.
• Temperature cannot be sensed everywhere.

37. EPFL, March 25th 2015
Problem statement
17
Objectives:
• Design an algorithm to recover the entire
thermal map from few measurements.
• Design a sensor placement algorithm to
minimize the reconstruction error.

38. EPFL, March 25th 2015
Problem statement
17
Given: a set of thermal distributions,
representing the workload of the processor.
, , . . . . ,
{ }
Objectives:
• Design an algorithm to recover the entire
thermal map from few measurements.
• Design a sensor placement algorithm to
minimize the reconstruction error.

39. EPFL, March 25th 2015
Low-dimensional linear model
18
We learn using PCA [Ranieri V.C.A.V. 2012]
↵1 +↵2 +↵3
+↵4 + . . . + K
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalue

40. EPFL, March 25th 2015
Low-dimensional linear model
18
We learn using PCA [Ranieri V.C.A.V. 2012]
↵1 +↵2 +↵3
+↵4 + . . . + K
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalue
Recover the parameters from
few measurements to recover
the thermal map.

41. EPFL, March 25th 2015
Low-dimensional linear model
18
We learn using PCA [Ranieri V.C.A.V. 2012]
↵1 +↵2 +↵3
+↵4 + . . . + K
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20 25 30 35
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalues
... 5 10 15 20
10
−8
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Eigenvalue
Recover the parameters from
few measurements to recover
the thermal map.
We use FrameSense to place the sensors.
optimized given the noise level.

42. EPFL, March 25th 2015
Performance evaluation
19
Reconstruction results on the 8-cores Niagara.
5 10 15 20 25 30
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Number of Sensors L
MeanSquaredError
FrameSense + PCA
[Nowroz 2010]

43. EPFL, March 25th 2015
Performance evaluation
19
Similar results on a 64-cores STM architecture.
Reconstruction results on the 8-cores Niagara.
5 10 15 20 25 30
10
−6
10
−4
10
−2
10
0
10
2
10
4
10
6
Number of Sensors L
MeanSquaredError
FrameSense + PCA
[Nowroz 2010]

44. EPFL, March 25th 2015
Results and future work
20
FrameSense (TSP 2014):
• A greedy algorithm based on the frame potential,
• First near-optimal algorithm w.r.t. MSE,
• Computationally efficient,
• State-of-the-art performance.
Applications
• Thermal monitoring of many-core processors (DAC 2012, TCOMP 2015),
• DASS: distributed adaptive sampling scheduling (TCOMM 2014).
Extensions
• Source placement for linear forward problems,
• Union of subspaces (EUSIPCO 2014).
Future work
• Sensor optimization for control theory,
• Tomographic sensing.

45. Thanks for your attention!
Questions?