Oct. 4, 2016•0 likes•305 views

Report

Science

These notes include some results from frame theory. These results are useful for sparse representations and compressive sensing theory. Errors and omissions are all mine.

- 1. RESULTS ON EQUIANGULAR TIGHT FRAMES SHAILESH KUMAR Contents 1. Introduction 1 1.1. Unit norm frame 4 1.2. Dual frame and pseudo inverse 6 1.3. Analysis and Synthesis 7 1.4. Tight frames 8 1.5. Coherence and Grassmannian frames 9 1.6. Equiangular tight frames 9 2. More Properties of Frames and Dictionaries 9 3. Vandermonde Matrices 11 4. Grassmannian frames 11 4.1. Existence of Grassmannian frames 11 4.2. Optimal Grassmannian frames 12 5. Full Spark Frames 17 5.1. Vandermonde matrices 17 6. Subspace angles 17 6.1. Principal angles 18 6.2. One dimensional subspaces 19 6.3. Optimal Grassmannian Frame 20 Appendix A. Linear Algebra 21 References 21 Index 22 1. Introduction We will be concerned with ﬁnite dimensional real or complex spaces RM or CM . Most of the results will be speciﬁed in terms of ﬁeld F which is either R or C. Thus, FM means either RM or CM . Some results will be speciﬁc to either RM or CM . A general introduction to frame theory can be found in [3]. While [3] builds the theory over general Hilbert spaces, our treatment will be restricted to ﬁnite dimensional spaces. In the sequel the results marked with are potentially original. Many of them are very simple. It is possible that some of them might have appeared in some prior work in some other context. The new results are listed here for quick reference • theorem 21 • theorem 31 • theorem 35 1
- 2. 2 SHAILESH KUMAR • theorem 36 Deﬁnition 1 A ﬁnite frame ΦN M in a real or complex M-dimensional Hilbert space FM is a sequence of N ≥ M vectors {φk}N k=1, φk ∈ FM , satisfying the following condition α x 2 2 ≤ N k=1 | x, φk |2 ≤ β x 2 2, ∀ x ∈ FM (1.1) where α, β are positive constants 0 < α ≤ β called the lower and upper frame bounds. In this work, we will only work with ﬁnite dimensional spaces and ﬁnite frames. Thus, whenever we say a frame, we mean a ﬁnite frame. Deﬁnition 2 The M × N matrix Φ = φ1 . . . φN with the frame vectors as the columns is known as the frame synthesis operator. In the sequel we will use the frame symbol ΦM N and its synthesis operator Φ inter- changeably. Deﬁnition 3 The frame analysis operator is a mapping from FM to FN deﬁned as y ΦH x. (1.2) yi = x, φi . The entries in y are known as the frame coeﬃcients which are the inner product of the frame vectors with x. Clearly y 2 2 = N k=1 | x, φk |2 = ΦH x 2 2 = xH ΦΦH x. (1.3) We can rewrite (1.1) as α x 2 2 ≤ ΦH x 2 2 ≤ β x 2 2, ∀ x ∈ FM (1.4) Deﬁnition 4 The frame operator S associated with the frame {φk}N k=1 is deﬁned by Sx N k=1 x, φk φk. (1.5) We note that N k=1 x, φk φk = ΦΦH x. (1.6)
- 3. ETF 3 Thus, S = ΦΦH ∈ FM×M (1.7) Also xH ΦΦH x = xH Sx. (1.8) Theorem 1 A frame spans FM . If the frame didn’t span FM , then the lower frame bound would be 0. This means that both Φ and S are full rank. Theorem 2 The frame operator is positive deﬁnite. Proof. We note that the frame bounds can be rewritten as α x 2 2 ≤ xH Sx ≤ β x 2 2, ∀ x ∈ FM . (1.9) Since α > 0, hence xH Sx > 0 ∀ x = 0. Theorem 3 The frame analysis operator is injective. Proof. Let x1 and x2 be two vectors in FN such that ΦH x1 = ΦH x2. Then ΦH (x1 − x2) = 0. Thus, ΦH (x1 − x2) 2 2 = 0. This can happen only if x1 − x2 = 0 and the two vectors are identical. This shows that ΦH is injective. Corollary 4 The frame analysis operator is bijective between FM and its range given by R(ΦH ). Theorem 5 The upper and lower frame bounds of a frame are given by the largest and smallest eigen values of the frame operator S = ΦΦH respectively. Alternatively they are the largest and smallest (non-zero) singular values of Φ. Proof. This can be seen from the frame bounds inequality (1.9). Deﬁnition 5 The operator G = ΦH Φ is called the Gram matrix associated with Φ. This is also known as the Grammian. The (i, j)-th entry in G is the inner product of φi and φj. The (i, i)-th entry is obviously the norm-squared of i-th frame vector given by φi 2 2 = φH i φi. Theorem 6 The non-zero eigen values of the frame operator S and the Gram matrix G are identical.
- 4. 4 SHAILESH KUMAR Proof. Let λ be a (non-zero) eigen value of S = ΦΦH . Then, there exists x = 0 such that ΦΦH x = λx. Multiplying both sides by ΦH and deﬁning y = ΦH x, we obtain ΦH Φy = λy. Since x = 0 =⇒ y = 0, hence y is an eigen vector of G with the same eigen value. We note that ΦH x, y = x, Φy . (1.10) Theorem 7 The rank of G = ΦH Φ is M. Exactly M eigen values of G are positive. Rest are zero. Proof. Let x ∈ N(Φ). Then Φx = 0 =⇒ ΦH Φx = 0 =⇒ N(Φ) ⊆ N(ΦH Φ). Let x ∈ N(ΦH Φ). Then, ΦH Φx = 0 =⇒ xH ΦH Φx = 0 =⇒ Φx 2 2 = 0 =⇒ x ∈ N(Φ). Thus N(Φ) = N(ΦH Φ). Since rank of Φ is M, hence rank(G) = M. Thus exactly M of eigen values of G are positive. Deﬁnition 6 The redundancy of a frame is deﬁned as ρ = N M . An orthonormal basis (ONB) for FM has exactly M vectors. The redundancy of a frame tells us about relative number of extra vectors in a frame viz. an ONB. Deﬁnition 7 When the frame vectors {φk}N k=1 are linearly independent, then the frame is called a Riesz basis. Theorem 8 If a ﬁnite frame is a Riesz basis, then M = N. This is obvious since any set of more than M vectors in FM is linearly dependent and any set of less than M vectors cannot span FM . 1.1. Unit norm frame. Deﬁnition 8 A frame is called unit norm frame (UNF) if each frame vector satisﬁes φk 2 = 1. A unit norm frame is also known as a dictionary.
- 5. ETF 5 The diagonal entries of the Grammian for a unit norm frame are 1. Naturally for UNF, the trace of the Grammian is given by tr(G) = N. (1.11) Theorem 9 The redundancy of a unit norm frame is bounded by α ≤ N M ≤ β. (1.12) Proof. Recall that S = ΦΦH ∈ FM×M . There are exactly M eigen values of S. All of these are positive since S is positive deﬁnite. Thus tr(S) = M k=1 λi is bounded by αM = λminM ≤ M k=1 λi = tr(ΦΦH ) ≤ λmaxM = βM. But tr(ΦΦH ) = tr(ΦH Φ) = tr(G) = N. We get αM ≤ N ≤ βM. Dividing by M, we obtain the result. Theorem 10 [1] For a UNF Φ the sum of non-zero eigen values of G = ΦH Φ is N. Let the eigen values of G be ordered in by decreasing size as λ1 ≥ · · · ≥ λN . Then M k=1 λk = N. (1.13) Further, the sum of square of eigen values is bounded by M k=1 λ2 k ≥ N2 M . (1.14) The lower bound is achieved only if λk = N M for all M positive eigen values. Proof. Recall that rank(G) = M. Thus, only M of these eigen values are non-zero. We have M k=1 λk = tr(G) = N k=1 | φk, φk | = N k=1 1 = N. Now set, ek = λk − N M . Then M k=1 ek = M k=1 λk − N M = N − M N M = 0.
- 6. 6 SHAILESH KUMAR Thus, M k=1 λ2 k = M k=1 N M + ek 2 = M k=1 N2 M2 + 2N M M k=1 ek + M k=1 e2 k = N2 M + M k=1 e2 k ≥ N2 M . The lower bound is achieved only if ek = 0, thus λk = N M for k = 1, . . . , M. 1.2. Dual frame and pseudo inverse. The reconstruction of x from its frame coeﬃcients ΦH x is calculated with a pseudo-inverse of ΦH . Theorem 11 If {φk}N k=1 is a frame but not a Riesz basis, then the correspond- ing frame analysis operator ΦH admits an inﬁnite number of left inverses. Proof. We know that N(Φ) = R(ΦH )⊥ . If {φk}N k=1 is a frame but not a Riesz basis, then {φk}N k=1 is linearly dependent. Thus, there exists an a ∈ N(Φ) with a = 0. Since the restriction of ΦH to R(ΦH ) is bijective thus invertible, hence ΦH admits a left inverse. Since the restriction of the left inverse to R(ΦH )⊥ may be arbitrary, hence there is an inﬁnite number of left inverses. The Moore-Penrose pseudo-inverse is deﬁned as the left inverse that is zero on R(ΦH )⊥ . Theorem 12 If {φk}N k=1 is a frame, then S = ΦΦH is invertible and the pseudo inverse satisﬁes ΦH† = (ΦΦH )−1 Φ. (1.15) Alternatively, the right pseudo-inverse for the frame synthesis operator Φ is given by Φ† = ΦH (ΦΦH )−1 . (1.16) Proof. Since S is positive deﬁnite, it is invertible. Next, we see that ΦH† ΦH x = (ΦΦH )−1 ΦΦH x = x. (1.17) Thus, ΦH† is a left inverse. For any a ∈ R(ΦH )⊥ = N(Φ), we have Φa = 0. Thus, ΦH† a = (ΦΦH )−1 Φa = 0. Thus, it is the pseudo inverse. Similarly, we note that ΦΦ† = ΦΦH (ΦΦH )−1 = I. (1.18) Thus, Φ† is the right pseudo-inverse of Φ.
- 7. ETF 7 Deﬁnition 9 Let {φk}N k=1 be a frame. The vectors of a dual frame are deﬁned by φk = (ΦΦH )−1 φk, 1 ≤ k ≤ N. (1.19) The dual frame consists of {φk}N k=1. The dual frame synthesis operator is deﬁned by Φ φ1 . . . φN = (ΦΦH )−1 Φ = S−1 Φ. (1.20) The dual frame analysis operator is given by ΦH . Theorem 13 The dual frame operator S = S−1 . Proof. ΦΦH = (ΦΦH )−1 Φ (ΦΦH )−1 Φ H = (ΦΦH )−1 ΦΦH (ΦΦH )−1 = (ΦΦH )−1 . Thus S = S−1 . Theorem 14 Let 0 < α ≤ β be the frame bounds of Φ. Then, the frame bounds of Φ are given by 1 β and 1 α . Speciﬁcally, 1 β x 2 2 ≤ N k=1 | x, φk |2 ≤ 1 α x 2 2, ∀ x ∈ FM (1.21) Proof. Note that N k=1 | x, φk |2 = ΦH x 2 2 = xH Sx = xH S−1 x. Since S is positive deﬁnite, hence S−1 is positive deﬁnite too. The eigen values of S−1 are the inverse of eigen values of S. Thus if α is the minimum eigen value of S then 1 α is the maximum eigen value of S−1 . Similarly for β. The result follows. 1.3. Analysis and Synthesis. Theorem 15 We have x = ΦΦH x = ΦΦH x. (1.22) This can also be expressed as x = N k=1 x, φk φk = N k=1 x, φk φk. (1.23) Proof. ΦΦH x = (ΦΦH )−1 ΦΦH x = x.
- 8. 8 SHAILESH KUMAR Similarly ΦΦH x = ΦΦH (ΦΦH )−1 x = x. 1.4. Tight frames. Deﬁnition 10 When the frame bounds α and β are equal, a frame is said to be tight. Deﬁnition 11 A tight frame is called unit norm tight frame (UNTF) if each frame vector satisﬁes φk 2 = 1. Theorem 16 The redundancy of a unit norm tight frame(UNTF) is given by α = β = N M . (1.24) We can rewrite (1.1) as N M x 2 2 = ΦH x 2 2 ∀ x ∈ FM . (1.25) Note that α and β are upper and lower bounds of the eigen values of S = ΦΦH . In a tight frame, they are same, thus all eigen values are identical. Theorem 17 Let Φ be a tight frame, then S = αI where α is the one and only eigen value of S. Proof. Let S = V ΛV H be the eigen value decomposition of S. Since α = β, hence all eigen values of S are equal. Thus Λ = αI. Thus S = αV IV H = αI. It is obvious that Φ = S−1 Φ = 1 α Φ. Thus, the reconstruction formula looks like x = N k=1 x, φk φk = 1 α N k=1 x, φk φk. (1.26) Compare this with the reconstruction formula for an orthonormal basis. Except for an extra factor of 1 α which is the inverse of the frame redundancy factor, the reconstruction formula is identical to ONB. Theorem 18 Rows of the frame synthesis operator Φ of a tight frame ΦN M are orthogonal. The squared norm of each row is α. Speciﬁcally, the norm of the rows of a UNTF is N M .
- 9. ETF 9 Proof. We have S = ΦΦH = αI. Clearly the inner product of distinct rows is 0 and squared norm of each row is α. 1.5. Coherence and Grassmannian frames. A good introduction to Grassman- nian frames can be found in [4]. Deﬁnition 12 Let {φk}N k=1 be a unit norm frame with the frame synthesis operator Φ. The coherence or the maximum correlation of the frame is deﬁned as µ(Φ) max k=l | φk, φl |. (1.27) Whenever the frame is obvious from the context, we may denote the coherence simply by µ. Deﬁnition 13 Fix N ≥ M. Let Ω denote the set of all unit norm frames over FM consisting of N vectors. A frame Φ ∈ Ω is called a Grassmannian frame if µ(Φ) = inf X∈Ω {µ(X)} (1.28) i.e. the coherence of Φ is the smallest possible over all frames in Ω. One can see that the idea of Grassmannian frames follows the mini-max principle (minimize the maximum correlation). 1.6. Equiangular tight frames. In orthonormal bases, any two basis vectors are orthogonal to each other. This notion is generalized in frames through the idea of equiangular tight frames. Deﬁnition 14 A unit norm tight frame {φk}N k=1 is called equiangular if | φk, φl | = p whenever k = l (1.29) i.e. the angle between the lines spanned by any two frame vectors is same. In other words, the absolute value of the oﬀ-diagonal entries in the Gram matrix is also the same. We note that 0 ≤ p < 1 where 0 is achieved only when Φ is an orthonormal basis. Further µ(Φ) = p. We also note that | φk, φl | = | φk, −φl |. Thus, a sign change in frame vectors doesn’t make any diﬀerence. 2. More Properties of Frames and Dictionaries We have already deﬁned the coherence or maximum correlation of a dictionary.
- 10. 10 SHAILESH KUMAR Deﬁnition 15 The spark of a frame is the size of the smallest linearly de- pendent subset of frame vectors. spark(Φ) = min{ x 0 : Φx = 0, x = 0}. (2.1) Deﬁnition 16 Let {φk}N k=1 be a unit norm frame with the frame synthesis operator Φ. The minimum correlation of the frame is deﬁned as µmin(Φ) min k=l | φk, φl |. (2.2) Theorem 19 The coherence of a frame ΦN M is invariant to unitary transfor- mation. Proof. Let Ψ be an orthonormal basis for FM . Let ΨΦ be the frame constructed by the orthonormal transformation. Then Ψφk, Ψφl = φk, ΨH Ψφl = φk, φl since the inner products are invariant to unitary transformation. It is obvious that max k=l | Ψφk, Ψφl | = max k=l | φk, φl |. Theorem 20 The Grammian of a frame ΦN M is invariant to unitary transfor- mation. Proof. Let Ψ be an orthonormal basis for FM . Let ΨΦ be the frame constructed by the orthonormal transformation. Then (ΨΦ)H (ΨΦ) = ΦH ΨH ΨΦ = ΦH Φ. This result is simply due to the invariance of inner products over orthonormal transformation. Theorem 21 Let ΦN M be a frame in FM . Let Γ be a diagonal matrix whose diagonal entries are chosen from {1, −1} when F = R or {eiθ : 0 ≤ θ < 2π} when F = C. Let α, β be frame bounds of Φ. Then α, β are also frame bounds of ΦΓ. Moreover: (1) If Φ is a unit norm frame, then so is ΦΓ. (2) If Φ is a tight frame, then so is ΦΓ. (3) If Φ is a unit norm tight frame, then so is ΦΓ. (4) If Φ is an equiangular tight frame then so is ΦΓ.
- 11. ETF 11 Proof. Let the vectors in ΦΓ be γkφk where γk is drawn from {1, −1} or {eiθ : 0 ≤ θ < 2π}. Clearly |γk| = 1. Now, (ΦΓ)H x 2 2 = N k=1 | x, γkφk |2 = N k=1 |γk|2 | x, φk |2 = N k=1 | x, φk |2 = ΦH x 2 2. Thus, the frame bounds are identical. For other results we note that: (1) If φk is unit norm vector then so is γkφk. (2) Since frame bounds are same, hence tight frame property is preserved. (3) Application of the above two statements. (4) The absolute value of the inner product is identical: | φk, φl | = | γkφk, γlφl |. Hence equiangular property is preserved also. 3. Vandermonde Matrices A Vandermonde matrix has the following form. V = 1 1 . . . 1 γ1 γ2 . . . γN ... ... ... ... γM−1 1 γM−1 2 . . . γM−1 N (3.1) A square Vandermonde matrix (N = M) has the following determinant: det(V ) = 1≤i<j≤M (γi − γj). (3.2) When N ≥ M, then every M × M submatrix of V is also Vandermonde and the submatrices are non-singular as long as the bases {γk}N k=1 are distinct. Theorem 22 The only Vandermonde matrices that are equal norm and tight have bases in the complex unit circle. Among these, the frames with the small- est worst case coherence have bases that are equally spaced in the complex unit circle, provided N ≥ 2M. Proof. Since the matrix is a tight frame, hence it is full rank. Its rows have equal norms. 4. Grassmannian frames 4.1. Existence of Grassmannian frames. Let SM−1 = {x ∈ FM : x 2 = 1}denote the unit sphere in FM . A unit norm frame is constructed by picking up vectors from SM−1 .
- 12. 12 SHAILESH KUMAR We demonstrate that maximum correlation of a sequence of vectors is a contin- uous function. We deﬁne the function f :FM × · · · × FM → [0, 1] f(x1, . . . , xN ) = µ({xk}N k=1). Deﬁne a norm over X = FM × · · · × FM (N times) given by {xk}N k=1 X = N k=1 xk 2. It is easy to check that the norm so deﬁned satisﬁes all properties of a norm. Theorem 23 [1] The maximum correlation function from X to [0, 1] is con- tinuous. Proof. We only provide the proof outline here. Fix any point {xk}N k=1 in X. We need to show that for any > 0 there exists a δ > 0 such that |µ({xk}N k=1) − µ({yk}N k=1)| < whenever {xk}N k=1 − {yk}N k=1 X < δ. Theorem 24 for N ≥ M, a Grassmannian frame exists. Proof. Note that SM−1 is a compact subset of FM . This implies S = {SM−1 × · · · × SM−1 } is a compact subset of X. Since, µ is continuous over X, hence it is bounded over S and a minimum as well maximum value exists. Thus, unit norm sequences {xk}N k=1 exist in S with minimum coherence. What is left to show that they also span FM . If {xk}N k=1 doesn’t span FM , then we can easily replace one of the vectors in the sequence with a unit vector from the orthogonal complement of {xk}N k=1 in FM . This process doesn’t increase coherence. We can keep doing it till the sequence spans FM . Thus there are indeed some sequences which satisfy all the criteria for a frame with minimum coherence. This completes the proof. Theorem 25 Let ΦN M be a Grassmannian frame and let Ψ be an orthonormal basis (unitary matrix) for FM . Then, ΨΦ is also a Grassmannian frame. Proof. Since (due to theorem 19) coherence is invariant to unitary transformation, hence ΦN M and ΨΦN M have identical coherence. Thus, ΨΦN M is also a Grassmannian frame. 4.2. Optimal Grassmannian frames.
- 13. ETF 13 Theorem 26 Let ΦN M with N ≥ M be a unit norm frame. Then µ(Φ) ≥ N − M M(N − 1) . (4.1) The equality holds if and only if Φ is equiangular tight frame with frame bounds α = β = N M . (4.1) gives a lower bound to the coherence of any unit norm frame. Thus, a Grass- mannian frame can achieve this bound if and only if it is an equiangular tight frame with the speciﬁed frame bound. Such a Grassmannian frame is known as optimal Grassmannian frame. Proof. Let Φ be a Grassmannian frame and G = ΦH Φ ∈ CN×N be its Grassman- nian. G is symmetric and positive semi-deﬁnite. Its eigen-values are non-negative. Let the eigen values of G be ordered in by decreasing size as λ1 ≥ · · · ≥ λN . From theorem 10, we have M k=1 λk = N and M k=1 λ2 k ≥ N2 M . The lower bound is achieved only if λk = N M for all M positive eigen values. Let G = g1 . . . gN in terms of its column vectors. Consider the matrix G2 . Its eigen values are λ2 1 ≥ · · · ≥ λ2 N . Since G2 = GH G. Thus, M k=1 λ2 k = tr(G2 ) = N k=1 gH k gk. But gH k gk = gk 2 2 N l=1 | φk, φl |2 . We get N2 M ≤ M k=1 λ2 k = N k=1 N l=1 | φk, φl |2 . (4.2) Since G is Hermitian, hence | φk, φl | = | φl, φk |. We can rewrite N2 M ≤ k=l | φk, φl |2 + k<l | φk, φl |2 + k>l | φk, φl |2 = N + 2 k<l | φk, φl |2 ≤ N + 2 N(N − 1) 2 max k=l | φk, φl |2 . Rearranging this, we obtain µ2 (Φ) ≥ N − M M(N − 1) .
- 14. 14 SHAILESH KUMAR We now show that equality holds only if Φ is an equiangular tight frame. We note that µ2 (Φ) = N−M M(N−1) forces that N2 M = M k=1 λ2 k = N k=1 N l=1 | φk, φl |2 . This can happen only if λk = N M for k = 1, . . . , M. Immediately, since the largest and smallest eigen values of the frame operator ΦΦH are equal, hence Φ is a tight frame. To see that Φ is also equiangular, we proceed as follows. Recall from above that N + 2 k<l | φk, φl |2 = N2 M when Φ is tight. Hence k<l | φk, φl |2 = N(N − M) 2M . Since we are assuming that max k=l | φk, φl |2 = N−M M(N−1) , hence for any k = l, | φk, φl |2 = N − M M(N − 1) − εk,l where εk,l ≥ 0. Summing over k < l, we get N(N − M) 2M = k<l N − M M(N − 1) − εk,l = N(N − 1) 2 N − M M(N − 1) − k<l εk,l = N(N − M) 2M − k<l εk,l. Therefore, k<l εk,l = 0 and since εk,l is non-negative, hence each of it has to be zero. Thus, Φ is equiangular and | φk, φl |2 = N − M M(N − 1) . We have shown that if the lower bound for coherence is satisﬁed, then Φ has to be an ETF. We now show the converse. Let Φ be an equiangular tight frame. Then the frame bounds are α = β = N M . Further, there is µ ∈ [0, 1), such that | φk, φl |2 = µ2 for k = l. Since Φ is tight, hence λk = N M for k = 1, . . . , M and zero otherwise. We have N2 M = M k=1 λ2 k = N k=1 N l=1 | φk, φl |2 = N + N(N − 1)µ2 . Rearranging, we get the lower bound for µ. Theorem 27 For real matrices, if N > M(M+1) 2 , then Φ can’t be equiangular.
- 15. ETF 15 Proof. Let ΦN M be an equiangular frame. Let Pk : RM → RM be the projection of x on to the line spanned by φk, i.e., Pkx = x, φk φk (or Pk = φkφT k ). Let V be the vector space of symmetric linear mappings RM → RM . Then dim(V ) = M(M+1) 2 . We deﬁne an inner product over V as C, D V = tr(CD). Since Φ is equiangular, there exists µ ∈ [0, 1] such that φk, φl = ±µ for k = l. If µ = 1, then we have N = 2 since the vectors in Φ are assumed to be distinct and unit norm. Clearly for M ≥ 2, N = 2 < 3 ≤ M(M+1) 2 . µ = 0 is possible only if all vectors are orthogonal to each other. This gives us an orthonormal basis with N = M ≤ M(M+1) 2 . We now consider µ ∈ (0, 1). Now, Pk, Pl V = φk, φl = 1 if k = l µ2 if k = l Then, the Grammian G of the set {P1, . . . , PN } ⊂ V is [G]k,l == 1 if k = l µ2 if k = l Determinant of G is given by det(G) = (1 + (N − 1)µ2 )(1 − µ2 )N−1 = 0. Therefore G is invertible and full rank. We now have N = rank(G) = dim(span{P1, . . . , PN }) ≤ dim(V ) = M(M + 1) 2 . We have shown that if Φ is ETF, then N ≤ M(M+1) 2 . By contraposition, the result is proved. Theorem 28 For complex matrices, if N > M2 , then Φ can’t be equiangular. Proof. The proof follows theorem 27. Main diﬀerences are as follows. We let V to be the real vector space of Hermitian linear mappings CM → CM . Then, dim(V ) = M2 . The projection operators for φk are deﬁned as Pk = φkφH k . The inner product deﬁnition remains the same. Rest of the argument is identical. Theorem 29 For optimal Grassmannian frame, the minimum and maximum correlations are identical. Proof. The reason is that all oﬀ-diagonal entries in G = ΦH Φ have same magnitude.
- 16. 16 SHAILESH KUMAR Theorem 30 [4] Let M, N ∈ N with N ≥ M. Assume R is a Hermitian N × N matrix with entries Rk,k = 1 and Rk,l = ± N−M M(N−1) if F = R ±i N−M M(N−1) if F = C , (4.3) for k, l = 1, . . . , N; k = l. If the eigenvalues λ1, . . . , λN of R are such that λ1 = · · · = λM = N M and other eigen values are 0, then there exists a frame ΦN M that achieves the bound (4.1). Proof. Since R is Hermitian, it has a spectral factorization of the form R = WΛWH . Without loss of generality, we can assume that the non-zero eigen values of R are contained in the ﬁrst M diagonal entries of Λ. Let W be formed by the ﬁrst M columns of W. Then, it is clear that R = W N M IWH . Alternatively R = N M W N M W H . Let φk = N M {Wk,l}M l=1. In other words, φk is the k-th row (transposed) of N M W . Clearly, φk, φl = φH l φk = φT k ¯φl = Rl,k. Hence, {φk}N k=1 is equiangular. {φk}N k=1 is also tight since the non-zero eigen values of R are identical. Hence by theorem 26, {φk}N k=1 achieves the optimal Grassmannian frame coherence bound. Theorem 31 Let ΦN M be an optimal Grassmannian frame with coherence µ. Then for every vector unit norm vector x ∈ FM , there exists at least one vector φ in ΦN M such that | x, φ | ≥ µ. (4.4) Proof. Since ΦN M spans FM , hence x is a linear combination of M or less vectors from ΦN M . If x is a scalar multiple of one of the vectors in ΦN M (say φk) , then | x, φk | = 1 > µ. We now consider the case where x is a linear combination of more than one vectors in ΦN M . Thus, it is diﬀerent from all vectors in ΦN M . For the sake of contradiction, assume that | x, φ | < µ for every vector in ΦN M . Now construct a new frame XN M by replacing one of the vectors (say φ1) in ΦN M with x. Clearly coherence of XN M is also µ (the absolute inner product of any pair of vectors in XN M excluding x). Thus, XN M achieves the lower bound in (4.1). But this is possible only if XN M is equiangular, i.e. | x, φk | = µ for k = 2, . . . , N. We reach a contradiction. Hence, there exists at least one vector φ in ΦN M such that | x, φ | ≥ µ.
- 17. ETF 17 5. Full Spark Frames This section considers the design of full spark frames. i.e. unit norm frames ΦM N such that every set of M frame vectors in the frame is linearly independent and spans FM . A frame with larger spark can provide unique sparse representations for signals with larger sparsity levels. Deﬁnition 17 A frame ΦM N is called full spark if spark(Φ) = M + 1. For a full-spark frame, every M × M submatrix is invertible. 5.1. Vandermonde matrices. Theorem 32 A Vandermonde matrix is full spark if and only if its bases are distinct. 6. Subspace angles Consider an arbitrary vector x ∈ FM . The angle between x and standard basis vectors is given by cos θi = |xi| x 2 where we ignore the sign / phase of xi. We consider the angle between x and the vector closest to it in the standard basis. The minimum angle is given by cos θmin = max 1≤i≤M |xi| x 2 . For a general orthonormal basis Ψ, we have cos θmin = max 1≤i≤M |ψH i x| x 2 . If x is one of the basis vectors in Ψ, then θmin is zero degree. How large can θmin be? Since, the angle calculation is independent of the norm of x, hence let us assume x to be a unit norm vector. We will call this angle as the principal angle and the corresponding vector as the principal vector. Theorem 33 The largest principal angle between any vector x and an or- thonormal basis is given by cos θ = 1√ M . Proof. Let x = Ψv where v 2 = 1. Since Ψ is unitary, x 2 = v 2 = 1. Also v = ΨH x, thus vi = ψH i x. The principal angle is given by cos θmin = max 1≤i≤M |ψH i x| = max 1≤i≤M |vi|.
- 18. 18 SHAILESH KUMAR Choose v = 1√ M [1, . . . , 1]T ∈ FM . Clearly, the angle between x = Ψv and each of the basis vectors is cos θ = 1√ M . Suppose, there is a unit norm vector y = Ψu, such that y is further away from all vectors in Ψ compared to x. Then, max 1≤i≤M |ui| < 1 √ M . But this means that |ui| < 1√ M , resulting in u 2 < 1 which is a contradiction. The minimum angle can be generalized for a unit norm frame as cos θmin = max 1≤k≤N |φH k x| x 2 . Let Φ be an optimal Grassmannian frame (i.e. it is also a unit norm equiangular tight frame). Let x be an arbitrary unit norm vector. What is the minimum angle of x with the columns of Φ? To start with, if x is a vector in the sequence Φ, then the minimum angle is 0◦ . TODO complete this. 6.1. Principal angles. Deﬁnition 18 [2] Let U and V be given subspaces of FM . Assume that p = dim(U) ≥ dim(V) = q ≥ 1. The smallest principal angle θ1(U, V) = θ1 ∈ [0, π/2] between U and V is deﬁned by cos θ1 = sup u∈U,v∈V uH v = uH 1 v1, u 2 = 1, v 2 = 1. (6.1) For k = 2, . . . , q, the k-th principal angle θk is deﬁned by cos θk = sup u∈Uk−1,v∈Vk−1 uH v = uH k vk, u 2 = 1, v 2 = 1. (6.2) where Uk(Vk) is orthogonal complement of span{u1, . . . , uk} (span{v1, . . . , vk}) in U (V) respectively. The vectors {u1, . . . , uk} and {v1, . . . , vk} are called principal vectors. If we have U and V as matrices whose ranges are U and V respectively, then in order to ﬁnd the principal angles, we construct orthonormal bases QU and QV . We then compute the inner product matrix G = QH U QV . The SVD of G gives the principal angles. In particular, the smallest principal angle is given by cos θ1 = σ1, the largest singular value. We state the result in the following theorem. Theorem 34 Let U and V be subspaces as in deﬁnition 18. Let QU and QV be orthonormal bases for U and V respectively. Put G = QH U QV . Let the SVD of this p × q matrix be G = Y ΣZH , Σ = diag(σ1, . . . , σq).
- 19. ETF 19 We assume that σ1 ≥ · · · ≥ σq. Then the principal angles and principal vectors associated with this pair of subspaces are given by cos θk = σk, U = QU Y, V = UV Z. (6.3) Proof. Note that QH U QU = Ip and QH V QV = Iq. Also, Y H Y = Y Y H = Ip and ZH Z = ZZH = Iq. The singular values and singular vectors of a matrix G are characterized by σk = max y 2=1, z 2=1 (yH Gz) = yH k Gzk, subject to yH yj = zH zj = 0, j = 1, . . . , k − 1. Let u = QU y ∈ U and v = QV z ∈ V, then it follows that u 2 = y 2, v 2 = z 2 and yH yi = uH ui, zH zi = vH vi. Since yH Gz = yH QH U QV z = uH v, hence we can rewrite σk = max u 2=1, v 2=1 (uH v) = uH k vk, subject to uH uj = vH vj = 0, j = 1, . . . , k − 1. This is exactly the deﬁnition of the principal angles and vectors which concludes the proof. 6.2. One dimensional subspaces. The principal angle between the subspaces spanned by two vectors x1 , x2 ∈ FM is given by cos θx = | x1 , x2 | x1 2 x2 2 . (6.4) Let Φ be a unit norm tight frame. Then with y1 = ΦH x1 and y2 = ΦH x2 , the angle between y1 and y2 in FN is given by cos θy = | y1 , y2 | y1 2 y2 2 . (6.5) The frame bounds give us N M x 2 2 = ΦH x 2 2. Thus ΦH x 2 = N M x 2. Thus, we have y1 2 y2 2 = N M x1 2 N M x2 2 = N M x1 2 x2 2. Further, y1 , y2 = ΦH x1 , ΦH x2 = x2H ΦΦH x1 = x2H Sx1 = x2H N M x2H x1 . Thus y1 , y2 = N M x1 , x2 .
- 20. 20 SHAILESH KUMAR This gives us cos θy = | y1 , y2 | y1 2 y2 2 . = N M | x1 , x2 | N M x1 2 x2 2 = cos θx. This result is summarized in following theorem. Theorem 35 Let Φ be a unit norm tight frame. Then the principal angle between the one dimensional subspaces spanned by two vectors x1 and x2 in FM is same as the principal angle between the one dimensional subspaces spanned by the vectors y1 = ΦH x1 and y2 = ΦH x2 in FN . 6.3. Optimal Grassmannian Frame. Recall that an optimal Grassmannian frame is an equiangular tight frame (with unit norm columns). The correlation between any two vectors of the frame is given by µ = N−M M(N−1) . Let ΦN M be an optimal Grassmannian frame. It is obvious that the angle between any two one dimensional subspaces spanned the vectors in ΦN M is given by cos θ1 = N−M M(N−1) . Following discussion focuses on subspace angles between independent subspaces created by choosing atoms from ΦN M . Let us consider two subspaces as U = span{φk, φl} and V = span{φm} where k, l, m are diﬀerent. If U and V are dependent (happens when φm ∈ U), then the smallest principal angle is 0◦ . Let us now assume that U and V are independent. What is the smallest principal angle between U and V? We restrict the following discussion to real optimal Grass- mannian frames. We have qk, ql = ±µ. If qk, ql = −µ, then lets just replace ql by −ql to ensure that qk, ql = µ. This change doesn’t impact U. Also see theorem 21. Similarly, let’s change qm by −qm if necessary to ensure that qk, qm = µ. Consider two vectors qa = qk + ql and qb = qk − ql. We have qk ± ql 2 2 = qk 2 2 + ql 2 2 ± 2 qk, ql = 2(1 ± µ). Let qa and qb be obtained by normalizing qa and qb respectively. If ql, qm = µ, then qa, qm = 2µ and qb, qm = 0. We have qa, qm = 2 1 + µ µ. On the other hand if ql, qm = −µ, then qa, qm = 0 and qb, qm = 2µ. We have qb, qm = 2 1 − µ µ. In both cases, we have found a vector in U whose inner product with qm is larger than µ. If µ 1, then in both cases, the inner product is approximated by √ 2µ. This result is summarized in following theorem.
- 21. ETF 21 Theorem 36 Let ΦN M be a real optimal Grassmannian frame with coherence µ where µ 1. Let us consider two subspaces as U = span{φk, φl} and V = span{φm} where k, l, m are diﬀerent. Further, assume that U and V V V are independent. Then the smallest principal angle between U and V V V is approximated by cos θ1 ≈ √ 2µ. The question now is, can we generalize this result for more vectors in the subspaces U and V? Appendix A. Linear Algebra Some results from linear algebra are collected here for reference. Theorem 37 Let Hn be an n × n matrix with 1 on the main diagonal and p elsewhere. Let Cn be a matrix identical to Hn except that its (1, 1)-th element equals p. Then det(Hn) = (1 + (n − 1)p)(1 − p)n−1 (A.1) and det(Cn) = p(1 − p)n−1 . (A.2) References [1] John J Benedetto and Joseph D Kolesar. Geometric properties of grassmannian frames for and. EURASIP Journal on Advances in Signal Processing, 2006(1): 049850, 2006. [2] ke Bj¨orck and Gene H Golub. Numerical methods for computing angles between linear subspaces. Mathematics of computation, 27(123):579–594, 1973. [3] Stephane Mallat. A wavelet tour of signal processing: the sparse way. Access Online via Elsevier, 2008. [4] Thomas Strohmer and Robert W Heath. Grassmannian frames with appli- cations to coding and communication. Applied and computational harmonic analysis, 14(3):257–275, 2003.
- 22. Index Coherence, 9 Dictionary, 4 Dual frame, 7 Equiangular tight frame, 9 Frame, 2 Frame analysis operator, 2 Frame coeﬃcients, 2 Frame operator, 2 Frame redundancy, 4 Frame synthesis operator, 2 Full spark frame, 17 Gram matrix, 3 Grammian, 3 Grassmannian frame, 9 Maximum correlation, 9 Minimum correlation, 10 Principal angle, 18 Principal vector, 18 Redundancy, 4 Riesz basis, 4 Smallest principal angle, 18 Spark, 10 Tight frame, 8 Unit norm frame, 4 Unit norm tight frame, 8 22