Lecture 12 orhogonality - 6.1 6.2 6.3

2. Slide 6.1- 2© 2012 Pearson Education, Inc. INNER PRODUCT  If u and v are vectors in Rn, then we regard u and v as nx1 matrices  The number uTv is called the inner product of u and v, and it is written as u∙v  aka dot product  Scalar

5. INNER PRODUCT - Properties Theorem 1: Let u, v, and w be vectors in Rn, and let c be a scalar. Then a. u•v = v•u b. (u + v)•w = u•w + v•w c. (cu)•v = c(u•v) = u•(cv) d. u•u ≥0, and =0 iff u = 0  Properties (b) and (c) can be combined several times to produce the following useful rule: (c1u1 + c2u2 + … + cnun)•v = c1(u1•v) + … + cp(up•v)

6. Slide 6.1- 6© 2012 Pearson Education, Inc. THE LENGTH OF A VECTOR Definition: The length (or norm) of v: 𝐯 = 𝐯 ∙ 𝐯 = 𝑣1 2 + 𝑣2 2 + ⋯ + 𝑣 𝑛 2  ||cv|| = |c| ||v||  Unit vector has length 1  Any vector divided by its length is a unit vector in same direction  normalizing

9. Slide 6.1- 9© 2012 Pearson Education, Inc. DISTANCE IN Rn Definition: For u and v in Rn, the distance between u and v, written as dist (u, v), is the length of the vector u-v. i.e., dist(u, v) = ||u – v|| Example: Find the distance between u=(7,1) & v=(3,2)

10. Orthogonal Vectors Definition: 2 vectors u and v are orthogonal if u•v = 0. Theorem 6-2: Pythagorean Theorem: 2 vectors u and v are orthogonal iff 𝐮 + 𝐯 2 = 𝐮 2 + 𝐯 2 Slide 6.1- 10© 2012 Pearson Education, Inc.

11. Slide 6.1- 11© 2012 Pearson Education, Inc. Orthogonal Complement  If a vector z is orthogonal to every vector in a subspace W of Rn, then z is said to be orthogonal to W.  The set of all vectors z that are orthogonal to W is called the orthogonal complement of W and is denoted by W┴ (and read as “W perpendicular” or simply “W perp”).

12. ORTHOGONAL COMPLEMENTS Theorem 3: Let A be an mxn matrix. The orthogonal complement of the row space of A is the null space of A, and the orthogonal complement of the column space of A is the null space of AT: Proof: ai = ith row of A Row(A) = Span{rows of A} Nul(A) = x:Ax = 0 ai • x = 0 for every i x is orthogonal to each row of A (Row ) NulA A  (Col ) Nul T A A 

16. Slide 6.2- 16© 2012 Pearson Education, Inc. ORTHOGONAL SETS  A set of vectors {u1,…,up} in Rn is said to be an orthogonal set if each pair of distinct vectors from the set is orthogonal, i.e, if ui •uj = 0 if i ≠ j.

18. Theorem 6-4 – Orthogonal Set as a Basis Theorem 4: If S = {u1, …, up} is an orthogonal set of nonzero vectors in Rn , then S is linearly independent and hence is a basis for the subspace spanned by S. Slide 6.1- 18© 2012 Pearson Education, Inc.

19. Slide 6.2- 19© 2012 Pearson Education, Inc. ORTHOGONAL Basis Definition: An orthogonal basis for a subspace W of Rn is a basis for W that is also an orthogonal set. Theorem 5: Let {u1,…,up} be an orthogonal basis for a subspace W of Rn. For each y in W, the weights in the linear combination are given by y = c1u1+ … + cpup 𝑐𝑗 = 𝐲∙𝐮 𝒋 𝐮 𝒋∙𝐮 𝒋 (𝑗 = 1, … , 𝑝)

20. Orthogonal Basis – Example 2 Same S as in Example 1: 𝐮1 = 3 1 1 , 𝐮2 = −1 2 1 , 𝐮1 = −1/2 −2 7/2 write 𝐲 = 6 1 −8 as a linear combination of the vectors in S. Slide 6.1- 20© 2012 Pearson Education, Inc.

23. Slide 6.2- 23© 2012 Pearson Education, Inc. ORTHOGONAL PROJECTION - Example Example 1: 𝐲 = 7 6 , 𝐮 = 4 2 a) Find the orthogonal projection of y onto u. b) Write y as the sum of two orthogonal vectors, one in Span {u} and one orthogonal to u.

24. ORTHOGONAL PROJECTION - Example 7 8 1 6 4 2                    ˆyy ˆ(y y)

26. Slide 6.2- 26© 2012 Pearson Education, Inc. ORTHONORMAL SETS  A set {u1,…,up} is an orthonormal set if it is an orthogonal set of unit vectors (||u||=1).  If W is the subspace spanned by such a set, then {u1,…,up} is an orthonormal basis for W, since the set is automatically linearly independent, by Theorem 4.  The simplest example of an orthonormal set is the standard basis {e1,…,en} for Rn.

27. Slide 6.2- 27© 2012 Pearson Education, Inc. ORTHONORMAL SETS - Example Example 2: Show that {v1, v2, v3} is an orthonormal basis of R3, where 1 3/ 11 v 1/ 11 1/ 11             2 1/ 6 v 2 / 6 1/ 6             3 1/ 66 v 4 / 66 7 / 66            

28. Slide 6.2- 28© 2012 Pearson Education, Inc. ORTHONORMAL SETS  When the vectors in an orthogonal set of nonzero vectors are normalized to have unit length, the new vectors will still be orthogonal, and hence the new set will be an orthonormal set.

30. Slide 6.2- 30© 2012 Pearson Education, Inc. ORTHONORMAL SETS Theorem 7: Let U be an mxn matrix with orthonormal columns, and let x and y be in Rn. Then a. ||Ux|| = ||x|| b. (Ux)•(Uy) = x•y c. (Ux)•(Uy) = 0 iff x•y=0  Properties (a) and (c) say that the linear mapping x Ux preserves lengths and orthogonality.

32. Orthogonal Matrix  If U is square and U-1=UT  Called “Orthogonal Matrix”  Really should be Orthonormal matrix  Orthonormal columns  Orthonormal rows as well Slide 6.1- 32© 2012 Pearson Education, Inc.

34. Slide 6.3- 34© 2012 Pearson Education, Inc. ORTHOGONAL PROJECTIONS  Extend ideas of previous section from R2 to Rn  Given a vector y and a subspace W in Rn, there is a vector ŷ in W such that  ŷ is the unique vector in W for which y - ŷ is orthogonal to W  ŷ is the unique vector in W closest to y. See the following figure. ˆy

35. Slide 6.3- 35© 2012 Pearson Education, Inc. THE ORTHOGONAL DECOMPOSITION THEOREM Theorem 8: Let W be a subspace of Rn . Then each y in Rn can be written uniquely in the form y = ŷ + z where ŷ is in W and z is in W┴ . If {u1,…,up} is any orthogonal basis of W, then 𝐲 = 𝐲 ∙ 𝐮1 𝐮1 ∙ 𝐮1 𝐮1 + ⋯ + 𝐲 ∙ 𝐮 𝑝 𝐮 𝑝 ∙ 𝐮 𝑝 𝐮 𝑝 and z = y - ŷ

36. Slide 6.3- 36© 2012 Pearson Education, Inc. THE ORTHOGONAL DECOMPOSITION THEOREM  The vector ŷ is called the orthogonal projection of y onto W and often is written as projWy. See the following figure.

37. Slide 6.3- 37© 2012 Pearson Education, Inc. THE ORTHOGONAL DECOMPOSITION THEOREM - Example Example 1: Write y as the sum of a vector in W = span{u1,u2}, and a vector orthogonal to W. 𝐮1 = 2 5 −1 , 𝐮2 = −2 1 1 , 𝐲 = 1 2 3

38. Slide 6.3- 38© 2012 Pearson Education, Inc. PROPERTIES OF ORTHOGONAL PROJECTIONS  If {u1,…,up} is an orthogonal basis for W and if y happens to be in W, then the formula for projWy is exactly the same as the representation of y given in Theorem 5 in Section 6.2.  If y is in W = Span {u1,…,up}, projwy = y

39. Slide 6.3- 39© 2012 Pearson Education, Inc. THE BEST APPROXIMATION THEOREM Theorem 9: Let W be a subspace of Rn, let y be any vector in Rn, and let ŷ be the orthogonal projection of y onto W. Then ŷ is the closest point in W to y, in the sense that ||y – ŷ|| < ||y – v|| for all v in W distinct from ŷ.

40. The Best Approximation Theorem  ŷ in Theorem 9 is called the best approximation to y by elements of W.  The distance from y to v, given by ||y – v||, can be regarded as the “error” of using v in place of y.  Theorem 9 says that this error is minimized when v=ŷ  Note – does not depend of basis of W used Slide 6.1- 40© 2012 Pearson Education, Inc.

Lecture 12 orhogonality - 6.1 6.2 6.3

njit-ronbrown

Lecture 12 orhogonality - 6.1 6.2 6.3