Orthogonal projection in statistics a power point lecture (Rajshahi University)

1. Projection Md. Sahidul Islam Ripon Department of statistics Rajshahi University Email: ripon.ru.statistics@gmail.com

4. Orthogonal vector x y x + y Pythagoras, This always not true. This only true when

5. Example Are the vector (1,2,2) T and (2,3,-4) T are orthogonal ? Are the vector (4,2,3) T and (7,3,-4) T are orthogonal ?

6. Theorem 1: An orthogonal set of non zero vectors in a vector space is linearly independent.

7. Subspace S is orthogonal to subspace T How??? row space is orthogonal to null space Orthogonal Subspace

9. Orthonormal vector Theorem: Let { u 1 , …, u n } be an orthonormal basis for a vector space V. Let v be a vector in V. v can be written as a linear combination of these vectors as follows. Proof: Since { u 1 , …, u n } is a basis there exist scalars c 1 ,…,c n such that v= c 1 u 1 +…+c n u n We shall show that, c 1 =v 1 .u 1 ,…,c n =v n .u n

14. Projection Graph

15. Orthogonal projection

19. Projection on to a plan

20. Find the orthogonal projection of Y=(7,7,8) on to the plane spanned by vector X 1 =(5,6,4) and X 2 =(9,5,1). Solution: Since must lie in the plane spanned by X 1 and X 2

24. Gram Schmit Orthogonalization Let be a basis for vector space V. The set of vector defined as follows is orthogonal. To obtain a orthogonal basis for V, normalized each of the vector

25. Fig: First two steps of Gram schmidt orthogonalization Geometric Interpretation

26. Consider the following set of vectors in R 2 (with the conventional inner product) Now, perform Gram–Schmidt, to obtain an orthogonal set of vectors: We check that the vectors u 1 and u 2 are indeed orthogonal: noting that if the dot product of two vectors is 0 then they are orthogonal. We can then normalize the vectors by dividing out their sizes as shown above: ; Example

31. Modified Gram-Schmidt orthogonalization When this process is implemented on a computer, the vectors u k are often not quite orthogonal, due to rounding errors . For the Gram–Schmidt process as described above (sometimes referred to as "classical Gram–Schmidt") this loss of orthogonality is particularly bad; therefore, it is said that the (classical) Gram–Schmidt process is numerically unstable . The Gram–Schmidt process can be stabilized by a small modification; this version is sometimes referred to as modified Gram-Schmidt or MGS. This approach gives the same result as the original formula in exact arithmetic and introduces smaller errors in finite-precision arithmetic. Instead of computing the vector u k as it is computed as Each step finds a vector orthogonal to . Thus is also orthogonalized against any errors introduced in computation of .

32. Modified Gram-Schmidt orthogonalization

34. Classical Vs Modified

35. Classical Vs Modified