Lecture 12 orhogonality - 6.1 6.2 6.3

6
6.1
© 2012 Pearson Education, Inc.
Orthogonality
INNER PRODUCT, LENGTH,
AND ORTHOGONALITY

1- 2© 2012 Pearson Education, Inc.
INNER PRODUCT
 If u and v are vectors in Rn, then we regard u
and v as nx1 matrices
 The number uTv is called the inner product of
u and v, and it is written as u∙v
 aka dot product
 Scalar

INNER PRODUCT
If 𝐮 =
𝑢1
𝑢2
⋮
𝑢 𝑛
and 𝐯 =
𝑣1
𝑣2
⋮
𝑣 𝑛
then the inner product of u and v is
𝑢1 𝑢2 ⋯ 𝑢 𝑛
𝑣1
𝑣2
⋮
𝑣 𝑛
= 𝑢1 𝑣1 + 𝑢2 𝑣2 + ⋯ + 𝑢 𝑛 𝑣 𝑛

Inner Product - Example
 Find u∙v & v∙u, for:
𝐮 =
2
−5
−1
, 𝐯 =
3
2
3
Slide 6.1- 4© 2012 Pearson Education, Inc.

INNER PRODUCT - Properties
Theorem 1: Let u, v, and w be vectors in Rn, and let c
be a scalar. Then
a. u•v = v•u
b. (u + v)•w = u•w + v•w
c. (cu)•v = c(u•v) = u•(cv)
d. u•u ≥0, and =0 iff u = 0
 Properties (b) and (c) can be combined several
times to produce the following useful rule:
(c1u1 + c2u2 + … + cnun)•v = c1(u1•v) + … + cp(up•v)

THE LENGTH OF A VECTOR
Definition: The length (or norm) of v:
𝐯 = 𝐯 ∙ 𝐯 = 𝑣1
2 + 𝑣2
2 + ⋯ + 𝑣 𝑛
2
 ||cv|| = |c| ||v||
 Unit vector has length 1
 Any vector divided by its length is a unit vector in
same direction  normalizing

Normalization – Example 1
Example 1: Let v = (1, -2, 2, 0). Find a unit vector
u in the same direction as v.

Normalization – Example 2
Example 2: Let W be a subspace of R2 spanned
by x =
2/3
1
. Find a unit vector z that is a basis for
W.

DISTANCE IN Rn
Definition: For u and v in Rn, the distance between u
and v, written as dist (u, v), is the length of the vector
u-v. i.e.,
dist(u, v) = ||u – v||
Example: Find the distance between u=(7,1) & v=(3,2)

Orthogonal Vectors
Definition: 2 vectors u and v are orthogonal
if u•v = 0.
Theorem 6-2: Pythagorean Theorem: 2
vectors u and v are orthogonal iff
𝐮 + 𝐯 2
= 𝐮 2
+ 𝐯 2

Orthogonal Complement
 If a vector z is orthogonal to every vector in a
subspace W of Rn, then z is said to be
orthogonal to W.
 The set of all vectors z that are orthogonal to W
is called the orthogonal complement of W and
is denoted by W┴ (and read as “W
perpendicular” or simply “W perp”).

ORTHOGONAL COMPLEMENTS
Theorem 3: Let A be an mxn matrix. The orthogonal
complement of the row space of A is the null space of
A, and the orthogonal complement of the column
space of A is the null space of AT:
Proof: ai = ith row of A
Row(A) = Span{rows of A}
Nul(A) = x:Ax = 0
ai • x = 0 for every i
x is orthogonal to each row of A
(Row ) NulA A

(Col ) Nul T
A A


ANGLES & Inner Products
u•v = ||u|| ||v|| cosϑ

ANGLES & Inner Products - Example
Find the angle between
1
1
0
,
0
1
1

6
6.1
Orthogonal Sets

ORTHOGONAL SETS
 A set of vectors {u1,…,up} in Rn is said to be an
orthogonal set if each pair of distinct vectors
from the set is orthogonal, i.e, if
ui •uj = 0 if i ≠ j.

Orthogonal Sets - Example
Show that S={u1,u2,u3} is an orthogonal set.
𝐮1 =
3
1
1
, 𝐮2 =
−1
2
1
, 𝐮1 =
−1/2
−2
7/2

Theorem 6-4 – Orthogonal Set as a Basis
Theorem 4: If S = {u1, …, up} is an
orthogonal set of nonzero vectors in Rn ,
then S is linearly independent and hence is
a basis for the subspace spanned by S.

ORTHOGONAL Basis
Definition: An orthogonal basis for a subspace W
of Rn is a basis for W that is also an orthogonal set.
Theorem 5: Let {u1,…,up} be an orthogonal basis for
a subspace W of Rn. For each y in W, the weights in
the linear combination
are given by y = c1u1+ … + cpup
𝑐𝑗 =
𝐲∙𝐮 𝒋
𝐮 𝒋∙𝐮 𝒋
(𝑗 = 1, … , 𝑝)

Orthogonal Basis – Example 2
Same S as in Example 1:
𝐮1 =
3
1
1
, 𝐮2 =
−1
2
1
, 𝐮1 =
−1/2
−2
7/2
write 𝐲 =
6
1
−8
as a linear combination of the
vectors in S.

AN ORTHOGONAL PROJECTION
Decompose y into 2 vectors:
 One in direction of u  ŷ = αu
 One orthogonal to u  z = y - ŷ

Orthogonal Projections
 ŷ is the projection of y onto u
 If L is subspace spanned by u
 ŷ is the projection of y onto L
𝐲 = proj 𝐿 𝐲 =
𝐲 ∙ 𝐮
𝐮 ∙ 𝐮
𝐮

ORTHOGONAL PROJECTION - Example
Example 1: 𝐲 =
7
6
, 𝐮 =
4
2
a) Find the orthogonal projection of y onto u.
b) Write y as the sum of two orthogonal vectors,
one in Span {u} and one orthogonal to u.

ORTHOGONAL PROJECTION - Example
7 8 1
6 4 2
     
      
     
ˆyy ˆ(y y)

AN ORTHOGONAL PROJECTION
 Check: ŷ•z = 0?
 Distance from y to L?
 Distance from y to u?

ORTHONORMAL SETS
 A set {u1,…,up} is an orthonormal set if it is an
orthogonal set of unit vectors (||u||=1).
 If W is the subspace spanned by such a set, then
{u1,…,up} is an orthonormal basis for W, since
the set is automatically linearly independent, by
Theorem 4.
 The simplest example of an orthonormal set is the
standard basis {e1,…,en} for Rn.

ORTHONORMAL SETS - Example
Example 2: Show that {v1, v2, v3} is an
orthonormal basis of R3, where
1
3/ 11
v 1/ 11
1/ 11
 
 
  
 
  
2
1/ 6
v 2 / 6
1/ 6
 
 
  
 
  
3
1/ 66
v 4 / 66
7 / 66
 
 
  
 
  

ORTHONORMAL SETS
 When the vectors in an orthogonal set of
nonzero vectors are normalized to have unit
length, the new vectors will still be orthogonal,
and hence the new set will be an orthonormal
set.

ORTHONORMAL SETS
Theorem 6: An mxn matrix U has orthonormal
columns if and only if UTU = I

ORTHONORMAL SETS
Theorem 7: Let U be an mxn matrix with
orthonormal columns, and let x and y be in Rn.
Then
a. ||Ux|| = ||x||
b. (Ux)•(Uy) = x•y
c. (Ux)•(Uy) = 0 iff x•y=0
 Properties (a) and (c) say that the linear
mapping x Ux preserves lengths and
orthogonality.

Orthonormal Sets - Example
𝑈 =
1 2 2 3
1 2 2/3
0 1/3
, 𝐱 = 2
3

Orthogonal Matrix
 If U is square and U-1=UT
 Called “Orthogonal Matrix”
 Really should be Orthonormal matrix
 Orthonormal columns
 Orthonormal rows as well

6
6.1
Orthogonal Projections

ORTHOGONAL PROJECTIONS
 Extend ideas of previous section from R2 to Rn
 Given a vector y and a subspace W in Rn, there
is a vector ŷ in W such that
 ŷ is the unique vector in W for which y - ŷ is
orthogonal to W
 ŷ is the unique vector in W closest to y. See the
following figure.
ˆy

THE ORTHOGONAL DECOMPOSITION
THEOREM
Theorem 8: Let W be a subspace of Rn . Then
each y in Rn can be written uniquely in the form
y = ŷ + z
where ŷ is in W and z is in W┴ .
If {u1,…,up} is any orthogonal basis of W, then
𝐲 =
𝐲 ∙ 𝐮1
𝐮1 ∙ 𝐮1
𝐮1 + ⋯ +
𝐲 ∙ 𝐮 𝑝
𝐮 𝑝 ∙ 𝐮 𝑝
𝐮 𝑝
and
z = y - ŷ

THEOREM
 The vector ŷ is called the orthogonal
projection of y onto W and often is written as
projWy. See the following figure.

THEOREM - Example
Example 1: Write y as the sum of a vector in W =
span{u1,u2}, and a vector orthogonal to W.
𝐮1 =
2
5
−1
, 𝐮2 =
−2
1
1
, 𝐲 =
1
2
3

PROPERTIES OF ORTHOGONAL
PROJECTIONS
 If {u1,…,up} is an orthogonal basis for W and if y
happens to be in W, then the formula for projWy
is exactly the same as the representation of y
given in Theorem 5 in Section 6.2.
 If y is in W = Span {u1,…,up}, projwy = y

THE BEST APPROXIMATION THEOREM
Theorem 9: Let W be a subspace of Rn, let y be any
vector in Rn, and let ŷ be the orthogonal projection of
y onto W. Then ŷ is the closest point in W to y, in the
sense that
||y – ŷ|| < ||y – v||
for all v in W distinct from ŷ.

The Best Approximation Theorem
 ŷ in Theorem 9 is called the best
approximation to y by elements of W.
 The distance from y to v, given by ||y – v||,
can be regarded as the “error” of using v in
place of y.
 Theorem 9 says that this error is minimized
when v=ŷ
 Note – does not depend of basis of W used

THE BEST APPROXIMATION THEOREM

Best Approximation - Example
Example 2: Find the distance from y to W = Span
{u1,u2} for:
𝐮1 =
5
−2
1
, 𝐮2 =
1
2
−1
, 𝐲 =
−1
−5
10

Lecture 12 orhogonality - 6.1 6.2 6.3

More Related Content

What's hot

Viewers also liked

Similar to Lecture 12 orhogonality - 6.1 6.2 6.3

More from njit-ronbrown

Recently uploaded

Lecture 12 orhogonality - 6.1 6.2 6.3