Orthogonal Vector Spaces

Adv. Engg. Mathematics
MTH-812 Orthogonal Subspaces
Dr. Yasir Ali (yali@ceme.nust.edu.pk)
DBS&H, CEME-NUST
November 20, 2017
Dr. Yasir Ali (yali@ceme.nust.edu.pk) Adv. Engg. Mathematics

Geometric concepts of length, distance and perpendicularity
Vectors in Rn
Let u and v be vectors in Rn then they must be of the matrices of
order n × 1.
Transpose of u and v is of order 1 × n
The product of uT v is of order 1 × 1, which is a scalar.
Let
u =





u1
u2
...
un





and v =





v1
v2
...
vn





uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar

Inner Product of u and v
Inner Product of u and v is written as u, v and is deﬁned as follows
uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar

uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar
Similarly
v, u = vT
.u = v1u1 + v2u2 + · · · + vnun

uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar
Similarly
v, u = vT
.u = v1u1 + v2u2 + · · · + vnun
Compute u.v and v.u, where
u = 2 −5 −1
T
and v = 3 2 −3
T

Theorem
Let u, v and w be vectors in Rn and c be any scalar then
1 u, v = v, u
2 u, v , w = u, < v, w
3 cu, v = c u, v = u, cv
4 u, u ≥ 0, and u, u = 0 if and only if u = 0

Length of a vector in Rn
If v ∈ Rn with entries v1, v2, · · · , vn, then
The length of v is a nonnegative scalar ||v|| deﬁned by
||v|| = v2
1 + v2
2 + · · · + v2
n =
n
i=1
v2
i
1
2
||v||2
= v, v also ||cv|| = c||v||

If ||v|| = 1 or, equivalently, if v, v = 1, then v is called a unit vector
and it is said to be normalized.
Every nonzero vector v in V can be multiplied by the reciprocal of its
length to obtain the unit vector
ˆv =
1
||v||
v
which is a positive multiple of v. This process is called normalizing v.
Example
Let v = (1, −2, 2, 0). Find a unit vector in the direction of v.

Distance between u and v
For u and v in Rn, the distance between u and v, written as dist(u.v), is
the length of vector u − v. That is
dist(u, v) = ||u − v||

dist(u, v) = ||u − v||
In R2 and R3 the deﬁnition of distance coincides with usual formulae
of Euclidean distance between two points.

dist(u, v) = ||u − v||
In R2 and R3 the deﬁnition of distance coincides with usual formulae
of Euclidean distance between two points.
Example
Compute the distance between u = (7, 1) and v = (3, 2).

Perpendicular Lines
Consider two lines through origin determined by the vectors u and v.

Perpendicular Lines
Two lines are geometri-
cally perpendicular if dis-
tance between u and v is
same as distance between
u and −v. That is
dist(u, v) = dist(u, −v) ⇒ ||u − v|| = ||u − (−v)||
||u − v|| = ||u + v|| ⇒ u − v, u − v = u + v, u + v

Perpendicular Lines
Two lines are geometri-
cally perpendicular if dis-
tance between u and v is
same as distance between
u and −v. That is
dist(u, v) = dist(u, −v) ⇒ ||u − v|| = ||u − (−v)||
||u − v|| = ||u + v|| ⇒ u − v, u − v = u + v, u + v
Simpliﬁcation gives us
||u||2
+ ||v||2
− 2 u, v = ||u||2
+ ||v||2
+ 2 u, v
u, v = 0

Two vectors u and v in Rn
are orthogonal(to each other) if
u, v = 0.
The Pythagorean Theorem
Two vectors u and v in Rn are orthogonal if and only if
||u + v||2
= ||u||2
+ ||v||2

Orthogonal Complements
If a vector z is orthogonal to every vector in a subspace W then z is said to
be Orthogonal to W
The set of all vectors z orthogonal to W is said to be Orthogonal
Complement of W and is denoted by W⊥
W⊥
= z z, w = 0, ∀w ∈ W
Two important features of W⊥
A vector x is in W⊥ if x is orthogonal to every vector in a set that
span W
W⊥ is a subspace of Rn.

Orthogonal and Orthonormal
Consider a set S = {u1, u2, · · · , ur} of nonzero vectors in an inner product
space V .
S is called orthogonal if each pair of vectors in S are orthogonal
S is called orthonormal if S is orthogonal and each vector in S
has unit length.
That is,
Orthogonal: ui, uj = 0 for i = j
Orthonormal: ui, uj =
0, i = j;
1, otherwise.

Important Results
Theorem
Suppose S is an orthogonal set of nonzero vectors. Then S is linearly
independent.

Important Results
Theorem
Suppose S is an orthogonal set of nonzero vectors. Then S is linearly
independent.
Theorem (Pythagoras)
Suppose {u1, u2, · · · , ur} is a set of orthogonal vectors then
||u1 + u2 + · · · + ur||2
= ||u1||2
+ ||u2||2
+ · · · + ||ur||2

Orthogonal Basis
An orthogonal basis for W is basis for W that is also an orthogonal set.

Orthogonal Basis
Example
Check whether or not following vectors, of R3, are orthogonal basis
u1 = (1, 2, 1), u2 = (2, 1, −4), u3 = (3, −2, 1).

Orthogonal Basis
Example
Check whether or not following vectors, of R3, are orthogonal basis
u1 = (1, 2, 1), u2 = (2, 1, −4), u3 = (3, −2, 1).
We have to check following three conditions that given vectors are:
1 Linearly independent
2 Spans R3
3 Orthogonal set

Theorem
Let {u1, u2, · · · , ur} be an orthogonal basis of V . Then, for any v ∈ V ,
v =
v, u1
u1, u1
u1 +
v, u2
u2, u2
u2 + · · · +
v, ur
ur, ur
ur.
The scalar ki = v,ui
ui,ui
is called the Fourier coeﬃcient of v with respect
to ui, because it is analogous to a coeﬃcient in the Fourier series of a
function.

Projection
Let V be an inner product space. Suppose w is a given nonzero vector in
V , and suppose v is another vector.
We seek the “projection of v along w”, which, as indicated in Fig,
will be the multiple cw of w such that
v = v − cw is orthogonal to w.
This means
v , w = 0
v − cw, w = 0
v, w − c w, w = 0
c =
v, w
w, w

Accordingly, the projection of v along w is denoted and deﬁned by
proj(v, w) = cw =
v, w
w, w
w

Theorem (Gram-Schmidt Orthogonalization Process)
Suppose Suppose {u1, u2, · · · , un} is a basis of an inner product space
V . One can use this basis to construct an orthogonal basis
{w1, w2, · · · , wn} of V as follows. Set
w1 = u1
w2 = u2 −
u2, w1
w1, w1
w1
w3 = u3 −
u3, w1
w1, w1
w1 −
u3, w2
w2, w2
w2
. . .
wn = un −
un, w1
w1, w1
w1 −
un, w2
w2, w2
w2 − · · · −
un, wn−1
wn−1, wn−1
wn−1
In other words, for k = 2, 3, · · · n, we deﬁne
wk = uk − ck1w1 − ck2w2 − · · · − ckk−1wk−1, where cki =
uk, wi
wi, wi

Inner Product Spaces
Function Space C[a, b] and Polynomial Space P(t)
The notation C[a, b] is used to denote the vector space of all continuous
functions on the closed interval [a, b], that is, where a ≤ t ≤ b. The
following deﬁnes an inner product on C[a, b], where f(t) and g(t) are
functions in C[a, b]
f, g =
b
a
f(t)g(t)dt.
It is called the usual inner product on C[a, b].
Example (Consider f(t) = 3t − 5 and g(t) = t2 in P(t) with the inner
product
f, g =
1
0
f(t)g(t)dt.
Find f, g and ||f||.)

Function Space C[a, b] and Polynomial Space P(t)
The notation C[a, b] is used to denote the vector space of all continuous
functions on the closed interval [a, b], that is, where a ≤ t ≤ b. The
following deﬁnes an inner product on C[a, b], where f(t) and g(t) are
functions in C[a, b]
f, g =
b
a
f(t)g(t)dt.
It is called the usual inner product on C[a, b].
Example (Consider f(t) = 3t − 5 and g(t) = t2 in P(t) with the inner
product
f, g =
1
0
f(t)g(t)dt.
Find f, g and ||f||.)
f, g is given whereas ||f||2 = f, f

Matrix Space M = Mm×n
Let M = Mm×n, the vector space of all real m × n matrices. An inner
product is deﬁned on M by
A, B = tr(BT
A)
where, as usual, tr() is the trace ⇒ the sum of the diagonal elements.
Hilbert Space
Let V be the vector space of all inﬁnite sequences of real numbers
(a1, a2, a3, · · · ) satisfying
∞
i=1
a2
i = a2
1 + a2
2 + a2
3 + · · · < ∞
that is, the sum converges. This inner product space is called l2-space
or Hilbert space.

Orthogonal and Positive Deﬁnite Matrices
Orthogonal Matrices
A real matrix P is orthogonal if P is nonsingular and
P−1
= PT
, or, in other words, if PPT
= PT
P = I.

Orthogonal and Positive Deﬁnite Matrices
Orthogonal Matrices
A real matrix P is orthogonal if P is nonsingular and
P−1
= PT
, or, in other words, if PPT
= PT
P = I.
Let P be a real matrix. Then the following are equivalent:
(a) P is orthogonal
(b) the rows of P form an orthonormal set
(c) the columns of P form an orthonormal set.

Positive Deﬁnite Matrices
Let A be a real symmetric matrix; that is, AT A. Then A is said to be
positive deﬁnite if, for every nonzero vector u in Rn,
u, Au = uT
Au > 0.

Norms on Rn
and Cn
The following define three important norms on Rn and Cn
:
||(a1, · · · , an)||∞ = max (|ai|)
||(a1, · · · , an)||1 = |a1| + |a2| + · · · + |an|
||(a1, · · · , an)||2 = |a1|2 + |a2|2 + · · · + |an|2
The norms ||.||∞, ||.||1, and ||.||2 are called the infinity-norm, one-norm,
and two-norm, respectively.
The distance between two vectors u and v in V is denoted and defined
by d(u, v) = ||u − v||

Consider the Cartesian plane R2 shown
in Fig
Let D1 be the set of points
u = (x, y) in R2 such that
||u||2 = 1, that is,
x2 + y2 = 1(unit circle).
||u||1 = 1, that is, |x| + |y| = 1.
Thus, D2 is the diamond inside
the unit circle
||u||∞ = 1, that is,
max (|x|, |y|) = 1. Thus, D3 is the
square circumscribing the unit
circle

Orthogonal Vector Spaces

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Orthogonal Vector Spaces

Similar to Orthogonal Vector Spaces (20)

More from Sohaib H. Khan

More from Sohaib H. Khan (12)

Recently uploaded

Recently uploaded (20)

Orthogonal Vector Spaces