Orthogonal Vector Spaces

Adv. Engg. Mathematics
MTH-812 Orthogonal Subspaces
Dr. Yasir Ali (yali@ceme.nust.edu.pk)
DBS&H, CEME-NUST
November 20, 2017
Dr. Yasir Ali (yali@ceme.nust.edu.pk) Adv. Engg. Mathematics

Geometric concepts of length, distance and perpendicularity
Vectors in Rn
Let u and v be vectors in Rn then they must be of the matrices of
order n × 1.
Transpose of u and v is of order 1 × n
The product of uT v is of order 1 × 1, which is a scalar.
Let
u =





u1
u2
...
un





and v =





v1
v2
...
vn





uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar

Inner Product of u and v
Inner Product of u and v is written as u, v and is deﬁned as follows
uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar

uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar
Similarly
v, u = vT
.u = v1u1 + v2u2 + · · · + vnun

uT
.v = u1 u2 · · · un .





v1
v2
...
vn





= u1v1 + u2v2 + · · · + unvn
scalar
Similarly
v, u = vT
.u = v1u1 + v2u2 + · · · + vnun
Compute u.v and v.u, where
u = 2 −5 −1
T
and v = 3 2 −3
T

Theorem
Let u, v and w be vectors in Rn and c be any scalar then
1 u, v = v, u
2 u, v , w = u, < v, w
3 cu, v = c u, v = u, cv
4 u, u ≥ 0, and u, u = 0 if and only if u = 0

Length of a vector in Rn
If v ∈ Rn with entries v1, v2, · · · , vn, then
The length of v is a nonnegative scalar ||v|| deﬁned by
||v|| = v2
1 + v2
2 + · · · + v2
n =
n
i=1
v2
i
1
2
||v||2
= v, v also ||cv|| = c||v||

If ||v|| = 1 or, equivalently, if v, v = 1, then v is called a unit vector
and it is said to be normalized.
Every nonzero vector v in V can be multiplied by the reciprocal of its
length to obtain the unit vector
ˆv =
1
||v||
v
which is a positive multiple of v. This process is called normalizing v.
Example
Let v = (1, −2, 2, 0). Find a unit vector in the direction of v.

Distance between u and v
For u and v in Rn, the distance between u and v, written as dist(u.v), is
the length of vector u − v. That is
dist(u, v) = ||u − v||

dist(u, v) = ||u − v||
In R2 and R3 the deﬁnition of distance coincides with usual formulae
of Euclidean distance between two points.

dist(u, v) = ||u − v||
In R2 and R3 the deﬁnition of distance coincides with usual formulae
of Euclidean distance between two points.
Example
Compute the distance between u = (7, 1) and v = (3, 2).

Perpendicular Lines
Consider two lines through origin determined by the vectors u and v.

Perpendicular Lines
Two lines are geometri-
cally perpendicular if dis-
tance between u and v is
same as distance between
u and −v. That is
dist(u, v) = dist(u, −v) ⇒ ||u − v|| = ||u − (−v)||
||u − v|| = ||u + v|| ⇒ u − v, u − v = u + v, u + v

Perpendicular Lines
Two lines are geometri-
cally perpendicular if dis-
tance between u and v is
same as distance between
u and −v. That is
dist(u, v) = dist(u, −v) ⇒ ||u − v|| = ||u − (−v)||
||u − v|| = ||u + v|| ⇒ u − v, u − v = u + v, u + v
Simpliﬁcation gives us
||u||2
+ ||v||2
− 2 u, v = ||u||2
+ ||v||2
+ 2 u, v
u, v = 0

Two vectors u and v in Rn
are orthogonal(to each other) if
u, v = 0.
The Pythagorean Theorem
Two vectors u and v in Rn are orthogonal if and only if
||u + v||2
= ||u||2
+ ||v||2

Orthogonal Complements
If a vector z is orthogonal to every vector in a subspace W then z is said to
be Orthogonal to W
The set of all vectors z orthogonal to W is said to be Orthogonal
Complement of W and is denoted by W⊥
W⊥
= z z, w = 0, ∀w ∈ W
Two important features of W⊥
A vector x is in W⊥ if x is orthogonal to every vector in a set that
span W
W⊥ is a subspace of Rn.

Orthogonal and Orthonormal
Consider a set S = {u1, u2, · · · , ur} of nonzero vectors in an inner product
space V .
S is called orthogonal if each pair of vectors in S are orthogonal
S is called orthonormal if S is orthogonal and each vector in S
has unit length.
That is,
Orthogonal: ui, uj = 0 for i = j
Orthonormal: ui, uj =
0, i = j;
1, otherwise.

Important Results
Theorem
Suppose S is an orthogonal set of nonzero vectors. Then S is linearly
independent.

Important Results
Theorem
Suppose S is an orthogonal set of nonzero vectors. Then S is linearly
independent.
Theorem (Pythagoras)
Suppose {u1, u2, · · · , ur} is a set of orthogonal vectors then
||u1 + u2 + · · · + ur||2
= ||u1||2
+ ||u2||2
+ · · · + ||ur||2

Orthogonal Basis
An orthogonal basis for W is basis for W that is also an orthogonal set.

Orthogonal Basis
Example
Check whether or not following vectors, of R3, are orthogonal basis
u1 = (1, 2, 1), u2 = (2, 1, −4), u3 = (3, −2, 1).

Orthogonal Basis
Example
Check whether or not following vectors, of R3, are orthogonal basis
u1 = (1, 2, 1), u2 = (2, 1, −4), u3 = (3, −2, 1).
We have to check following three conditions that given vectors are:
1 Linearly independent
2 Spans R3
3 Orthogonal set

Theorem
Let {u1, u2, · · · , ur} be an orthogonal basis of V . Then, for any v ∈ V ,
v =
v, u1
u1, u1
u1 +
v, u2
u2, u2
u2 + · · · +
v, ur
ur, ur
ur.
The scalar ki = v,ui
ui,ui
is called the Fourier coeﬃcient of v with respect
to ui, because it is analogous to a coeﬃcient in the Fourier series of a
function.

Projection
Let V be an inner product space. Suppose w is a given nonzero vector in
V , and suppose v is another vector.
We seek the “projection of v along w”, which, as indicated in Fig,
will be the multiple cw of w such that
v = v − cw is orthogonal to w.
This means
v , w = 0
v − cw, w = 0
v, w − c w, w = 0
c =
v, w
w, w

Accordingly, the projection of v along w is denoted and deﬁned by
proj(v, w) = cw =
v, w
w, w
w

Theorem (Gram-Schmidt Orthogonalization Process)
Suppose Suppose {u1, u2, · · · , un} is a basis of an inner product space
V . One can use this basis to construct an orthogonal basis
{w1, w2, · · · , wn} of V as follows. Set
w1 = u1
w2 = u2 −
u2, w1
w1, w1
w1
w3 = u3 −
u3, w1
w1, w1
w1 −
u3, w2
w2, w2
w2
. . .
wn = un −
un, w1
w1, w1
w1 −
un, w2
w2, w2
w2 − · · · −
un, wn−1
wn−1, wn−1
wn−1
In other words, for k = 2, 3, · · · n, we deﬁne
wk = uk − ck1w1 − ck2w2 − · · · − ckk−1wk−1, where cki =
uk, wi
wi, wi

Inner Product Spaces
Function Space C[a, b] and Polynomial Space P(t)
The notation C[a, b] is used to denote the vector space of all continuous
functions on the closed interval [a, b], that is, where a ≤ t ≤ b. The
following deﬁnes an inner product on C[a, b], where f(t) and g(t) are
functions in C[a, b]
f, g =
b
a
f(t)g(t)dt.
It is called the usual inner product on C[a, b].
Example (Consider f(t) = 3t − 5 and g(t) = t2 in P(t) with the inner
product
f, g =
1
0
f(t)g(t)dt.
Find f, g and ||f||.)

Function Space C[a, b] and Polynomial Space P(t)
The notation C[a, b] is used to denote the vector space of all continuous
functions on the closed interval [a, b], that is, where a ≤ t ≤ b. The
following deﬁnes an inner product on C[a, b], where f(t) and g(t) are
functions in C[a, b]
f, g =
b
a
f(t)g(t)dt.
It is called the usual inner product on C[a, b].
Example (Consider f(t) = 3t − 5 and g(t) = t2 in P(t) with the inner
product
f, g =
1
0
f(t)g(t)dt.
Find f, g and ||f||.)
f, g is given whereas ||f||2 = f, f

Matrix Space M = Mm×n
Let M = Mm×n, the vector space of all real m × n matrices. An inner
product is deﬁned on M by
A, B = tr(BT
A)
where, as usual, tr() is the trace ⇒ the sum of the diagonal elements.
Hilbert Space
Let V be the vector space of all inﬁnite sequences of real numbers
(a1, a2, a3, · · · ) satisfying
∞
i=1
a2
i = a2
1 + a2
2 + a2
3 + · · · < ∞
that is, the sum converges. This inner product space is called l2-space
or Hilbert space.

Orthogonal and Positive Deﬁnite Matrices
Orthogonal Matrices
A real matrix P is orthogonal if P is nonsingular and
P−1
= PT
, or, in other words, if PPT
= PT
P = I.

Orthogonal and Positive Deﬁnite Matrices
Orthogonal Matrices
A real matrix P is orthogonal if P is nonsingular and
P−1
= PT
, or, in other words, if PPT
= PT
P = I.
Let P be a real matrix. Then the following are equivalent:
(a) P is orthogonal
(b) the rows of P form an orthonormal set
(c) the columns of P form an orthonormal set.

Positive Deﬁnite Matrices
Let A be a real symmetric matrix; that is, AT A. Then A is said to be
positive deﬁnite if, for every nonzero vector u in Rn,
u, Au = uT
Au > 0.

Norms on Rn
and Cn
The following define three important norms on Rn and Cn
:
||(a1, · · · , an)||∞ = max (|ai|)
||(a1, · · · , an)||1 = |a1| + |a2| + · · · + |an|
||(a1, · · · , an)||2 = |a1|2 + |a2|2 + · · · + |an|2
The norms ||.||∞, ||.||1, and ||.||2 are called the infinity-norm, one-norm,
and two-norm, respectively.
The distance between two vectors u and v in V is denoted and defined
by d(u, v) = ||u − v||

Consider the Cartesian plane R2 shown
in Fig
Let D1 be the set of points
u = (x, y) in R2 such that
||u||2 = 1, that is,
x2 + y2 = 1(unit circle).
||u||1 = 1, that is, |x| + |y| = 1.
Thus, D2 is the diamond inside
the unit circle
||u||∞ = 1, that is,
max (|x|, |y|) = 1. Thus, D3 is the
square circumscribing the unit
circle

Orthogonal Vector Spaces

More Related Content

What's hot

Similar to Orthogonal Vector Spaces

More from Sohaib H. Khan

Recently uploaded

Orthogonal Vector Spaces