3. 1 Metric Spaces
Definition (Metric Space). The pair (X, d), where X is a set and the function
d:X ×X →R
is called a metric space if
1. d(x, y) ≥ 0
2. d(x, y) = 0 iff x = y
3. d(x, y) = d(y, x)
4. d(x, y) ≤ d(x, z) + d(z, y)
Example 1.1 (Metric Spaces).
1. d(x, y) = |x − y| in R.
n 1
2. d(x, y) = [ i=1 (xi − yi )2 ] 2 in Rn .
3. d(x, y) = x − y in a normed space.
4. Let (X, ρ), (Y, σ) be metric spaces and define the Cartesian product X × Y =
{(x, y)|x ∈ X, y ∈ Y }. Then the product measure τ ((x1 , y1 ), (x2 , y2 )) = [ρ(x1 , x2 )2 +
1
σ(y1 , y2 )2 ] 2 .
¯ ¯
5. (Subspace) (Y, d) of (X, d) if Y ⊂ X and d = d|Y ×Y .
6. l∞ . Let X be the set of all bounded sequences of complex numbers, i.e., x =
(ξi ) and |ξi | ≤ cx , ∀ i. Then
d(x, y) = sup |ξi − ηi |
i∈N
defines a metric on X.
7. X = C[a, b] and
d(x, y) = max |x(t) − y(t)|.
t∈[a,b]
8. (Discrete metric)
1 x=y
d(x, y) = .
0 x=y
3
4. ∞
9. lp . x = (ξi ) ∈ lp if i=1 |ξi |
p < ∞, (p ≥ 1, fixed),
1
∞ p
d(x, y) = |ξi − ηi |p .
i=1
Problem 1.
¯
1. Show that d is a metric on C[a, b], where
b
¯
d(x, y) = |x(t) − y(t)|dt.
a
2. Show that the discrete metric is a metric.
3. Sequence space s: set of all sequences of complex numbers with the metric
∞
1 |ξi − ηi |
d(x, y) = . (1)
2i 1 + |ξi − ηi |
i=1
Solution.
¯
1. d(x, y) = 0 iff |x(t) − y(t)| = 0 for all t ∈ [a, b] because of the continuity. We have
¯ ¯ ¯
d(x, y) ≥ 0 and d(x, y) = d(y, x) trivially. We can argue the triangle inequality as
follows::
b b b
¯
d(x, y) = |x(t) − y(t)|dt ≤ |x(t) − z(t)|dt + ¯ ¯
|z(t) − y(t)|dt = d(x, z) + d(z, y).
a a a
2. Left as an exercise.
3. We show only the triangle inequality. Let a, b ∈ R. Then we have the inequalities
|a + b| |a| + |b| |a| |b|
≤ ≤ + ,
1 + |a + b| 1 + |a| + |b| 1 + |a| 1 + |b|
where in the first step we have used the monotonicity of the function
x 1
f (x) = =1− , for x > 0.
1+x 1+x
Substituting a = ξi − ζi and b = ζi − ηi , where x = (ξi ), y = (ηi ), and z = (ζi ) we get
|ξi − ηi | |ξi − ζi | |ζi − ηi |
≤ + .
1 + |ξi − ηi | 1 + |ξi − ζi | 1 + |ζi − ηi |
1
If we multiply both sides by 2i
and sum over from i = 1 to ∞ we get the stated
result.
4
5. 1.1 Open Sets, Closed Sets
Definition (Open Ball, Closed Ball, Sphere).
1. B(x0 , r) = {x ∈ X|d(x, x0 ) < r}
¯
2. B(x0 , r) = {x ∈ X|d(x, x0 ) ≤ r}
3. S(x0 , r) = {x ∈ X|d(x, x0 ) = r}
Definition (Open, Closed, Interior).
1. M is open if contains a ball about each of its points.
2. K ⊂ X is closed if K c = X − K is open.
3. B(x0 ; ε) denotes the ε neighborhood of x0 .
4. Int(M ) denotes the interior of M .
Remark 1.2 (Induced Topology). Consider the set X with the collection τ of all open
subsets of X. Then we have
1. ∅ ∈ τ, X ∈ τ .
2. The union of any members of τ is a member of τ .
3. The finite intersection of members of τ is a member of τ .
We call the pair (X, τ ) a topological space and τ a topology for X. It follows that a metric
space is a topological space.
¯
Definition (Continuous). Let X = (X, d) and Y = (Y, d) be metric spaces. The mapping
T : X → Y is continuous at x0 ∈ X if for every ε > 0 there is δ > 0 such that
¯
d(T x, T x0 ) < ε , ∀ x such that d(x, x0 ) < δ.
Theorem 1.3 (Continuous Mapping). T : X → Y is continuous if and only if the inverse
image of any open subset of Y is an open subset of X.
Proof.
1. Suppose that T is continuous. Let S ⊂ Y be open S0 the inverse image of S. Let
S0 = ∅ and take x0 ∈ S0 . We have T x0 = y0 ∈ S. Since S is open there exists an
ε-neighborhood of y0 , say N ⊂ S such that y0 ∈ N . The continuity of T implies
that x0 has a δ-neighborhood N0 which is mapped into N . Since N ⊂ S we get that
N0 ⊂ S0 , and it follows that S0 is open.
5
6. 2. Assume that the inverse image of every open set in Y is an open set in X. Then
∀ x0 ∈ X, and N (ε-neighborhood of T x0 ) the inverse image N0 of N is open.
Therefore N0 contains a δ-neighborhood of x0 . Thus T is continuous.
Some more definitions:
Definition (Accumulation Point). x ∈ M is said to be an accumulation point of M if
∃ (xn ) ⊂ M s.t. xn → x.
Definition (Closure). M is the closure of M .
Definition (Dense Set). M ⊂ X is in X dense if M = X.
Definition (Separable Space). X is separable if there is a countable subset which is dense
in X.
Remark 1.4.
1. If M is dense, then every ball in X contains a point of M .
2. R, C are separable.
3. A discrete metric space is separable if and only if it is countable.
Theorem 1.5. ℓ∞ is not separable.
Proof. Let y = (ηi ) where ηi = 0, 1. There are uncountably many y’s. If we put small balls
1
with radius 3 at the y’s they will not intersect. It follows that if M ⊂ l∞ is dense in l∞ ,
then M is uncountable. Therefore l∞ is not separable.
Problem 2. Show that lp , 1 ≤ p < ∞ is separable.
Solution. Let M the set of all sequences of the form x = (ξ1 , ξ2 , ..., ξn , 0, 0, ...), where n is
any positive integer and the ξi s are rational. M is countable. We argue that M is dense in
lp as follows. Let y = (ηi ) ∈ lp be arbitrary. Then for every ε > 0 there is an n such that
∞
εp
|ηi |p < .
2
i=n+1
Since the rationals are dense in R, for each ηi there is a rational ξi close to it. Hence there
is an x ∈ M such that
n
εp
|ηi − ξi | < .
2
i=1
It follows that d(y, x) < ε.
6
7. 1.2 Convergence, Cauchy Sequence, Completeness
Definition (Convergent Sequence). We say that the sequence (xn ) is convergent in the
metric space X = (X, d) if
lim d(xn , x) = 0 for some x ∈ X.
n→∞
We shall also use the notation xn → x.
Definition (Diameter). For a set M ⊂ X we define the diameter δ(M ) by
δ(M ) = sup d(x, y).
x,y∈M
M is said to be bounded if δ(M ) is finite.
Lemma 1.6.
1. A convergent sequence (xn ) in X is bounded and its limit x is unique.
2. If xn → x and yn → y in X, then d(xn , yn ) → d(x, y).
Problem 3. Prove Lemma 1.6.
Solution.
1. Suppose that xn → x. Then, taking ε = 1 we can find an N such that d(xn , x) < 1
for all n > N . By the triangle inequality we have
d(xn , x) < 1 + max d(x1 , x), d(x2 , x), ..., d(xN , x).
Therefore (xn ) is bounded. If xn → x and xn → z, then
0 ≤ d(x, z) ≤ d(xn , x) + d(xn , z) → 0 as n → ∞
and uniqueness of the limit follows.
2. We have
d(xn , yn ) ≤ d(xn , x) + d(x, y) + d(y, yn ),
and hence
d(xn , yn ) − d(x, y) ≤ d(xn , x) + d(yn , y).
Interchanging xn and x, yn and y, and multiplying by −1 we get
d(x, y) − d(xn , yn ) ≤ d(xn , x) + d(yn , y).
Combining the two inequalities we get
|d(xn , yn ) − d(x, y)| ≤ d(xn , x) + d(yn , y) → 0 as n → ∞.
7
8. Definition (Cauchy Sequence). (xn ) in (X, d) is a Cauchy sequence if ∀ ε > 0 ∃ N =
N (ε) s.t.
m, n > N ⇒ d(xm , xn ) < ε.
Definition (Complete). X is complete if every Cauchy sequence in X converges in X.
Theorem 1.7. R, C are complete metric spaces.
Theorem 1.8. Every convergent sequence in a metric space is a Cauchy sequence.
Theorem 1.9. Let M ⊂ X = (X, d), X is a metric space and let M denote the closure of
M in X. Then
1. x ∈ M iff ∃ (xn ) ∈ M s.t. xn → x.
2. M is closed iff xn ∈ M and xn → x imply that x ∈ M .
Theorem 1.10. Let X be a complete metric space and M ⊂ X. M is complete iff M is
closed in X.
˜
Theorem 1.11. Let T be a mapping from (X, d) into (Y, d). T : X → Y is continuous at
x0 ∈ X iff xn → x0 implies T xn → T x0 .
Problem 4.
1. Prove Theorem 1.10
2. Prove Theorem 1.11
Solution.
1. Let M be complete. Then for every x ∈ M there is a sequence (xn ) which converges
to x. Since (xn ) is Cauchy and M is complete xn → x ∈ M , therefore M is closed.
Conversely, let M be closed and (xn ) be Cauchy in M . Then xn → x ∈ X, which
implies x ∈ M , and therefore x ∈ M . Thus M is complete.
2. Assume that T is continuous at x0 . Then for ε > 0 ∃ δ > 0 such that
˜
d(x, x0 ) < δ implies d(T x, T x0 ) < ε.
Take a sequence (xn ) such that xn → x0 . Then ∃ N s.t. ∀ n > N we have
d(xn , x0 ) < δ and hence
˜
∀ n > N , d(T xn , T x)) < ε.
Conversely, assume that xn → x) ⇒ T xn → T x0 and show that T is continuous
at x0 . Otherwise, ∃ ε > 0 such that ∀ δ > 0, ∃ x = x0 ) satisfying d(x, x0 ) < δ
˜ 1 1
but d(T x, T x0 ) ≥ ε. Let δ = n . Then there is an xn such that d(xn , x0 ) < n but
˜
d(T xn , T x0 ) ≥ ε contrary to our assumption.
8
9. 1.3 Completeness Proofs
Theorem 1.12. Rn , Cn are complete.
Theorem 1.13. ℓ∞ is complete.
Proof. Let (xm ) be a Cauchy sequence in ℓ∞ .
(m) (n)
⇒ d(xm , xn ) = sup |xi − xi | < ε, for m, n > N (ε).
i
(m) (m)
⇒ for fixed i, the sequence (ξi ) is Cauchy in R, and therefore we have that ξi →
ξi , for i = 1, 2, 3, ... Let x = (ξi ). It follows easily that x ∈ ℓ∞ and xm → x.
Theorem 1.14. The space of convergent sequences x = (ξi ) of complex numbers with the
metric induced from ℓ∞ is complete.
Theorem 1.15. ℓp is complete if 1 ≤ p < ∞.
Theorem 1.16. C[a, b] is complete.
Proof. Let (xm ) be a Cauchy sequence in C[a, b]. Then
∀ ε > 0 ∃ N s.t. for m, n > N ⇒ d(xm , xn ) = max |xm (t) − xn (t)| < ε. (2)
t∈[a,b]
⇒ for any fixed t0 ∈ [a, b] we have that |xm (t0 ) − xn (t0 )| < ε. It follows that xm (t0 ) is
Cauchy and therefore xm (t0 ) → x(t0 ). Show that x(t) ∈ C[a, b] and xm → x. From (2)
with n → ∞ we get
max |xm (t) − x(t)| ≤ ε.
t∈[a,b]
Therefore xm (t) converges uniformly to x(t) on [a, b]. Since the xm ’s are continuous on
[a, b] and the convergence is uniform x(t) is continuous, and thus belongs to C[a, b].
Remark 1.17. Note that in C[a, b] convergence is uniform convergence; we also use the
terminology uniform metric for the metric generated by the “sup”-norm.
Example 1.18 (Incomplete Metric Spaces).
1. Q
2. The polynomials
3. C[a, b] with the · 2 norm
9
10. 1.4 Completion of Metric Spaces
¯ ¯ ¯
Definition (Isometry). Let X = (X, d) and X = (X, d) be metric spaces. The mapping
T :X→X ¯
¯ is an isometry if d(T x, T y) = d(x, y). The metric spaces X and X are isometric
¯
if there is a bijective isometry of X onto X ¯
Theorem 1.19. Let X = (X, d) be a metric space. Then there exists a complete metric
ˆ ˆ ˆ ˆ ˆ
space X = (X, d) which has a subspace W that is isometric with X and is dense in X. X
is unique except for isometries.
Proof. This proof is divided into steps.
′
ˆ
1. Construction of X. Let (xn ) and (xn ) be equivalent Cauchy sequences, i.e.,
′
lim d(xn , xn ) = 0.
n→∞
ˆ
Define X to be the set of all equivalence classes x of Cauchy sequences. Define now
ˆ
ˆx ˆ
d(ˆ, y ) = lim d(xn , yn ) where (xn ) ∈ x and (yn ) ∈ y .
ˆ ˆ (3)
n→∞
We show that limit in (3) exists and independent of the particular choice of the
representatives. We have using the triangle inequality
d(xn , yn ) ≤ d(xn , xm ) + d(xm , ym ) + d(ym , yn ).
It follows that
d(xn , yn ) − d(xm , ym ) ≤ d(xn , xm ) + d(ym , yn ).
Exchanging the role of n and m we get
d(xm , ym ) − d(xn , yn ) ≤ d(xn , xm ) + d(ym , yn ).
The two inequality together yield
|d(xn , yn ) − d(xm , ym )| ≤ d(xn , xm ) + d(ym , yn ).
It follows that (d(xn , yn )) is a Cauchy sequence in R and therefore it converges, i.e.,
the limit in (3) exists.
We show now that the limit in (3) is independent of the particular choice of repre-
′ ′
sentatives. Let (xn ), (xn ) ∈ x and (yn ), (yn ) ∈ y . Then
ˆ ˆ
′ ′ ′ ′
|d(xn , yn ) − d(xn , yn )| ≤ d(xn , xn ) + d(yn , yn ) → 0 as n → ∞,
which implies
′ ′
lim d(xn , yn ) = lim d(xn , yn ).
n→∞ n→∞
10
11. ˆ ˆ
2. Show d is a metric on X. Left as an exercise.
ˆ
3. Construction of an isometry T : X → W ∈ X. With each b ∈ X we associate the
class ˆ ∈ X which contains the constant Cauchy sequence (b, b, b, ...). This defines
b ˆ
ˆ
the mapping T : X → W onto the subspace W = T (X) ∈ X. The mapping T is
given by b → ˆ = T b, where (b, b, ...) ∈ ˆ According to (3) d(ˆ c) = d(b, c), so T is
b b. ˆ b, ˆ
an isometry. T is onto W since T (X) = W . ( W and X are isometric.)
ˆ ˆ
Show that W is dense in X. Let x ∈ X be arbitrary and let (xn ) ∈ x. For every
ˆ ˆ
ε
ε > 0 there is an N such that d(xn , xN ) < 2 , for n > N . Let (xN , xN , ...) ∈ xN .
ˆ
Then xN ∈ W , and by (3)
ˆ
ˆx ˆ ε
d(ˆ, xN ) = lim d(xn , xN ) ≤ < ε.
n→∞ 2
ˆ ˆ
4. Completeness of X. Let (ˆn ) be an arbitrary Cauchy sequence in X. Since W is
x
ˆ
dense in X for every xn there is a zn ∈ W such that
ˆ ˆ
ˆx ˆ 1
d(ˆn , zn ) < .
n
It follows using the triangle inequality and the Cauchyness of (ˆn ) that (ˆn ) is Cauchy.
x z
ˆ ˆ ˆ
Then (zn ), where zn = T −1 zn is Cauchy in X. Let x ∈ X be the class to which (zn )
belongs. We have
ˆx ˆ ˆx ˆ ˆz ˆ 1 ˆz ˆ
d(ˆn , x) ≤ d(ˆn , zn ) + d(ˆn , x) < + d(ˆn , x) < ε
n
for sufficiently large n.
ˆ ¯ ¯ ¯
5. Uniqueness of X. If (X, d) is another complete metric space with a subspace W dense
¯ ¯ ¯
in X and isometric with X, then W is isometric with W and the distances on X and
Xˆ must be the same. Hence X and X are isometric.
¯ ˆ
ˆ ˆ
Problem 5. Show now that d is a metric on X.
ˆx ˆ ˆx ˆ ˆx ˆ
Solution. Clearly, d(ˆ, y ) ≥ 0 because of the definition (3). Also, d(ˆ, x) = 0, and d(ˆ, y ) =
ˆ y , x), because of the properties of d.
d(ˆ ˆ
ˆx ˆ ˆx ˆ ˆz ˆ
d(ˆ, y ) ≤ d(ˆ, z ) + d(ˆ, y )
because we have
d(xn , yn ) ≤ d(xn , zn ) + d(zn , yn ) for all n.
11
12. Problem 6. A homeomorphism is a continuous bijective mapping T : X → Y whose inverse
is continuous. If such mapping exists, then X and Y are homeomorphic.
1. Show that is X and Y are isometric, then they are homeomorphic.
2. Illustrate with an example that a complete and an incomplete metric space may be
homeomorphic.
Solution. ˆ ˆ
1. If f : (X, d) → (Y, d) is an isometry, then d(f (x), f (y)) = d(x, y) < ǫ
ˆ
whenever d(x, y) < δ, so f is continuous. If (X, d) and (Y, d) are isometric, then
there exists a bijective isometry f : X → Y . It follows that f −1 : Y → X is
also an isometry, and that both f and f −1 are continuous. Therefore X and Y are
homeomorphic.
2. Let f : (− π , π ) → R be given by f (x) = tan x. f is cintinuous and bijective.
2 2
Moreover, f −1 : R → (− π , π ) given by f −1 (x) = tan−1 (x) is also continuous. So, R
2 2
is homeomorphic to (− π , π ).
2 2
¯ ¯
Problem 7. If (X, d) is complete, show that (X, d), where d = d
is complete.
1+d
¯
Solution. It is easy to see that if (xn ) is Cauchy in (X, d), then (xn ) is Cauchy in (X, d).
¯
Since (X, d) is complete, xn → x, but then we also have xn → x in (X, d). It follows that
¯ is complete.
(X, d)
12
13. 2 Normed Spaces
Vector spaces, linear independence, finite- and infinite-dimensional vector spaces.
Definition (Basis). If X is any vector space, and B is a linearly independent subset of X
which spans X, then B is called a basis (or Hamel basis) for X.
Definition (Banach Space). A complete normed space.
Definition (norm). A real-valued function · : X → R satisfying
1. x ≥ 0
2. x = 0 iff x = 0
3. αx = |α| x
4. |x + y| ≤ |x| + |y|
Definition (Induced Metric). In a normed space (X, d) we can always define a metric by
d(x, y) = x − y .
Remark 2.1. All previous results about metric spaces apply to normed spaces with the
induced metric.
Problem 8. The norm is continuous, that is · : X → R by x → x is a continuous.
Solution. The triangle inequality implies that
| x − y | ≤ x−y .
Example 2.2 (Norm Spaces). Rn , Cn , ℓp , ℓ∞ , C[a, b], L2 [a, b], Lp [a, b].
Remark 2.3 (Induced Norm is Translation Invariant). A metric induced by a norm on
a normed space is translation invariant, i.e., d(x + a, y + a) = d(x, y) and d(αx, αy) =
|α|d(x, y). For example the metric (1) is not coming from a norm.
Theorem 2.4. A subspace Y of a Banach space X is complete iff the set Y is closed in
X.
Definition (Convergent, Cauchy, Absolutely Convergent).
1. (xn ) is convergent if xn − x → 0. (x is the limit of (xn )).
2. (xn ) is Cauchy if xm − xn < ǫ for m, n > N (ǫ).
13
14. Definition (Infinite series). Let (xk ) be a sequence and consider the partial sums
sn = x1 + x2 + ... + xn .
∞
If sn converges, say sn → s, then k=1 xk is convergent and
∞
s= .
k=1
∞
If x1 + x2 + ... + xn + ... converges, then k=1 xk is absolutely convergent.
Remark 2.5. Absolute convergence implies convergence iff X is complete.
Definition (Schauder basis). If X contains a sequence (en ) such that for every x ∈ X
there exists a unique sequence (αn ) with the property that
x − (α1 e1 + ... + αn en ) → 0 as n → ∞,
then (en ) is called a Schauder Basis for X. The representation
∞
x= αk ek
k=1
is called the expansion of x with respect to (en ).
Remark 2.6. If X has a Schauder basis, then X is separable.
ˆ
Theorem 2.7. Let X = (X, · ) be a normed space. Then there is a Banach space X and
ˆ which is dense in X. X is unique, except
an isometry A from X onto a subspace W ⊂ X ˆ ˆ
for isometries.
Theorem 2.8. Let X be a normed space, where the absolute convergence of any series
always implies convergence. Then X is complete.
Proof. Let (sn ) be any Cauchy sequence in X. Then for every k ∈ N there exists nk
such that sn − sm < 2−k , (m, n > nk ) and we can choose nk+1 > nk for all k. Then
(snk ) is a subsequence of (sn ) and is the sequence of the partial sums of xk , where
x1 = sn1 , ..., xk = snk − snk−1 , .... Hence
xk ≤ x 1 + x 2 + 2−k = x1 + x2 + 1.
It follows that xk is absolutely convergent. By assumption, xk converges, say, snk →
s ∈ X. Since (sn ) is Cauchy, sn → s because
s n − s ≤ s n − s nk + s nk − s .
Since (sn ) was arbitrary, sn → s shows that X is complete.
14
15. 2.1 Finite dimensional Normed Spaces or Subspaces
Theorem 2.9 (Linear Combinations). Let X be a normed space, and let
{x1 , . . . , xn }
be a linearly independent set of vectors in X. Then ∃ c > 0 s.t.
α1 x1 + · · · + αn xn ≥ c (|α1 | + · · · + |αn |) .
Proof. Let
s = |α1 | + · · · + |αn |.
If s = 0 we are done, so assume s > 0. Define
αi
βi = .
s
Clearly we have that
n
|βi | = 1.
i=1
Then we can reduce the above inequality to
n
βi xi ≥ c. (4)
i=1
We will prove the result for Theorem 4.
Suppose Theorem 4 is not true. Then ∃ (ym )
n
(m)
ym = βi xi
i=1
such that
ym → 0 as m → ∞.
n (m) (m)
Note that i=1 |βi | = 1 ⇒ |βi | ≤ 1 ∀ i. Hence for each fixed i the sequence
(m) (1) (2)
βi = βi , βi , . . .
(m)
is bounded. Therefore β1 has a convergent subsequence (B-W). Let β1 be the limit, and
let (y1,m ) be the corresponding subsequence of (ym )
n
(m) (m)
(y1,m ) = γ1 x1 + βi xi
i=2
15
16. (m)
γ1 → β1 .
Then there is a there exist corresponding β2 , (y2,m ) where
(m)
β2 → β2 as n → ∞
n
(m) (m) (m)
(y2,m ) = γ1 x1 + γ2 x2 + βi xi .
i=3
Since n is finite this process will terminate with
n
(m)
(yn,m ) = γi xi
i=1
with (yn,m ) a subsequence of (ym ). By construction,
(m)
γi → βi ∀ i
n
|βi | = 1.
i=1
Therefore (yn,m ) has a limit, say
n
y= βi xi .
i=1
Since ym → 0 by assumption we must also have yn,m → 0, We know y = 0 since
n
i=1 |βi | = 1 and {xi } is a basis, a contradiction.
Theorem 2.10 (Completeness). Every f.d.s.s. Y of a n.s. X is complete. In particular,
every f.d.n.s. is complete.
Proof. Let (ym ) be a Cauchy sequence in Y . We must find a limit y such that
ym → y
y ∈ Y.
Let n = dim Y , and let {e1 , . . . , en } be a basis for Y . Then ∀ m we can represent ym as
n
(m)
ym = αi ei .
i=1
(m)
But then for each fixed i sequence (αi ) is Cauchy in R. So each of these sequence
converges, say
(m)
αi → αi .
16
17. Then define
n
y= αi ei .
i=1
Clearly, y ∈ Y and ym → y.
Theorem 2.11. Every f.d.s.s. Y of a closed n.s. X is closed in X.
Proof. By Theorem 2.10 Y must be complete. Therefore Y is closed in X.
Note that finiteness in the previous proof is essential. Infinite dimensional subspaces
need not be closed in X. For example consider
X = C[0, 1]
Y = span ({1, t, ..., tn , ..}) .
Then the sequence (ti ) converges to
0 t<1
f (t) =
1 t=1
which is not in Y .
Definition (Equivalent Norms). · 1 and · 2 are equivalent if ∃ a, b > 0 s.t.
a x 2 ≤ x 1 ≤b x 2, ∀ x ∈ X.
Theorem 2.12 (Finite Dimesional ⇒ All Norms are Equivalent). On a f.d.n.s. X every
two norms · 1 , · 2 are equivalent.
Proof. Let n = dim X, let {e1 , . . . , en } be a basis for X with ei 1 = 1, let x ∈ X be
represented as
n
x= αi ei .
i=1
Then ∃ c > 0 s.t.
n
x 1 ≥c |αi |.
i=1
Also note
n n
x 2 ≤ |αi | ei 2 ≤k |αi |
i=1 i=1
where
k = max { ei 2 } .
17
18. Combining these inequalities we obtain
n
k 1
x 2 ≤ c |αi | ≤ x 1,
c a
i=1
where
c
a=
k
and we obtain the inequality involving a. A similar argument yields for the inequality
involving b.
Problem 9. What is the largest possible c > 0 in
n n
αi xi ≥ c |αi |
i=1 i=1
if
1. X = R2 , x1 = (1, 0), x2 = (0, 1).
2. X = R3 , x1 = (1, 0, 0), x2 = (0, 1, 0), x3 = (0, 0, 1).
1 1
Solution. Find min{ βi xi : |βi | = 1} and obtain √ , √ ,
2 3
respectively.
Problem 10. Show that if · 1 , · 2 are equivalent norms on X then the Cauchy sequences
in (X, · 1 ), (X, · 2 ) are the same.
Solution. Suppose that (xn ) is a Cauchy sequence in · 2 . Then for every ǫ > 0 there
ǫ
exists N such that xn − xm 2 < b for all m, n ≥ N , where b is a constant for which
x 1 ≤ b x 2 . Then xn − xm 1 ≤ b xn − xm 2 < ǫ for all m, n ≥ N . It follows that (xn )
is Cauchy in · 1 . A similar argument works in the other direction.
2.2 Compactness and Finite Dimension
Definition (Compactness). A metric space (X, d) is compact if every sequence in X has
a convergent subsequence. M ⊂ X is compact if M is compact as a subspace of X, i.e., if
every sequence in M has a convergent subsequence which converges to an element in M .
Theorem 2.13. If M is compact then M is closed and bounded.
Proof. First we show closed. For all x ∈ M ∃ (xn ) ⊂ M s.t. xn → x. M is compact ⇒
x ∈ M , hence M is closed.
To show boundedness assume it is not true. Then fix b ∈ M and choose yn ∈
M s.t. d(yn , b) > n. Then (yn ) ⊂ M is a sequence with no convergent subsequence,
contradicting the assumption that M is compact. Thus M is bounded.
18
19. Note the converse of this theorem is false. Consider M ⊂ ℓ2 as
M = xn ∈ ℓ2 s.t. xn = (en ), (en )i = δni .
Then M is bounded since xn = 1 and M is closed because there are no accumulation
points √
xn − xm = 2.
But the sequence (xn ) has no convergent subsequence, so M is not compact. In some sense,
there is too much room in the infinite dimensional setting.
Theorem 2.14 (Compactness). Let X be a f.d.n.s. . Then M ⊂ X is (sequentially)
compact iff M is closed and bounded.
Proof. The “⇒” case was proved by the previous theorem. For the other direction, assume
M is closed and bounded.
For the other direction (“⇐”), assume n = dim X, {e1 , . . . , en } is a basis for X, and
(xm ) ⊂ M is an arbitrary sequence. Then there is a representation
m
(m)
xm = ξi ei , ∀ m.
i=1
Since M is bounded, (xm ) is bounded so ∃ k ≥ 0 s.t.
xm ≤ k.
So then
k ≥ xm
n
(m)
= ξi ei
i=1
n
(m)
≥ c |ξi |
i=1
(m) (m)
and therefore ∀ i, (ξi ) ⊂ R is bounded and therefore ∀ i, (ξi ) has an accumulation
point z. Since M is closed, z ∈ M and there is a subsequence (zm ) ⊂ (xm ) s.t. zm → z,
say,
n
z= ξi ei .
i=1
Since (xm ) was arbitrary, M is compact.
We shall need the following technical result to prove subsequent theorems.
19
20. Theorem 2.15 (Riesz). Let Y, Z be subspaces of X, and suppose Y is closed and is a
proper subspace of Z. Then ∀ θ ∈ (0, 1) ∃ z s.t.
z =1
z − y ≥ θ, ∀ y ∈ Y.
Proof. Let v ∈ Z − Y be arbitrary and let
a = inf v − y .
y∈Y
Note if a = 0 then v ∈ Y since Y is closed. Therefore a > 0.
Choose θ ∈ (0, 1). Then ∃ y0 ∈ Y s.t.
a
a ≤ v − y0 ≤ .
θ
Let
1
z = c(v − y0 ) where c = .
v − y0
By construction, z = 1. We want to show z − y ≥ θ, ∀ y ∈ Y . So
z−y = c(v − y0 ) − y
y
= c v − y0 −
c
= c v − y1
for some y1 ∈ Y, ⇒ v − y1 ≥ a ⇒
z−y = v − y1
≥ ca
a
=
v − y0
a
≥ a
θ
= θ.
Theorem 2.16 (Finite Dimension and Compactness). Let X be a normed space. Then
the closed unit ball
M = {x s.t. x ≤ 1}
is compact iff dim X < ∞.
20
21. Proof. “⇒”. Suppose M is compact, but dim X = ∞. Pick x1 , x1 = 1. Then x1
generates a closed, 1-dimensional subspace X1 X. Then also ∃ x2 ∈ X s.t. x2 = 1
and
1
x 2 − x1 ≥ θ = .
2
Then x1 , x2 generate a closed, 2-dimensional subspace X2 X. Then ∃ x3 ∈ X s.t. x3 =
1 and
1
x3 − xi ≥ θ = , ∀ i = 1, 2.
2
In this way construct xn s.t.
1
xn − xi ≥ θ = , ∀ i = 1, . . . , n − 1.
2
Then (xn ) ⊂ X is a sequence such that xm − xn ≥ 1 , ∀ m, n. Therefore (xn ) has no
2
accumulation point, contradicting that M was compact.
“⇐” was proved by Theorem 2.14.
Theorem 2.17 (Continuous Preserves Compact). Let T : X → Y be continuous. Then
M ⊂ X compact ⇒ T (M ) ⊂ Y compact.
Proof. Let (yn ) ⊂ T (M ) be arbitrary. Then ∀ n ∃ xn s.t. yn = T xn . Then (xn ) ⊂ M
has a convergent subsequence (xnk ). Since T is continuous, the image (ynk ) = T (xnk ) also
converges. Therefore T (M ) is compact.
Corollary 2.18 (Max, Min). Let T : X → R be continuous and let M ⊂ X be compact.
Then T obtains its max and min on M .
Proof. T (M ) is compact ⇒ T (M ) is closed and bounded. Therefore
inf T (M ) ∈ T (M )
sup T (M ) ∈ T (M ).
Problem 11. Show that a discrete metric space with infinitely many points is not compact.
Solution. The space contains an infinite sequence (xn ), xn = xm (n = m) which cannot
have a convergent subsequence since d(xn , xm ) = 1.
Problem 12 (Local Compactness). A metric space X is said to be locally compact if every
point x ∈ X has a compact neighborhood. Show that R, Rn , C, Cn are locally compact.
¯
Solution. The closed ball B(x, ǫ) around an arbitrary point x is compact.
21
22. 2.3 Linear Operators
into
Definition (Linear, Domain and Range). Let X, Y be vector spaces and let T : X −→ Y .
Then
1. If ∀ x, y ∈ D (T ) and scalars α, β
T (αx + βy) = αT x + βT y
then T is said to be a linear operator.
2. The null space of T is N (T ) = {x ∈ D (T ) s.t. T x = 0}
3. The domain of T , D (T ), is a vector space
4. The range of T , R (T ), lies in a vector space
Note that obviously D (T ) ⊂ X. It is customary to write
onto
T : D (T ) −→ R (T ) .
Also, T 0 = T (0 + 0) = T 0 + T 0 ⇒ T 0 = 0 ⇒ N (T ) = ∅.
Example 2.19 (Differentiation). Let X be the space of polynomials on [a, b]. Then define
onto
T : X −→ X
d
T x(t) = x(t).
dt
into
Example 2.20 (Integration). Let X = C[a, b]. Then define T : X −→ X by
t
T x(t) = x(s)ds.
a
into
Example 2.21 (Multiplication). Let X = C[a, b]. Then define T : X −→ X by
T x(t) = tx(t).
Example 2.22 (Matrices). Let A = (aij ). Then define T : Rm → Rn by
T x = Ax.
Theorem 2.23 (Range and Null Space). Let T be a linear operator.
1. R (T ) is a vector space.
2. If dim D (T ) = n < ∞ then dim R (T ) ≤ n.
22
23. 3. N (T ) is a vector space.
Proof. 1. Let y1 , y2 ∈ R (T ) and let α, β be scalars. Since y1 , y2 ∈ R (T ) , ∃ x1 , x2 ∈
D (T ) s.t.
T x1 = y 1
T x2 = y 2 .
Since D (T ) is vector space, αx1 + βx2 ∈ D (T ), so
T (αx1 + βx2 ) ∈ R (T ) .
But
T (αx1 + βx2 ) = αT x1 + βT x2 = αy1 + βy2
means R (T ) is a vector space.
2. Pick n + 1 elements arbitrarily in R (T ). Then we have
yi = T xi , ∀ i = 1, . . . , n + 1
for some {x1 , . . . , xn+1 } in D (T ). Since dim D (T ) = n, this set must be linearly
dependent. Similarly, the points in the range are also linearly dependent. Therefore,
dim R (T ) ≤ n.
3. Let x1 , x2 ∈ N (T ) , i.e. , T x1 = T x2 = 0, and let α, β be scalars. But then
T (αx1 + βx2 ) = αT x1 + βT x2 = 0 ⇒ αx1 + βx2 ∈ N (T ) .
Therefore N (T ) is a vector space.
The next result shows linear operators preserve linear independence.
into
Definition (Injective (One-to-one)). T : D (T ) −→ Y is injective (or one-to-one) if
∀ x1 , x2 ∈ D (T ) s.t. x1 = x2
T x1 = T x2 .
Equivalently,
T x 1 = T x 2 ⇒ x1 = x2 .
into onto
So therefore if T : D (T ) −→ Y is injective then ∃ T −1 : R (T ) −→ D (T )
y 0 → x0
y 0 = T x0
and therefore
T −1 T x = x, ∀ x ∈ D (T )
T T −1 y = y, ∀ y ∈ R (T ) .
23
24. into
Theorem 2.24 (Inverses). Let X, Y be vector spaces and T : D (T ) −→ Y be linear with
D (T ) ⊂ X and R (T ) ⊂ Y.
onto
1. T −1 : R (T ) −→ D (T ) exists iff T x = 0 ⇒ x = 0.
2. If T −1 exists then it is linear.
3. If dim D (T ) = n < ∞ and T −1 exists then dim R (T ) = dim D (T ).
Proof.
1. Suppose T x = 0 ⇒ x = 0. Then
T x1 = T x2 ⇒ T (x1 − x2 ) = 0 ⇒ x1 = x2
So T is injective T −1 exists.
Conversely, if T −1 exists then T x1 = T x2 ⇒ x1 = x2 . Therefore
T x = T 0 = 0 ⇒ x = 0.
2. We assume that T −1 exists and show that it is linear in a straightforward fashion.
3. It follows from the earlier result applied to both T and T −1 .
Theorem 2.25 (Inverse of Product (Composition)). Let T : X → Y and S : Y → Z be
bijections. Then (ST )−1 : Z → X exists and
(ST )−1 = T −1 S −1 .
2.4 Bounded and Continuous Linear Operators
into
Definition (Bounded Linear Operator). Let X, Y be normed spaces and T : D (T ) −→
Y, D (T ) ⊂ X. Then T is said to be bounded if ∃ k > 0 s.t.
T x ≤ k x , ∀ x ∈ D (T ) .
into
Definition (Induced Operator Norm). Let X, Y be normed spaces and T : D (T ) −→
Y, D (T ) ⊂ X. Then the norms on X, Y induce a norm for operators
Tx
T = sup .
x∈D(T ) x
x=0
Equivalently,
T = sup Tx .
x∈D(T )
x =1
24
25. The preceding definition implies that
Tx ≤ T x , ∀ x.
Example 2.26 (Differention is Unbounded). Let T : C[0, 1] → C[0, 1],be defined by
′
T x(t) = x (t). Then T is linear but unbounded. Indeed, let xn (t) = tn . Then xn = 1
and T xn = n, i.e., T is not bounded.
Example 2.27 (Integration is Bounded). Let T : C[0, 1] → C[0, 1], k ∈ C[0, 1]2 , y = T x
via
1
y(t) = k(t, τ )x(τ )dτ.
0
Since k is continuous, ∃ k0 > 0 s.t.
|k(t, τ )| ≤ k0 .
So then
y = Tx
1
≤ max |k(t, τ )| x(t) dτ
t∈[0,1] 0
≤ k0 x
Theorem 2.28 (Finite Dimension). Let X be normed and dim X = n < ∞. Then every
linear operator on X is bounded.
Proof. Let e1 , . . . , en be a basis for X and let x ∈ X be arbitrary and represented as
n
x= αi ei .
i=1
Then
n
n
Tx = αi T ei ≤ k |αi |,
i=1
i=1
where k = maxi=1,...,n T ei . Also,
n
1
|αi | ≤ x .
c
i=1
It follows that T is bounded.
Next, we illustrate a general method for computing bounds on operators.
25
26. Example 2.29 (Calculating Bounds on Operators). Let
n
x= ξi ei .
i=1
Then
n
Tx = ξi T ei
i=1
n
≤ |ξi | T ei
i=1
n
≤ max T ek |ξi | .
k=1,...,n
i=1
Then using Theorem 2.9 we can conclude
n n
1 1
x = ξi ei ≥ |ξi |
c c
i=1 i=1
which implies
1
Tx ≤ γ x where γ = max T ek .
c k=1,...,n
into
Definition (Continuity for Operators). Let T : D (T ) −→ Y . Then T is said to be
continuous at x0 ∈ D (T ) if
∀ ε > 0 ∃ δ > 0 s.t. x − x0 < δ ⇒ T x − T x0 < ε.
T is said to be continuous if it is continuous at every x0 ∈ D (T ).
Theorem 2.30 (Continuity and Boundedness). Let X, Y be vector spaces D (T ) ⊂ X, and
into
T : D (T ) −→ Y be linear.
1. T is continuous iff T is bounded
2. T is continuous at a single point ⇒ T is continuous everywhere
Proof.
1. The case T = 0 is trivial. Assume T = 0, then T = 0. Suppose T is bounded and
let x0 ∈ D (T ) , ε > 0 be arbitrary. Then because T is linear, ∀ x ∈ D (T ) s.t.
ε
x − x0 < δ =
T
26
27. we have that
T x − T x0 = T (x − x0 ) ≤ T x − x0 < T δ = ε.
Therefore T is continuous.
Conversely, suppose T is continuous at x0 ∈ D (T ). Then ∀ ε > 0 ∃ δ > 0 s.t.
x − x0 ≤ δ ⇒ T x − T x0 ≤ ε.
So choose any y ∈ D (T ) , y = 0 and set
y
x = x0 + δ .
y
Then
y
x − x0 = δ ⇒ x − x0 = δ,
y
so
T x − T x0 = T (x − x0 )
δ
= T y
y
δ
= Ty
y
≤ ε.
Therefore we have
ε
Ty ≤ c y where c = .
δ
2. Above we only used continuity at a single point to show boundedness. Boundedness
implies continuity.
into
Corollary 2.31. Let T : D (T ) −→ Y be a bounded, linear operator. Then
1. (xn ) ⊂ D (T ) , x ∈ D (T ) and xn → x ⇒ T xn → T x
2. N (T ) is closed.
Proof. 1. T xn − T x = T (xn − x) ≤ T xn − x → 0
2. ∀ x ∈ N (T ) ∃ (xn ) ⊂ N (T ) s.t.
xn → x ⇒ T xn → Tx .
But xn ∈ N (T ) ⇒ T xn = 0, ∀ n ⇒ T x = 0 ⇒ x ∈ N (T ). Therefore N (T ) is
closed.
27
28. Compare to the notion of subadditivity (triangle inequality).
Remark 2.32 (Operator Norm is Submultiplicative).
1. TS ≤ T S
n
2. Tn ≤ T
Remark 2.33 (Equality of Operators). Two operators T, S are said to be equal and we
write T = S if
D (T ) = D (S) and T x = Sx, ∀ x ∈ D (T ) .
into
Definition (Rescriction). Let T : D (T ) −→ Y and let B ⊂ D (T ). We define the restric-
into
tion of T to B, denoted T |B : B −→ Y , as
T |B x = T x, ∀ x ∈ B.
into
Definition (Extension). Let T : D (T ) −→ Y and D (T ) ⊂ M . An extension of T is
ˆ into
T : M −→ Y s.t.
ˆ
T |D (T ) = T
ˆ
T x = T x, ∀ x ∈ D (T ) .
Theorem 2.34 (Bounded Linear Extension). Let D (T ) ⊂ X, X a normed space, Y a
into
Banach space, and T : D (T ) −→ Y be linear. Then there is a unique bounded, linear
ˆ
extension T of T s.t.
ˆ
T = T .
Proof. Let x ∈ D (T ). Then ∃ (xn ) ∈ D (T ) s.t. xn → x. Since T is linear and bounded
we have
T xn − T x = T (xn − x) ≤ T xn − x ,
so (T xn ) is Cauchy. Since Y is Banach, (T xn ) converges, say
T xn → y ∈ Y.
In this way, we define
ˆ
T x = y.
ˆ ˆ
Because limits are unique, T is uniquely defined ∀ x ∈ D (T ). Of course T inherits linearity
and
ˆ
T x = T x, ∀ x ∈ D (T )
ˆ
so T is a genuine extension.
28
29. ˆ
Finally, we need to show T is bounded. Note
T xn ≤ T xn
and recall
x→ x
ˆ
is continuous. Therefore T xn → y = T x and
ˆ
Tx ≤ x x .
ˆ ˆ ˆ
Therefore T is bounded and T ≤ T . By definition, T ≥ T so
ˆ
T = T .
2.5 Linear Functionals
Definition (Linear Functional). A linear functional f is a linear operator with D (f ) ⊂ X
where X is vector space and R (f ) ⊂ K where K is the scalar field corresponding to X (R
or C). Then
into
f : D (f ) −→ K.
into
Definition (Bounded Linear Functional). Let f : D (f ) −→ K be a linear functional.
Then f is said to be bounded if ∃ c > 0 s.t.
|f (x)| ≤ c x , ∀ x ∈ D (f ) .
Definition (Norm of a Linear Functional). Let f be a linear functional. Then
|f (x)|
f = sup = sup |f (x)| ,
x∈D(f ) x x∈D(f )
x=0 x =1
so
|f (x)| ≤ f x .
Theorem 2.35 (Continuity and Boundedness). Let f be a linear functional. f is contin-
uous iff f is bounded.
29
30. Example 2.36 (Dot Product). Let a ∈ R, a = 0 be fixed and let f : Rn → R be defined
as
f (x) = x · a.
Then f is linear functional and
|f (x)| ≤ x a .
On the other hand, if we take x = a
2
|f (x)| = a ⇒ f ≥ a .
Example 2.37 (Integral). Define f : C[a, b] → R by
b
f (x) = x(t)dt.
a
Then
|f (x)| ≤ (b − a) x .
Further, if x = 1
|f (x)| = b − a ⇒ f = b − a.
Example 2.38 (Point Evaluation). Let t0 ∈ [a, b] be fixed and define f : C[a, b] → R by
f (x) = x(t0 ).
Choosing the sup-norm, we see
|f (x)| = |x(t0 )| ≤ x ,
and since f (1) = 1 = 1 , we have
f = 1.
Example 2.39 (Point Evaluation in L2 ). Let f : L2 [a, b] → R by
f (x) = x(a).
Choose the L2 norm, and let x ∈ L2 [a, b] be such that x(a) = 1 but then x rapidly decreases
to 0. Then f (x) = 1 but
1
b 2
x = |x(t)|2 dt
a
and for any ε > 0 we can choose an x such that x < ε. Therefore there is no such
c > 0 s.t.
|f (x)| = 1 ≤ c x .
So point evaluation in this space with this norm is unbounded.
30
31. Example 2.40 (Inner Product in ℓ2 ). Fix a ∈ ℓ2 and define f : ℓ2 → R by
∞
f (x) = xi ai = x, a .
i=1
Then
∞
|f (x)| ≤ |xi ai |
i=1
∞ ∞
≤ x2
i a2
i
i=1 i=1
= x a .
Definition (Algebraic Dual). Let X be a vector space. Then we denote X ∗ as the space
of linear functionals on X. X ∗∗ represents the dual of X ∗ , et cetera.
Space General Element Value at a Point
X x n/a
X∗ f f (x)
X ∗∗ g g(f )
Definition (Isomorphism = Bijective Isometry). Let X, Y be vector spaces over the same
scalar field K. Then an isomorphism T : X → Y is a bijective mapping which preserves
the two algebraic operations of vector spaces, i.e.
T (αx + βy) = αT x + βT y.
So T is a bijective linear operator and we say X, Y are isomorphic.
If in addition X, Y are normed spaces, then T must also preserve the norms
x = Tx .
We can obtain g ∈ X ∗∗ picking a fixed x ∈ X and setting
g(f ) = gx (f ) = f (x), ∀ f ∈ X ∗ .
This g is linear, and x ∈ X ⇒ gx ∈ X ∗∗ .
Definition (Canonical Mapping/Embedding). Define C : X → X ∗∗ by
x → gx .
Then C is injective and linear. Therefore C is an isomorphism of X and R (C) ⊂ X ∗∗ .
31
32. An interesting and important problem is when R (C) = X ∗∗ , i.e., X, X ∗∗ are isomor-
phic.
Definition (Embeddable). Let X, Y be vector spaces. X is said to be embeddable in Y if
it is isomorphic with a subspace of Y .
From Definition 2.5 we can see X is always embeddable in X ∗∗ . C is also called the
canonical embedding of X into X ∗∗ .
Definition (Algebraic Reflexive). If the canonical mapping C is surjective (and hence
bijective) then
R (C) = X ∗∗
and X is said to be algebraically reflexive.
Problem 13. If Y is a subspace of a vector space X and f is a linear functional on X such
that f (Y ) is not the whole scalar field K, show
f (y) = 0, ∀ y ∈ Y.
Solution. Suppose that f (y) = b = 0 for some y ∈ Y . For arbitrary a ∈ R we have a y ∈ Y
b
and f ( a y) = a. It follows that f (Y ) = R; a contradiction.
b
Problem 14. Let f = 0 be a linear functional on a vector space X, and let x0 ∈ X − N (f )
be fixed. Show that every x ∈ X has a unique representation
x = αx0 + y, some y ∈ N (f ) .
Solution. Let f = 0 and consider x0 ∈ X − N (f ). Then f (x0 ) = c = 0. Let a = f (x) =
f ( a x0 ). It follows that x − a x0 ∈ N (f ). We can write
c c
a a
x = x0 + y , where y = x − x0 ∈ N (f ).
c c
2.6 Finite Dimensional Case
Let X, Y be finite dimensional vector spaces over the same field K, and let T : X → Y be
a linear operator. Let E = {e1 , . . . , en } , B = {b1 , . . . bn } be bases for X, Y respectively.
Then x ∈ X and y = T x ∈ Y and T ek ∈ Y, ∀ k have the unique representations
n
x = ξk ek
k=1
n
T ek = τik bi
i=1
n
y = ηi bi .
i=1
32
33. Then calculate
y = Tx
n
= T ξk ek
k=1
n
= ξk T ek
k=1
which implies
n n n
y= ηi bi = ξk τik bi
i=1 k=1 i=1
n n
= τik ξk bi
i=1 k=1
yielding
n
ηi = τik ξk .
k=1
So if we label
TEB = (τik ), x = (ξk ), y = (ηi )
¯ ¯
then
y = TEB x.
¯ ¯
We say TEB represents T with respect to the bases E, B.
Remark 2.41 (Finite Dimensional Linear Operators are Matrices). The above argument
shows that every f.d.l.o. can be represented as a matrix.
2.6.1 Linear Functionals
Let X be a vector space, n = dim X, and let {e1 , . . . , en } be a basis for X. Label
αi = f (ei ).
Then
n n
f (x) = f ξi ei = ξi αi .
i=1 i=1
(f is uniquely determined by its values αi at the n basis vectors of X).
33
34. Definition (Dual Basis). Define fk by
fk (ei ) = δik , ∀ k = 1, . . . , n.
Then
bij
{f1 , . . . , fn } ↔ {e1 , . . . , en } .
Theorem 2.42 (Dimension of X ∗ ). Let X be a vector space, n = dim X, and let E =
{e1 , . . . , en } be a basis for X. Then {f1 , . . . , fk } is a basis for X ∗ and dim X ∗ = n. So
then if f ∈ X ∗ then we can represent
n
f= αi fi .
i=1
Proof. See Theorem 2.41.
Lemma 2.43 (Zero Vector). Let X be a fdvs. If x0 ∈ X has the property that f (x0 ) =
0, ∀ f ∈ X ∗ then x0 = 0.
Proof. Let {e1 , . . . , en } be a basis for X and let
n
x0 = ξ0i ei .
i=1
Then
n
f (x0 ) = ξ0i αi .
i=1
By assumption f (x0 ) = 0, ∀ f ∈ X ∗ ⇒ ξ0i = 0, ∀ i = 1, . . . , n.
Theorem 2.44 (Algebraic Reflexivity). A fdvs X is algebraically reflexive, i.e., X is
isomorphic to X ∗∗ .
into
Proof. By construction C : X −→ X ∗∗ is linear. Then
Cx0 = 0 ⇒ (Cx0 )(f ) = gx0 (f ) = f (x0 ) = 0, ∀ f ∈ X ∗
and therefor x0 = 0 by the previous lemma. Therefore C has an inverse C −1 : R (C) → X.
Therefore dim R (C) = dim X. Therefor C is an isomorphism and X is algebraically
reflexive.
Problem 15. Find a basis for the null space of functional f : R3 → R given by
f (x) = α1 ξ1 + α2 ξ2 + α3 ξ3
where α1 = 0.
Solution. Let (α1 , α2 , α3 ) with α1 = 0 be a fixed point in R3 and let f (x) = ai xi . The
vectors (− α2 , 1, 0) and (− α3 , 0, 1) form a basis for N (f ).
α1 α2
34
35. 2.7 Dual Space
Definition (B(X, Y )). Let X, Y be vector spaces. Then we define B(X, Y ) as the set of
all bounded linear operators from X to Y .
Remark 2.45 (B(X, Y ) is a Vector Space). (αS + βT )x = αSx + βT x
Theorem 2.46 (B(X, Y ) is a Normed Space). Let X, Y be normed spaces. The vector
space B(X, Y ) is also a normed space with norm
Tx
T = sup = sup T x .
x∈X x x∈X
x=0 x =1
Theorem 2.47 (B(X, Y ) is a Banach Space). If in the assumptions of the previous proof
Y is Banach, then B(X, Y ) is also a Banach space.
Proof. Let (Tn ) be an arbitrary Cauchy sequence in B(X, Y ). Then
∀ ε > 0 ∃ N s.t. Tn − Tm < ε, ∀ m, n > N.
Therefore ∀ x ∈ X, and m, n > N
Tn x − Tm x = (Tn − Tm )x
≤ Tn − Tm x
< ε x .
So fix x ∈ X. We see ∀ x ∈ X that (Tn x) ⊂ Y is Cauchy. Y is complete so ∃ y ∈
Y s.t. Tn x → y. We need to construct a limit for (Tn ). Define T : X → Y by
T x = y.
By construction, T is linear
lim Tn (αx + βy) = lim (αTn x + βTn x)
n→∞ n→∞
= α lim Tn x + β lim Tn x
n→∞ n→∞
= αT x + βT x.
The following
Tn x − T x = Tn x − lim Tm x
m→∞
= lim Tn x − Tm x
m→∞
≤ ε x
35
36. implies that Tn − T is bounded. Since Tn ∈ B(X, Y ) ⇒ Tn is bounded and
T = Tn − (Tn − T )
T is also bounded. Therefore T ∈ B(X, Y ).
Finally,
Tn x − T x
≤ε
x
implies
Tn − T ≤ ε ⇒ Tn − T → 0.
Definition (Normed Dual Space X ′ ). Let X be a n.s.. Then the set of all bounded linear
functionals on X constitutes a n.s. with norm
|f (x)|
f = sup = sup |f (x)| .
x∈X x x∈X
x=0 x =1
This is the normed dual of X.
Theorem 2.48 (X ′ is a Banach Space). Let X be a n.s. and let X ′ be the dual of X.
Then X ′ is a Banach space.
Proof. See Theorem 2.47.
Example 2.49 (Rn ). The dual of Rn is Rn .
Proof. We claim Rn∗ = Rn′ and ∀ f ∈ Rn∗ we have
n
f (x) = ξk αk , αk = f (ek ).
k=1
So then
n n
|f (x)| ≤ |ξk αk | ≤ x α2 ,
k
k=1 k=1
which implies
n
f ≤ α2 .
k
k=1
If we choose x = (αk ) then
n n
|f (x)| = α2
k ⇒ f = α2 .
k
k=1 k=1
36
37. So take the onto mapping from Rn′ to Rn
f → α = (αk ) = (f (ek ))
which is norm-preserving, linear, and a bijection (an isomorphism), which completes the
proof.
Problem 16.
′
1. Show ℓ1 = ℓ∞
1 1
2. Show ℓp′ = ℓq where 1 < p < ∞ and p + q =1
Solution.
1. A Schauder basis for ℓ1 is (ek ), where ek = (δki ). Then every x ∈ ℓ1 has a unique
representation
∞
x= ξk ek .
k=1
′ ′
We consider any f ∈ ℓ1 , where ℓ1 is the dual space of ℓ1 . Since f is linear and
bounded,
∞
f (x) = ξk γk , where γk = f (ek ).
k=1
It is easy to see that (γk ) ∈ ℓ∞ .
On the other hand, for every b = (βk ) ∈ ℓ∞ ,
∞
g(x) = ξk βk , where x = (ξk ) ∈ ℓ1
k=1
′
is a bounded linear functional on ℓ1 , i.e., g ∈ ℓ1 .
It is also easy see that f = c ∞ , where c = (γk ) ∈ ℓ∞ . It follows that the bijective
′
linear mapping of ℓ1 onto ℓ∞ defined by f → c is an isomorphism.
2. Take the same Schauder basis and representation for x and f as in 1. Let q be the
(n)
conjugate of p and consider xn = (ξk ) with
(n) |γk |q
ξk = , if k ≤ n and γk = 0,
γk
and
(n)
ξk = 0 if k > n or γk = 0.
37
38. Then we have
∞ n n
(n) 1
f (xn ) = ξk γk = |γk |q ≤ f ( |γk |q ) p .
k=1 k=1 k=1
1 1
Using 1 − p = q we get
n
1
( |γk |q ) q ≤ f .
k=1
Since n is arbitrary, letting n → ∞, we obtain
∞
1
( |γk |q ) q ≤ f ,
k=1
and therefore (γk ) ∈ ℓq .
Conversely, for any b = (βk ) ∈ ℓq we may define g on ℓp by
∞
g(x) = ξk βk , where x = (ξk ) ∈ ℓp .
k=1
′
Then g is linear, and boundedness follows from the Holder inequality. Hence g ∈ ℓp .
From the Holder inequality we get
∞
1
|f (x)| ≤ x ( |γk |q ) q .
k=1
It follows that f = c q , where c = (γk ) ∈ ℓq and γk = f (ek ). The mapping of
′
ℓp onto ℓq defined by f → c is linear, bijective, and norm-preserving, so it is an
isomorphism.
38
39. 3 Hilbert Spaces
Definition (Inner Product). Let X be vector space with scalar field K. ·, · : X × X → K
is called an inner product if
1. x + y, z = x, z + y, z
2. αx, y = α x, y
3. x, y = y, x
4. x, x ≥ 0, x, x = 0 iff x = 0
Definition (Hilbert Space). A Hilbert space is a complete inner product space.
Remark 3.1.
1. A vector space X with inner product ·, · has the induced norm and metric:
x = x, x
d(x, y) = x − y .
2. If X is real then x, y = y, x .
3. From the definition of inner product, we obtain
αx + βy, z = α x, z + β y, z
x, αy = α x, y
x, αy + βz = α x, y + β x, z
So the inner product is linear in the first argument and conjugate linear in the second
argument.
4. Parallelogram Equality.
2 2 2 2
x+y + x−y =2 x + y (5)
Any norm which is induced from an inner product must satisfy this equality.
5. Orthogonality. x, y ∈ X are said to be orthogonal if x, y = 0 and we write
x ⊥ y.
For subsets A, B ⊂ X we say A ⊥ B if
a ⊥ b ∀ a ∈ A, b ∈ B.
39
40. Example 3.2.
1. Rn with inner product x, y = xT y.
2. Cn with inner product x, y = xT y.
3. Real L2 [a, b] with
b
x, y = x(t)y(t)dt.
a
4. Complex L2 [a, b] with
b
x, y = x(t)y(t)dt.
a
5. ℓ2 with inner product
∞
x, y = xi y i .
i=1
Problem 17.
1. Show ℓp with p = 2 is not a Hilbert space.
2. Show C[a, b] is not a Hilbert space.
Solution.
1. Consider x = (1, 1, 0, ..., 0, ..) and y = (1, −1, 0, ..., 0, ...). Then x, y ∈ lp and x p =
1
y p = 2 , and x+y p = x−y
p
p = 2. It follows that if p = 2, then the paralelogram
equality is not satisfied.
t−a t−a
2. Let x(t) = b−a and y(t) = 1 − b−a . Then we have x = y = x + y = x − y = 1
and the paralelogram equality is not satisfied.
Problem 18. Show that x = x, x defines a norm.
Solution. It follows from the definition of the inner product that x ≥ 0 and x = 0 iff
x = 0. Also, we have using the definiton of the inner product that
1 1 1
αx = ( αx, αx ) 2 = (α¯ x, x ) 2 = (|α|2 x, x ) 2 = |α| x .
α
Concerning the triangle inequality we get
2 2 2
x+y = x + y, x + y = x, x + x, y + y, x + y, y = x + 2Re x, y + y
2 2
≤ x +2 x y + y = ( x + y )2 ,
from which the result follows.
40
41. Lemma 3.3 (Continuity of Inner Product). If in an inner product space yn → y and
xn → x then
xn , yn → x, y .
Theorem 3.4 (Completion). For any inner product space X there is a Hilbert space H
and isomorphism A from X onto a dense subspace W ⊂ H. The space H is unique up to
isomorphisms.
Theorem 3.5 (Subspace). Let Y ⊂ H with H a Hilbert space. Then
1. Y is complete iff Y is closed in H.
2. If Y is finite dimensional then Y is complete.
3. If H is separable then Y is separable.
Definition (Convex Subset). M ⊂ X is said to be convex if
αx + (1 − α)y ∈ M, ∀ α ∈ [0, 1], a, b ∈ M.
Theorem 3.6. Let X be an inner product space and M = ∅ a convex subset which is
complete. Then ∀ x ∈ X ∃! y ∈ M s.t.
δ = inf x−y = x−y .
ˆ
y ∈M
ˆ
Proof.
1. Existence. By definition of inf,
∃ (yn ) s.t. δn → δ, where δn = x − yn .
We will show (yn ) is Cauchy. Write vn = yn − x. Then vn = δn and
vn + vm = yn + ym − 2x
1
= 2 (yn + ym ) − x ≥ 2δ
2
Furthermore, yn − ym = vn − vm , ⇒
2 2
yn − ym = vn − vm
2 2
= − vn + vm + 2( vn + vm )
2 2 2
≤ −(2δ) + 2(δn + δm ).
Since M is complete, yn → y ∈ M . Then x − y ≥ δ. Also, we have
x−y ≤ x − yn + yn − y
= δn + yn − y
→ δ+0
therefore x − y = δ.
41
42. 2. Uniqueness. Suppose that two such elements exist, y1 , y2 ∈ M and from above,
x − y1 = δ = x − y2 .
By (5) we have
2 2
y1 − y2 = (y1 − x) + (x − y2 )
2 2 2
= 2 y1 − x + 2 y2 − x − (y1 − x) + (y2 − x)
2
1
= 2δ2 + 2δ2 − 22 (y1 + y2 ) − x
2
≤ 0
since
2
1
22 (y1 + y2 ) − x ≥ 4δ2 .
2
Therefore y1 = y2 .
Lemma 3.7 (Orthogonality). With the setup of the previous theorem, z = x − y is orthog-
onal to M .
Proof. Suppose z ⊥ M were false, then ∃ w ∈ M s.t.
z, w = β = 0.
and so w = 0. For any α,
2
z − αw = z − αw, z − αw
= z, z − α z, w − α [ w, z − α w, w ]
2
= z − αβ − α [β − α w, w ] .
If we choose
β
α= ,
w, w
and continue the equality above,
2 |β|2
z − αw = δ2 − − α [0]
w, w
< δ2
contradicting the minimality of y, δ. Therefore we must have z ⊥ M .
42
43. Definition (Direct Sum). A vector space X is said to be direct sum of subspaces Y and
Z, written
X = Y ⊕ Z,
if ∀ x ∈ ∃! y ∈ Y, z ∈ Z s.t.
x = y + z.
Definition (Orthogonal Complement). Let Y be a closed subspace of X. Then the or-
thogonal complement of Y in X is
Y ⊥ = {z ∈ X s.t. z ⊥ Y } .
Theorem 3.8 (Direct Sum). Let Y be a closed subspace of H. Then
H = Y ⊕ Y ⊥.
Proof.
1. Existence. H is complete and Y is closed implies Y is complete. Further, we know
Y is convex. Therefore ∀ x ∈ H ∃ y ∈ Y s.t.
x = y + z, where z ∈ Y ⊥ .
2. Uniqueness. Assume x = y + z = y1 + z1 . Then
y − y1 ∈ Y
z − z1 ∈ Y ⊥ .
But Y ∩ Y ⊥ = {0} ⇒ y − y1 = z − z1 = 0.
Definition (Orthogonal Projection). Let Y be a closed subspace of H, so H = Y ⊕ Y ⊥ ,
and let
x=y+z
y∈Y
z ∈ Y ⊥.
Then the orthogonal projection onto Y is P : H → Y given by
P x = y.
Clearly P is bounded. Further P is idempotent, that is, P 2 = P , which we call idempo-
tent.
43
44. Definition (Annihilator). Let X be an inner product space and let M be a nonempty
subset of X. Then the annihilator of M is
M ⊥ = {x ∈ X s.t. x ⊥ M } = {x ∈ X s.t. x, v = 0 ∀ v ∈ M } .
Theorem 3.9. Let M, X be as the definition above and denote (M ⊥ )⊥ = M ⊥⊥ . Then
1. M ⊥ is a vector space.
2. M ⊥ is closed.
3. M ⊂ M ⊥⊥ .
Theorem 3.10. If M is a closed subspace of a Hilbert space H then
M ⊥⊥ = M.
Lemma 3.11 (Dense Set). Let M = ∅ be a subset of a Hilbert space H. Then
span (M ) = H iff M ⊥ = {0} .
Proof.
1. ⇒. Assume x ∈ M ⊥ , V = span (M ) is dense in H. Then x ∈ V = H ⇒ ∃ (xn ) ⊂
V s.t. xn → x. Now x ∈ M ⊥ and M ⊥ ⊥ V ⇒ xn , x = 0. But ·, · is continuous
⇒ xn , x → x, x = 0 ⇒ x = 0 ⇒ M ⊥ = {0}, since x ∈ M ⊥ was arbitrary.
2. ⇐. Suppose M ⊥ = {0} and write V = span (M ). Then
x⊥V ⇒ x⊥M
⇒ x ∈ M⊥
⇒ x=0
⇒ V ⊥ = {0}
⇒ V = H.
Definition (Orthonormal Set/Sequence). M is said to be an orthogonal set if ∀ x, y ∈ M
x = y ⇒ x, y = 0.
M is said to be orthonormal if ∀ x, y ∈ M
0 x=y
x, y = δxy = .
1 x=y
If M is countable and orthogonal (resp. orthonormal) then we can write M = (xn ) and we
say (xn ) is an orthogonal (resp. orthonormal) sequence.
44
45. Remark 3.12 (Pythagorean Relation, Linear Independence). Let M = {x1 , . . . , xn } be an
orthogonal set. Then
1.
n 2 n
2
xi = xi .
i=1 i=1
2. M is linearly independent.
Example 3.13 (Orthonormal Sequences).
1. {(1, 0, . . . , 0), (0, 1, 0, . . . , 0), . . . , (0, . . . , 0, 1, 0), (0, . . . , 0, 1)} ⊂ R.
2. (en ) ⊂ ℓ2 where en = δni .
2π
3. Let X = C[0, 2π] with x, y = 0 x(t)y(t)dt. Then the functions
(sin nt), n = 1, 2, . . . , (cos nt), n = 0, 1, 2, . . .
form an orthogonal sequence.
Remark 3.14 (Unique Representation with Orthonormal Sequences). Let X be an inner
product space and let (ek ) be an orthonormal sequence in X, and suppose x ∈ span ({e1 , . . . , en })
where n is fixed. Then we can represent
n n
x= αk ek ⇒ x, ei = αk ek , ei = αi
k=1 k=1
n
⇒ x= x, ek ek .
k=1
Now let x ∈ X be arbitrary and take y ∈ span ({e1 , . . . , en }), where
n
y= x, ek ek .
k=1
Define z by setting x = y + z ⇒ z ⊥ y because
z, y = x − y, y
= x, y − y, y
2
= x, x, ek ek − y
2
= x, x, ek ek − y
= x, ek x, ek − | x, ek |2
= 0.
45
![Functional Analysis
July 12, 2007
Contents
1 Metric Spaces 3
1.1 Open Sets, Closed Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Convergence, Cauchy Sequence, Completeness . . . . . . . . . . . . . . . . . 7
1.3 Completeness Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Completion of Metric Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Normed Spaces 13
2.1 Finite dimensional Normed Spaces or Subspaces . . . . . . . . . . . . . . . 15
2.2 Compactness and Finite Dimension . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Linear Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4 Bounded and Continuous Linear Operators . . . . . . . . . . . . . . . . . . 24
2.5 Linear Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6 Finite Dimensional Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.6.1 Linear Functionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.7 Dual Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3 Hilbert Spaces 39
3.1 Representation of Functionals on Hilbert Spaces . . . . . . . . . . . . . . . 49
3.2 Hilbert Adjoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4 Fundamental Theorems 59
4.1 Bounded, Linear Functionals on C[a, b] . . . . . . . . . . . . . . . . . . . . . 68
4.2 Adjoint Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3 Reflexive Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.4 Baire Category Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.5 Strong and Weak Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.6 The Open Mapping Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.7 Closed Graph Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
1](https://image.slidesharecdn.com/notes-121009204506-phpapp01/85/notes-1-320.jpg)
