Download 1 Normed Linear Spaces

Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
1
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
Normed Linear Spaces
1.1
Vector Spaces
Typically, we think of quantities with a magnitude and direction, such as wind speeds, velocities,
flows or displacements as vectors, and things like mass, temperature, area and volume as scalars.
However, this distinction is not really that true in practice! A good working definition of a vector
space is “a collection of things that can be added and scaled”. This includes not only the usual
notions of points in a plane, or rather the distances and directions between points in a plane, but
also tuples (ordered sets) of real or complex numbers, matrices of all shapes and sizes and functions.
In this course we will take many examples from pairs and tuples of real numbers, but our main
interest is in the various vector spaces of functions.
For a more precise definition, we must digress into Algebra. Here, a vector space is defined as
a set of objects that satisfy certain axioms regarding addition and scaling. For fun, the list below
gives a heirarchy of abstract spaces building up to a proper definition of a vector space. You won’t
need to memorize this list, or know any of these terms for this course, but if you scan the list you
should recognize these axioms as familiar properties of numbers (both real and complex) and of
transformations (like rotations, scalings and reflections).
Definition 1.1 (From Hungerford). A semigroup is a set V together with a binary operation
V × V → V , denoted by +, which is
i) associative: (x + y) + z = x + (y + z) for every x, y and z in V .
A monoid is a semigroup with an identity element 0 satisfying
ii) 0 + x = x + 0 = x for every x in V .
A group is a monoid V such that for every x in V there is an inverse (−x) in V satisfying
iii) x + (−x) = (−x) + x = 0 for every x in V .
A group is abelian if the operation is
iv) commutative: x + y = y + x for every x and y in V .
Remark 1.1. Since the above objects are comprised of a set together with an operation, they are
sometimes denoted as a pair. For example (V, +). However, more often a single letter is used with
no explicit mention of the operator.
Definition 1.2. A field is a set F together with two binary operations, addition and multiplication,
denoted here by + and · respectively, such that (F, +) is an abelian group with identity 0, (F/{0}, ·)
is an abelian group with identity 1, and multiplication distributes over addition
Definition 1.3. Let F be a field with addition denoted by + and multiplication denoted by
juxtaposition. An abelian group (V, +) together with a binary operation F × V → V , also denoted
by juxtaposition, forms a vector space over F if this binary operation satisfies the following axioms.
v) (a + b)x = ax + bx for all x in V , a and b in F ,
vi) a(x + y) = ax + ay for all x and y in V and every a in F ,
vii) a(bx) = (ab)x for all x in V , a and b in F ,
page 1 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
viii) 1x = x for all x in V and the multiplicative identity, 1, for F .
Remark 1.2. The elements of V are called vectors, and the elements of F are called scalars. Our
scalars will be either real numbers or complex numbers, depending on the set V , and we will refer
to V as a real or complex vector space respectively.
Definition 1.4. Let V be a real or complex vector space, and let {v1 , v2 , . . . , vn } be a set of
elements of V , and let {a1 , a2 , . . . , an } be a set of scalars.
i) A linear combination of the vectors v1 , . . . , vn is any sum of the form a1 v1 + · · · + an vn .
ii) A subset S of V is called linearly dependent if there are nonzero scalars a1 , . . . , an and distinct
elements v1 , . . . , vn in S such that a1 v1 + · · · + an vn = 0. Otherwise S is called linearly
independent.
iii) For any subset S of vectors in V , the span of S, denoted span(S), is the set of all (finite) linear
combinations of vectors in S.
iv) A subset S of V is called a subspace of V if it is closed under (finite) linear combinations. That
is, if span(S) = S.
v) A set B of vectors in V is a basis for V if B is linearly independent and span(B) = V .
Theorem 1.5. If {u1 , u2 , . . . , um } and {v1 , v2 , . . . , vn } are both bases for a given space V then
m = n.
Remark 1.3. The number of elements in any basis for a vector space V is called its dimension. If
there is no finite set that spans V then the space is said to be infinite dimensional.
Theorem 1.6. If V is a vector space with basis B, then every element v of V can be written in
one and only one way as a linear combination of the elements of B.
Remark 1.4. More specifically, if B = {u1 , u2 , . . . , um } then we can write v = a1 u1 + a2 u2 + · · · +
am um for some scalars a1 , . . . , an . Furthermore, there is only one such m-tuple of scalars. This
tuple is referred to as the coordinates of v with respect to the basis B and often denoted by the
‘column vector’ [v]B = [a1 , a2 , . . . , an ]T .
Exercise 1.1. Show that the set Q of rational numbers is a vector space if we also take scalars
from Q. Similarly, the sets R and C also form vector spaces over R and C respectively.
Exercise 1.2. Given two vector spaces X and Y , show that their Cartesian product, defined as
the set of ordered pairs X × Y = {(x, y)|x ∈ X and y ∈ Y }, forms a vector space with addition
and scalar multiplication defined entrywise.
Exercise 1.3. Define the Euclidian space Rn as the set of ordered n-tuples of real numbers x =
(x1 , x2 , . . . , xn ) with each xi in R. Show that with the usual definition of vector addition and scalar
multiplication Rn forms a vector space of dimension n over R. The complex counterparts, Cn , are
similarly defined. We will restrict the definition of the Euclidian spaces further when we introduce
norms in the next section.
Exercise 1.4. Let l2 be the set of all square summable sequences (xn ) and define addition in l2
by entrywise addition ofPsequences. Show that if (xn ) and (yn ) are two sequences in l2 , then the
sum (xn + yn ) satisfies
|xn + yn |2 < ∞. Hence l2 is a (complex) vector space. We say that l2 is
infinite dimensional, since it has no finite basis.
page 2 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
Exercise 1.5. Let C[a, b] denote the set of all continuous functions on the interval [a, b] and define
addition and scalar multiplication in the obvious way. Verify that C[a, b] is a vector space. If we
restrict ourselves to real-valued functions, then C[a, b] is a real vector space.
Exercise 1.6. Let Pn [a, b] denote the set of all polynomial functions of degree n or less on the
interval [a, b].
1. Verify that Pn [a, b] is a vector space of dimension n + 1.
2. Verify that Pn [a, b] is a subspace of Pm [a, b] for all m > n.
3. Verify that Pn [a, b] is a subspace of C[a, b] for every n. What does this imply about the
dimension of C[a, b]?
Exercise 1.7. Verify that the set C0 [a, b] of all continuous functions f with domain [a, b] and range
R (or C) and f (a) = f (b) = 0 forms a vector space over R (or C). This space is a subspace of the
space of all continuous functions on [a, b] and is relevant for certain boundary value problems.
Exercise 1.8. Verify the set Ck [a, b] of all functions with a continuous kth derivative and domain
[a, b] is a subspace of C[a, b].
Exercise 1.9. Let X be the set of all solutions to the differential equation y ′′ + y = 0, 0 ≤ x ≤ π.
Show that X is a subspace of C 2 [0, π].
Example 1.1. Consider the set S = {x, 1, x2 − 1, x + 1} as a set of elements from C[0, 1]. To
determine if the set is linearly independent, look for a trivial linear combination.
ax + b + c(x2 − 1) + d(x + 1) = 0.
This simplifies to cx2 + (a + d)x + (b − c + d) = 0. If this equation is to be satisfied at every point
in the interval [0, 1], or indeed in any interval, we must have
c = 0,
a + d = 0,
b − c + d = 0.
Since this system has non-trivial solutions (for example, a = c = 0, b = d = 1), the set S is linearly
dependent. This can also be seen from inspection, since x + 1 is a linear combination of x and
the function 1. Throwing away x + 1 (since x + 1 is an element of Span{1, x}), we see that the
remaining vectors are linearly independent as follows:
ax + b + c(x2 − 1) = 0 =⇒ a + 2cx = 0
=⇒ 2c = 0
by taking the derivative
by taking another derivative
Hence c = 0, which implies a = 0 by the second relation and finally b = 0 by the first. Hence,
the space V = spanS is spanned by three linearly independent vectors and so has dimension 3.
Equivalently, V has a basis with three elements, namely 1, x and x2 − 1.
Example 1.2. Consider the vector space spanned by {sin x, cos x, sin 2x}, where the domain of
the functions is some arbitrary interval in R. The space is dimension 3, since the three vectors are
linearly independent. This can be shown as follows. Suppose there are scalars (real numbers) a, b
and c such that
a sin(x) + b cos(x) + c sin(2x) = 0.
(1)
page 3 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
We can’t evaluate the functions at points, since we aren’t given a specific interval, but we can take
derivatives. Hence,
a cos(x) − b sin(x) + 2c cos(2x) = 0
(2)
−a sin(x) − b cos(x) − 4c sin(2x) = 0
(3)
Adding (3) to (1) gives c = 0. Multiplying (2) by sin(x) and (1) by cos(x) and taking the difference
gives b = 0. All we have left is a, which must be zero by (1).
Definition 1.7. If u and v are differentiable, their Wronskian is the function [u, v] defined by
[u, v](x) = u(x)v ′ (x) − u′ (x)v(x).
Theorem 1.8. If u and v are differentiable and their Wronskian, [u, v] is nonzero for at least one
value of x, then the functions u and v must be linearly independent.
Proof. If u and v are differentiable then for any two scalars a and b, au+bv = 0 implies au′ +bv ′ = 0.
In matrix form, we have
u v
a
0
=
′
′
u v
b
0
Since the Wronskian is simply the determinant of this matrix, the system has nontrivial solutions
only if [u, v] is zero. Recall that if a homogeneous system of linear equations has a trivial solution,
then the determinant of the associated matrix must be zero (the matrix is singular).
Exercise 1.10. Show that the reverse implication of Theorem 1.8 need not be true. Consider the
function u defined by
(
x2 , x > 0
u(x) =
0,
x≤0
and define v by v(x) = u(−x).
page 4 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
1.2
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
Norms, sequences and convergence
The abstract vector space includes no notion of a distance between points, or of a length of a
vector. However, this concept is central for any discussion of sequences and convergence. A set
with a suitable notion of distance is termed a metric space. Metric spaces do not necessarily have
any algebraic structure. Since we are interested mainly in vector spaces rather than more general
sets, we will start with normed spaces, one step up the rung from metric spaces. A normed vector
space, or normed linear space is constructed by defining a length, or norm, for every vector in the
space.
Definition 1.9. A norm on a vector space V is a function k·k : V → R satisfying
(N0) kxk ≥ 0 for every x in V ,
(N1) kxk = 0 if and only if x = 0,
(N2) kaxk = |a| kxk for every x in V and every scalar a,
(N3) kx + yk ≤ kxk + kyk for every x and y in V .
Remark 1.5. A vector space equipped with a norm is called a normed linear space
Remark 1.6. In vector spaces, it is natural to define the distance between two points x and y as
the length of their difference, kx − yk.
Definition 1.10. Since the norm is a map from the vector space into R, the notion of convergence
developed for real numbers can be easily adapted to linear spaces.
i) A sequence (xn ) in (a normed space) V converges to a point x in V if given any real number
ǫ > 0 there is a natural number N such that kxn − xk < ǫ for every n > N .
ii) A sequence (xn ) is called a Cauchy sequence if, given ǫ > 0, there is a number N such that
kxn − xm k < ǫ for every n, m ≥ N .
iii) A space is complete if every Cauchy sequence converges.
iv) A complete normed linear space is called a Banach Space
Remark 1.7. If the sequence (xn ) converges to the point x we call x the limit of the sequence and
write lim xn = x or xn → x.
Exercise 1.11. Show that the real numbers form a vector space with both vectors and scalars
taken to be real numbers and the usual interpretations of addition and multiplication. Further,
show that the absolute value function is a norm on R, and R is a complete normed linear space
with this norm.
Exercise 1.12. Show that the set of complex numbers, C, with the norm |z| = zz̄ is a Banach
space.
Exercise 1.13 (Euclidean Spaces). Let Rn denote the set of all ordered n-tuples of real numbers
x = (x1 , . . . , xn ). Show that Rn with the Euclidean norm
!1/2
n
X
(4)
|xi |2
kxk =
i=1
forms a Banach space.
page 5 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
Remark 1.8. We can define the Complex Euclidean spaces, Cn , in the same fashion.
Exercise 1.14. Show that a sequence (xn ) in Rn is Cauchy (convergent) if and only if each of the
sequences xni , i = 1, . . . , d is Cauchy (convergent). Hence, since R is complete, it follows that Rn is
also complete. A similar argument shows that the complex Euclidean space Cn is a Banach space.
Exercise 1.15. Another interesting space is Rn with the supremum norm
kxk∞ = max |xi |.
(5)
i=1,...,n
Verify that this satisfies the properties of a norm.
Remark 1.9. If we think of points in Rn as portions of a discrete signal, then the supremum norm
measures the maximum difference in two signals, whereas the Euclidean norm of the previous
example measures the root mean squared difference. In statistics, the supremum norm is related
to the median and the Euclidean norm is related to the mean.
Exercise 1.16. An experiment to determine the age of an urn is repeated n times to yield the n
real numbers (x1 , . . . , xn ). Obviously, outcomes of the experiment can be associated with points in
Rn , and our goal is to estimate the true age of the urn. A reasonable approach is to let M be the
subset of Rn consisting of all points whose entries are identical (I.e., points of the form (a, . . . , a)),
and to find the point, y, in M that minimizes kx − yk, where x is the result of the experiment. Do
this for both the supremum and the Euclidean norms.
Exercise 1.17 (Sequence Spaces). Let l∞ denote the normed space formed by the set of all bounded
sequences of complex numbers, x = (x1 , x2 , . . . ), together with the norm kxk∞ = sup |xi |. Verify
that this is a normed linear space. Similarily, for p ≥ 1, let lp denote the normed space formed
∞
X
|xi |p < ∞, with the norm
by the set of all sequences of complex numbers x = (xi ) such that
i=1
defined by
kxkp =
∞
X
|xi |p
i=1
Verifty that this is a norm.
!1/p
.
(6)
Exercise 1.18. Let C[a, b] denote the set of all real-valued functions of a real variable that are
defined and continuous on the interval [a, b] and define the uniform norm by
kf k = max |f (x)|.
(7)
x∈[a,b]
Verify that C([a, b]) is a normed space (Kreyszig, Section 1.2).
Remark 1.10. The same notation is used to denote the space of all complex-valued functions of a
real variable with the uniform norm.
Remark 1.11. Other norms can be applied to the space of continuous functions on [a, b]. For
instance, define Lp [a, b] as this space with the norm
kxkp =
Z
b
p
|xi |
a
1/p
.
(8)
page 6 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
Exercise 1.19. Show that the space l∞ defined in example 1.17 is a Banach space. That is, show
that the norm satisfies the axioms of definition 1.9 and that every Cauchy sequence of points in l∞
has a limit in l∞ . (See Sections 1.2 and 1.5 of Kreyszig.)
Exercise 1.20. The set of all polynomials considered as functions of x on the interval [a, b] with
the uniform norm is a subspace of C([a, b]). Show that this space is not complete by giving an
example of a sequence of polynomials that converges to a limit that is not a polynomial.
Exercise 1.21. Let X be the set of all continuous, real-valued functions on the interval [0, 1].
(i) Show that the function
kf k =
Z
1
|f (x)| dx
(9)
0
is a norm on X. (I.e, show d satisfies the axioms of definition 1.9.)
(ii) Show that the sequence (fn ), where
fn (x) =
n
t−1/2
if 0 ≤ x ≤ n−2 ,
if n−2 ≤ x ≤ 1,
(10)
is Cauchy, but not convergent. Hence, X with the above norm is an incomplete normed space.
page 7 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
1.3
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
Uniform convergence
Let C[a, b] denote the set of all complex-valued functions of a real variable that are defined and
continuous on the closed interval [a, b]. Unless otherwise specified, the usual norm associated with
this vector space is the uniform norm defined by
kf k∞ = sup |f (x)|.
(11)
a≤x≤b
In practice, this norm measures the maximum difference, or deviation, between two functions.
It is instructive to compare convergence with respect to the uniform norm with convergence of
the functions at each point in the domain.
Definition 1.11 (Pointwise Convergence). A sequence of functions, (fn ), is said to converge pointwise to a function f if kfn (x) − f (x)k → 0 as n → ∞ for every x in X.
Definition 1.12 (Uniform Convergence). A sequence of functions, (fn ) is said to converge uniformly to a function f if given any ǫ > 0 there is a number, N , such that |fn (x) − f (x)| < ǫ for
every x in X and n > N .
Note the difference between the two is that the choice of N can depend on x for pointwise
convergence, but must be independent of x for uniform convergence. Hence, if (fn ) converges
pointwise and N can be chosen independently of x for every ǫ, then the convergence is also uniform.
It follows that uniform convergence implies pointwise convergence.
There is an interesting quirk in the definitions above. In the definition of uniform convergence,
there is no assumption on the continuity of the limiting function f . Whereas, convergence with
respect to the uniform norm requires that the limit be in C[a, b] and therefore continuous. The
following theorem states that the limiting function will be continuous and that C[a, b] is a complete
normed space (a Banach space).
Theorem 1.13. If (fn ) is a Cauchy sequence in C[a, b] (with respect to the uniform norm), then
(fn ) converges uniformly to the pointwise limit f (x) = lim fn (x) and, by implication, f is continn→∞
uous on [a, b].
Proof. This proof consists of three steps: first, show that the pointwise limit exists, second that it
is a continuous function, and third, that the sequence converges uniformly to the pointwise limit.
Suppose (fn ) is a Cauchy sequence in C[a, b], and define the pointwise limit f by f (x) = lim fn (x).
First, note that if the sequence (fn ) is Cauchy in C[a, b], then (fn (x)), which is a sequence of
complex numbers, is Cauchy in C for each x, a ≤ x ≤ b. Since C is complete, (fn (x)) converges for
each x and thus, f is defined on [a, b].
To show that the pointwise limit, f , is continuous at each point in [a, b], consider
|f (x) − f (y)| = |f (x) − fn (x) + fn (x) − fn (y) + fn (y) − f (y)|
≤ |f (x) − fn (x)| + |fn (x) − fn (y)| + |fn (y) − f (y)|.
Suppose we are given a positive real number ǫ. We must show there is a positive real number δ
for which |f (x) − f (y)| < ǫ whenever |x − y| < δ and both x and y are in the interval [a, b]. Since
fn is continuous on [a, b], there is a δ for which |fn (x) − fn (y)| < ǫ/3 whenever |x − y| < δ and
both x and y are in [a, b]. Since (fn ) converges uniformly to f , there is a number N for which
|f (x) − fn (x)| < ǫ/3 and |f (y) − fn (y)| < ǫ/3 whenever n > N . Note that since convergence is
uniform, N can be chosen independently of x and y. It follows that |f (x) − f (y)| < ǫ whenever
|x − y| < δ. Thus, f is continuous on [a, b].
page 8 of 9
Math 3003, Winter 2013
Topics: vectors spaces, norms, convergence in norm
Applied Analysis - Norms and Convergence
Revised: February 11, 2013
Finally, to show that the sequence converges to the pointwise limit, we will show by contradiction
that for any real number ǫ > 0, there is choice of N for which |fn (x) − f (x)| < ǫ for every n > N
and x in [a, b]. The assumption that (fn ) is Cauchy means that for any given positive real number
ǫ, there is a number N for which kfn − fm k∞ < ǫ for every n and m greater than N . Suppose that
for some n > N and x in [a, b] |fn (x) − f (x)| > ǫ. Since we have picked n and x and the inequality
is strict, we can now choose a positive real number ǫ2 so that
|fn (x) − f (x)| > ǫ + ǫ2 .
(12)
That is, there is a number, ǫ2 , that is smaller than the difference between |fn (x)−f (x)| and ǫ. Since
(fm (x)) converges to f (x), we can choose a number m larger than n for which |fm (x) − f (x)| < ǫ2 ,
and, since both n and m are larger than N , |fn (x)−fm (x)| < ǫ. However, by the triangle inequality,
|fn (x) − f (x)| < |fn (x) − fm (x)| + |fm (x) − f (x)| < ǫ + ǫ2 .
(13)
This contradicts the previous inequality and we conclude that there are no such n and x. It must
therefore be that |fn (x) − f (x)| < ǫ for every n > N and x in [a, b] and convergence of (fn ) to f is
uniform.
Remark 1.12. More generally, this theorem can be restated for more functions between more general
vector spaces. In this case, the interval [a, b] is replaced by a compact subset of the space in question.
page 9 of 9