Download Analysis III

Analysis III
Lecturer: Prof. Oleg Zaboronski, Typesetting: David Williams
Term 1, 2015
Contents
I
Integration
2
1 The Integral for Step Functions
1.1 Definition of the Integral for Step Functions . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Properties of the Integral for Step Functions . . . . . . . . . . . . . . . . . . . . . . . . . .
2
2
4
2 The Integral for Regulated Functions
2.1 Definition of the Integral for Regulated Functions . . . . . . . . . . . . . . . . . . . . . . .
2.2 Properties of the Integral for Regulated Functions . . . . . . . . . . . . . . . . . . . . . .
6
6
8
3 The Indefinite Integral & Fundamental Theorem of Calculus
10
3.1 The Indefinite Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 The Fundamental Theorem of Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4 Practical Methods of Integration
12
4.1 Integration by Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.2 Integration by Substitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.3 Constructing Convergent Sequences of Step Functions . . . . . . . . . . . . . . . . . . . . 14
5 The Characterisation of Regulated Functions
15
5.1 Characterisation & Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6 Improper & Riemann Integrals
17
6.1 Improper Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.2 The Riemann Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7 Uniform & Pointwise Convergence
21
7.1 Definitions, Properties and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
7.2 Uniform Convergence & Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
7.3 Uniform Convergence & Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8 Functional Series
28
8.1 The Weierstrass M-Test & Other Useful Results . . . . . . . . . . . . . . . . . . . . . . . . 28
8.2 Taylor & Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
II
Norms
33
9 Normed Vector Spaces
33
9.1 Definition & Basic Properties of a Norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.2 Equivalence & Banach Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
10 Continuous Maps
40
10.1 Definition & Basic Properties of Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . 40
10.2 Spaces of Bounded Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
11 Open and Closed Sets of Normed Spaces
11.1 Definition & Basic Properties of Open Sets . . . . . . . . . . . . . . . . . . . . . . . . . .
11.2 Continuity, Closed Sets & Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.3 Lipschitz Continuity, Contraction Mappings & Other Remarks . . . . . . . . . . . . . . .
43
43
44
45
12 The
12.1
12.2
12.3
12.4
46
46
47
50
51
Contraction Mapping Theorem & Applications
Statement and Proof . . . . . . . . . . . . . . . . . . .
The Existence and Uniqueness of Solutions to ODEs .
The Jacobi Algorithm . . . . . . . . . . . . . . . . . .
The Newton-Raphson Algorithm . . . . . . . . . . . .
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Part I
Integration
1
The Integral for Step Functions
1.1
Definition of the Integral for Step Functions
Motivation:
“Integrals = Areas Under Curves”
Consider constant functions of the form fc : x 7→ f0 ∈ R for x ∈ [a, b]. Let D be the area under the line
at f0 from a to b. Then it makes sense to define
b
Z
fc (x) dx ..= sign(f0 ) · Area(D)
a
= sign(f0 ) · (|f0 | · (b − a))
= f0 · (b − a)
Now this definition will be extended to piecewise constant or step functions.
Definition 1: ϕ : [a, b] → R is called a step function if there exists a finite set P ⊂ [a, b] with
a, b ∈ P such that ϕ is constant on the open sub-intervals of [a, b] \ P . P is called a partition of [a, b].
More
concretely, P = {a = p0 , p1 , p2 , . . . , pk−1 , pk = b}, with p0 < p1 < p2 < . . . < pk−1 < pk , and
ϕ
= ci (an arbitrary constant), for i = 1, 2, . . . , k − 1, k.
(pi−1 ,pi )
Notes: If φ is a step function then there are many partitions of [a, b] such that φ is constant on the
open sub-intervals defined by the partition, call one P . There exists at least one other point greater than
a in P (since both a and b belong to P by definition), call the first p. Then φ is also constant on the
open intervals of [a, b] \ (P ∪ {n}), where a < n < p. Let Q be a partition of [a, b], with Q ⊃ P . Then Q
is called a refinement of P (so (P ∪ {n}) is a refinement of P ). The minimal common refinement of P
and Q is denoted by P ∪ Q.
If φ is a step function on [a, b], then
any partition P = {a = p0 , p1 , p2 , . . . , pk−1 , pk = b}, with p0 < p1 <
p2 < . . . < pk−1 < pk , such that φ
= ci (an arbitrary constant), for i = 1, 2, . . . , k − 1, k, is called
(pi−1 ,pi )
compatible with, or adapted to, φ.
Proposition 1: Let S[a, b] be the space of step functions on [a, b]. If f, g ∈ S[a, b], then ∀µ, λ ∈ R,
(λf + µg) ∈ S[a, b] (so S[a, b] is closed under taking finite linear combinations of its elements).
Proof: f ∈ S[a, b], so ∃P = {a = p0 , p1 , p2 , . . . , pk−1 , pk = b}, with p0 < p1 < p2 < . . . < pk−1 < pk ,
which is compatible with f , and g ∈ S[a, b] so ∃Q = {a = q0 , q1 , q2 , . . . , ql−1 , ql = b}, with q0 < q1 <
q2 < . . . < ql−1 < ql , which is compatible with g. Consider P ∪ Q = {a = r0 , r1 , r2 , . . . , rm−1 , rm = b},
with r0 < r1 < r2 < . . . < rm−1 < rm . Then this partition is compatible with (λf + µg). It remains to
show that (λf + µg)
= ci (an arbitrary constant), for i = 1, 2, . . . , m − 1, m.
(ri−1 ,ri )
P ⊂ (P ∪ Q), so (ri−1 ,ri ) ⊂ (pa−1 , pa ) for some
a, and Q ⊂ (P ∪ Q), so (ri−1, ri ) ⊂ (qb−1 , qb ) for
some b. Then, since f is constant, f is constant, and since g is constant,
(pa−1 ,pa ) (ri−1 ,ri )
(qb−1 ,qb )
g
is constant, so (λf + µg)
is constant. Since the choice of i was arbitrary, this proves
(ri−1 ,ri )
(ri−1 ,ri )
the proposition. 2
Notes: This demonstrates that S[a, b] is a vector space over R. Define C[a, b] as the space of continuous
functions on [a, b], and define B[a, b] as the space of bounded functions on [a, b]. These are also vector
spaces. It was demonstrated in Analysis II that C[a, b] ⊂ B[a, b], and by definition S[a, b] ⊂ B[a, b].
Definition
2: For φ ∈ S[a, b], let P = {pi }ki=0 be a partition compatible with φ. Define (temporarily)
Rb
φi ..= φ
. Then a : S[a, b] → R is defined as
(pi−1 ,pi )
Z
b
φ ..=
a
Remark: If φ ≡ φ0 ∈ R, then P = {a, b}, so
the motivation.
Lemma 2:
The value of
Rb
a
k
X
φi (pi − pi−1 )
i=1
Rb
a
φ = φ0 (b − a), which is precisely the area described in
φ does not depend on the choice of partition compatible with [a, b].
Proof: For any φ ∈ S[a, b], let P and Q be partitions compatible with [a,b]. Suppose there are k points
Rb in Q that are not in P . Denote them n1 , n2 , . . . , nk . Temporarily let a φ be the integral of φ from a
P
to b defined by the partition P which is compatible with [a, b].
Now, consider P ∪ {n1 }. Since Q is compatible with [a, b], we know that n1 ∈ [a, b], so ∃i ∈ N such that
n1 ∈ (pi−1 , pi ) for some pi−1 , pi ∈ P . φ is constant on (pi−1 , pi ), and so is also constant on (pi−1 , n1 )
and (n1 , pi ), so P ∪ {n1 } is compatible with φ. By definition
Z
a
b
φ
Z
P ∪{n1 }
−
a
b
Z
φ = φi (n1 − pi−1 ) + φi (pi − n1 ) − φi (pi − pi−1 ) = 0, so
P
a
b
φ
Z
P ∪{n1 }
=
a
b
φ
P
The proof is completed by induction, since there are a finite number of points. This is left as an
exercise. 3
1.2
Properties of the Integral for Step Functions
1. Proposition 3 - Additivity: For any φ ∈ S[a, b], ∀v ∈ (a, b),
Z b
Z v
Z b
φ=
φ+
φ
a
a
v
2. Proposition 4 - Linearity: For any φ, ψ ∈ S[a, b], ∀λ, µ ∈ R,
Z b
Z b
Z b
λφ + µψ = λ
φ+µ
ψ
a
a
a
3. Proposition 5 - The Fundamental Theorem of Calculus: LetR φ ∈ S[a, b], and let P be a
x
partition of [a, b] compatible with φ. Consider I : [a, b] → R, I : x 7→ a φ. Then
(I) I is continuous on [a, b].
Sk
Sk
(II) I is differentiable on i=1 (pi−1 , pi ), and, ∀x ∈ i=1 (pi−1 , pi ), I 0 (x) = φ(x).
Proof:
1. Let P = {pi }ki=0 be a partition compatible with φ ∈ S[a, b]. Then there are two cases:
(i) If v ∈ P , then by definition
b
Z
φ=
a
k
X
φi (pi − pi−1 ) =
v
X
φi (pi − pi−1 ) +
v
Z
φi (pi − pi−1 ) =
Z
φ+
b
φ
a
i=v+1
i=1
i=1
k
X
v
(ii) If v ∈
/ P , then ∃i ∈ N, 1 ≤ i ≤ k, such that v ∈ (pi−1, pi ). Then {a = p0 , . . . , pi−1 , v},
{v, pi , . . . , pk = b} are partitions compatible with φ, so φ
∈ S[a, v], φ
∈ S[v, b], so
(a,v)
(v,b)
their integrals are well-defined on those intervals. Then
Z
b
φ=
a
k
X
φj (pj − pj−1 )
j=1
= φ1 (p1 − p0 ) + φ2 (p2 − p1 ) + . . . + φi (pi − pi−1 ) + . . . + φk (pk − pk−1 )
= φ1 (p1 − p0 ) + . . . + φi (pi + v − v − pi−1 ) + . . . + φk (pk − pk−1 )
= φ1 (p1 − p0 ) + . . . + φi (v − pi−1 ) + φi (pi − v) + . . . + φk (pk − pk−1 )


# "
i−1
k
X
X
=
φj (pj − pj−1 ) + φi (v − pi−1 ) + φi (pi − v) +
φj (pj − pj−1 )
j=1
Z
=
j=i+1
v
Z
b
φ+
a
φ
v
2. Let P be a partition compatible with φ ∈ S[a, b], and let Q be a partition compatible with ψ ∈
S[a, b]. Then, as was proved in the previous section, P ∪ Q is compatible with both φ and ψ.
Let
P ∪ Q = {a = x0, x1 , . . . , xk−1 , xk = b}, with a = x0 < x1 < . . . < xk−1 < xk = b. Then
φ
= φi and ψ = ψi , so (λφ + µψ)
= (λφ + µψ)i , so P ∪ Q is compatible
(xi−1 ,xi )
(xi−1 ,xi )
(xi−1 ,xi )
with λφ + µψ. Then
Z
b
λφ + µψ =
a
k
k
X
X
(λφ + µψ)j (xj − xj−1 ) =
(λφj + µψj )(xj − xj−1 )
j=1
=λ
k
X
j=1
φj (xj − xj−1 ) + µ
j=1
k
X
j=1
4
Z
ψj (xj − xj−1 ) = λ
b
Z
φ+µ
a
b
ψ
a
Sk
3. Let P = {pi }ki=0 be a partition compatible with φ ∈ S[a, b]. Choose x ∈ j=1 (pj−1 , pi ), then
∃i ∈ N, 1 ≤ i ≤ k such that x ∈ (pi−1 , pi ). Then
Z x
Z pi−1
Z x
φ = I(pi−1 ) + φi (x − pi−1 ) = φi x + constant
φ+
φ=
I(x) =
pi−1
a
a
Sk
so I is continuous on j=1 (pj−1 , pi ) and I 0 (x) = φi = φ(x). Furthermore, limx↓pi−1 I(x) = I(pi−1 ),
Rp
so I is right continuous on [a, b). Similarly, limx↑pi I(x) = I(pi−1 ) + φi (pi − pi−1 ) = a i−1 φ +
Rp
R pi
Rp
φi (pi − pi−1 ) = a i−1 φ + pi−1
φ = a i φ = I(pi ), so I is left continuous on (a, b], so I is continuous
on [a, b]. Definition 3: Let f ∈ B[a, b]. Then the supremum norm (or sup norm) of f , denoted by kf k∞ , with
k·k∞ : B[a, b] → R is defined as
kf k∞ ..= sup f (x)
x∈[a,b]
Proposition 6:
Let φ ∈ S[a, b] be bounded between m, M ∈ R, so m ≤ φ(x) ≤ M ∀x ∈ [a, b]. Then
Z
Z b
b φ ≤ M (b − a), in particular φ ≤ kφk∞ (b − a).
m(b − a) ≤
a a
Proof: For φ ∈ S[a, b], choose a compatible partition P . Now
Z
b
φ=
a
k
X
φi (pi − pi−1 )
i=1
then
k
X
m(pi − pi−1 ) ≤
i=1
k
X
φi (pi − pi−1 ) ≤
k
X
M (pi − pi−1 )
i=1
i=1
so, due to telescoping sums on the upper and lower bounds,
m(b − a) ≤
k
X
Z
φi (pi − pi−1 ) ≤ M (b − a), so m(b − a) ≤
b
φ ≤ M (b − a).
a
i=1
If kφk∞ is given, then, by definition, −kφk
R ∞ ≤ φ(x) ≤ kφk∞ , so using the same argument as above,
Rb
b −kφk∞ (b − a) ≤ a φ ≤ kφk∞ (b − a), so a φ ≤ kφk∞ (b − a). Rb
Rb .
.
φ has been defined for
R b >
a. Define, for a > b, a φ = − a φ. Then the bound proved
b in the previous proposition becomes a φ ≤ kφk∞ |b − a|. It is a simple exercise to demonstrate that
such an extension preserves the three properties of the integral for step functions proved previously.
Note:
So far
Rb
a
5
2
2.1
The Integral for Regulated Functions
Definition of the Integral for Regulated Functions
Definition 4: A function f on [a, b] is called
a regulated
function if, ∀ε > 0, ∃φ ∈ S[a, b] such that
kφ − f k∞ < ε, or, alternatively, if, ∀ε > 0, f (x) − φ(x) < ε ∀x ∈ [a, b]. We denote the set of regulated
functions on [a, b] by R[a, b].
Definition 4.1: A sequence (φn )n≥1 ⊂ S[a, b] is said to converge uniformly to a function f : [a, b] → R
if limn→∞ kf − φn k∞ = 0.
Lemma : f : [a, b] → R is regulated iff ∃(φn )n≥1 ⊂ S[a, b] such that, as n → ∞, (φn ) converges
uniformly to f .
Proof: The proof is routine and is left as an exercise. Definition 4.2: A function f : A ⊂ R → R is called uniformly continuous on A if, ∀ε > 0, ∃δ such
that ∀x, y ∈ A, |x − y| < δ and f (x) − f (y) < ε.
Note: In general, uniform continuity implies continuity, but not the other way around. For example,
f : (0, 1] → R, f : x 7→ x1 , is continuous, but not uniformly continuous. However, on closed intervals, the
two are equivalent.
Lemma:
For a function f : A → R, if A = [a, b], continuity implies uniform continuity.
Proof: Suppose f is continuous on [a, b] but not uniformly
continuous. Then ∃ε > 0 such that ∀δ > 0,
∃x, y ∈ [a, b] such that |x − y| < δ but f (x) − f (y) ≥ ε. Taking δn = n1 → 0 as n → ∞ yields two
sequences (xn )n≥1 , (yn )n≥1 ⊂ [a, b] with |xn − yn | < δn . Since (xn )n≥1 is bounded, it converges by the
Bolzano-Weierstrass Theorem, so ∃(xnk )k≥1 ⊂ (xn )n≥1 which converges in [a, b], since [a, b] is closed. So
limk→∞ xnk = u ∈ [a, b]. Choose (ynk )k≥1 , which converges by a similar argument, and choosing δ = n1k
demonstrates that limk→∞ xnk − ynk = 0, so limk→∞ ynk = u. But f is continuous,
so limk→∞
f (xnk ) =
f (u) = limk→∞ f (ynk ), which contradicts the condition that |x − y| < δ but f (x) − f (y) ≥ ε, so f is
uniformly continuous on [a, b]. Proposition 7:
If a function f : [a, b] → R is continuous on [a, b], then it is also regulated on [a, b].
k
=
{pi }i=0 be
a partition of [a, b] such that |pi − pi−1 | < δ, where δ
f (x) − f (y) < ε. Such a δ exists because f is continuous and so
 φ
= f (pi−1 ) for i = 1, . . . , k
[pi−1 ,pi )
uniformly continuous on [a, b]. Let φ ∈ S[a, b] be such that
. Now,
φ(b) = f (b)
∀x ∈ [a, b), ∃j such
that x ∈ [pj−1 , pj ), then f (x)− φ(x) = f (x) − f (pj−1 ) < ε since x − pj−1 < δ.
Also, f (b) − φ(b) = 0, so, ∀x ∈ [a, b], f (x) − φ(x) < ε, so f is regulated by definition. Proof: For any ε > 0, let P
is such that |x − y| < δ =⇒
Exercise:
Prove that any monotone function is regulated.
6
Proposition 8:
For any f, g ∈ R[a, b] and λ, µ ∈ R, λf + µg ∈ R[a, b].
Proof: First prove that ∀f, g ∈ B[a, b], ∀λ ∈ R, kf + gk∞ ≤ kf k∞ +kgk∞ and kλf k∞ = |λ|kf k∞ . The
remainder of the proof is left as an exercise. Definition 5:
Let f ∈ R[a, b]. Then
Rb
a
: R[a, b] → R is defined as
Z
b
f
a
..=
Z
n→∞
b
φn
lim
a
where (φn )n≥1 ⊂ S[a, b] converges to f uniformly as n → ∞.
Rb
Proposition 9: limn→∞ a φn exists and is independent of the choice of (φn )n≥1 ⊂ S[a, b], which
converges to f ∈ R[a, b] uniformly as n → ∞.
Proof: Let f ∈ R[a, b] and (φn )n≥1 , (ψn )n≥1 ⊂ S[a, b] converge to f uniformly as n → ∞. (φn ) conε
, so
verges to f uniformly, so for any ε > 0, eventually there will be an n such that kφn − f k∞ < 2(b−a)
eventually there will be n and m, when sufficiently large, such thatkφn − φm k∞ = (φn − f ) + (f − φm )∞
R
R
Rb
b
b
ε
≤ kφn − f k∞ +kf − φm k∞ ≤ b−a
. Then a φn − a φm = a (φn − φm ) ≤ kφn − φm k∞ (b − a) < ε, so
Rb
( a φn )n≥1 is Cauchy, so the limit exists.
Let ωn = (φn , ψn )n−1 = (φ1 , ψ1 , φ2 , ψ2 , . . .) ⊂ S[a, b]. Then ωn converges to f uniformly as n → ∞,
Rb
Rb
the proof of this fact is routine and is left as an exercise. limn→∞ a ωn exists, so the subsequences a φn
Rb
Rb
Rb
Rb
and a ψn must converge to the same limit a f , so a φn = a ψn . 7
2.2
Properties of the Integral for Regulated Functions
Proposition 10:
For any f ∈ R[a, b], u ∈ [a, b],
Z b
Z
f=
a
u
Z
b
f+
a
f
u
converges
Proof: f is regulated, so ∃(φn )n≥1 ⊂ S[a, b] which converges uniformly to f . Then (φn )
[a,u]
. So
, since restrictions are regulated. The same holds for (φn )
uniformly to f [u,b]
[a,u]
u
Z
Z
b
a
Z
u
u
n→∞
Z
b
φn
φn +
f = lim
f+
!
Z
u
a
n→∞
b
Z
a
b
f
φn =
= lim
a
by the additivity of integrals of step functions. Proposition 11:
The map I : R[a, b] → R, I : f 7→
Rb
a
f is linear.
Proof: The proof is similar to a combination of the last proof and the proof for the linearity of the
integral for step functions, and so is left as an exercise. Note:
Not all functions are regulated.
Example:
Consider f : [0, 1] → R, with
(
f : x 7→
f is also denoted by
1
0
x∈Q
x∈
/Q
1Q - the indicator of Q. Let φ be any step function
on [a, b], with a compatible
f
− φ1 . Then kf − φk∞ ≥ = max(|φ1 | ,|φ1 − 1|) ≥
(p0 ,p1 )
(po ,p1 )
∞
there cannot exist a sequence of step functions converging uniformly to f .
partition P , and let φ1 = φ
Proposition 12:
Then
1
2,
so
Suppose that f ∈ R[a, b], and that, ∀x ∈ [a, b], m ≤ f (x) ≤ M for some m, M ∈ R.
b
Z
m(b − a) ≤
f ≤ M (b − a)
a
Proof: f ∈ R[a, b], so, ∀ε > 0, ∃φε ∈ S[a, b] such that kf − φε k∞ < ε, so ∀x ∈ [a, b], f (x) − ε ≤ φε (x) ≤
f (x) + ε, so m − ε ≤ φε (x) ≤ M + ε, now using the result proved for step functions
Z b
(m − ε)(b − a) ≤
φε ≤ (M − ε)(b − a)
a
Choose ε =
then
1
n
for n ∈ N, then (φn )n≥1 converges uniformly to f . Take the limit of the above with ε =
Z
m(b − a) ≤
b
f ≤ M (b − a) a
Exercise:
Prove that, for any f ∈ R[a, b],
Z
b f ≤ kf k∞ (b − a)
a 8
1
n,
Example: f (x) = x is uniformly continuous on R, since f (x) − f (y) = |x − y|, so when|x − y| < δ = ε,
f (x) − f (y) < ε.
Proposition 13: Let fα : [1, ∞) → R, fα : x 7→ xα for some α ≥ 0. Then fα is uniformly continuous
for any α ≥ 0 but not uniformly continuous for α > 1.
Proof: For α > 1, fα (x+δ)−fα (x) = (x+δ)α −xα = xα ((1+ xδ )α −1) ≥ xα (1+α xδ −1) = αδxα−1 → ∞
as x → ∞, which contradicts the condition for uniform continuity.
α
For α < 1, α 6= 0, 0 ≤ fα (x + δ) − fα (x) = xα ((1 + xδ )α − 1) ≤ xα (1 + αδ
x − 1), (α − 1) < 0, so x → 0
ε
α
as x → ∞. For any ε > 0, choose δ = 2αδ , then x is uniform continuous.
The case when α = 0 is trivial and is left as an exercise.
Proposition 14:
If f : [a, b] → R is weakly increasing, then f ∈ R[a, b].
(a)
. Let P be a partition with n + 1 elements such that
Proof: For f (b) > f (a), let d = f (b)−f
n
p0 = a, pj = supx∈[a,b] {x : f (x) ≤ f (a) + dj}, and pn+1 = b. By construction, since pj > pj−1 , then
= f (a)+dj and
∀x ∈ (pj−1 , pj ), f (a)+d(j −1) ≤ f (x) ≤ f (a)+dj. Let ϕn ∈ S[a, b] such that ϕn (pj−1 ,pj )
ϕn (pj ) = f (pj ). Now, let x ∈ (pj−1 , pj ). Then, using the previous inequality, −d ≤ f (x) − ϕn (x) ≤ 0, so
(a)
kf − ϕn k∞ ≤ d = f (b)−f
→ 0 as n → ∞, so f is regulated. n
Exercise: Prove that if f : [a, b] → R is weakly decreasing, then f is regulated. This proves that all
monotonic functions on bounded intervals are regulated.
Example:
Let Q ∩ [0, 1] = { 01 ,
1 1 1 2 1 3 1
1, 2, 3, 3, 4, 4, 5,
(
1
1(x > qn ) =
0
Let f : [0, 1] → R be given by
f : x 7→
∞
X
, . . .} = {q1 , q2 , q3 , . . .}, and let
x > qn
x ≤ qn
2−n 1(x > qn )
n=1
which exists ∀ x ∈ [0, 1]. Then f is non-decreasing, and is regulated, but jumps an infinitely many times,
so is impossible to draw.
9
3
3.1
The Indefinite Integral & Fundamental Theorem of Calculus
The Indefinite Integral
Definition 6:
Let f ∈ R[a, b]. Define F : [a, b] → R by
Z x
F : x 7→
f
a
F is called the indefinite integral of f .
Lemma:
R[a, b] ⊂ B[a, b]
Proof: f ∈ R[a, b], so ∃φ ∈ S[a, b] such that kφ − f k∞ ≤ 1, so ∀ x ∈ [a, b], φ(x) − 1 ≤ f (x) ≤ φ(x) + 1.
Since step functions are bounded, φ ∈ B[a, b], so f ∈ B[a, b]. Proposition 15:
F is continuous on [a, b].
Proof: Let x, y ∈ [a, b]. Then
Z
F (x) − F (y) = a
x
y
Z
f−
a
Z x f = f ≤ kf k∞ |x − y| ,
y so by the sequential definition of continuity, f is continuous.
Note: Let f : [a, b] → R. Then, if ∃ L > 0 such that, ∀ x, y ∈ [a, b], f (x) − f (y) ≤ L|x − y|. Then f
is called Lipschitz continuous or Lipschitz. Lipschitz continuity implies continuity, but the reverse is not
generally true.
Example:
f (x) = e−x is Lipschitz on [0, 1], but g(x) =
10
√
x is not Lipschitz on [0, 1].
3.2
The Fundamental Theorem of Calculus
Theorem 16 - FTC1: Let f ∈ R[a, b] such that f is continuous at c ∈ (a, b). Then F (x) =
differentiable at c and F 0 (c) = f (c).
Rx
a
f is
Proof: f is continuous at c, so, ∀ε > 0, ∃δ > 0 such that, ∀x ∈ (c − δ, c + δ), f (x) − f (c) < ε, so
f (c) − ε < f (x) < f (c) + ε. Choose 0 < h < δ. Then, by integrating the previous statement
Z
c+h
Z
c+h
Z
a
c
c
f = F (c + h) − F (c) < h(f (c) + ε)
f−
f < h(f (c) + ε), so h(f (c) − ε) <
h(f (c) − ε) <
a
So, for any h ∈ (0, δ), h(f (c) − ε) < F (c + h) − F (c) < h(f (c) + ε), so
−ε <
F (c + h) − F (c)
− f (c) < ε
h
So F is differentiable at c, and F 0 (c) = f (c). The proof for the case where −δ < h < 0 is similar and so
is left as an exercise. Corollary 17:
If f ∈ C[a, b], then f is differentiable on (a, b) and F 0 = f .
Proof: f is continuous, so it is regulated. Applying FTC1 completes the proof.
Theorem 18 - FTC2: Let f : [a, b] → R be continuous, and let g : [a, b] → R be differentiable, with
g 0 (x) = f (x) ∀x ∈ [a, b]. Then
Z b
f = g(b) − g(a)
a
This is the Newton-Leibniz Formula, and g is called the antiderivative of f . This is sometimes written
Z
b
g 0 = g(b) − g(a)
a
Rx
Proof: Let δ : [a, b] → R, with δ(x) = g(x)− a f . Then δ is continuous on [a, b]. it is also differentiable
on (a, b) by FTC1. By the MVT, δ(b) − δ(a) = δ(ξ)(b − a) for some ξ ∈ (a, b). But δ 0 (ξ) = g 0 (ξ) − f (ξ) by
Ra
Rb
Rb
FTC1, so δ 0 (ξ) = f (ξ) − f (ξ) = 0, so δ(a) = δ(b), so g(a) − a f = g(b) − a f , so a f = g(b) − g(a). 11
4
Practical Methods of Integration
4.1
Integration by Parts
Proposition 20:
If f, g ∈ R[a, b], then f · g ∈ R[a, b].
Proof: kf + gk∞ ≤ kf k∞ +kgk∞ . Then ∃(ϕn ), (ψn ) ∈ S[a, b] suchthat (ϕn ) → f, (ψn ) → g uniformly.
0 ≤ kf · g − ϕn ψn k∞ = kf · g − ϕn g + ϕn g − ϕn ψn k∞ ≤ (f − ϕn )g ∞ + ϕn (g − ψn )∞ .
Since supx∈[a,b] a(x) · b(x) ≤ supx∈[a,b] a(x)·supx∈[a,b] b(x), this is less than or equal tokgk∞ kf − ϕn k∞ +
kϕn k∞ kg − ψn k∞ . Now, kgk∞ is regulated, and so is bounded. kf − ϕn k∞ ,kg − ψn k∞ → 0 as n → ∞,
and kϕn k∞ = kϕn − f + f k∞ ≤ kϕn − f k∞ + kf k∞ . Also, kϕn − f k∞ → 0 as n → ∞, and kf k∞ is
regulated, and so is bounded. Taken together, this means that kf · g − ϕn ψn k∞ → 0 as n → ∞, so
(ϕn ψn ) converges uniformly to f · g, so f · g is regulated. Corollary 19 - Integration by Parts: If F and G are differentiable such that F 0 and G0 are continuous, hence regulated, then
b
Z
a
Proof:
b
Z
F 0 G = F (b)G(b) − F (a)G(a) −
F G0
a
(F G)0 = F G0 + F 0 G. Then
b
Z
(F G)0 =
a
Z
b
F G0 + F 0 G =
b
Z
a
F G0 +
a
Z
b
F 0G
a
By FTC2
Z
b
F (b)G(b) − F (a)G(a) =
0
Z
0
FG + F G =
a
b
Z
0
a
a
b
F G = F (b)G(b) − F (a)G(a) −
Note:
Z
FG +
a
So
b
Z
0
F G0
a
F (b)G(b) − F (a)G(a) can be denoted by F G|ba .
Example:
The Gamma Function Γ : (0, ∞) → R is given by
Z
Γ : x 7→
∞
tx−1 e−t dt ..=
0
Z
lim
ε↓0,L→∞
L
tx−1 e−t dt
ε
Then
Z
Γ(x) =
lim
ε↓0,L→∞
ε
L
1 x
t
x
0
e−t dt
1
lim (tx e−t )|L
ε −
x ε↓0,L→∞
Z
1 ∞ x −t
=
t e dt
x 0
1
= Γ(x + 1)
x
Z
=
So Γ(x + 1) = xΓ(x), and Γ(1) = 1, so Γ(n + 1) = n!
12
ε
L
tx 1(−e−t ) dt
b
F 0G
4.2
Integration by Substitution
Corollary 21 - Integration by Substitution: Let f ∈ C[a, b], and let g : [c, d] → [a, b] be differentiable with Im g ⊂ [a, b]. Then
d
Z
f (g(x)) · g 0 (x) dx =
f (t) dt
g(c)
c
Proof: Let F (x) =
by FTC2
Rx
g(d)
Z
f . Since f is continuous, F is differentiable and F 0 (x) = f (x) by FTC1, then
a
Z
g(d)
f (t) dt = F (g(d)) − F (g(c))
g(c)
d
Also, dx
F (g(x)) = F 0 (g(x)) · g 0 (x) = f (g(x)) · g 0 (x) which is the initial integrand. Now, by FTC2,
d
F (g(x))|c = F (g(d)) − F (g(c)), so
d
Z
Z
0
g(d)
f (g(x)) · g (x) dx =
f (t) dt
c
g(c)
Note: g does not need to be injective or surjective.
Example:
Let g(x) = x2 . Then
Z
a
f (x2 )(x2 )0 dx = F (x2 )|a−a
−a
Example:
Let
G(a, j) ..=
Z
∞
2
e−ax
Z
+2jx
dx =
−∞
L1
2
e−ax
lim
L1 →∞,L2 →∞
+2jx
dx
−L2
for some a > 0, j ∈ R. Then
∞
Z
2
G(a, j) =
−∞
Z ∞
=
−∞
∞
Z
e−ax
+2jx
e−a(x
2
dx
+ 2j
a x)
j 2
e−a((x− a )
=
dx
2
j
−a
2)
dx
−∞
∞
Z
j2
j 2
e−a(x− a ) dx
=ea
−∞
Now, substituting g(x) = x − aj ,
e
j2
a
Z
∞
e
j 2
−a(x− a
)
dx = e
j2
a
Z
−∞
Now, substituting h(t) =
∞
e
−at2
dt = e
−∞
j2
a
Z
∞
e−(
√
at)2
−∞
√
at,
Z
2
e
j
a
∞
√
−( at)2
e
−∞
dt = e
j2
a
1
√
a
Z
e
−∞
So
j2
G(a, j) = e a
13
∞
r
π
a
−y 2
dy = e
j2
a
r
π
a
dt
4.3
Constructing Convergent Sequences of Step Functions
Theorem 22:
Let f : [a, b] → R be regulated. Then
lim
n→∞
n
X
b−a
h
j=1
Z
b
f (x) dx, where aj = a +
f (aj ) =
a
b−a
j
h
Proof: First, consider φ ∈ S[a, b]. Let P be a partition compatible with φ, with |P | = k. Then
Z b
n
X
b−a
b
−
a
φ−
φ(aj ) ≤ k
2kφk∞ → 0 as n → ∞
0 ≤
h
n
a
j=1
since errors only occur and contribute to the difference at jumps on the step function, of which there are
k, and the maximum error this can be is twice the maximum value of the step function across the whole
section. Then
Z b
n
X
b−a
φ(aj )
φ = lim
n→∞
h
a
j=1
Now, for f ∈ R[a, b], let
(n)
Rf
=
n
X
b−a
j=1
h
f (aj ), where aj = a +
b−a
j
h
Since f is regulated, then ,∀ε > 0, ∃φ ∈ S[a, b] such that kf − φk∞ < 3ε . Then
Z b
Z b Z b
Z b (n)
(n)
(n)
(n)
φ+
f = Rf − Rφ + Rφ −
φ−
f
Rf −
a
a
a
a
Z b Z b
(n)
(n)
(n) ≤ Rf − Rφ + Rφ −
φ + φ − f
a
a
Now,
ε
(n)
(n) Rf − Rφ ≤ kφ − f k∞ (b − a) < (b − a)
3
Z b (n)
ε
φ < (b − a) eventually in n
Rφ −
3
a
and
and
n
Z
n
b
X
X
b
−
a
≤
f (aj ) − φ(aj ) b − a < ε (b − a)
φ − f = (f (aj ) − φ(aj ))
a
n j=1
n
3
j=1
since f (aj ) − φ(aj ) < 3ε ,
so
Z b (n)
f < ε(b − a) eventually in n
Rf −
a
so
(n)
lim R
n→∞ f
Z
=
f
a
14
b
5
The Characterisation of Regulated Functions
5.1
Characterisation & Proof
Proposition 23: A function f ∈ R[a, b] iff, ∀x ∈ (a, b), f (x±) both exist, and both f (a+) and f (b−)
exist, where f (x+) = limy↓x f (y) (the right limit), and f (x−) = limy↑x f (y) (the left limit). Alternatively,
this can be stated as “all one-sided limits exist”.
Note: This characterisation only requires the existence of these limits, not equality, f (x+) 6= f (x−) is
acceptable so long as they both exist.
Proof =⇒ : f is regulated, so, ∀ε > 0, ∃φ ∈ S[a, b] such that kf − φk∞ < ε. Let x ∈ (a, b).
There are now
two cases, φ is continuous at x, or φ is discontinuous at x. In either case, ∃δ > 0
= φ0 , a constant. Let (xn )n≥1 ⊂ R be any sequence such that, ∀n ∈ N, xn > x,
such that φ
(x,x+δ)
f (xn ) − f (xm ) =
and
lim
x
=
x.
Eventually
x
∈
(x,
x
+
δ).
Then,
eventually
in
n
and
m,
n→∞
n
n
f (xn ) − φ0 + φ0 − f (xm ) ≤ f (xn ) − φ0 + f (xm ) − φ0 < 2ε, since f (xn ) − φ0 ,f (xm ) − φ0 < ε,
so f (xn )n≥1 is Cauchy, and so converges. If yn is any other sequence with yn ↓ x as n → ∞, then
(f (yn ))n≥1 converges to the same limit. The proof of this fact is routine after considering (xn , yn )n≥1 ,
and so is left as an exercise. It has been shown that for any (xn )n≥1 ⊂ R such that xn ↓ x as n → ∞,
limn→∞ f (xn ) = f (x+) exists and does not depend on xn . A similar argument can be used to prove
this result for the left limit, and not much modification is required for x = a, b, so these proofs are left
as exercises.
Proof ⇐= :
All one-sided limits exist. Now, fix any ε > 0. Let
:
Aε = {γ ∈ [a, b] ∃ψ ∈ S[a, b] such that (f − ψ)
[a,γ]
< ε}
∞
The proof shall follow from proving three statements:
(i) Aε 6= ∅: f (a+) exists, so, ∀ε > 0, ∃δ > 0 such that, ∀x ∈ (a, a + δ), f (x) − f (a+) < ε. Take
γ = 2δ . Then define
(
f (a)
x=a
φ(x) =
f (a+) x ∈ (a, a + 2δ )
that φ(x)
∈ S[a, a + 2δ ]. Since f (x) − f (a+) < ε and f (a) − φ(a) = 0,
It is clear
< ε, so γ = δ ∈ Aε , so Aε 6= ∅.
(f − φ)
2
δ [a,a+ 2 ] ∞
(ii) sup Aε = b: Assume
that c ..= sup Aε < b. Then, ∀δ > 0, (c − δ) ∈ Aε , so ∃φ ∈ S[a, c − δ] such that
(f − φ)
< ε. But f has limits f (c±) at c, so ∀ε > 0, ∃δ1 , δ2 such that, ∀x ∈ (c, c + δ2 ),
[a,c−δ] ∞
f (x) − f (c+) < ε and, ∀x ∈ (c − δ1 , c), f (x) − f (c−) < ε. Choose δ < δ1 . Now, let ψ ∈ S[a, b]
be given by


f (c−) x < c

ψ(x) = f (c)
x=c


f (c+) x > c
and let φ̃ ∈ S[a, b] be given by
(
φ(x) x ∈ [a, c − δ]
φ̃(x) =
ψ(x) x ∈ (c − δ, c +
Then (c +
δ2
2 )
∈ Aε , with
δ2
2
δ2
2 ]
> 0, which contradicts the fact that c = sup Aε , so b = c.
15
< ε. But
(iii) b ∈ Aε : It has been shown that, ∀δ > 0, ∃φ ∈ S[a, b − δ] such that (f − φ)
[a,b−δ] ∞
f (b−) exists, so ∃δ1 > 0 such that, ∀x ∈ (b − δ1 , b), f (x) − f (b−) < ε. Choose δ < δ1 . Now, let
ψ ∈ S[a, b] be given by
(
f (b)
x=b
ψ(x) =
f (b−) x < b
and let φ̃ ∈ S[a, b] be given by
(
φ̃(x) =
Now, by definition, f − φ̃
∞
φ(x) x ∈ [a, b − δ]
ψ(x) x ∈ (b − δ, b]
< ε, so b ∈ Aε .
So γ = b, so ∃ϕ ∈ S[a, b] such that kf − ϕk∞ < ε, so f ∈ R[a, b].
Exercise: Prove that if f ∈ R[a, b], then the set of discontinuities of f is countable. Hint: Check that,
∀ε > 0, the set of jumps of size ε or greater is countable.
16
6
6.1
Improper & Riemann Integrals
Improper Integrals
Motivation: It is possible to generalise the regulated integral to unbounded functions and/or unbounded domains.
Example:
1
Z
1
x− 2 dx = 2
0
because, although x
− 21
1
is not regulated on (0, 1], ∀ε > 0, x− 2 ∈ R[ε, 1]. Then
1
Z
1
1
1
x− 2 dx = 2x 2 |1ε = 2 − 2ε 2 → 2 as ε → 0
ε
Example:
∞
Z
1
because, ∀L > 1,
1
x2
1
dx = 1
x2
∈ R[1, L]. Then
L
Z
1
L
1
1 1
= 1 − → 1 as L → ∞
dx
=
−
2
x
x 1
L
Example:
Z
1
0
1
dx = lim
ε↓0
x
1
Z
ε
1
1
dx = lim log x|1ε = lim log
ε↓0
ε↓0
x
ε
which does not exist, therefore the integral is not defined.
Definition 7.1:
If, for any ε > 0, f : (a, b] → R is regulated on [a + ε, b] and
l
exists, then
Rb
a
b
lim
ε↓0
f
a+ε
R∞
Rb
a
f diverges.
If, ∀L > 0, f ∈ R[a, L] and
l
a
Z
f converges and is equal to l. If the limit does not exist, then
Definition 7.2:
exists, then
..=
..=
Z
lim
L→∞
L
f
a
f converges and is equal to l. If the limit does not exist, then
17
R∞
a
f diverges.
R∞
Rc
R∞
Note: Let f ∈ R[−L1 , L2 ] for some L1 , L2 > 0. Then −∞ f converges if, ∀c ∈ R, −∞ f and c f
Rc
R∞
R∞
Rc
R∞
converge. In this case, −∞ f + c f = −∞ f . As an exercise, show that if −∞ f and c f exist, then
Rc
R∞
f + c f does not depend on the choice of c.
−∞
Let f (x) = x ∈ R[−L, L] ∀L > 0. Then
Z L
Z
x dx = 0, so lim
Example:
L→∞
−L
L
x dx = 0
−L
However, take any c ∈ R. Then
L
Z
Z
c
x dx = ∞, and lim
lim
L→∞
L→∞
c
so do not exist. Then
Z
x dx = −∞
−L
∞
x dx
−∞
does not exist, since the limits must exist independently.
Rb
Suppose f : (a, b) → R, and, for any ε1 , ε2 > 0, f ∈ R[a + ε1 , b − ε2 ]. Then a f exists if, for
Rc
Rb
Rb
Rb
Rc
Rb
any c ∈ (a, b), a f and c f both exist (converge). Then a f converges, and a f = a f + c f .
Rc
Rb
Rc
Suppose f is defined on [a, b) ∪ (b, c]. Then a f exists (converges) if a f and b f both exist (indepenRc
Rb
Rc
dently), with a f = a f + b f .
Notes:
f (x) = √1
Example:
Note:
|x|
for x ∈ [−1, 0) ∪ (0, 1].
Let f : (a, b) → R such that, for any ε1 , ε2 > 0, f ∈ [a + ε1 , b − ε2 ]. Then ∃c ∈ (a, b) such that
Z c
Z b−ε2
l1 (c) = lim
f and l2 (c) = lim
f
ε1 ↓0
0
0
ε2 ↓0
a+ε1
c
0
exist iff, ∀c ∈ (a, b), l1 (c ) and l2 (c ) exist. The first formulation is often the more convenient in practice
Rb
Rb
however. If this formulation is true of f , then a f converges, and a f = l1 (c) + l2 (c). As an exercise,
check that l1 (c) + l2 (c) − l1 (c0 ) − l2 (c0 ) = 0 to demonstrate that this integral is independent of the choice
of c.
Definition 7.3:
Let f ∈ R[−L1 , L2 ] for some L1 , L2 > 0. Then
Z ∞
Z L1
Z
.
.
f = lim
f + lim
−∞
Definition 7.4:
L1 →∞
If f : [a, b) ∪ (b, c] → R, then
Z c
Z
f ..= lim
a
ε1 ↓0
c
L2 →∞
b−ε1
Z
c
c
f + lim
ε2 ↓0
a
f
−L2
f
b+ε2
Note:
All integrals in the limit must make sense as regulated or improper integrals.
Note:
It is possible to define the integral of functions with domains such as
[
f:
(j, j + 1) → R
j∈Z
Situations like this occur when studying gamma functions.
18
6.2
The Riemann Integral
Let f : [a, b] → R. Then
Definition:
Uf ..=
Lf ..=
b
Z
φ:φ≥f
inf
φ∈S[a,b]
(“upper sum”)
a
b
Z
φ:φ≤f
sup
φ∈S[a,b]
(“lower sum”)
a
f : [a, b] → R is Riemann integrable if Uf = Lf . Then
Z b
R f ..= Uf = Lf
Definition:
a
Proposition:
If f ∈ R[a, b], then it is Riemann integrable, and
Z b
Z b
R f=
f
a
a
Proof: f ∈ R[a, b], so, ∀ε > 0, ∃φ ∈ S[a, b] such that, ∀x ∈ [a, b], φ(x) − ε ≤ f (x) ≤ φ(x) + ε. Since
φ(x) − ε, φ(x) + ε ∈ S[a, b],
Z b
Z b
φ − ε(b − a) ≤ Lf ≤ Uf ≤
φ + ε(b − a)
a
a
Taking ε ↓ 0,
b
Z
Lf = Uf =
Z b
f= R f
a
Note:
a
There are functions which are Riemann integrable but not regulated.
Example:
Let f : [0, 1] → R be given by
(
1 x = 2−n for n = 0, 1, 2, . . .
f : x 7→
0 otherwise
Then f is not regulated since, ∀p > 0, ∃x ∈ (0, p) such that f (x) = 1 and ∃y ∈ (0, p) such that f (y) = 0,
so a step function can get no closer than a half to both “sections” of the function on any interval, so
Rb
f is not defined.
a
However, let
φN
(
1
x ≤ 2−N
=
f (x) x > 2−N
Now, φN ≥ f ≥ 0, so
Z
1
0 ≤ Lf ≤ Uf ≤
Z
1
φN = lim
N →∞
0
Then Lf = Uf = 0, so
Z 1
R f =0
0
so the Riemann integral of f is defined.
19
φN = 0
2−N
Note:
There are functions which are not Riemann integrable.
Example:
Let
1Q (x) =
Then U1Q = 1 6= 0 = L1Q , so
(
1
0
x∈Q
x∈R\Q
1Q is not Riemann integrable. However, it is Lebesgue integrable, with
Z b
L 1Q = 0
a
20
7
7.1
Uniform & Pointwise Convergence
Definitions, Properties and Examples
Example:
Let (fn )n≥1 be a sequence of functions on [-1,1], with
1
fn : x 7→ x− 2n −1 for n = 1, 2, . . .
Fix x and consider



1
lim fn (x) = 0
n→∞


−1
x>0
x = 0 , so lim fn (x) = sign x.
n→∞
x<0
So the limit of (fn (x))n≥1 as n → ∞ exists ∀x ∈ [−1, 1], and fn is continuous ∀n ∈ N, but the limit is
not continuous at 0.
lim lim fn (x) = 0 6= ±1 = lim sign x = lim lim fn (x)
n→∞ x→0±
x→0± n→∞
x→0±
This function has non-commutative limits and a sequence of continuous functions converges to a discontinuous function, both of which are undesirable properties of pointwise convergence.
Definition 8: Let (fn )n≥1 be a sequence of functions, and let A ⊂ R. Then fn → f pointwise as
n → ∞ if, ∀x ∈ A, limn→∞ fn (x) = f (x).
Note: Pointwise convergence is very “loose”, it does not preserve properties of the fn s such as continuity
1
(as in the case of x− 2n −1 → sign x). Pointwise convergence can also be very non-uniform (the same
example converges pointwise very slowly near 0).
Note: Let f and (fn )n≥1 be functions on A ⊂ R. If (fn ) → f pointwise as n → ∞, then f can fail to
be continuous, even if all fn s are. Alternatively, ∃x0 ∈ A such that
lim lim fn (x) 6= lim lim fn (x)
n→∞ x→x0
x→x0 n→∞
Let fn : [0, 1] → R, with


2n2 x

1
fn (x) = n − 2n2 (x − 2n
)


0
Example (The Witch’s Hat):
fn (0) = 0 and
1
x ∈ [0, 2n
]
1 1
x ∈ ( 2n , n ]
x > n1
Then fn ∈ C[0, 1]. Fix x > 0, then limn→∞ fn (x) = 0. Now, as n → ∞, fn → f ≡ 0 on [0, 1], which is
R1
R1
1
continuous. However, 0 f = 0, but 0 fn = n · 2n
= 12 , so
1
= lim
2 n→∞
Z
1
Z
fn 6=
0
1
lim fn = 0
0 n→∞
It is clear from examples such as this that pointwise convergence must be strengthened.
Definition 9: Let f, fn : A ⊂ R →
R for n = 1, 2, . . ., then (fn )n≥1 → f converges uniformly as n → ∞
if, ∀ε > 0, ∃Nε such that, ∀n > Nε , fn (x) − f (x) < ε ∀x ∈ A, or, equivalently, if limn→∞ kf − fn k∞ = 0.
21
Note: For the remainder of this section, assume, unless specified otherwise, that f, fn : A ⊂ R → R
for n = 1, 2, . . ..
Remark: Uniform convergence =⇒ pointwise convergence, but pointwise convergence =⇒
6
uniform
convergence.
Definition 9.1: A sequence
(fn )n≥1 (of functions on A ⊂ R) is called uniformly Cauchy if, ∀ε > 0,
∃Nε such that, ∀n, m > Nε , fn (x) − fm (x) < ε ∀x ∈ A.
Note:
For a fixed x ∈ A, if (fn (x))n≥1 is Cauchy, then it is pointwise convergent (to a common limit f ).
Lemma:
A uniformly Cauchy sequence converges uniformly.
Proof: Let (fn )n≥1 , (fm )m≥1 be uniformly Cauchy sequences converging to f . Then, ∀ε > 0, ∃Nε
such that, ∀n, m > Nε , fm (x) − 2ε ≤ fn (x) ≤ fm (x) + 2ε ∀x ∈ A. Take m → ∞, then fm → f , so
f (x) − 2ε ≤ fn (x) ≤ f (x) + 2ε ∀x ∈ A, so fn (x) − f (x) < ε ∀x ∈ A, so (fn )n≥1 converges uniformly. Theorem 24:
Let (fn )n≥1 ⊂ R[a, b]. Assume that fn → f uniformly. Then
(i) f ∈ R[a, b]
Rb
Rb
Rb
(ii) limn→∞ a fn = a limn→∞ fn = a f
Proof:
(i) fn ∈ R[a, b], so, ∀ε > 0, ∃φn ∈ S[a, b] such thatkfn − φn k∞ < 2ε . Now,kf − φn k∞ = kf − fn + fn − φn k∞ ≤
kf − fn k∞ +kfn − φn k∞ . Assume that n is large enough such that kf − fn k∞ < 2ε , which is guaranteed since fn → f , so kf − φn k∞ < ε, so f ∈ R[a, b].
Rb
(ii) It was shown in (i) that f is regulated, so a f is well defined and
Z
Z b b
0 ≤
f−
fn ≤ kf − fn k∞ (b − a)
a
a
kf − fn k∞ → 0 as n → ∞, so
Z
lim
n→∞
b
Z
fn =
a
b
Z
lim fn =
a n→∞
b
f
a
Theorem 25: Suppose that (fn )n≥1 ⊂ C[a, b], and fn → f uniformly as n → ∞. Then f ∈ C[a, b].
This theorem also holds for some arbitrary A ⊂ R in place of [a, b], and shall be proven for this more
general case.
Proof: fn → f uniformly, so ∀ε > 0, ∃Nε such that, ∀n > Nε , fn(x) − f (x) < ε ∀x ∈ A, so fn is
ε
continuous ∀n ∈ N. Then ∃δε0 such that, ∀x ∈ (x0 − δε0 , x0 + δε0 ) ∩ A, fn (x) − fn (x0 ) <
3
0
0
0
0
0
∀x ∈ (x0 − δε , x0 + δε ) ∩ A. Set δε = δε . Then,
∀x ∈ (x0 − δε, x0 + δε ) ∩ A, f (x) − f (x0 ) =
f (x) − fn (x) + fn (x) − fn (x0 ) + fn (x0 ) − f (x0 ) ≤ fn (x) − fn (x0 )+fn (x) − fn (x0 )+fn (x0 ) − f (x0 ).
fn → f uniformly as n → ∞, so the first
terms are less than 3ε , and the middle term
and last of these
ε
3ε
was shown to be less than 3 earlier, so f (x) − f (x0 ) < 3 = ε, so f is continuous at x0 . 22
Example (Cantor’s Function or The Devil’s Staircase): Let (fn )n≥1 ⊂ C[0, 1] be such that
f0 ≡ 1 and


 21 fn (3x)
x ∈ [0, 31 )

1
fn+1 (x) = 2
x ∈ [ 31 , 23 ]


 1 + 1 fn (3x − 2) x ∈ ( 2 , 1]
2
2
3
Proposition:
limn→∞ fn exists and is continuous. It is called Cantor’s function.
Proof: Clearly f0 ∈ C[0, 1]. Now, suppose that fn ∈ C[0, 1]. Then fn+1 is clearly continuous on ( 13 , 23 ).
Now, fn is continuous on [0, 13 ) by assumption, so 12 fn (3·) is continuous, so fn+1 is continuous on [0, 31 ).
Also, fn is continuous on ( 23 , 1] by assumption, so 12 + 12 fn (3· − 2) is continuous, so fn+1 is continuous
on ( 23 , 1]. Now the only points remaining to investigate the continuity of are 31 and 23 .
fn+1 (1) =
1
2
+ 12 fn (1), which is 1 if fn (1) = 1. f0 (1) = 1, so fn (1) = 1 ∀n ∈ N by induction.
Concerning the continuity of 13 , fn+1 ( 31 ) = 12 , and fn+1 ( 13 +) = 12 , so fn+1 is right-continuous at 13 , and
fn+1 ( 13 −) = 12 fn (3( 31 −)) = 12 fn (1−) = 12 fn (1) = 12 since fn is continuous at 1, so fn+1 is left-continuous
at 31 , so fn+1 is continuous at 13 . The proof for 23 is similar and is left as an exercise.
Now,
kfn+1 − fn k∞
= max (fn+1 − fn )
(f
−
f
)
= max n+1
n
1 x∈[0, )
3
∞
x∈[0, 1 )
∞
3
,
(fn+1 − fn )
(f
−
f
)
, 0,
n+1
n

1
1

= max  fn (3·) − fn−1 (3·) 2
2
x∈[0, 1 )
3
∞
x∈[ 1 , 2 ]
3 3
∞
,
(fn+1 − fn )
!
x∈( 2 ,1]
3
∞
!
2
x∈( ,1]
3
∞

1

1

, fn (3· − 2) − fn−1 (3· − 2) 2 2
2
x∈( ,1]
3
∞
1
≤ kfn − fn−1 k∞
2
So, for some c, d ∈ R,
kfn−1 − fn k∞ ≤
1
1
c
d
kfn − fn−1 k∞ ≤ 2 kfn−1 − fn−2 k∞ ≤ . . . ≤ n+1 kf1 − f0 k∞ ≤ n+1
2
2
2
2
The constants are introduced to correct for f0 .
Now, fix n > m, then, for some D ∈ R,
0 ≤ kfn − fm k∞ = kfn − fn+1 + fn+1 − fn−2 + . . . + fm−1 − fm k∞
≤ kfn − fn−1 k∞ +kfn−1 − fn−2 k∞ + . . . +kfm+1 − fm k∞
n−m−1
X 1
1
1
1
D
≤ D n + n−1 + . . . + m+1 ≤ m+1
2
2
2
2
2k
k=0
≤
D
2m+1
∞
X
D
1
= m → 0 as m → ∞
k
2
2
k=0
Since kfn − fm k∞ can be made arbitrarily small by taking n > m large enough, (fn ) is uniformly
Cauchy. Therefore, f = limn→∞ fn exists and is continuous, since a uniformly Cauchy sequence converges
uniformly and fn is continuous ∀n ∈ N. The limit is called Cantor’s function or the Devil’s Staircase.
23
Now, f ∈ C[0, 1], so f ∈ R[0, 1]. Taking the limit of fn (x),

1


 2 f (3x)
f (x) = 12


 1 + 1 f (3x − 2)
2
2
So
Z
1
Z
f=
0
1
3
2
3
Z
f+
0
Z
1
f+
1
3
Z
f=
2
3
0
1
3
1
f (3x) dx +
2
it is clear that
x ∈ [0, 13 )
x ∈ [ 13 , 23 ]
x ∈ ( 23 , 1]
2
3
Z
1
3
1
dx +
2
Z
1
2
3
1 1
+ f (3x − 2) dx
2 2
Let y = 3x, z = 3x − 2. Then
Z 1
Z
Z
Z
Z
Z 1
1 1 1
1 1 1
1 1 1
2 1
1
1
f= +
f (y) dy + +
f (z) dz = +
f , so
f = , so
f=
6
6
6
6
3
3
3
3
2
0
0
0
0
0
0
Remark: Assume that f is constant on E ⊂ [0, 1]. Then f is constant on ( 21 , 23 ) ∪ 13 E ∪ ( 23 + 13 E), so
|E| = 13 + 13 |E| + 31 |E|, so 13 |E| = 13 , so |E| = 1, so f is locally constant on some E ⊂ [0, 1] of length 1,
but f : [0, 1] → [0, 1] continuously.
Note: It is possible to map [0, 1] ⊂ R onto (surjectively) [0, 1] × [0, 1] ⊂ R2 continuously. If γ :
[0, 1] → [0, 1] × [0, 1] is such a map, then, ∀x ∈ [0, 1] × [0, 1], ∃t ∈ [0, 1] such that γ(t) = x. A
surjective map is relatively simple to construct, such as τ : [0, 1] → [0, 1] × [0, 1], with τ : 0.x1 x2 x3 . . . 7→
(0.x1 x3 x5 . . . , 0.x2 x4 x6 . . .), but τ is not continuous. A continuous map satisfying these criteria is called
a space filling curve, and examples were constructed by Hilbert and Peano in the 1890s.
24
7.2
Uniform Convergence & Integration
Note:
Throughout this section, let D = [a, b] × [c, d] ⊂ R2 , and (x, t) ∈ D.
Definition 10: A function f : D → R is continuous at (x0 , t0 ) ∈ D if, ∀ε > 0, ∃δε (x
0 , t0 ) > 0 such that,
∀(x0 , t0 ) ∈ D with |x − x0 | < δε (x0 , t0 ) and |t − t0 | < δε (x0 , t0 ), f (x, t) − f (x0 , t0 ) < ε. An equivalent
definition, which shall be introduced later, uses Euclidean distance.
Definition 10.1: A function f : D → R is uniformly
continuous on D if, ∀ε > 0, ∃δε such that,
∀(x, t), (x0 , t0 ) ∈ D with |x − x0 | < δε and |t − t0 | < δε , f (x, t) − f (x0 , t0 ) < ε.
Lemma 26.2:
If f is continuous on D, then it is also uniformly continuous if D is closed.
Proof: The proof is left as an exercise. Hint: Fix t ∈ [c, d], and consider f (·, t) : [a, b] → R with
f : x 7→ f (x, t). Exercise:
Prove that, if f is continuous on D, then f (·, t) is continuous on [a, b] (so f (·, t) ∈ C[a, b]).
Lemma 26.1:
If a map I is such that
Z
b
I : t ∈ [c, d] 7→
f (x, t) dx
a
then, if f ∈ C(D), I ∈ C[c, d] (I is often called the integral depending on a parameter ).
Proof: Let (tn )n≥1 ⊂ [c, d] with limn→∞ tn = t0 , and let fn : [a, b] → R with fn : x 7→ f (x, tn ). Now,
f is continuous on D, so f is uniformly
continuous on D since D is closed, so, ∀ε > 0, ∃δε>0 such that,
if |x − x0 | < δε and |t − t0 | < δε , then f (x, t) − f (x0 , t0 ) < ε. Since (tn ), (tn+m) → t0 as n → ∞, then,
eventually in n, |tn − tn+m | < δε , so fn (x) − fn+m (x) = f (x, tn ) − f (x, tn+m ) < ε eventually in n, so
(fn ) is uniformly Cauchy, so (fn ) converges uniformly to f . Now,
Z
lim I(tn ) = lim
n→∞
n→∞
b
Z
f (x, tn ) dx = lim
n→∞
a
b
b
Z
fn (x) dx =
a
Z
b
lim fn (x) dx =
a n→∞
f (x, t0 ) = I(t0 )
a
So I is continuous at t0 , and since t0 ∈ [c, d] was arbitrary, I is continuous on [c, d].
Proposition 27:
If f, ∂f
∂t are continuous on [a, b] × [c, d], then, ∀t ∈ (c, d),
∂
∂t
Z
Z
b
b
Z
f (x, t) dx =
a
a
b
∂f
(x, t) dx
∂t
Or, alternatively, ∀t ∈ (c, d),
F (t) =
Z
f (x, t) dx and G(t) =
a
a
both exist on (c, d), F is differentiable on (c, d), and F 0 = G.
25
b
∂f
(x, t) dx
∂t
Proof: Fix t ∈ (c, d). Then f (·, t), ∂f
∂t (·, t) ∈ C[a, b], so F and G exist. Now, ∃[c1 , d1 ] ⊂ [c, d] such that
is
continuous
on
[a, b] × [c, d], and so is uniformly continuous on [a, b] × [c, d]. Then
t ∈ (c1 , d1 ). But ∂f
∂t
Z b
F (t + h) − F (t)
f
(x,
t
+
h)
−
f
(x,
t)
∂f
− G(t) = −
(x, t) dx
a
h
h
∂t
Z
b ∂f
∂f
MVT = (x, τ ) −
(x, t) dx
a ∂τ
∂t
for some τ ∈ (t, t + h). Now,
with |h| < δε ,
∂f
∂τ
is uniformly continuous on [a, b] × [c, d], so ∀ε > 0, ∃δε > 0 such that ∀h
Z
Z
b ∂f
b
∂f
∂f
∂f
(x, τ ) −
< ε, so
ε = ε(b − a)
<
(x,
t)
(x,
τ
)
−
(x,
t)
dx
∂τ
a ∂τ
∂t
∂t
a
so
F (t + h) − F (t) = G(t)
lim
h→0
h
so F 0 (t) = G(t).
Theorem 28 - Fubini’s Theorem: If f : D → R is continuous, then
!
!
Z b Z d
Z d Z b
f (x, y) dy dx =
f (x, y) dx dy
a
c
c
Proof: Let
t
Z
Z
a
!
d
d
Z
f (x, y) dy dx −
F (t) =
a
!
t
Z
f (x, y) dx dy
c
c
a
for t ∈ [a, b]. Now, F (a) = 0, and
0
FTC
Z
d
d
Z
f (t, y) dy −
F (t) =
c
f (t, y) dy = 0
c
F is continuous on [a, b] and differentiable on (a, b), and F 0 (t) = 0, so F (b) − F (a) = 0, so
!
!
Z b Z d
Z d Z b
f (x, y) dy dx =
f (x, y) dx dy a
Note:
c
c
a
f must be continuous on the whole of D, or counterexamples such as the following can arise:
Let D = [0, 2] × [0, 1], and let f (x, y) : D → R with

 xy(x2 −y2 )
(x2 +y 2 )3
f : (x, y) 7→
0
Then
Z
0
2
Z
0
1
!
(x, y) 6= (0, 0)
(x, y) = (0, 0)
1
1
f (x, y) dy dx = =
6
=
5
20
26
Z
1
Z
2
!
f (x, y) dx dy
0
0
7.3
Uniform Convergence & Differentiation
Note: Suppose that (fn )n≥1 is a sequence of functions on [a, b], and that (fn ) → f uniformly as n → ∞.
Then, even if every fn is differentiable on [a, b], f is not necessarily differentiable on [a, b]. And even if
f is smooth, (fn0 ) does not necessarily converge to f 0 .
Example: Suppose that fn (x) = n1 cos nx on R. This function is smooth, and fn0 (x) = sin nx. Let
f : x 7→ 0. Then kfn − f k∞ = kfn k∞ = n1 kcos n·k∞ = n1 → 0 as n → ∞, so (fn ) → f uniformly as
n → ∞. All fn s are smooth, and f is smooth, since f 0 ≡ 0. But (fn0 ) 6→ 0 as n → ∞, in fact, (fn0 ) does
not converge at all, except for at certain points such as x = kπ for k ∈ N.
q
Let fn : R → R, with fn : x 7→ x2 + n1 . Then (fn (x)) → |x| pointwise as n → ∞. Let
|fn (x)2 −f (x)2 |
f (x) = |x|. Then fn (x)2 − f (x)2 = x2 + n1 − x2 = n1 . fn (x) − f (x) = f (x)+f (x) ≤ n1 √1 1 = √1n →
|n
|
n
0 as n → ∞, so (fn ) → f uniformly as n → ∞. Now, all fn s are smooth, and (fn ) → f uniformly as
n → ∞, but f is not differentiable at 0.
Example:
Note: If a function f : [a, b] → R is continuously differentiable, it is called a C 1 function, or, alternatively, f ∈ C 1 [a, b] is written, where C 1 [a, b] is the space of continuously differentiable functions on
[a, b].
Theorem 29: Let (fn ) be a sequence of C 1 functions on [a, b], with (fn ) → f pointwise as n → ∞ on
[a, b], and suppose that (fn0 ) converges uniformly. Then f ∈ C 1 [a, b] and limn→∞ (fn0 ) = f 0 .
Proof: Let g = limn→∞ (fn0 ). Now, ∀n ∈ N, fn0 ∈ C[a, b], so g ∈ C[a, b] since (fn0 ) → g uniformly, so
Z x
Z x
Z x
FTC
0
g=
lim (fn ) = lim
fn0 = lim [fn (x) − fn (a)] = f (x) − f (a)
a
a n→∞
n→∞
n→∞
a
so, since
Z
x
f (x) = f (a) +
g
a
Rx
a
g is continuous, so by the FTC, f 0 = g = limn→∞ (fn ), so
0
lim (fn )
n→∞
FTC
= f0 =
27
lim (fn0 ) n→∞
8
8.1
Functional Series
The Weierstrass M-Test & Other Useful Results
P∞
Note: Let (fn )n≥1 be a sequence of functions on A ⊂ R. Then thei functional
series k=1 fk converges
Pn
pointwise (or uniformly) on A if the sequence of partial sums Sn = ( k=1 fk )n≥1 converges pointwise
(or uniformly).
Theorem 240 : Let (fn )n≥1 ⊂ R[a, b]. Suppose that Sn =
P
∞
k=1 fk ∈ R[a, b] and
Z bX
∞
∞ Z b
X
fk =
fk
a k=1
Proof: If (fk ) ∈ R[a, b], then Sn =
Pn
k=1
k=1
Pn
fk ∈ R[a, b] ∀n ∈ N, so
x→x0
Proof: Sn ∈ C[a, b] ∀n ∈ N, so limx→x0
Theorem 290 :
∞
X
fk (x) =
k=1
∞
X
k=1
P∞
k=1
If (fn )n≥1 ⊂ C 1 [a, b] and Sn0 =
Pn
k=1
R b P∞
k=1
a
Pn
k=1
fk =
P∞ R b
k=1 a
fk . fk converges uniformly. Then
lim fk (x)
x→x0
P∞
fk (x) =
fk converges uniformly. Then
a
Theorem
250 : Suppose that (fn )n≥1 ⊂ C[a, b] such that Sn =
P∞
k=1 fk ∈ C[a, b], and, ∀x ∈ [a, b],
lim
k=1
k=1
limx→x0 fk (x). fk0 converges uniformly on [a, b], then
∞
∞
k=1
k=1
X d
d X
fk (x) =
fk (x)
dx
dx
Proof: The proof is left as an exercise. Theorem 30 - Weierstrass M-Test:
Let
⊂ R. Then, if
(fn )n≥1 bePa∞sequence of functions on
PA
∞
∃(Mk )k≥1 ⊂ R such that, ∀x ∈ A, fn (x) < Mn and
M
converges,
then
k
k=1
n=1 fn converges
uniformly.
P∞
Pn
Mk )n≥1 is Cauchy, so,
k=1 Mk converges, so (
∀εn > 0 ∃Nεn such thatP∀n, m > Nεn ,
k=1
Pn
P
P
m
n
n
with n > m, k=1 Mk − k=1 Mk < εn . Then k=m+1 Mk < εn , so, since MK ≥ 0, k=m+1 Mk <
εn . Therefore
n
n
m
n
n
X
X
X
X
X
≤
fk (x) ≤
f
(x)
−
f
(x)
=
f
(x)
Mk < εn
k
k
k
k=1
k=m+1
k=m+1
k=1
k=m+1
Proof:
This holds ∀x ∈ A, so the sequence of partial sums of fk is uniformly Cauchy, so
uniformly to its pointwise limit. Corollary 30.1:
If (fn )n≥1 is such that
P∞
k=1 kfk k∞
28
converges, then
P∞
k=1
P∞
k=1
fk converges
fk converges uniformly.
8.2
Taylor & Fourier Series
Motivation:
Taylor polynomials are of the form
n X
1 (k)
k
Sn =
f (a)(x − a)
k!
k=0
and were studied in Analysis II.
Fourier series are of the form
n
X
1
a0 +
(ak cos kx + bk sin kx), x ∈ [0, 2π]
2
k=1
Applications of functional series include solving PDEs, ODEs and time series.
Definition:
For any f ∈ R[−π, π], the series
∞
a0 X
+
(ak cos kx + bk sin kx)
2
k=1
with coefficients given by
Z
1 π
f (x) cos kx dx, k ≥ 0
ak =
π −π
Z π
1
f (x) sin kx dx, k > 0
bk =
π −π
is called the Fourier series generated by f .
Note: It is not immediately obvious for which classes of functions this series is convergent, or if it
converges to f .
P∞
P∞
Theorem 31: Let (ak )k≥0 , (bk )k≥1 ⊂ R, with k=0 |ak | and k=1 |bk | convergent. Then
P∞
• a20 + k=1 (ak cos kx + bk sin kx) converges uniformly on R.
• The pointwise limit f : R → R is continuous.
• f is 2π-periodic (f (x + 2π) = f (x)).
Moreover,
Z
1 π
f (x) cos kx dx, k ≥ 0
π −π
Z
1 π
bk =
f (x) sin kx dx, k > 0
π −π
ak =
so the terms of the series can be recovered.
ProofP(Convergence):
∀x ∈ R, |aP
cos kx| + |bk sin kx| ≤ |ak | + |bk | ..= Mk .
k cos kx + bP
k sin kx| ≤ |akP
P∞
∞
∞
∞
∞
Then k=1 Mk = k=1 |ak | +|bk | = k=1 |ak | + k=1 |bk |, so k=1 Mk converges, so the Fourier series
converges by the M-test.
Proof (Continuity): Let f (x) ..=
a0
2
+
P∞
k=1 (ak
cos kx + bk sin kx). Then f ∈ C(R).
Pn
Proof (Periodicity): Let SN (y) ..= a20 + k=1 (ak cos kx + bk sin kx). Then f (x + 2π) − f (x) =
limn→∞ Sn (x+2π)−limn→∞ Sn (x). Both of these limits exist, so f (x+2π)−f (x) = limn→∞ Sn (x + 2π)−
Sn (x) = 0, since cos (k(x + 2π)) = cos kx and sin (k(x + 2π)) = sin kx.
29
Lemma:
∀m ∈ Z
π
Z
−π
eimx dx = 2π 1(m = 0)
where 1(m = 0) is the indicator function for m = 0.
Proof: For m ∈ Z \ {0}:
Z
Z π
Z π
imx
cos mx + i sin mx dx =
e
dx =
For m = 0:
Z
π
e
So
π
2 sin πm
=0
m
π
e dx =
1 dx = 2π
−π
eimx dx = 2π 1(m = 0) −π
Proof (Recovery):
Z π
Z
f (x) dx =
Z
−π
Z
sin mx dx =
−π
0
dx =
π
cos mx dx + i
π
Z
imx
−π
−π
Z
−π
−π
−π
π
∞
π
a0 X
+
(ak cos kx + bk sin kx) dx
−π 2
k=1
" Z
#
Z
Z π
∞
π
X
a0 π
1 dx +
ak
cos kx dx + bk
sin kx dx = πa0
=
2 −π
−π
−π
k=1
so
a0 =
1
π
π
Z
f (x) dx =
−π
1
π
Z
π
f (x) cos 0x dx
−π
Now,
" Z
# ∞ " Z
#
Z π
Z π
∞
π
π
X
X
1
f (x) cos mx dx = a0
cos mx dx+
ak
cos kx cos mx dx +
bk
sin kx cos mx dx
2
−π
−π
−π
−π
k=1
k=1
and
π
Z
π
Z
sin kx cos mx dx =
−π
−π
1
1 ikx
(e − e−ikx ) (eimx − e−imx ) dx
2i
2
Z
1 π i(k−m)x
=
e
− ei(m−k)x + ei(k+m)x − ei(m+k)x dx
4i −π
1
= (2π 1(k − m = 0) − 2π 1(m − k = 0)) = 0
4i
and
Z
π
Z
π
cos kx cos mx dx =
−π
−π
1
1 ikx
(e + e−ikx ) (eimx − e−imx ) dx
2
2
Z
1 π i(k−m)x
=
e
+ ei(m−k)x + ei(k+m)x + ei(m+k)x dx
4 −π
1
= 2π((1(k − m = 0) + 1(m − k = 0)) = π 1(k = m)
4
then
Z
π
f (x) cos mx dx =
−π
∞
X
(ak π 1(k = m)) = πam
k=1
so
am =
1
π
Z
π
f (x) cos mx dx
−π
The proof for bk is similar and so is left as an exercise. 30
Note: This proof relies on an application of Theorem 24, where the sum is multiplied by a bounded
function. The proof that the result holds is left as an exercise.
Corollary 31.1: If f : [−π, π] → R is a C 2 function, ( or f 0 ∈ C[−π, π]), and f (π) = f (−π). Then the
Fourier series generated by f converges to f uniformly.
Note:
It must be established that
(i) The Fourier series converges uniformly.
(ii) The limit is f .
However, (ii) is a fairly clunky argument using existing tools, and so shall be omitted and left as an
exercise, whilst (i) shall be proved.
Proof:
!
Z π
Z π
π
1
1 0
0
f (x) cos kx dx
f (x) sin kx dx =
f (x) cos kx dx =
f (x) cos kx −π −
πk −π
πk
−π
−π
!
Z π
Z π
Z π
0
π
1
1
1
0
0
0
00
=
f (x) cos kx dx =
f (x) sin kx dx =
f (x) sin kx −π −
f (x) sin kx dx
πk −π
πk 2 −π
πk 2
−π
Z π
1
=− 2
f 00 (x) sin kx dx
πk −π
1
bk =
π
So
Z
π
Z
1 π
1 f 00 · sin k · · 2π = 2 f 00 ..= cb
|bk | =
f
(x)
sin
kx
dx
≤
∞
∞
πk 2
πk 2 −π
k2
k2
and cb is independent
A similar argument establishes that |ak | ≤ kca2 for k > 0, where ca is also
P∞of k.
ca +cb
1
independent of k.
k=1 k2 converges, so the Fourier series converges by the M-test with Mk =
k2
for k > 0. Example (Temperature Equilibration on a Non-Uniformly Heated Ring): Consider the unit
circle S 1 . Let any point on it be described by the angle ϕ ∈ [−π, π] it makes with the x-axis, and let
the initial temperature on the ring be given by f (ϕ) ∈ C 2 [−π, π] such that f (−π) = f (π). Then the
temperature T at time t at the point at ϕ, for t > 0, ϕ ∈ [−π, π], is given by the Fourier heat equation
∂T
∂2T
(t, ϕ) −
(t, ϕ) = 0
∂t
∂ϕ2
Since the point at π is the same as the point at −π, there are some boundary conditions to consider also:
T (t, −π) = T (t, π)
∂T
∂T
(t, −π) =
(t, π)
∂ϕ
∂ϕ
The initial condition is given by
T (0, ϕ) = f (ϕ)
Throughout this example,
Ṫ ..=
∂T
∂T
∂2T
, T 0 ..=
, T 00 ..=
∂t
∂ϕ
∂ϕ2
31
Now, (cos kϕ)0 = −k sin kϕ, and (cos kϕ)00 = −k 2 cos kϕ, so A(t) cos kϕ seems a suitable ansatz to
make for a solution. Substituting this into the heat equation yields Ȧ cos kϕ + k 2 A cos kϕ = 0, so
Ȧ(t) + k 2 A(t) = 0, which is an ODE. The boundary conditions are in fact satisfied by either this
solution, or one using the ansatz B(t) sin kϕ, but neither of these satisfy the initial conditions. Since a
finite linear combination of any of these functions will have the same shortcoming, it seems sensible to
take the next ansatz as being an infinite linear combination of them, such as
∞
T (t, ϕ) =
a0 (t) X
+
(ak (t) cos kϕ + bk (t) sin kϕ)
2
k=1
Assume that T , Ṫ , T 0 and T 00 all converge uniformly for t > 0, ϕ ∈ [−π, π]. Then substituting T into
the heat equation and differentiating pointwise yields
∞
∞
k=1
k=1
X
a˙0 X
+
(a˙k cos kϕ + ·bk sin kϕ) +
(ak k 2 cos kϕ + bk k 2 sin kϕ) = 0
2
So
∞
i
a˙0 X h
+
(a˙k + ak k 2 ) cos kϕ + (b˙k + bk k 2 ) sin kϕ = 0
2
k=1
which is the Fourier series generated by 0, so
a˙0 = 0
2
a˙k + k ak = 0
b˙k + k 2 bk = 0
for k > 0. This yields infinitely many ODEs, which require infinitely many initial conditions.
Expanding f (ϕ) ∈ C 2 [−π, π] by a Fourier series yields
f (ϕ) =
∞ (f )
X
a0
(f )
(f )
+
ak cos kϕ + bk sin kϕ
2
k=1
(f )
where nk denotes the coefficient nk dependent on the function f . This gives the required amount of
initial conditions. Then
(f )
a0 (t) = a0
(f )
2
(f )
2
ak (t) = ak e−k
bk (t) = bk e−k
So
(f )
T (t, ϕ) =
t
t
∞
X
2
a0
(f )
(f )
+
e−k t ak cos kϕ + bk sin kϕ
2
k=1
Whilst this is a solution, its uniqueness must also be established, however that shall not be done here.
Exercise:
Note:
Check that T converges uniformly for t ≥ 0, and that Ṫ , T 0 , T 00 do so also for t > 0.
Over time, the temperature becomes distributed evenly across the whole ring, since
(f )
a
1
lim T (t, ϕ) = 0 =
t→∞
2
2π
Z
π
f (ϕ) dϕ = Average Temperature.
−π
The approach to this equilibrium is exponential.
32
Part II
Norms
9
Normed Vector Spaces
9.1
Definition & Basic Properties of a Norm
Definition 11: Let V be a vector space over R. Then a function k·k : V → R, with k·k : v ∈ V 7→ kvk
which satisfies the following:
(i) ∀v ∈ V , kvk ≥ 0 (positivity). Moreover, kvk = 0 ⇐⇒ v = 0 ∈ V (the ability to separate points).
(ii) ∀λ ∈ R, ∀v ∈ V , kλvk = |λ| ·kvk (absolute homogeneity).
(iii) ∀u, v ∈ V , ku + vk ≤ kuk +kvk (triangle inequality).
is called a norm on V . A pair (V,k·k) is called a normed vector space, or a normed space.
Remarks:
• (ii) and (iii) use the linear structure of V .
abs.
• Positivity follows from (ii) and (iii), since, ∀v ∈ V , 0 = 0 ·kvk = k0 · vk −kv − vk ≤ kvk +k−vk =
hom.
kvk +|−1| ·kvk = kvk +kvk = 2kvk, so 0 ≤ kvk.
Examples:
1. |·|, the absolute value, is a norm on R. The triangle inequality is satisfied since |x + y| ≤ |x| +|y|.
The proofs of the other properties are left as an exercise.
2. k·k∞ , the sup norm, is a norm on B[a, b].
(i) : ∀f ∈ B[a, b], if kf k∞ = 0, then supx∈[a,b] f (x) = 0, so, ∀x ∈ [a, b], 0 ≤ f (x) ≤ 0, so f ≡ 0
on [a, b], so k·k∞ can separate points (kf − gk∞ = 0 ⇐⇒ f = g, kf − gk∞ > 0 ⇐⇒ f 6= g).
(ii) : ∀λ ∈ R, ∀f ∈ B[a, b],kλf k∞ = supx∈[a,b] λf (x) = supx∈[a,b] |λ|f (x) = |λ| supx∈[a,b] f (x) =
λkf k∞ , so k·k∞ has absolute homogeneity.
(iii) ∀f, g ∈ B[a, b],kf + gk∞ = supx∈[a,b] f (x) + g(x) = supx∈[a,b] (f (x)+g(x)) ≤ supx∈[a,b] f (x)+
supx∈[a,b] g(x) = kf k∞ + kgk∞ , so kf + gk∞ ≤ kf k∞ + kgk∞ , so k·k∞ satisfies the triangle
equality.
Note:
Norms generalise the notion of magnitude of vectors in Rn .
Proposition 34: Let x = (x1 , . . . , xn ) ∈ Rn (a vector ). Then the following functions on Rn are norms:
1.
kxk1 =
n
X
|xi |
(the taxicab or Manhattan norm)
i=1
2.
v
u n
uX
kxk2 = t
xi 2
(the Euclidean norm or Euclidean distance)
i=1
3.
kxk∞ = max |xi |
1≤i≤n
33
Proof: Triangle inequality:
1.
ku + vk1 =
n
X
|ui + vi | ≤
n
X
i=1
n
X
(|ui | +|vi |) =
i=1
2. For any u, v ∈ Rn ,
u·v =
n
X
|ui | +
i=1
n
X
|vi | = kuk1 +kvk1
i=1
ui · vi = kukn ·kvk2 · cos θ ≤ kuk2 ·kvk2
i=1
for some θ ∈ [0, 2π]. This is known as the Cauchy-Schwarz inequality for Rn . Then
2
ku + vk2 = (u + v) · (u + v)
n
X
=
(ui + vi ) · (ui + vi )
i=1
= u · (u + v) + v · (u + v)
Cauchy
≤
Schwarz
kuk2 ·ku + vk2 +kvk2 ·ku + vk2
So
2
ku + vk2 ≤ kuk2 ·ku + vk2 +kvk2 ·ku + vk2
The case where u + v = 0 is trivial to check, otherwise
ku + vk2 ≤ kuk2 +kvk2
3.
ku + vk∞ = max |ui + vi | ≤ max (|ui | +|vi |) ≤ max |ui | + max |vi | = kuk∞ +kvk∞
1≤i≤n
1≤i≤n
1≤i≤n
The verification of the other properties is left as an exercise.
Remark:
1≤i≤n
All of these norms are instances of the following family of norms:
 p1

n
X
p
kxkp =  |xi |  , p = 1, 2, . . .
i=1
In particular,
kxk∞ = lim kxkp
p→∞
That this family of functions is a family of norms follows from the Minkowski inequality:

 p1

 p1 
 p1
n
n
n
X
X
X
 |xi + yi |p  ≤  |xi |p  +  |yi |p 
i=1
i=1
∀x, y ∈ Rn
i=1
Note : Rn can be thought of as the space of functions on Nn = {1, 2, . . . , n} by considering x : Nn → R,
with x : j 7→ xj .
34
9.2
Equivalence & Banach Spaces
Definition 12: Norms k·ka and k·kb on V are called equivalent (or Lipschitz equivalent) if ∃k1 , k2 ≥ 0
such that, ∀v ∈ V , k1 ·kvkb ≤ kvka ≤ k2 ·kvkb . This equivalence is denoted by k·ka ∼ k·kb .
Remark:
The equivalence of norms is an equivalence relation on the set of all norms on V . That is,
1. k·ka ∼ k·ka (choose k1 = k2 = 1).
2. k·ka ∼ k·kb ⇐⇒ k·kb ∼ k·ka .
3. If k·ka ∼ k·kb and k·kb ∼ k·kc , then k·ka ∼ k·kc .
The proof is left as an exercise.
Lemma 35: k·k1 , k·k2 and k·k∞ are all equivalent on Rn .
Proof: Because equivalence is an equivalence relation, it is enough to prove that k·k1 ∼ k·k∞ and
k·k2 ∼ k·k∞ . Now, ∀x ∈ Rn ,
kxk∞ = max |xi | ≤ kxk1 =
1≤i≤n
n
X
|xi | ≤
i=1
n
X
i=1
max xj = nkxk∞
1≤j≤n
So k·k1 ∼ k·k∞ with k1 = 1, k2 = n.
There is a similar argument for the second part of the proof, although a much more general statement
will be proved shortly, and so this shall be left as an exercise.
Definition 13:
Let (V,k·k) be a normed space. Then
(i) A sequence (σn )n≥1 ∈ V converges to σ ∈ V if limn→∞ σn − σ = 0.
(ii) A sequence (σn )n≥1 ∈ V is Cauchy if, ∀ε > 0, ∃Nε such that, ∀n, m > Nε , σn − σm < ε.
(iii) If every Cauchy sequence in V converges, then (V,k·k) is called Banach (or complete). A complete
normed space is also called a Banach space.
Remarks:
• Any Cauchy sequence on (R,|·|) converges, so (R,|·|) is Banach.
• Consider (S[a, b],k·k∞ ). Let (sn )n≥1 ⊂ S[a, b] be a uniform Cauchy sequence, so (sn ) is Cauchy
with respect to k·k∞ . Then sn → r ∈ R[a, b] as n → ∞, so, in general, (sn ) doesn’t converge to
a “point” in S[a, b] because not all regulated functions are step functions, so (S[a, b],k·k∞ ) is not
Banach.
Note: All finite-dimensional normed spaces are complete. A proof for the case when the space is over
R shall be given later.
Remarks:
1. If limn→∞ σn exists, then it is unique.
σn → a ∈ V and σn → b ∈ V as n → ∞. Then 0 ≤ ka − bk = a − σn + σn − b ≤
Suppose that
a − σn + σn − b → 0 as n → ∞, so ka − bk = 0, so a = b by separation of points.
2. (R[a, b],k·k∞ ) is complete, because a uniformly Cauchy sequence of regulated functions converges to
a regulated function. Also, (C[a, b],k·k∞ ) and (B[a, b],k·k∞ ) are complete by results demonstrated
earlier.
35
3. Suppose that k·ka ∼ k·kb . Then, if limn→∞ σn = σ with respect to k·ka , then limn→∞ σn = σ with
respect to k·kb . The proof is left as an (important) exercise.
4. If k·ka ∼ k·kb , then (V,k·ka ) is Banach iff (V,k·kb ) is Banach. This is essentially a consequence of
the previous remark.
P
5. (V,kP
σn which
·k)∞ is Banach iff any series ∞
n=1
converges absolutely converges to an element in
P∞
converges
absolutely
if
V.
σ
n
n=1 σn converges. The proof of this is left as an exercise,
n=1
although the ⇐= direction is difficult.
Theorem 36:
All norms on Rn are (Lipschitz) equivalent.
Pn
Proof: Let (e1 , . . . , en ) ⊂ Rn be the standard basis of Rn . Then, ∀x ∈ Rn , x = k=1 xk ek . Now,
X
X
n
n
n
n
X
n
X
X
ek ≤
x
e
x
e
=
e
≤
max
x
e
=
kxk
|x
|
kxk = k k
k k
k
j
k
k
∞
1≤j≤n
k=1
k=1
k=1
k=1
k=1
Pn Let k2 = k=1 ek . Then ∃k2 such that, ∀x ∈ Rn , kxk ≤ k2 kxk∞ .
Now, let
kxk
, A ..=
J ..= inf
x6=0 kxk∞
)
kxk
: x 6= 0 , B ..= kxk : kxk∞ = 1
kxk∞
(
Clearly B ⊂ A by definition. If α ∈ A, then ∃u, v ∈ Rn such that
1
kuk
α=
=
· u = kvk
kuk∞ kuk∞ Then
kvk∞
u =
kuk∞ =
∞
1
·kuk∞ = 1
kuk∞
so α ∈ B, so A ⊂ B, so B = A. Then J = inf A = inf B, and clearly J ≥ 0.
Suppose that J = 0. Then ∃x1 , x2 , x3 , . . . ∈ Rn with xk = k1 (so limk→∞ xk = 0) and xk ∞ = 1 ∀k.
Now, it must be established that there is a convergent subsequence xkp
⊂ (xk )k≥1 such that
p≥1 (1)
(2)
(n)
xkp → x = (x(1) , x(2) , . . . , x(n) ) ∈ Rn as p → ∞ with respect to k·k∞ . Let xk = xk , xk , . . . , xk .
(1) (1)
(1)
Since xk ∞ = 1, xk ≤ 1, so xk
is bounded, so by the Bolzano-Weierstrass theorem ∃ xkp
⊂
k≥1
p≥1
(1)
which converges to x(1) ∈ R. Applying this same argument to each component of xk
xk
k≥1
(j)
yields xkp
such that xkp
converges to x(j) ∈ R for any 1 ≤ j ≤ n, so xkp
→ x
p≥1
p≥1 p≥1
(j)
“component-wise” as p → ∞. Since xkp
→ x(j) , then, ∀ε > 0, ∃Nε (j) such that, ∀n > Nε (j),
p≥1
(j)
(j)
xkp − x(j) < ε. Let Nε = max1≤j≤n Nε (j). Then, ∀k > Nε , xkp − x = max1≤j≤n xkp − x(j) < ε,
∞
→ x as p → ∞ with respect to k·k∞ .
so limp→∞ xkp − x = 0, so xkp
∞
p≥1
Now, 1 = xkp = xkp − x + x ≤ xkp − x + kxk∞ , so kxk∞ ≥ 1 − xkp − x > 0 since
∞
∞
∞
∞
xkp − x → 0 as p → ∞, so x 6= 0 by the separation of points property of k·k .
∞
∞
Also, 0 ≤ kxk = x − xkp + xkp ≤ xkp +x − xkp ≤ xkp +k2 x − xkp . (xkp ) ⊂ (xk ), so xkp →
∞
0 as p → ∞, and it was established that x − xkp → 0 as p → ∞, so xkp + k2 x − xkp → 0 as
∞
p → ∞, so kxk = 0, so x = 0 by the separation of points property of k·k.
∞
However, this is a contradiction, so J > 0, so let k1 = J. Then, from the definition of J, k1 kxk∞ ≤ kxk
∀x ∈ Rn \ {0}, and if x = 0, then kxk∞ = kxk = 0, so k1 kxk∞ = kxk, so k1 kxk∞ ≤ kxk ∀x ∈ Rn , so
k1 kxk∞ ≤ kxk ≤ k2 kxk∞ ∀x ∈ Rn , so k·k ∼ k·k∞ on Rn , so all norms are equivalent on Rn . 36
Proposition 37:
The space (Rn ,k·k) is Banach for any norm k·k on Rn .
Proof: It is enough to show that (Rn ,k·k∞ ) is complete, since if k·ka ∼ k·kb , then (V,k·ka ) is Banach
⇐⇒ (V,k·kb ) is Banach, and k·k∞ ∼ k·k for all norms k·k on Rn as shown previously.
(j)
(j) Consider any Cauchy subsequence (xk )k≥1 ⊂ Rn . Then, for any 1 ≤ j ≤ n, 0 ≤ xk+m − xk ≤
(j)
(j) (j)
max1≤j≤n xk+m − xk = xk+m − xk → 0 as k → ∞ since (xk )k≥1 is Cauchy in Rn , so xk
∞
k≥1
(j)
(1)
(2)
(n)
n
is Cauchy in R, so ∃x(j) ∈ R such that xk → x(j) as k → ∞.
Let x= (x , x , . . . , x ) ∈ R . It
was shown in the proof of the previous theorem that limk→∞ xk − x ∞ = 0, so (xk )k≥1 converges in
Rn with respect to k·k∞ , so every Cauchy sequence in Rn converges with respect to k·k∞ , so (Rn ,k·k∞ )
is complete, so (Rn ,k·k) is Banach for any norm k·k on Rn . Corollary: (V,k·k) is Banach for any norm k·k on V , provided dim V is finite. This is because any
n-dimensional vector space V over R is isomorphic to Rn , simply choose a basis in V .
∀f, g ∈ C[a, b],
s
Z b
Z b
f2
f g ≤ kf k2 kgk2 , where kf k2 =
Lemma (Cauchy-Schwarz Inequality for Integrals):
a
a
Proof: ∀λ ∈ R,
λ2
b
Z
g 2 + 2λ
a
Z
b
Z
b
fg +
a
f2 =
a
b
Z
(f + λg)2 ≥ 0
(1)
a
This is a polynomial in λ of degree 2 with at least one real root, so the discriminant
Z
D=4
!2
b
fg
a
2
2
− 4kgk2 kf k2 ≤ 0, so
Z
a
b
f g ≤ kf k2 kgk2
Proposition 38:
1. The following functions are norms on C[a, b]:
(a)
kf k∞ = sup f (x)
(the sup norm)
x∈[a,b]
(b)
b
Z
kf k1 =
|f |
(a generalisation of the taxicab norm)
f2
(a generalisation of the Euclidean norm)
a
(c)
s
Z
kf k2 =
b
a
2. (C[a, b],k·k∞ ) is Banach, but (C[a, b],k·k1,2 ) is not Banach.
3. ∀f ∈ C[0, 1], kf k1 ≤ kf k2 ≤ kf k∞ .
4. No pair of norms (a), (b) or (c) is equivalent.
37
Proof:
1. (a) It was demonstrated that k·k∞ is a norm on B[a, b], and C[a, b] ⊂ B[a, b], so k·k∞ is a norm
on C[a, b].
Rb
(b) Separation of points: Assume that ∃f such
that
kf
k
=
0.
Then
|f | = 0. Assume that
1
a
f 6≡ 0. Then ∃x0 ∈ [a, b] such that f (x0 ) > 0. But f ∈ C[a, b], so ∃[t1 , t2 ] ⊂ [a, b] such that
R
R
> 0. Now, 0 = kf k = b |f | ≥ t2 |f | > 0, so 0 > 0, which is clearly a contradiction,
f 1
t1
a
[t1 ,t2 ]
so f ≡ 0.
Rb
Rb
|λ · f | = a (|λ| ·|f |) = |λ| · a |f | = |λ| ·kf k1
Rb
Rb
Rb
Rb
Triangle inequality: ∀f, g ∈ C[a, b], kf + gk1 = a |f + g| ≤ a (|f | + |g|) = a |f | + a |g| =
kf k1 +kgk1
Absolute homogeneity: ∀λ ∈ R, kλ · f k1 =
Rb
a
(c) The proofs of the first two properties are very similar to those in (b), and so they are left as
exercises.
Triangle inequality: Using the Cauchy-Schwarz inequality
Z b
Z b
Z b
2
g(f + g) ≤ kf k2 kf + gk2 +kgk2 kf + gk2
f (f + g) +
(f + g)2 =
kf + gk2 =
a
a
a
Then, assuming that (f + g) 6≡ 0,
kf + gk2 ≤ kf k2 +kgk2
If f + g ≡ 0 then the proof is trivial.
2. Consider the sequence of functions (fk )k≥1 ⊂ C[a, b] given by

1


a ≤ x < a+b
−1
2 − k
a+b
1
a+b
a+b
fk (x) = k(x − 2 )
2 − k ≤x≤ 2 +


a+b
1
1
2 + k <x≤b
1
k
Then, for any k, m ∈ N,
Z
kfk − fk+m k1 =
b
Z
|fk − fk+m | =
a+b
1
2 +k
a+b
1
2 −k
a
|fk − fk+m |
since the functions are equal outside of these bounds. Now, fk is bounded by [−1, 1], so|fk − fk+m | ≤
2, so
1
Z a+b
2 +k
2
4
kfk − fk+m k1 ≤
2 = 2 · = → 0 as k → ∞
a+b
1
k
k
2 −k
b]. Then, ∀ε > 0,
so(fk )k≥1 is Cauchy withrespect to
k·k1 . Suppose that it has a limit f ∈ C[a,
Rb
a+b
a+b
1
f 2 − ε = −1 and f 2 + ε = 1, since otherwise, for any k > ε , a |fk − f | > 0 which
contradicts f being the limit of (fk )k≥1 . But then f is not continuous, since
−1 = lim f (x) 6= lim f (x) = 1
x↓ a+b
2
x↑ a+b
2
This is a contradiction, so (fk )k≥1 does not have a limit in C[a, b] with respect to k·k1 , but it is
Cauchy with respect to k·k1 , so (C[a, b,k·k1 ) is not Banach.
3. For any f ∈ C[0, 1], and some x0 ∈ [0, 1]
Z 1
Z 1
kf k1 =
|f | ≤
kf k∞ = kf k∞
0
kf k1 =
0
1
Z
|f | · 1 ≤ kf k2 ·k1k2 = kf k2
0
Z
kf k2 =
! 12
1
f
0
2
12
≤ f 2 =
∞
! 21
2
sup f (x)
x∈[0,1]
38
12 = f (x0 )2
= f (x0 ) = kf k∞
4. This is mostly left as an exercise, however a proof that k·k1 6∼ k·k2 will be given. For simplicity,
this shall be proven with respect to C[0, 1], the generalisation is left as an easy exercise.
Let, for any ε > 0,
fε (x) =

 √1
0≤x<ε
ε
 √1x
ε≤x≤1
∈ C[0, 1]
Then
Z
kfε k1 =
2
kfε k2
1
Z
ε
|fε | =
0
Z
=
0
1
fε2
Z
0
ε
=
0
Z 1
√
√ ε
√
1
1
√ dx +
√ dx =
ε+2 x 1 =2− ε≤2
ε
x
ε
Z 1
1
1
1
1
dx +
dx = [1 + log x]ε = 1 + log → ∞ as ε ↓ 0
ε
x
ε
ε
So @k > 0 such that kf k2 ≤ kkf k1 ∀f ∈ C[0, 1] .
Exercise:
Using a proof similar to the one in 3, show that (C[a, b],k·k2 ) is not Banach.
39
10
10.1
Continuous Maps
Definition & Basic Properties of Linear Maps
Definition 14: Let f : (V,k·kV ) → (W,k·kW ) be a map from V to W . Then f is called (k·kV ,k·kW )
continuous (or simply continuous when the spaces in question are clear) at u ∈ V if, ∀ε > 0, ∃δε > 0
such that, ∀v ∈ V , ku − vkV < δε =⇒ f (u) − f (v)W < ε.
Example:
Let I : (C[a, b],k·k∞ ) → (R,|·|) be given by
Z
b
f ∈R
I : f ∈ C[a, b] 7→
a
It was demonstrated in the first section that this map is linear.
Theorem 39:
equivalent:
Let T : (V,k·kV ) → (W,k·kW ) be a linear map. Then the following conditions are
(i) T is continuous at 0V ∈ V .
(ii) T is continuous on V (T is continuous at every point of V ).
o
n
(iii) The set T (x)W : x ∈ V such that kxkV ≤ 1 ⊂ R is bounded (then T is called bounded ).
Proof (i) ⇐⇒ (ii):
The ⇐= direction is clear, if T is continuous on V , it is continuous at 0V ∈ V .
For the =⇒ direction,
suppose that T is continuous at 0V . T (0V ) = 0W , so, ∀ε > 0, ∃δε > 0 such that
kxkV < δε =⇒ T (x)W < ε. Choose some u ∈ V . Then, for any v ∈ V , let x = u − v. Now, ∀ε > 0,
∃δε > 0 such that ku − vkV < δε =⇒ T (u) − T (v)W = T (u − v)W < ε, so T is continuous at u.
Since the choice of u was arbitrary, T is continuous on V .
Proof (i) ⇐⇒ (iii): For the =⇒ direction, by continuity,
∀ε> 0, ∃δε > 0 such that, ∀x ∈ V ,
x ε
kxkV ≤ δ˜ε = δ2ε =⇒ T (x)W < ε. Then δx ≤ 1 =⇒ T δε ≤ δε , so ∀x ∈ V , kxkV ≤ 1 =⇒
ε V
W
n
o
T (x) < ε , so T (x) : x ∈ V such that kxk ≤ 1 is bounded by ε , so it is bounded.
˜
V
W
W
δε
δ˜ε
For the ⇐= direction, T is bounded, so ∃B > 0 such that ∀u, kukV ≤ 1 =⇒ T (u)W < B. Then,
εu ε
for any ε > 0, multiplying this inequality by Bε yields εu
< ε, so ∀v ∈ V ,
B V ≤ B =⇒ T B W
ε
ε
=⇒ T (v) < ε, so T is continuous at 0V with δε = . kvk < δε =
V
Example:
B
B
W
For any f ∈ C[a, b], kf k∞ ≤ 1,
Z
a
b
≤ kf k∞ ·|b − a| ≤ |b − a|
so I : C[a, b] → R,
Z
I : f ∈ C[a, b] 7→
f ∈R
a
is bounded, so I is continuous on (C[a, b],k·k∞ ).
40
b
Proposition 40: Let V and W be vector spaces, with norms k·kV1 ∼ k·kV2 and k·kW1 ∼ k·kW2
respectively, and let T : V → W be a linear map. Then
(i) T is (k·kV1 ,k·kW1 ) continuous iff T is (k·kV2 ,k·kW2 ) continuous.
(ii) Any linear map (Rn ,k·k1 ) → (W,k·kW ) is continuous.
(iii) Any linear map (Rn ,k·k) → (W,k·kW ) is continuous, where k·k is an arbitrary norm.
(iv) Let F : (C[0, 1],k·k1,∞ ) → (R,|·|) be an evaluation map with F : f 7→ f (0) ∈ R. Then F is
(k·k∞ ,|·|) continuous but not (k·k1 ,|·|) continuous.
Proof:
(i) T is (k·kV1 ,k·kW1 ) continuous, so T is bounded with respect to (k·kV1 ,k·kW1 ), so ∃B ≥ 0 such that
∀u ∈ V , kukV1 ≤ 1 =⇒ T (u)W1 ≤ B. Since k·kV1 ∼ k·kV2 , ∃K > 0 such that, ∀u ∈ V , kukV1 ≤
KkukV2 . Consider all u ∈ V such that KkukV2 ≤ 1. Then kukV1 ≤ 1, so T (u)W1 ≤ B. Now,
k·kW1 ∼ k·kW2 , so ∃L > 0 such that, ∀v ∈ W , LkvkW2 ≤ kvkW1 , so LT (u)W2 ≤ T (u)W1 ≤ B.
Then, KkukV2 ≤ 1 =⇒ LT (u)W2 ≤ B. Let w = Ku. Then, ∀w ∈ V , kwkV2 ≤ 1 =⇒
0
T (w)
≤ KB
L = B , so T is bounded with respect to (k·kV2 ,k·kW2 ), so T is (k·kV2 ,k·kW2 )
W2
continuous.
(ii) For any x ∈ Rn , ∃x1 , . . . , xn ∈ R such that, for some basis e1 , . . . , en ∈ Rn ,
x=
n
X
xi ei , so kxk1 =
i=1
n
X
|xi |
i=1
Now, consider any x ∈ Rn with kxk1 ≤ 1. Then


n
n
X
X
T (x) = T 

xi T (ei )
xi ei = W
i=1
i=1
W
≤
n
X
W
n
X
|xi | ≤ max T (ei ) ..= B
|xi |T (ei )W ≤ max T (ei )
1≤i≤n
i=1
i=1
1≤i≤n
So ∃B > 0 such that, ∀x ∈ Rn with kxk1 ≤ 1, T (x)W ≤ B, so T is bounded, so T is continuous.
(iii) This is a direct consequence of (i) and (ii), since all norms on Rn are equivalent.
(iv) For F : (C[0, 1],k·k∞ ) → (R,|·|), take any f ∈ C[0, 1] such that kf k∞ ≤ 1. Then f (0) ≤ 1, so
F (f ) ≤ 1, so F is bounded by 1, so F is continuous.
For F : (C[0, 1],k·k1 ) → (R,|·|), let (fn )n≥1 ⊂ C[a, b] be the sequence given by fn : [0, 1] → R, fn :
x 7→ ( n1 − n)x + n. Then
1
1
1
1
− n x + n dx = ≤ 1 ∀n ∈ N
n
n
0
0
n
o
However, F (fn ) = fn (0) = n → ∞ as n → ∞, so F (f ) : kf k1 ≤ 1 is not bounded, so F is
not continuous. Z
kfn k1 =
Z
fn =
41
10.2
Spaces of Bounded Linear Maps
Theorem 41:
Suppose (V,k·kV ) and (W,k·kW ) are normed spaces, let
L(V, W ) = {T : V → W : T is linear and continuous/bounded}
and let
k·k : T ∈ L(V, W ) 7→
sup
T (x)
W
x∈V :kxkV ≤1
Then (L(V, W ),k·k) is a normed vector space.
Proof: For any λ, µ ∈ R, T, G ∈ L(V, W ), x ∈ V , (λT + µG)(x) = λT (x) + µG(x) - a map from V
to W . Take any α, β ∈ R, u, v ∈ V . Then (λT + µG)(αu + βv) = λT (αu + βv) + µG(αu + βv) =
so (λT + µG)
λ(αT (u) + βT (v)) + µ(αG(u) + βG(v)) = α(λT
+ µG)(v),
is linear.
+ µG)(u) + β(λT
Now, kλT + µGk = supkxkV ≤1 λT (x) + µG(x)W ≤ |λ| supkxkV ≤1 T (x)W + |µ| supkxkV ≤1 G(x)W =
|λ|kT k +|µ|kGk. Then, since T and G are both bounded, λT + µG is bounded, so λT + µG is continuous,
so L(V, W ) is closed under linear combinations of maps. Verification of the remaining axioms of a vector
space is left as an exercise.
It remains to show that k·k is a norm. Taking λ = µ = 1 in the previous section of the proof yields
kT + Gk ≤ kT k+kGk which proves the triangle inequality. The proof of absolute homogeneityis leftas an
easy exercise. To prove separation of points, suppose that kT k = 0L(V,W ) . Then supkukV ≤1 T (u)W =
0L(V,W ) , so T (u)W = 0L(V,W ) ∀u ∈ V such that kukV ≤ 1. Since k·kW is a norm, T (x) = 0L(V,W ) ∀u ∈
v
V such that kukV ≤ 1. Now, take any v ∈ V such that kvkV > 1. Then T (v) W = kvkV · T kvk V
= 0L(V,W ) , so T (x) = 0L(V,W ) ∀x ∈ V , so T = 0L(V,W ) , which completes the proof.
W
Remarks:
1. The norm k·k is called an operator norm.
2. Consider L(Rm , Rn ) - this is the space Rm×n of all m × n matrices.
3. Suppose then that m =√n and that both of these Rn s are equipped with the k·k2 norm. Then for
any T ∈ Rn×n , kT k = λmax , where λmax is the maximum eigenvalue of T T 0 , with T 0 being the
transpose of T .
4. If (W,k·kW ) is Banach, then (L(V, W ),k·k) is Banach also.
Exercise:
Prove 4 and that kT k =
√
λmax as given in 3 is a norm.
42
11
Open and Closed Sets of Normed Spaces
11.1
Definition & Basic Properties of Open Sets
Definition 15:
Let (V,k·k) be a normed space, and let u ∈ V , δ > 0. Then the set
B(u, δ) = {v ∈ V : kv − uk < δ} ⊂ (V,k·k)
is called an open ball of radius δ centred at u. B(u, δ,k·k) denotes an open ball defined with respect to
k·k, B(u, δ) is shorthand when the norm in question is obvious.
Examples:
1. For (R,|·|), B(x, δ) = (x − δ, x + δ).
2. For (R2 ,k·k1 ), B1 (0, 1) = {(x1 , x2 ∈ R2 : x1 + x2 < 1)}.
p
For (R2 ,k·k2 ), B2 (0, 1) = {(x1 , x2 ∈ R2 : x1 2 + x2 2 < 1)}.
For (R2 ,k·k∞ ), B∞ (0, 1) = {(x1 , x2 ∈ R2 : max(x1 , x2 ) < 1)}.
3. For (C[a, b],k·k∞ ), B(g, ε) = {f ∈ C[a, b] : supx∈[a,b] f (x) − g(x) < ε}
Definition 15.1: Let (V,k·k) be a normed space. A subset U ⊂ V is called open if, ∀x ∈ V , ∃δx such
that B(x, δx ) ⊂ U .
Exercise:
Prove that the empty set is open.
Lemma 42:
Let (V,k·k) be a normed space. Then
(i) Open balls in V are open sets.
(ii) ∀u ∈ V, δ > 0, B(u, δ) = u + δB(0, 1) ..= {v ∈ V : v = u + δw, w ∈ B(0, 1)}.
Proof:
(i) v ∈ B(u, δ) ⇐⇒ kv − uk < δ. Let δ 0 = δ − kv − uk > 0. Consider B(v, δ 0 ). Then, for any
w ∈ B(v, δ 0 ), kw − vk < δ 0 . Then ku − wk = ku − v + v − wk ≤ ku − vk +kv − wk < ku − vk + δ 0 =
ku − vk + δ −kv − uk = δ, so B(v, δ 0 ) ⊂ B(u, δ). Then, since the choice of v was arbitrary, B(u, δ)
is open.
u−v
(ii) Take v ∈ B(u, δ), then ku − vk < δ, so u−v
δ < 1. Let w = δ . Then w ∈ B(0, 1), so v = u + δw
with w ∈ B(0, 1), so v ∈ u + δB(0, 1), so B(u, δ) ⊆ u + δB(0, 1).
Now, take any v ∈ u + δB(0, 1). Then ∃w ∈ B(0, 1) such that v = u + δw, so ku − vk = kδwk =
δkwk < δ, so v ∈ B(u, δ). Then u + δB(0, 1) ⊆ B(u, δ), so B(u, δ) = u + δB(0, 1). Lemma 43: Let k·kA ∼ k·kB be equivalent norms on V . Then U ⊂ V is open with respect to k·kA iff
U ⊂ V is open with respect to k·kB .
Proof: Assume without loss of generality that V is open with respect to k·kA . Then, ∀u ∈ U , ∃δu > 0
such that B(u, δu ,k·kA ) ⊂ U . Now, consider some δu0 > 0. Then v ∈ B(u, δu0 ,k·kB ) ⇐⇒ kv − ukB < δu0 .
Sincek·kA ∼ k·kB , ∃K > 0 such thatkv − ukA < Kkv − ukB < Kδu0 . Let δu0 =
B(u, δu ,k·kA ), so V is open with respect to k·kB . δu
K.
Then B(u, δu0 ,k·kB ) ⊂
Example: If v ∈ Rn is open with respect to k·k1 , then V is open with respect to an arbitrary norm
k·k on Rn since all norms on Rn are equivalent.
43
11.2
Continuity, Closed Sets & Convergence
Definition 15.2: A function f : (V,k·kV ) → (W,k·kW ) is continuous at x ∈ V if, ∀ε > 0, ∃δ > 0 such
that f (B(x, δ,k·kV )) ⊂ B(f (x), ,k·kW ).
Exercise:
Prove that this definition is equivalent to the previous definition of continuity.
Definition 15.3: A set U ⊂ (V,k·k) is bounded if it is contained in a ball of finite radius. Equivalently,
U ⊂ V is bounded if ∃x ∈ V, δ > 0 such that U ⊂ B(x, δ).
Proposition 44: Let f : (V,k·kV ) → (W,k·kW ). Then f is continuous on V iff the preimage of any
open subset U ⊂ W is open. Equivalently, f : V → W is continuous iff, ∀U ⊂ W such that U is open,
f −1 (U ) ..= {x ∈ V : f (x) ∈ U } is open.
Proof =⇒ : Let U ⊂ W be an open subset. Then, either f −1 (U ) = ∅ which is open, or f −1 (U ) 6= ∅.
If f −1 (U ) 6= ∅, then let u ∈ f −1 (U ) ⊂ V be any point in the preimage. Let v = f (u) ∈ U . Then ∃ε > 0
such that B(v, ε,k·kW ) ⊂ U . But f is continuous on V , so f is continuous at u, so ∃δε > 0 such that
f (B(u, δε ,k·kV )) ⊂ B(v, ε,k·kW ) ⊂ U , so B(u, δε ,k·kV ) ⊂ f −1 (U ), so f −1 (U ) is open.
Proof ⇐= : Take any u ∈ V , and let v = f (u). For any ε > 0, consider B(v, ε,k·kW ). f −1 (B(v, ε,k·kW ))
is open, and u ∈ f −1 (B(v, ε,k·kW )). Since f −1 (B(v, ε,k·kW )) is open, ∃δε such that B(u, δε ,k·kV ) ⊂
f −1 (B(v, ε,k·kW )), so f (B(u, δε ,k·kV )) ⊂ (B(v, ε,k·kW ), so f is continuous on V . Definition 16:
A set U ⊂ (V,k·k) is called closed if V \ U is open.
Examples:
(i) [a, b] ⊂ R is closed since R \ [a, b] = (−∞, a) ∪ (b, ∞) is open.
(ii) [a, b) ⊂ R is neither open nor closed (it is not open since B(a, ε) 6⊂ [a, b) for any ε > 0, and it is
not closed since R \ [a, b) = (−∞, a) ∪ [b, ∞) is not open).
Theorem 45: Let (V,k·k) be a normed space. Then U ⊂ V is closed iff, for any sequence (xn ) ⊂ U
which converges to x ∈ V , x ∈ U .
Proof =⇒ : Let U ⊂ V be a closed subset. Assume that U 6= ∅, the case when U= ∅ is trivial.
Then let (xn ) ⊂ U be a convergent sequence, with x = limn→∞ xn , so limn→∞ x − xn = 0. Suppose
that x ∈
/ U , then x ∈ V \ U . Since U is closed, V \ U is open,
so ∃δ> 0 such that B(x, δ) ⊂ V \ U .
However, since xn → x as n → ∞, ∃Nδ such that, ∀n > Nδ , x − xn < δ, so xn ∈ B(x, δ) ⊂ V \ U ,
which is a contradiction since xn ∈ U , so x ∈ U .
Proof ⇐= : Now, any sequence in U which converges does so to a point in U . Suppose that U is not
closed. Then V \ U is not open, so ∃x ∈ V \ U such that, ∀ε > 0, B(x, ε) 6⊂ V \ U , so B(x, ε)
∩ U 6= ∅. Let
x − xn < εn ,
(εn ) be anynull sequence.
Then,
for
any
n
∈
N,
B(x,
ε
)
∩
U
=
6
∅,
so
∃x
∈
U
such
that
n
n
so limn→∞ x − xn = 0. Then (xn ) ⊂ U is a sequence which converges to a point x ∈
/ U , which is a
contradiction, so U is closed. Notes: Consider the normed space (V,k·k). For any x ∈ V and any δ > 0, B(x, δ) ⊂ V , so V is open.
However, any sequence in V which converges does so in V , so V is closed also. ∅ is also both closed and
open, since ∅ = V \ V . These sets are called clopen.
44
11.3
Lipschitz Continuity, Contraction Mappings & Other Remarks
Remarks:
Let (V,k·k) be a normed space. Then
1. The map k·k : (V,k·k) → (R,|·|) is continuous. To see this, fix any u ∈ V . Then,
for any
v ∈ V,
kuk −kvk ≤ ku − vk by the triangle inequality. Now, if ku − vk < ε, then kuk −kvk < ε, so
B(u, ε,k·k) ⊂ B kuk , ε,|·| , so setting δε = ε demonstrates that k·k : V → R is continuous.
2. Suppose that, for a map f : (V,k·k) → (V,k·k), ∃K ∈ (0, 1) such that, ∀u, v ∈ V , f (u) − f (v) ≤
ε
Kku − vk. Then f is continuous on V . To see this, note that f (B(v, 2K
)) ⊂ B(f (v), ε), the details
of the proof are left as an exercise. Such a map is called a contraction mapping. In fact, if this
condition holds for any k > 0, continuity is preserved, and the map is called Lipschitz continuous.
3. A closed ball of radius δ centred at u is defined as B(u, δ) ..= {v ∈ V : ku − vk ≤ δ} ⊂ (V,k·k). This
set is closed. To see this, let (un )n≥1 ⊂ B(u, δ)
un → w ∈ V as n → ∞. Then, by the
such that
continuity of k·k at u ∈ V , kw − uk = limn→∞ un − u ≤ δ, so w ∈ B(u, δ), so B(u, δ) is closed.
4. Let U ⊂ (V,k·k). Then w ∈ V is a boundary point of U if, ∀ε > 0, B(w, ε) ∩ U 6= ∅ and
B(w, ε) ∩ V \ U 6= ∅. w ∈ V is a limit point of U if ∃(xn )n≥1 ⊂ U which converges to w.
45
12
12.1
The Contraction Mapping Theorem & Applications
Statement and Proof
Theorem 46 - The Contraction Mapping/Banach Fixed Point Theorem: Let (V,k·k) be a
Banach space, U ⊂ V a closed subset of this space, and f : U → U a contraction mapping. Then
∃!w ∈ U such that f (w) = w. Moreover, for any u ∈ U , the sequence (un )n≥0 ⊂ U converges to w as
n → ∞, where u0 = u and un+1 = f (un ).
Proof:
mapping, so
v ∈ U , f (u)
f is a contraction
∃K ∈ (0, 1) such that, ∀u,
− f (v) ≤ Kku − vk.
Then un+1 − un = f (un ) − f (un−1 ) ≤ K un − un−1 ≤ K 2 un−1 − un−2 ≤ . . . ≤ K n u1 − u0 .
Now,
un+m − un = un+m − un+m−1 + un+m−1 − un+m−2 + un+m−2 − . . . + un+1 − un ≤ un+m − un+m−1 + . . . + un+1 − un ≤ (K n+m−1 + K n+m−2 + . . . + K n )u1 − u0 ∞
X
≤
K n+j u1 − u0 j=0
=
Kn u1 − u0 → 0 as n → ∞,
1−K
so (un )n≥0 is Cauchy. Since (V,k·k) is Banach and U is closed, (un ) → w ∈ U as n → ∞. Now,
consider un+1 = f (un ). Then limn→∞ un+1 = limn→∞ f (un ). limn→∞ un+1 = w, and limn→∞ f (un ) =
f (limn→∞ un ) = f (w) since f is a contraction mapping and therefore Lipschitz continuous. Then
w = f (w), so w is a fixed point of f .
It remains toestablish the uniqueness of w. Suppose that ∃v ∈ U such that v = f (v). Then kw − vk =
f (w) − f (v) ≤ Kkw − vk, so (1 − K)kw − vk ≤ 0. Now, (1 − K) > 0 and kw − vk ≥ 0, so kw − vk = 0,
so w = v since k·k separates points. Example: Let α ∈ (0, 1). Then, using the Contraction Mapping Theorem, it can be shown that the
equation x = e−αx , x ∈ [0, 1], has a unique solution x∗ ∈ (0, 1).
Now, (R,|·|) is Banach, and U = [0, 1] is closed. It remains to show
that
f : [0, 1] → [0,
1], f : x ∈ [0, 1] 7→
e−αx , is a contraction mapping. ∀x, y ∈ [0, 1], f (x) − f (y) = f 0 (ξ)(x − y) = −αe−αξ (x − y) ≤
α|x − y| for some ξ ∈ (x, y) by the Mean Value Theorem, which can be applied since f is continuously
differentiable. Then, since α ∈ (0, 1), f is a contraction mapping, so the Contraction Mapping Theorem
can be applied, so ∃!x∗ ∈ (0, 1) such that x∗ = f (x∗ ). Then x∗ solves the equation x = e−αx .
However, the Contraction Mapping Theorem not only guarantees the existence of x∗ , it provides a
method for constructing it, or at least an approximation to it. Let α = 21 and take x0 = 21 . Then
1
2
1
x1 = e− 4 ≈ 0.78
x0 =
x2 ≈ 0.68
x3 ≈ 0.71
x4 ≈ 0.70
x5 ≈ 0.70
It happens that x∞ = x∗ = W ( 21 ), where W is Lambert’s W -function, with W ( 12 ) ≈ 0.70.
46
12.2
The Existence and Uniqueness of Solutions to ODEs
Definition:
An ordinary differential equation (ODE) is of the form
dy
(t) = ẏ(t) = F (t, y(t)), t ∈ A ⊂ R
dt
y(t0 ) = y0 , t0 ∈ A
with y(t) the unknown to be solved for subject to the conditions stated. The second equality is known
as the intial condition(s).
Remark:
Applying
Rt
t0
to the first equality in the ODE yields
Z
t
Z
t
ẏ = y(t) − y(t0 ) =
t0
F (τ, y(τ )) dτ
t0
so
t
Z
F (τ, y(τ )) dτ
y(t) = y(t0 ) +
t0
Suppose that F : R2 → R is continuous, and that y ∈ C[t0 − δ, t0 + δ] for some δ > 0. Then y(t0 ) +
Rt
F (τ, y(τ )) dτ is well-defined and differentiable, and so y(t) solves the ODE.
t0
Definition:
The operator
T (f ) ..= y0 +
Z
t
F (τ, f (τ )) dτ
t0
acting from C[t0 − δ, t0 + δ] to C[t0 − δ, t0 + δ] is called the Pickard operator. An operator is simply a
mapping from one vector space to another, and in many contexts they are linear, however the Pickard
operator is not.
Remark:
A fixed point of T occurs when f = T (f ), so
Z t
f (t) = y0 +
F (τ, f (τ )) dτ
t0
so f solves the ODE.
Theorem 47 - The Picard-Lindelöf Theorem: Suppose that F ∈ C[t0 − a, t0 + a] × [y0 − ε, y0 + ε]
for some a, ε > 0, and that F is Lipschitz continuous
on D = [t0 − a,t0 + a]
× [y0 − ε, y0 + ε] with respect
to y, that is, ∃L > 0 such that, ∀(t, y), (t, y 0 ) ∈ D, F (t, y) − F (t, y 0 ) ≤ Ly − y 0 . Then ∃δ ∈ (0, a] such
that the ODE as given in the definition has a unique solution in C[t0 − δ, t0 + δ].
Remark:
Suppose that
∂F
∈ C(D)
∂y
Then, ∀(t, y)(t, y 0 ) ∈ D and for some ξ(t, y, y 0 ) ∈ D,
∂F
∂F 0 0 F (t, y) − F (t, y 0 ) MVT
y − y 0 = (ξ(t, y, y )) y − y ≤ ∂y
∂y ∞
so F is Lipschitz continuous with respect to y, with
∂F L =
∂y ∞
47
Proof: The aim of the proof is to find some δ > 0 such that T acts on C[t0 − δ, t0 + δ] and is a
contraction mapping for some closed U ⊂ C[t0 − δ, t0 + δ].
For any δ > 0, T : C[t0 − δ, t0 + δ] → C[t0 − δ, t0 + δ]. Let y0 represent the constant function of value y0 ,
and consider B(y0 , ε) = {f ∈ C[t0 − δ, t0 + δ] : kf − y0 k∞ ≤ ε} ⊂ C[t0 − δ, t0 + δ]. For any f ∈ B(y0 , ε),
Z t
T (f ) − y0 = y0 +
≤ sup |F | ·|t − t0 | ≤ sup |F | · δ
F
(τ,
f
(τ
))
dτ
−
y
0
∞
(τ,y)∈D
(τ,y)∈D
t0
∞
So choose δ > 0 sufficiently small such that
sup |F | · δ ≤ ε
(τ,y)∈D
This can be done since
sup |F | < ∞
(τ,y)∈D
because F is continuous on D and D is bounded. Then T : B(y0 , ε) → B(y0 , ε).
Now, ∀f, g ∈ B(y0 , ε),
Z
t
T (f ) − T (g) = [F
(τ,
f
(τ
))
−
F
(τ,
g(τ
))]
dτ
∞
t0
∞
Z
t
≤ L f (τ ) − g(τ ) dτ t0
∞
≤ Lkf − gk∞ kt − t0 k∞
≤ Lδkf − gk∞
So choose δ > 0 sufficiently small such that Lδ < 1 also. Then
1. (C[t0 − δ, t0 + δ],k·k∞ ) is Banach.
2. B(y0 , ε) ⊂ C[t0 − δ, t0 + δ] is closed.
3. T : B(y0 , ε) → B(y0 , ε) is a contraction mapping.
Then, by the Contraction Mapping Theorem, T has a unique fixed point y ∈ C[t0 − δ, t0 + δ] such that
Z
t
y(t) = y0 +
F (τ, y(τ )) dτ
t0
which solves the ODE as given in the definition.
Note: Because of the Contraction Mapping Theorem, it is possible to construct this solution iteratively.
This process is known as Picard’s iteration. The iteration is given by y (n+1) (t) = T (y (n) (t)).
Example:
Consider the ODE
ẏ(t) =
p
y(t)
y(0) = 0
2
Then y1 ≡ 0 is a solution. However, y2 (t) = t4 is also a solution. This can occur because F is not
√
Lipschitz in this case, y is much greater than y close to 0, so there is no L which bounds the difference
between these functions.
48
Example:
Consider the ODE
ẏ(t) = 2ty(t)
y(0) = 1
with D = [−1, 1] × [0, 2]. Then
F (t, y) = 2ty
(t0 , y0 ) = (0, 1)
This satisfies the conditions of the Picard-Lindelöf Theorem, so a unique solution exists, and can be
constructed iteratively using Picard’s iteration, in this case
Z t
y (n+1) = T (y (n) ) = y0 +
2τ y (n) (τ ) dτ
0
Now, taking y (0) (t) = y0 = 1 the initial approximation to the solution,
y (0) = 1
y
Z
(1)
t
=1+
2τ y
(0)
Z
2τ dτ = 1 + t2
(τ ) dτ = 1 +
0
y (2) = 1 +
t
Z
0
t
2τ y (1) (τ ) dτ = 1 +
t
Z
0
2τ (1 + τ 2 ) dτ = 1 +
Z
0
t
2τ + 2τ 3 dτ = 1 + t2 +
0
t4
2
..
.
Using an inductive argument, it can be shown that


n
2n
2k
4
X
t
t
t
=

y (n) (t) = 1 + t2 + + . . . +
2
n!
k!
k=0
since y (0) is of this form, and, assuming the statement holds for y (n) ,
Z t
y (n+1) (t) = 1 +
2τ y (n) (τ ) dτ
0
Z
t
=1+
2τ
0
Z
=1+
τ 2n
τ4
+ ... +
1+τ +
2
n!
t
2τ + 2τ 3 + τ 5 + . . . +
0
= 1 + t2 +
!
2
dτ
2τ 2n
dτ
n!
t4
t2n
t2(n+1)
+ ... +
+
2
n!
(n + 1)!
which is of the correct form. Then, with respect to k·k∞ ,
y(t) = lim y (n) (t) =
n→∞
∞ 2k
X
t
k=0
k!
= et
2
which satisfies the ODE.
Exercise:
Check the existence and uniqueness of a solution to this ODE on C[−δ, δ], where δ ∈ (0, 14 ).
49
12.3
The Jacobi Algorithm
Motivation: Let A ∈ Rn×n be an n × n real-valued matrix, and let b ∈ Rn be a real-valued ndimensional vector. Then, if det A 6= 0, the equation Ax = b has the unique solution x = A−1 b for some
x ∈ Rn . However, inverting A is computationally expensive, typically O(n3 ). Whilst not a problem for
small n, if, for example, n = 108 , this quickly becomes impractical to compute.
However, if A = D + R for some D, R ∈ Rn×n , where D is diagonal and non-degenerate (det D 6= 0) and
R is “small”, so


 
0 r12 . . .
...
r1n
d11 0 . . . . . .
0

 0 d
0 r23
...
r2n 
0 ...
0 


  r21
22

 .

..   ..
.. 
.. ..
..
..
 .

.
.
.
.
r32
0
. + .
. 
A= .
 .



..
..
.. ..
..
..
 .

  .
.
.
.
.
.
.
0   ..
rn−1n 
 .
0
0 . . . 0 dnn
rn1 rn2 . . . rnn−1
0
then Dx + Rx = b, so Dx = b − Rx. If R is “small”, then Dx ≈ b. Let x(n) ⊂ Rn be a sequence of
approximations to x. Then it seems reasonable to set x(0) = D−1 b, and let
x(1)= D−1 (b − Rx(0) )
x(2)= D−1 (b − Rx(1) )
..
.
x(n+1)= D−1 (b − Rx(n) )
This is called the Jacobi iteration. Using the Contraction Mapping Theorem, under certain conditions
it can be shown that this sequence converges to x. Since diagonal inversion is only of order O(n), this
is very fast to compute, so these approximations are extremely useful in situations where inverting A is
not practical. Additionally, the sequence converges to x exponentially.
Proposition:
Suppose that
−1
D Rw
−1 ..
2
=λ<1
D R = sup
kwk2
w6=0
Then the Jacobi iteration as defined above converges to x, the solution to Ax = b.
Proof: Let T : Rn → Rn , T : x 7→ D−1 (b − Rx). T acts on (Rn ,k·k2 ), which is a Banach space, and
Rn ⊆ Rn is closed. Now, ∀u, v ∈ Rn ,
kT u − T vk2 = (D−1 b − D−1 Ru) − (D−1 b − D−1 Rv) = D−1 R(u − v)
2
2
−1 ≤ D Rku − vk2 = λku − vk2
Then, since λ < 1, T is a contraction mapping on Rn , so, by the Contraction Mapping Theorem, T has
a unique fixed point h ∈ Rn . h = T (h) = D−1 (b − Rh), so Dh = b − Rh, so (D + R)(h) = b, so Ah = b,
so the Jacobi iteration constructs a solution to Ax = b. Moreover
1 (n) (n−1) (n−1) n
(0) −n log λ
−1 −
x
≤
.
.
.
≤
λ
−
x
=
e
−
D
b
≤
λ
h − x = T (h) − T x
h
h
h
2
2
2
2
2
so the sequence converges exponentially, and the number of iterations needed does not depend on the
dimension of Rn . 50
12.4
The Newton-Raphson Algorithm
Motivation: For some continuously differentiable f : R → R, finding an x ∈ R satisfying the equation
f (x) = 0 is non-trivial, and a closed-form expression for x cannot always be found, quintic polynomials are
a simple example of this. It would be useful then to construct a series approximating x, and converging
to this solution.
Initially, choosing some x(0) “close” to the root, then taking the Taylor expansion about this point yields
g(x) ≈ g(x(0) ) + g 0 (x(0) )(x − x(0) ). Now, for some x(1) ∈ R, 0 ≈ g(x(1) ) = g(x(0) ) + g 0 (x(0) )(x(1) − x(0) ),
so it seems reasonable to set
g(x(0) )
x(1) − x(0) = − 0 (0)
g (x )
then
x(1) = x(0) −
g(x(0) )
g 0 (x(0) )
Now, let
x(n+1) = N (x(n) ) ..= x(n) −
g(x(n) )
g 0 (x(n) )
This is called the Newton-Raphson method. N : (R,|·|) → (R,|·|), and is called the Newton-Raphson
operator. (x(n) )n≥0 ⊂ R is a sequence of approximations to x, and, using the Contraction Mapping
Theorem, under certain conditions it can be shown that this sequence converges to x.
Note: This sequence does not always converge, especially if the initial guess is poor. For example, if
f is concave on the right and convex on the left, then the sequence can diverge if the initial guess is
particularly bad.
Proposition:
For any continuously differentiable f : R → R, if
f · f 00 (f 0 )2 = λ < 1
∞
then the Newton-Raphson method as defined above converges to x, the solution to f (x) = 0.
Proof: ∀x, y ∈ R and for some ξ ∈ (x, y),
f (ξ) · f 00 (ξ) 0
00
MVT 0
f
(ξ)
f
(ξ)
·
f
(ξ)
N (x) − N (y) = N (ξ)|x − y| = 1 −
|x − y|
+
|x − y| = f 0 (ξ)
f 0 (ξ)2
f 0 (ξ)2 f · f 00 ≤
(f 0 )2 |x − y| = λ|x − y|
∞
Then, since λ < 1, N is a contraction mapping on R, so, by the Contraction Mapping Theorem, N has
a unique fixed point h ∈ R.
f (h)
h = N (h) = h − 0
f (h)
so
f (h)
h−h=− 0
f (h)
so f (h) = 0. 51