Download Fixed-Point Iteration - MATH 375 Numerical Analysis

Fixed-Point Iteration
MATH 375 Numerical Analysis
J. Robert Buchanan
Department of Mathematics
Fall 2013
J. Robert Buchanan
Fixed-Point Iteration
Motivation
Several root-finding algorithms operate by repeatedly
evaluating a function until its value does not change (or
changes by a suitably small amount).
J. Robert Buchanan
Fixed-Point Iteration
Motivation
Several root-finding algorithms operate by repeatedly
evaluating a function until its value does not change (or
changes by a suitably small amount).
Definition
A fixed-point for a function f is a number p such that f (p) = p.
J. Robert Buchanan
Fixed-Point Iteration
Equivalence
The process of root-finding and the process of finding fixed
points are equivalent in the following sense.
Suppose g(x) is a function with a root at x = p, then
f (x) = g(x) + x has a fixed point at x = p.
Suppose f (x) is a function with a fixed point at x = p, then
g(x) = x − f (x) has a root at x = p.
J. Robert Buchanan
Fixed-Point Iteration
Examples
Find the fixed points (if any) of the following functions.
f (x) = x
f (x) = 2x
f (x) = x − sin πx
J. Robert Buchanan
Fixed-Point Iteration
Graphical Interpretation
y
2
1
x
-2
1
-1
-1
-2
J. Robert Buchanan
Fixed-Point Iteration
2
Existence and Uniqueness
Theorem (Brouwer)
Suppose g ∈ C[a, b] and g(x) ∈ [a, b] for all x ∈ [a, b]. Then
1
2
g has a fixed point in [a, b] (existence), and
if, in addition, g 0 (x) exists on (a, b) and if there exists a
constant 0 < k < 1 such that
|g 0 (x)| ≤ k ,
for all x ∈ (a, b),
then there is exactly one fixed point in [a, b] (uniqueness).
J. Robert Buchanan
Fixed-Point Iteration
Graphical Interpretation
y
b
p=gHpL
a
a
x
p
b
Note: g(a) ≥ a and g(b) ≤ b. If g(a) = a or if g(b) = b then g
has a fixed point in [a, b].
J. Robert Buchanan
Fixed-Point Iteration
Proof (1 of 2)
Suppose g(a) > a and g(b) < b.
Define h(x) = g(x) − x. The function h ∈ C[a, b].
h(b) < 0 < h(a) so by the Intermediate Value Theorem
there exists p ∈ (a, b) such that h(p) = 0
0 = h(p)
= g(p) − p
g(p) = p
J. Robert Buchanan
Fixed-Point Iteration
Proof (2 of 2)
Now suppose g 0 (x) exists on (a, b) and there exists a constant
0 < k < 1 such that |g 0 (x)| ≤ k for all x ∈ (a, b).
J. Robert Buchanan
Fixed-Point Iteration
Proof (2 of 2)
Now suppose g 0 (x) exists on (a, b) and there exists a constant
0 < k < 1 such that |g 0 (x)| ≤ k for all x ∈ (a, b).
For the purposes of contradiction suppose g has two fixed
points p and q with p 6= q.
J. Robert Buchanan
Fixed-Point Iteration
Proof (2 of 2)
Now suppose g 0 (x) exists on (a, b) and there exists a constant
0 < k < 1 such that |g 0 (x)| ≤ k for all x ∈ (a, b).
For the purposes of contradiction suppose g has two fixed
points p and q with p 6= q.
According to the Mean Value Theorem there exists a < c < b
for which
g(p) − g(q) p − q
0
=
p−q
p − q = 1 = |g (c)|.
This contradicts the assumption that |g 0 (x)| ≤ k < 1 for all
x ∈ (a, b).
J. Robert Buchanan
Fixed-Point Iteration
Example
Show that g(x) = 2−x has a unique fixed point on [0, 1].
J. Robert Buchanan
Fixed-Point Iteration
Example
Show that g(x) = 2−x has a unique fixed point on [0, 1].
g ∈ C[0, 1] and g 0 (x) = −(ln 2)2−x < 0 which implies g is
monotone decreasing.
g(0) = 1 and g(1) = 1/2 > 0. Since g is decreasing then
g(x) ∈ [0, 1] for all x ∈ [0, 1].
By the first conclusion of the Brouwer Theorem, g has a
fixed point in [0, 1].
J. Robert Buchanan
Fixed-Point Iteration
Example
Show that g(x) = 2−x has a unique fixed point on [0, 1].
g ∈ C[0, 1] and g 0 (x) = −(ln 2)2−x < 0 which implies g is
monotone decreasing.
g(0) = 1 and g(1) = 1/2 > 0. Since g is decreasing then
g(x) ∈ [0, 1] for all x ∈ [0, 1].
By the first conclusion of the Brouwer Theorem, g has a
fixed point in [0, 1].
|g 0 (x)| = | − (ln 2)2−x | ≤ ln 2 < 1 for x ∈ (0, 1).
By the second conclusion of the Brouwer Theorem, the
fixed point is unique.
J. Robert Buchanan
Fixed-Point Iteration
Estimating the Fixed Point (1 of 2)
Let p be the unknown fixed point of g(x) = 2−x and suppose p̂
is an approximation to p.
g(p) − g(p̂)
p − p̂
= g 0 (y ) (for some y between p and p̂)
J. Robert Buchanan
Fixed-Point Iteration
Estimating the Fixed Point (1 of 2)
Let p be the unknown fixed point of g(x) = 2−x and suppose p̂
is an approximation to p.
g(p) − g(p̂)
= g 0 (y ) (for some y between p and p̂)
p − p̂
|g(p) − g(p̂)| = |g 0 (y )| |p − p̂|
|p − 2−p̂ | = | − (ln 2)2−y | |p − p̂|
|
{z
}
<1
J. Robert Buchanan
Fixed-Point Iteration
Estimating the Fixed Point (1 of 2)
Let p be the unknown fixed point of g(x) = 2−x and suppose p̂
is an approximation to p.
g(p) − g(p̂)
= g 0 (y ) (for some y between p and p̂)
p − p̂
|g(p) − g(p̂)| = |g 0 (y )| |p − p̂|
|p − 2−p̂ | = | − (ln 2)2−y | |p − p̂|
|
{z
}
<1
|p − 2
−p̂
| < |p − p̂|
J. Robert Buchanan
Fixed-Point Iteration
Estimating the Fixed Point (1 of 2)
Let p be the unknown fixed point of g(x) = 2−x and suppose p̂
is an approximation to p.
g(p) − g(p̂)
= g 0 (y ) (for some y between p and p̂)
p − p̂
|g(p) − g(p̂)| = |g 0 (y )| |p − p̂|
|p − 2−p̂ | = | − (ln 2)2−y | |p − p̂|
|
{z
}
<1
|p − 2
−p̂
| < |p − p̂|
Conclusion: 2−p̂ is a better approximation to p than p̂ was.
J. Robert Buchanan
Fixed-Point Iteration
Estimating the Fixed Point (2 of 2)
Let g(x) = 2−x and let x0 = 1/2. Given xn−1 then define
xn = g(xn−1 ).
n
0
1
2
3
4
5
6
7
8
9
xn
0.5
0.707107
0.612547
0.654041
0.635498
0.643719
0.640061
0.641686
0.640964
0.641285
J. Robert Buchanan
g(xn )
0.707107
0.612547
0.654041
0.635498
0.643719
0.640061
0.641686
0.640964
0.641285
0.641142
Fixed-Point Iteration
Cobweb Diagram
y
0.8
0.7
0.6
0.5
x
0.5
J. Robert Buchanan
0.6
0.7
Fixed-Point Iteration
0.8
Fixed-Point Iteration
Given g(x):
INPUT p0 , tolerance , maximum iterations N
STEP 1 Set i = 1.
STEP 2 While i ≤ N do STEPS 3–6.
STEP 3 Set p = g(p0 ).
STEP 4 If |p − p0 | < then OUTPUT p; STOP.
STEP 5 Set i = i + 1.
STEP 6 Set p0 = p.
STEP 7 OUTPUT “The method failed after N iterations.”;
STOP.
J. Robert Buchanan
Fixed-Point Iteration
Example (1 of 2)
Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed
point iteration.
J. Robert Buchanan
Fixed-Point Iteration
Example (1 of 2)
Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed
point iteration.
If we let g(x) = x 3 − 1 then finding a fixed point of g is
equivalent to finding a root of the original equation.
J. Robert Buchanan
Fixed-Point Iteration
Example (1 of 2)
Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed
point iteration.
If we let g(x) = x 3 − 1 then finding a fixed point of g is
equivalent to finding a root of the original equation.
However; g(x) maps the interval [1, 2] to the interval [1, 7].
J. Robert Buchanan
Fixed-Point Iteration
Example (1 of 2)
Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed
point iteration.
If we let g(x) = x 3 − 1 then finding a fixed point of g is
equivalent to finding a root of the original equation.
However; g(x) maps the interval [1, 2] to the interval [1, 7].
Can you find another function g which maps [1, 2] into itself?
J. Robert Buchanan
Fixed-Point Iteration
Example (2 of 2)
x3 − x − 1 = 0
x3 = x + 1
√
3
x + 1 = g(x)
x =
Function g maps [1, 2] into [1, 2]. Starting with p0 = 1 the
sequence of approximations to the fixed point is
{1.0, 1.25992, 1.31229, 1.32235, 1.32427, 1.32463, 1.32470}
J. Robert Buchanan
Fixed-Point Iteration
Fixed-Point Theorem
Theorem (Fixed-Point)
Suppose g ∈ C[a, b] and g(x) ∈ [a, b] for all x ∈ [a, b].
Furthermore suppose that g 0 exists on (a, b) and that there
exists a constant 0 < k < 1 such that
|g 0 (x)| ≤ k ,
for x ∈ [a, b].
Then for any p0 ∈ [a, b] the sequence defined by
pn = g(pn−1 ),
for n ≥ 1
converges to the unique fixed point p in [a, b].
J. Robert Buchanan
Fixed-Point Iteration
Proof (1 of 2)
According to Brouwer’s Theorem the unique fixed point p
exists in [a, b].
The sequence {pn }∞
n=0 has the property that pn ∈ [a, b] for
all n.
By the MVT
g(p0 ) − g(p) = g 0 (ξ0 ) (with ξ0 ∈ [a, b])
p −p
0
|p1 − p| = g 0 (ξ0 ) |p0 − p|
|p1 − p| ≤ k |p0 − p| .
J. Robert Buchanan
Fixed-Point Iteration
Proof (2 of 2)
Suppose |pn − p| ≤ k n |p0 − p| for some n ≥ 1.
By the MVT
g(pn ) − g(p) = g 0 (ξn ) (with ξn ∈ [a, b])
pn − p
|pn+1 − p| = g 0 (ξn ) |pn − p|
|pn+1 − p| ≤ k |pn − p|
|pn+1 − p| ≤ k n+1 |p0 − p| .
Since 0 < k < 1, the lim |pn − p| = 0.
n→∞
J. Robert Buchanan
Fixed-Point Iteration
Error Estimate
While the Fixed-Point Theorem justifies that the algorithm will
converge to a fixed-point/solution of the function/equation, it
does not tell us anything directly about the error present in each
stage of the algorithm.
J. Robert Buchanan
Fixed-Point Iteration
Error Estimate
While the Fixed-Point Theorem justifies that the algorithm will
converge to a fixed-point/solution of the function/equation, it
does not tell us anything directly about the error present in each
stage of the algorithm.
Corollary
If g satisfies the hypotheses of the Fixed-Point Theorem, then
bounds for the error involved in using pn to approximate p are
|pn − p| ≤ k n max{p0 − a, b − p0 }
and
|pn − p| ≤
kn
|p1 − p|,
1−k
J. Robert Buchanan
for n ≥ 1.
Fixed-Point Iteration
Proof (1 of 2)
According to the Fixed-Point Theorem
|pn − p| ≤ k n |p0 − p| ≤ k n max{p0 − a, b − p0 }.
J. Robert Buchanan
Fixed-Point Iteration
Proof (1 of 2)
According to the Fixed-Point Theorem
|pn − p| ≤ k n |p0 − p| ≤ k n max{p0 − a, b − p0 }.
Using the MVT again we see that
|pn+1 − pn | = |g(pn ) − g(pn−1 )|
≤ k |pn − pn−1 |
≤ k 2 |pn−1 − pn−2 |
..
.
≤ k n |p1 − p0 |.
J. Robert Buchanan
Fixed-Point Iteration
Proof (2 of 2)
Assume 1 ≤ n < m then
|pm − pn | = |pm − pm−1 + pm−1 − · · · + pn+1 − pn |
≤ |pm − pm−1 | + |pm−1 − pm−2 | + · · · + |pn+1 − pn |
≤ k m−1 |p1 − p0 | + k m−2 |p1 − p0 | + · · · + k n |p1 − p0 |
= k n (1 + k + k 2 + · · · + k m−n−1 )|p1 − p0 |
!
∞
X
i
n
k |p1 − p0 |
≤ k
i=0
=
kn
|p1 − p0 |
1−k
J. Robert Buchanan
Fixed-Point Iteration
Example
Consider the equation x 3 + 4x 2 − 10 = 0.
Develop a fixed point function to approximate a solution to
this equation on the interval [1, 2].
Determine the number of iterations necessary to estimate
the solution to within 10−6 .
Estimate the solution.
J. Robert Buchanan
Fixed-Point Iteration
Solution
Among other possibilities:
0 = x 3 + 4x 2 − 10
x 2 (x + 4) = 10
r
x
=
J. Robert Buchanan
10
= g(x).
x +4
Fixed-Point Iteration
Solution
Among other possibilities:
0 = x 3 + 4x 2 − 10
x 2 (x + 4) = 10
r
x
=
10
= g(x).
x +4
Upon taking the derivative of g(x) we see that on [1, 2],
−5
0
≤ √ 5
|g (x)| = √
< 0.1415 = k .
10(x + 4)3/2 10 (5)3/2
√
If p0 = 1 then p1 = 2 and
kn
0.1415n √
|p1 − p0 | =
| 2 − 1| < 10−6
|p − pn | ≤
1−k
1 − 0.1415
n ≥ 7
J. Robert Buchanan
Fixed-Point Iteration
Solution
Among other possibilities:
0 = x 3 + 4x 2 − 10
x 2 (x + 4) = 10
r
x
=
10
= g(x).
x +4
Upon taking the derivative of g(x) we see that on [1, 2],
−5
0
≤ √ 5
|g (x)| = √
< 0.1415 = k .
10(x + 4)3/2 10 (5)3/2
√
If p0 = 1 then p1 = 2 and
kn
0.1415n √
|p1 − p0 | =
| 2 − 1| < 10−6
|p − pn | ≤
1−k
1 − 0.1415
n ≥ 7
p7 ≈ 1.365230
J. Robert Buchanan
Fixed-Point Iteration
Challenge
For a given equation f (x) = 0, find a fixed point function which
satisfies the conditions of the Fixed-Point Theorem.
J. Robert Buchanan
Fixed-Point Iteration
Homework
Read Section 2.2.
Exercises: 1, 2, 3, 7, 13, 16, 23
J. Robert Buchanan
Fixed-Point Iteration