Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Fixed-Point Iteration MATH 375 Numerical Analysis J. Robert Buchanan Department of Mathematics Fall 2013 J. Robert Buchanan Fixed-Point Iteration Motivation Several root-finding algorithms operate by repeatedly evaluating a function until its value does not change (or changes by a suitably small amount). J. Robert Buchanan Fixed-Point Iteration Motivation Several root-finding algorithms operate by repeatedly evaluating a function until its value does not change (or changes by a suitably small amount). Definition A fixed-point for a function f is a number p such that f (p) = p. J. Robert Buchanan Fixed-Point Iteration Equivalence The process of root-finding and the process of finding fixed points are equivalent in the following sense. Suppose g(x) is a function with a root at x = p, then f (x) = g(x) + x has a fixed point at x = p. Suppose f (x) is a function with a fixed point at x = p, then g(x) = x − f (x) has a root at x = p. J. Robert Buchanan Fixed-Point Iteration Examples Find the fixed points (if any) of the following functions. f (x) = x f (x) = 2x f (x) = x − sin πx J. Robert Buchanan Fixed-Point Iteration Graphical Interpretation y 2 1 x -2 1 -1 -1 -2 J. Robert Buchanan Fixed-Point Iteration 2 Existence and Uniqueness Theorem (Brouwer) Suppose g ∈ C[a, b] and g(x) ∈ [a, b] for all x ∈ [a, b]. Then 1 2 g has a fixed point in [a, b] (existence), and if, in addition, g 0 (x) exists on (a, b) and if there exists a constant 0 < k < 1 such that |g 0 (x)| ≤ k , for all x ∈ (a, b), then there is exactly one fixed point in [a, b] (uniqueness). J. Robert Buchanan Fixed-Point Iteration Graphical Interpretation y b p=gHpL a a x p b Note: g(a) ≥ a and g(b) ≤ b. If g(a) = a or if g(b) = b then g has a fixed point in [a, b]. J. Robert Buchanan Fixed-Point Iteration Proof (1 of 2) Suppose g(a) > a and g(b) < b. Define h(x) = g(x) − x. The function h ∈ C[a, b]. h(b) < 0 < h(a) so by the Intermediate Value Theorem there exists p ∈ (a, b) such that h(p) = 0 0 = h(p) = g(p) − p g(p) = p J. Robert Buchanan Fixed-Point Iteration Proof (2 of 2) Now suppose g 0 (x) exists on (a, b) and there exists a constant 0 < k < 1 such that |g 0 (x)| ≤ k for all x ∈ (a, b). J. Robert Buchanan Fixed-Point Iteration Proof (2 of 2) Now suppose g 0 (x) exists on (a, b) and there exists a constant 0 < k < 1 such that |g 0 (x)| ≤ k for all x ∈ (a, b). For the purposes of contradiction suppose g has two fixed points p and q with p 6= q. J. Robert Buchanan Fixed-Point Iteration Proof (2 of 2) Now suppose g 0 (x) exists on (a, b) and there exists a constant 0 < k < 1 such that |g 0 (x)| ≤ k for all x ∈ (a, b). For the purposes of contradiction suppose g has two fixed points p and q with p 6= q. According to the Mean Value Theorem there exists a < c < b for which g(p) − g(q) p − q 0 = p−q p − q = 1 = |g (c)|. This contradicts the assumption that |g 0 (x)| ≤ k < 1 for all x ∈ (a, b). J. Robert Buchanan Fixed-Point Iteration Example Show that g(x) = 2−x has a unique fixed point on [0, 1]. J. Robert Buchanan Fixed-Point Iteration Example Show that g(x) = 2−x has a unique fixed point on [0, 1]. g ∈ C[0, 1] and g 0 (x) = −(ln 2)2−x < 0 which implies g is monotone decreasing. g(0) = 1 and g(1) = 1/2 > 0. Since g is decreasing then g(x) ∈ [0, 1] for all x ∈ [0, 1]. By the first conclusion of the Brouwer Theorem, g has a fixed point in [0, 1]. J. Robert Buchanan Fixed-Point Iteration Example Show that g(x) = 2−x has a unique fixed point on [0, 1]. g ∈ C[0, 1] and g 0 (x) = −(ln 2)2−x < 0 which implies g is monotone decreasing. g(0) = 1 and g(1) = 1/2 > 0. Since g is decreasing then g(x) ∈ [0, 1] for all x ∈ [0, 1]. By the first conclusion of the Brouwer Theorem, g has a fixed point in [0, 1]. |g 0 (x)| = | − (ln 2)2−x | ≤ ln 2 < 1 for x ∈ (0, 1). By the second conclusion of the Brouwer Theorem, the fixed point is unique. J. Robert Buchanan Fixed-Point Iteration Estimating the Fixed Point (1 of 2) Let p be the unknown fixed point of g(x) = 2−x and suppose p̂ is an approximation to p. g(p) − g(p̂) p − p̂ = g 0 (y ) (for some y between p and p̂) J. Robert Buchanan Fixed-Point Iteration Estimating the Fixed Point (1 of 2) Let p be the unknown fixed point of g(x) = 2−x and suppose p̂ is an approximation to p. g(p) − g(p̂) = g 0 (y ) (for some y between p and p̂) p − p̂ |g(p) − g(p̂)| = |g 0 (y )| |p − p̂| |p − 2−p̂ | = | − (ln 2)2−y | |p − p̂| | {z } <1 J. Robert Buchanan Fixed-Point Iteration Estimating the Fixed Point (1 of 2) Let p be the unknown fixed point of g(x) = 2−x and suppose p̂ is an approximation to p. g(p) − g(p̂) = g 0 (y ) (for some y between p and p̂) p − p̂ |g(p) − g(p̂)| = |g 0 (y )| |p − p̂| |p − 2−p̂ | = | − (ln 2)2−y | |p − p̂| | {z } <1 |p − 2 −p̂ | < |p − p̂| J. Robert Buchanan Fixed-Point Iteration Estimating the Fixed Point (1 of 2) Let p be the unknown fixed point of g(x) = 2−x and suppose p̂ is an approximation to p. g(p) − g(p̂) = g 0 (y ) (for some y between p and p̂) p − p̂ |g(p) − g(p̂)| = |g 0 (y )| |p − p̂| |p − 2−p̂ | = | − (ln 2)2−y | |p − p̂| | {z } <1 |p − 2 −p̂ | < |p − p̂| Conclusion: 2−p̂ is a better approximation to p than p̂ was. J. Robert Buchanan Fixed-Point Iteration Estimating the Fixed Point (2 of 2) Let g(x) = 2−x and let x0 = 1/2. Given xn−1 then define xn = g(xn−1 ). n 0 1 2 3 4 5 6 7 8 9 xn 0.5 0.707107 0.612547 0.654041 0.635498 0.643719 0.640061 0.641686 0.640964 0.641285 J. Robert Buchanan g(xn ) 0.707107 0.612547 0.654041 0.635498 0.643719 0.640061 0.641686 0.640964 0.641285 0.641142 Fixed-Point Iteration Cobweb Diagram y 0.8 0.7 0.6 0.5 x 0.5 J. Robert Buchanan 0.6 0.7 Fixed-Point Iteration 0.8 Fixed-Point Iteration Given g(x): INPUT p0 , tolerance , maximum iterations N STEP 1 Set i = 1. STEP 2 While i ≤ N do STEPS 3–6. STEP 3 Set p = g(p0 ). STEP 4 If |p − p0 | < then OUTPUT p; STOP. STEP 5 Set i = i + 1. STEP 6 Set p0 = p. STEP 7 OUTPUT “The method failed after N iterations.”; STOP. J. Robert Buchanan Fixed-Point Iteration Example (1 of 2) Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed point iteration. J. Robert Buchanan Fixed-Point Iteration Example (1 of 2) Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed point iteration. If we let g(x) = x 3 − 1 then finding a fixed point of g is equivalent to finding a root of the original equation. J. Robert Buchanan Fixed-Point Iteration Example (1 of 2) Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed point iteration. If we let g(x) = x 3 − 1 then finding a fixed point of g is equivalent to finding a root of the original equation. However; g(x) maps the interval [1, 2] to the interval [1, 7]. J. Robert Buchanan Fixed-Point Iteration Example (1 of 2) Approximate a solution to x 3 − x − 1 = 0 on [1, 2] using fixed point iteration. If we let g(x) = x 3 − 1 then finding a fixed point of g is equivalent to finding a root of the original equation. However; g(x) maps the interval [1, 2] to the interval [1, 7]. Can you find another function g which maps [1, 2] into itself? J. Robert Buchanan Fixed-Point Iteration Example (2 of 2) x3 − x − 1 = 0 x3 = x + 1 √ 3 x + 1 = g(x) x = Function g maps [1, 2] into [1, 2]. Starting with p0 = 1 the sequence of approximations to the fixed point is {1.0, 1.25992, 1.31229, 1.32235, 1.32427, 1.32463, 1.32470} J. Robert Buchanan Fixed-Point Iteration Fixed-Point Theorem Theorem (Fixed-Point) Suppose g ∈ C[a, b] and g(x) ∈ [a, b] for all x ∈ [a, b]. Furthermore suppose that g 0 exists on (a, b) and that there exists a constant 0 < k < 1 such that |g 0 (x)| ≤ k , for x ∈ [a, b]. Then for any p0 ∈ [a, b] the sequence defined by pn = g(pn−1 ), for n ≥ 1 converges to the unique fixed point p in [a, b]. J. Robert Buchanan Fixed-Point Iteration Proof (1 of 2) According to Brouwer’s Theorem the unique fixed point p exists in [a, b]. The sequence {pn }∞ n=0 has the property that pn ∈ [a, b] for all n. By the MVT g(p0 ) − g(p) = g 0 (ξ0 ) (with ξ0 ∈ [a, b]) p −p 0 |p1 − p| = g 0 (ξ0 ) |p0 − p| |p1 − p| ≤ k |p0 − p| . J. Robert Buchanan Fixed-Point Iteration Proof (2 of 2) Suppose |pn − p| ≤ k n |p0 − p| for some n ≥ 1. By the MVT g(pn ) − g(p) = g 0 (ξn ) (with ξn ∈ [a, b]) pn − p |pn+1 − p| = g 0 (ξn ) |pn − p| |pn+1 − p| ≤ k |pn − p| |pn+1 − p| ≤ k n+1 |p0 − p| . Since 0 < k < 1, the lim |pn − p| = 0. n→∞ J. Robert Buchanan Fixed-Point Iteration Error Estimate While the Fixed-Point Theorem justifies that the algorithm will converge to a fixed-point/solution of the function/equation, it does not tell us anything directly about the error present in each stage of the algorithm. J. Robert Buchanan Fixed-Point Iteration Error Estimate While the Fixed-Point Theorem justifies that the algorithm will converge to a fixed-point/solution of the function/equation, it does not tell us anything directly about the error present in each stage of the algorithm. Corollary If g satisfies the hypotheses of the Fixed-Point Theorem, then bounds for the error involved in using pn to approximate p are |pn − p| ≤ k n max{p0 − a, b − p0 } and |pn − p| ≤ kn |p1 − p|, 1−k J. Robert Buchanan for n ≥ 1. Fixed-Point Iteration Proof (1 of 2) According to the Fixed-Point Theorem |pn − p| ≤ k n |p0 − p| ≤ k n max{p0 − a, b − p0 }. J. Robert Buchanan Fixed-Point Iteration Proof (1 of 2) According to the Fixed-Point Theorem |pn − p| ≤ k n |p0 − p| ≤ k n max{p0 − a, b − p0 }. Using the MVT again we see that |pn+1 − pn | = |g(pn ) − g(pn−1 )| ≤ k |pn − pn−1 | ≤ k 2 |pn−1 − pn−2 | .. . ≤ k n |p1 − p0 |. J. Robert Buchanan Fixed-Point Iteration Proof (2 of 2) Assume 1 ≤ n < m then |pm − pn | = |pm − pm−1 + pm−1 − · · · + pn+1 − pn | ≤ |pm − pm−1 | + |pm−1 − pm−2 | + · · · + |pn+1 − pn | ≤ k m−1 |p1 − p0 | + k m−2 |p1 − p0 | + · · · + k n |p1 − p0 | = k n (1 + k + k 2 + · · · + k m−n−1 )|p1 − p0 | ! ∞ X i n k |p1 − p0 | ≤ k i=0 = kn |p1 − p0 | 1−k J. Robert Buchanan Fixed-Point Iteration Example Consider the equation x 3 + 4x 2 − 10 = 0. Develop a fixed point function to approximate a solution to this equation on the interval [1, 2]. Determine the number of iterations necessary to estimate the solution to within 10−6 . Estimate the solution. J. Robert Buchanan Fixed-Point Iteration Solution Among other possibilities: 0 = x 3 + 4x 2 − 10 x 2 (x + 4) = 10 r x = J. Robert Buchanan 10 = g(x). x +4 Fixed-Point Iteration Solution Among other possibilities: 0 = x 3 + 4x 2 − 10 x 2 (x + 4) = 10 r x = 10 = g(x). x +4 Upon taking the derivative of g(x) we see that on [1, 2], −5 0 ≤ √ 5 |g (x)| = √ < 0.1415 = k . 10(x + 4)3/2 10 (5)3/2 √ If p0 = 1 then p1 = 2 and kn 0.1415n √ |p1 − p0 | = | 2 − 1| < 10−6 |p − pn | ≤ 1−k 1 − 0.1415 n ≥ 7 J. Robert Buchanan Fixed-Point Iteration Solution Among other possibilities: 0 = x 3 + 4x 2 − 10 x 2 (x + 4) = 10 r x = 10 = g(x). x +4 Upon taking the derivative of g(x) we see that on [1, 2], −5 0 ≤ √ 5 |g (x)| = √ < 0.1415 = k . 10(x + 4)3/2 10 (5)3/2 √ If p0 = 1 then p1 = 2 and kn 0.1415n √ |p1 − p0 | = | 2 − 1| < 10−6 |p − pn | ≤ 1−k 1 − 0.1415 n ≥ 7 p7 ≈ 1.365230 J. Robert Buchanan Fixed-Point Iteration Challenge For a given equation f (x) = 0, find a fixed point function which satisfies the conditions of the Fixed-Point Theorem. J. Robert Buchanan Fixed-Point Iteration Homework Read Section 2.2. Exercises: 1, 2, 3, 7, 13, 16, 23 J. Robert Buchanan Fixed-Point Iteration