Download Lecture3.pdf

Lecture 3: Interpolation
What is the problem ?
• Given a discrete set of ‘data’ points, find a continuous function which passes
through all the data points.
data
interpolant
(xk , yk )
k = 1...N
f (xk ) = yk , ∀k
What is the solution?
• There is no unique interpolant.
• Different interpolation algorithms give different interpolants.
• The algorithm is chosen on the basis of smoothness, accuracy and
computational expense.
• Common interpolation algorithms are linear, polynomial, trigonometric and
spline interpolation.
• A related question is that of extrapolation: given data (xk, yk) obtain data (xi, yi)
for xi outside the range (xmin, xmax).
• Both fit into the more general area of regression analysis (’curve fitting’).
Polynomial interpolation: the Lagrange formula
Pn (x) =
n
!
k=1


$ x − xj

 yk
xk − xj
j!=k
x − x2
x − x1
P2 (x) =
y1 +
y2
x1 − x2
x2 − x1
P3 (x) =
(x − x2 )(x − x3 )
(x − x3 )(x − x1 )
(x − x1 )(x − x3 )
y1 +
y2 +
y3
(x1 − x2 )(x1 − x3 )
(x2 − x3 )(x2 − x1 )
(x3 − x1 )(x3 − x2 )
....
Polynomial interpolation: monomial representation
Vandermonde matrix
Pn (x) = c1 xn−1 + c2 xn−2 + . . . + cn−1 x + cn
The coefficients are evaluated from the matrix equation:





xn−1
1
xn−1
2
xn−2
1
xn−2
2
...
...
x1
x2
xn−1
n
xn−2
n
...
xn
..
.
1
1
..
.
1
 
 
 
·
 
c1
c2
..
.
cn


 
 
=
 
y1
y2
..
.
yn
In practice, the polynomial is never directly evaluated, nor is the matrix
inverted. Instead an iterative procedure is used. See Numerical Recipes.





Polynomial interpolation: error analysis
N
!
f
(ξ)
f (x) − Pn (x) =
(x − xk )
(n + 1)!
(n+1)
k=0
• Applies to a funtion with n + 1 continuous derivatives in the bounded
interpolating interval I. With equispaced data points, the maximum error is
|f (n+1) (x)| n
h
max |f (x) − Pn (x)| ≤
4n + 1
Polynomial interpolation: Runge phenomenon
• A higher order polynomial interpolation does not always guarantee a smaller
error. Indeed, it may so happen that the error increases with the degree of the
interpolating polynomial. This is called the Runge phenomenon.
• Example: interpolating the Lorentzian
1
f (x) =
1 + x2
Polynomial interpolation: Runge phenomenon
Solution is to use non-uniform nodes.
Fourier interpolation: statement of the problem
• Assume f(x) is a function periodic in the interval (0, 2*pi), or admits of a such a
periodic extension without introducing discontinuities. We seek an interpolant
for f(x) in terms of trigonometric functions
2πk
, k = 0, . . . , n
Fn (xk ) = f (xk ), xk =
n+1
a0
+
Fn (x) =
2
m
!
[ak cos(kx) + bk sin(kx)]
(n = 2m)
k=0
• n + 1 points to determine n + 1 coefficients. Need to invert a matrix again?
Fourier interpolation: the Fourier formula
• Let us write the formula for the interpolant in complex notation:
Fn (x) =
m
!
ck exp(ikx)
k=−m
ak = ck + c−k , bk = i(ck − c−k )
• Then, the interpolation condition requires that
Also a polynomial expansion!
m
!
ck exp(ikxl ) = f (xl ), l = 0, . . . , n
k=−m
• Now use orthogonality of trigonometric functions, reduce to diagonal form!
Fourier interpolation: using the FFT
• The solution for the Fourier coefficients is:
n
1 !
ck =
f (xj ) exp(−ikxj ), k = −m, . . . , m
n + 1 j=0
• This operation can be done in O(n ln(n) ) operations using the Fast Fourier
Transform (FFT), which we will see in detail later. The interpolant is then
evaluated using the Inverse Fourier Transform (IFFT).
• The radix-2 FFT algorithm was discovered by Gauss, and rediscovered by
Cooley and Tukey in the 1950s. It enables most of todays digital technology to
function in real time.
Fourier interpolation: difference with Fourier series.
• In the usual Fourier series, we expand a continuous 2pi-periodic function in
terms of trigonometric functions. Given the function, the Fourier decomposition
is unique.
∞
a0 !
+
[ak cos(kx) + bk sin(kx)], x ∈ [0, 2π], k = 0, . . . , ∞
f (x) =
2
k=0
• In Fourier interpolation, we expand the interpolating function, given periodic
data, in terms of trigonometric functions. The data can be thought of as being
sampled from a periodic function. Sampling, or the discreteness of the
independent variable, brings new features.
• Foremost amongst these is aliasing: not all frequencies are distinguishable on a
discrete grid. In fact, high frequencies begin to look exactly like low
frequencies.
Fourier interpolation: aliasing
Fourier interpolation: aliasing
Fourier interpolation: aliasing
Fourier interpolation: aliasing
Fourier interpolation: aliasing
Trigonometric interpolation is not unique! There are infinitely many aliases.
Fourier interpolation: aliasing
Trigonometric interpolation is not unique! There are infinitely many aliases.
f (x)
= sin(5x)
2πk
g(xk ) = sin(5xk ), k =
, k = 0, . . . , 8.
9
h(x) = − sin(3x)
Fourier interpolation: aliasing error
f (x) = sin(x) + sin(5x)
f (x) = sin(x) − sin(3x)
Fourier interpolation: the sampling theorem
f (x) =
∞
!
k=−∞
π
f (xk )sinc (x − xk ) + e(x), xk = hk, k = −∞, . . . , ∞.
h
f (k) =
!
∞
f (x) exp(ikx)dx
−∞
Sampling theorem
π
=⇒ e(x) = 0.
f (k) = 0, f or |k| >
h
Spline interpolation: qualitative idea
• We have discussed both polynomial and Fourier interpolation. Polynomial
interpolation works best when the order is low, roughly below 10. At higher
orders, we are likely to get oscillations - the Runge phenomenon.
• Fourier interpolation is restricted to periodic functions or functions which can
be extended periodically without discontinuities. However, the discreteness of
the nodes gives rise to problems with aliasing.
• Piecewise linear interpolation avoids both these problems : there is no Runge
phenomenon, no aliasing problems. However, the derivatives are
discontinuous. This feature persists even if piecewise quadratic, cubic ...
interpolation is used.
• This is where splines come to the rescue (Bezier curves in PaintShop).
Spline interpolation: outline of algorithm
S3 (x) = c1 x3 + c2 x2 + c3 x + c4
• smoothness: the spline must have
continuous 1st and 2nd derivatives at nonboundary nodes. 2(n-1) conditions.
• We thus get a total of 4n - 2 conditions.
• We need four conditions per interval to
determine the spline interpolant.
• interpolation: the spline must interpolate
between adjacent nodes. (n + 1 conditions)
• continuity: the spline must be continuous at
non-boundary nodes. (n - 1 conditions)
• Remaining 2 are boundary conditions.
Natural splines enforce the equality of
second derivates at the boundary nodes.
• A matrix inversion needs to be done, but
this matrix is tridiagonal and can be
inverted in O(n) steps.
• See Wikipedia for details.
Summary
• Interpolation is to construct a function which passes through a prescribed set of points.
• For a set of n + 1 distinct nodes, there is an unique polynomial of degree not greater than n which
passes through these points. Polynomial interpolation of high degree is susceptible to the Runge
phenomenon. The cure is to use non-equispaced nodes, for example Chebyshev nodes.
• Fourier interpolation is useful for periodic functions. On a discrete grid, Fourier interpolation is
prone to aliasing error. However, if the grid is fine enough, the sampling theorem ensures that
Fourier interpolation is exact.
• Splines are piecewise polynomial interpolants which ensure continuity of derivatives. The cubic
spline is the most popular of the spline interpolants.
• Interpolation is used in many other areas of numerical analysis, apart from fitting data. Interpolation
forms the basis of numerical differentiation and integration, the solution of differential equations,
the spectral method and many more. It pays to know about interpolation!
Lab session 1: coding exercise 1
• These exercises illustrate how arithemetic with finite precision numbers is
different from arithmetic with real numbers.
• (1) Let a = 1 and b = 1. (2) Set b = b/2. (3) Test for the condition a + b = a. (4)
Repeat until (3) is true. For real numbers the condition is met only after an
infinite number of iterations. How many iterations does it take in Octave or
GSL? What is the value of b when (3) is satisfied ? Convert your answer for b
into base 2 and report the exponent.
• Let a = 1.0e+308, b = 1.1e+308 and c = -1.001e+308. (1) First calculate b + c
and then add a to the result. (2) First calculate a + b and then add c to the
result. Why are the results of (1) and (2) different ?
Lab session 1: coding exercise 2
• Consider the following algorithm for pi. Generate n pairs of random numbers
(x,y) lying in the interval [0, 1]. Compute the number m of them lying inside the
the first quadrant of the unit circle. Clearly,
π
πn
=
=
lim πn
n→∞
4m
n
• Write Octave or GSL code to compute this sequence and check the error for
increasing values of n.
Lab session 1: coding exercise 3
• An infinite series for pi is
π=
∞
!
m=0
16−m
"
4
2
1
1
−
+
+
8m + 1 8m + 4 8m + 5 8m + 6
#
• Write Octave or GSL code to calculate this to series to n terms. How large
must n be to get the value of pi correct to 16 decimal places ? Repeat this for
Ramanujan’s formula,
√ ∞
1
2 2 ! (4n)!(1103 + 26390n)
=
.
π
9801 n=0
(n!)4 3964n
• HOMEWORK
Lab session 1: coding exercise 4
• Consider the polynomial
f (x) = x7 − 7x6 + 21x5 − 35x4 + 35x3 − 21x2 + 7x − 1
• Evaluate it at 401 equispaced points between 1.0 - 2e-8 and 1.0 + 2e-8. Plot
the result. Is this what you expect the polynomial to look like in that interval? If
yes, justify. If not, explain the discrepancy.
• The answer to this question is related to the failure of a Patriot missile in Iraq.
• HOMEWORK
Lab session 1: coding exercise 5
• Plot the golden rectangle. This is rectangle whose length and breadth are in the
golden ratio,
√
1+ 5
φ=
2
• Name a rectangle in everyday use which has this ratio.
• HOMEWORK