Download Six Myths of Polynomial Interpolation and Quadrature

faug
July 12, 2011
10:51
Page 1
Six Myths of Polynomial Interpolation and
Quadrature
Lloyd N. Trefethen FRS, FIMA, Oxford University
t is a pleasure to offer this essay for Mathematics Today as a
On the face of it, this caution is justified by two theorems.
record of my Summer Lecture on 29 June 2011 at the Royal Weierstrass proved in 1885 [1] that any continuous function can
Society.
be approximated arbitrarily closely by polynomials. On the other
Polynomials are as basic a topic as any in mathematics, and hand, Faber proved in 1914 [2] that no polynomial interpolation
for numerical mathematicians like me, they are the starting point scheme, no matter how the points are distributed, will converge for
of numerical methods that in some cases go back centuries, like all such functions.
So it sounds as if there is something wrong with polynomial inquadrature formulae for numerical integration and Newton iterations for finding roots. You would think that by now, the basic terpolation. Yet the truth is, polynomial interpolants in Chebyshev
facts about computing with polynomials would be widely under- points always converge if f is a little bit smooth. (We shall call
stood. In fact, the situation is almost the reverse. There are indeed them Chebyshev interpolants.) Lipschitz continuity is more than
1885 that enough,
any continuous
can ≤
beL|x
approximated
arbitrarily
that is, |ffunction
(x) − f (y)|
− y| for some
constant Lclosely
and by polywidespread views about polynomials, but some of the important
they other
in 1914 that
no polynomial
interpolation
all x,
∈ [−1,hand,
1]. SoFaber
long proved
as f is Lipschitz
continuous,
as it will
ones are wrong, founded on misconceptions entrenched bynomials.
gener- On
scheme, nobematter
howany
thepractical
points are
distributed,
for all such functions.
in almost
application,
pn will
→ fconverge
is guaranteed.
ations of textbooks.
So it sounds
as ifisthere
is something
wrong
polynomial
interpolation. Yet the
There
indeed
a big problem
withwith
convergence
of polynomial
Since 2006, my colleagues and I have been solving mathemattruth
is, polynomial
interpolants
in Chebyshev
points always
convergepoints.
if f is a little bit
interpolants,
but it pertains
to interpolation
in equispaced
ical problems with polynomials using the Chebfun software
syssmooth (weAsshall
call
them
Chebyshev
interpolants).
Lipschitz
continuity
is more than
Runge showed in 1901 [3], equispaced interpolants may diverge
tem (www.maths.ox.ac.uk/chebfun). We have learned from
enough, that is, |f (x) − f (y)| ≤ L|x − y| for some constant L and all x, y ∈ [−1, 1].
daily experience how fast and reliable polynomials are. This en- exponentially, even if f is so smooth as to be analytic (holomorSo long as f is Lipschitz continuous, as it will be in almost any practical application,
phic). This genuinely important fact, known as the Runge phetirely positive record has made me curious to try to pinp down
n → f is guaranteed.
nomenon,
With Faber’softheorem
seeming
to
where these misconceptions come from, and this essay is anThere
at- is
indeed ahas
bigconfused
problempeople.
with convergence
polynomial
interpolants,
but justification,
real problem
with As
equispaced
polynomial
tempt to summarise some of my findings. Full details, including
it pertainsprovide
to interpolation
in aequispaced
points.
Runge showed
in 1901, equisinterpolants
has
been
overgeneralised
so
that
people
have
suspected
precise definitions and theorems, can be found in my draft
book
paced interpolants may diverge exponentially, even if f is so smooth
as to be analytic
to polynomial
interpolants
in general.
Approximation Theory and Approximation Practice, available
at it of applying
(holomorphic).
This genuinely
important
fact, known
as the Runge phenomenon, has
TheWith
smoother
f is,
the faster
its Chebyshev
www.maths.ox.ac.uk/~trefethen.
confused people.
Faber’s
theorem
seeming
to provide interpolants
justification,cona real problem
verge. polynomial
If f has νinterpolants
derivatives, has
with
the overgeneralized
ν th derivative so
being
The essay is organised around ‘six myths’. Each myth has
some
with
equispaced
been
thatofpeople have
−ν
suspected
it of applying
to polynomial
in ngeneral.
variation
V , then kf interpolants
− pn k = O(V
) as n → ∞. (By
truth in it – mathematicians rarely say things that are simply
false! bounded
The smoother
thethe
faster
its Chebyshev
kf − pn kf I is,
mean
maximum
of |f (x) −interpolants
pn (x)| for x converge.
∈ [−1, 1].) If f has ν
Yet each one misses something important.
−ν
ν th derivative
being of bounded
variation,
then
f is the
analytic,
the convergence
is geometric,
with kf
−kf
pn−p
k n=k = O(n )
Throughout the discussion, f is a continuous function derivatives,
defined Ifwith
−n
as
n
→
∞.
(By
kf
−
p
k
I
mean
the
maximum
of
|f
(x)
−
p
(x)|
for
x
∈
[−1,
1].)
If
f
n
n parameter ρ
ρ > 1. I will give details about the
on the interval [−1, 1], n + 1 distinct points x0 , . . . , xn in [−1, 1] O(ρ ) for some
−n
is
analytic,
the
convergence
is
geometric,
with
kf
−
p
k
=
O(ρ
)
for
some
ρ
>
1.
I
n
are given, and pn is the unique polynomial of degree at most n with under Myth 5.
will give details
about
the
parameter
ρ
under
Myth
5.
For example, here is the degree 10,000 Chebyshev interpolant
pn (xj ) = f (xj ) for each j. Two families of points will be of particFor example, here is the degree 10,000 Chebyshev interpolant pn to the sawtooth
−1
ular interest: equispaced points, xj = −1 + 2j/n, and Chebyshev pn to the sawtooth function f (x) defined as the integral from
function f (x) defined as the integral from −1 to x of sign(sin(100t/(2 − t))). This
to x of sign(sin(100t/(2 − t))). This curve may not look like a
points, xj = cos(jπ/n). I will also mention Legendre points,
decurve may not look like a polynomial, but it is! With kf − pn k ≈ 0.0001, the plot is
it is!
kf − pn k ≈ 0.0001, the plot is indisfined as the zeros of the degree n + 1 Legendre polynomialindistinguishable
Pn+1 . polynomial,
from but
a plot
of With
f itself.
The discussion of each myth begins with two or three represen- tinguishable from a plot of f itself.
tative quotations from leading textbooks, listed anonymously with
the year of publication. Then I say a word about the mathematical
truth underlying the myth, and after that, what that truth overlooks.
I
Myth 1. Polynomial interpolants diverge as n → ∞
Textbooks regularly warn students not to expect pn → f as
n → ∞.
‘Polynomial interpolants rarely converge to a general There is not
muchisuse
polynomial
interpolantsinterpolants
to functionstowith
so little smoothThere
notinmuch
use in polynomial
functions
continuous function.’ (1989)
ness as this,
butsomathematically
they
are trouble-free.
For smoother
like ex ,
with
little smoothness
as this,
but mathematically
they arefunctions
troucos(10x) orble
1/(1
+ For
25x2smoother
), we getfunctions
convergence
digits oforaccuracy
for2 ),small values
free.
like to
ex ,15
cos(10x)
1/(1+25x
‘Unfortunately, there are functions for which interof n (14, 34 and 182, respectively).
we get convergence to 15 digits of accuracy for small values of n
polation at the Chebyshev points fails to converge.’
(14, 34 and 182, respectively).
(1996)
Myth 2. Evaluating polynomial interpolants numerically is problematic
Interpolants in Chebyshev points may converge in theory, but aren’t there problems
on a computer? Textbooks warn students about this.
“Although Lagrangian interpolation is sometimes useful in theoretical investigations,
it is rarely
used in practical computations.” (1985)
faug
July 12, 2011
10:51
Page 2
Myth 2. Evaluating polynomial interpolants
numerically is problematic
Myth 3. Best approximations are optimal
This one sounds true by definition!
Interpolants in Chebyshev points may converge in theory, but aren’t
there problems on a computer? Textbooks warn students about this.
‘Polynomial interpolation has drawbacks in addition
to those of global convergence. The determination and
evaluation of interpolating polynomials of high degree
can be too time-consuming and can also lead to difficulty problems associated with roundoff error.’ (1977)
‘Since the Remes algorithm, or indeed any other algorithm for producing genuine best approximations,
requires rather extensive computations, some interest attaches to other more convenient procedures to
give good, if not optimal, polynomial approximations.’
(1968)
‘Although Lagrangian interpolation is sometimes useful in theoretical investigations, it is rarely used in
practical computations.’ (1985)
‘Minimal polynomial approximations are clearly suitable for use in functions evaluation routines, where it
is advantageous to use as few terms as possible in an
approximation.’ (1968)
‘Interpolation is a notoriously tricky problem from the
point of view of numerical stability.’ (1990)
‘Ideally, we would want a best uniform approximation.’ (1980)
The origin of this view is the fact that some of the methods one Though the statement of this myth looks like a tautology, there is
might naturally try for evaluating polynomial interpolants are slow content in it. The ‘best approximation’ is a common name for the
or numerically unstable or both. For example, if you write down unique polynomial p∗n that minimises kf − p∗n k. So a best approxthe Lagrange interpolation formula in its most obvious form and imant is optimal in the maximum norm, but is it really the best in
implement
on a computer
as problem
written, you
algorithm
thatof numerical
practice?
“Interpolation
is a itnotoriously
tricky
fromget
theanpoint
of view
requires O(n2 ) work per evaluation point. (Partly because of this,
stability.” (1990)
As the first quotation suggests, computing p∗n is not a trivbooks
warnview
readers
thatfact
theythat
should
useof
Newton
interpolation
for- naturally
The origin
of this
is the
some
the methods
one might
ial matter, since the dependence on f is nonlinear. By contrast,
mulae rather
than Lagrange
– another
myth.)
if you compute
try for evaluating
polynomial
interpolants
are slow
or And
numerically
unstable computing
or both. a Chebyshev interpolant with the barycentric formula is
‘linear
style’, byinterpolation
setting up a Vandermonde
maFor example,interpolants
if you write
downalgebra
the Lagrange
formula in its
mosteasy.
obvious
Here our prejudices about value for money begin to intrude.
n
trix whoseitcolumns
contain samples
of 1,you
x, x2get
, . . an
. , xalgorithm
at the grid
form and implement
on a computer
as written,
thatIfrequires
best approximations are hard to compute, they must be valuable!
O(n2 ) work points,
per evaluation
point. (Partly
of this, books
warnThis
readers
theyconsiderations make the truth not so simple. First of all,
your numerical
methodbecause
is exponentially
unstable.
is thatTwo
should use Newton
interpolationalgorithm
formulaeof
rather
than
— of
another
And
the ‘polyval/polyfit’
Matlab,
andLagrange
I am guilty
propakf
− p∗nmyth.)
k. the
So amaximum-norm
best approximant
is optimal
in thebetween
maximum
norm, but
is it really the
accuracy
difference
Chebyshev
interif you compute
interpolants
“linear
style”,
by setting
a Vandermonde
matrix
best
in practice?
gating
Myth 2 myself
by algebra
using this
algorithm
in my up
textbook
Specpolants and best approximations can never
be large, for Ehlich and
whose
columns
∗
n
a trivial
As
thenumerical
first quotation
suggests,
computing
pcannot
contain samples
1, x,
x2 , . . . , x
at on
thea computer
grid points,
your
tral Methods
in Matlabof[4].
Rounding
errors
destroy
n is not
Zeller
proved
in
1966
[7]
that
kf
−
p
k
exceed
kf − matter,
p∗n k by since the n
method
on fthan
is nonlinear.
By contrast,
computing
a Chebyshev
interpolant
with
is exponentially
This
is the
Matlab,
all accuracy ofunstable.
this method
even
for n“polyval/polyfit”
= 60, let alone n algorithm
= dependence
10,000 ofmore
the factor 2+(2/π)
log(n+1).
Usually
the difference
is
the in
barycentric
formula is easy. Here our prejudices about value for money begin to
and I am guilty
my textbook
as inof
thepropagating
plot above. Myth 2 myself by using this algorithm
less than that, and in fact, the best known error bounds for functions
intrude.
If best approximations
hard to compute, they must be valuable!
Spectral Methods
Matlab.
errors
on isa acomputer
all accuracy
of ν derivativesare
Orin
how
about nRounding
= 1,000,000?
Here
plot of thedestroy
polynomial
that have
or
are not
analytic,
the two smoothness classes
Two
considerations
make
the
truth
simple. First of all, the maximum-norm
this methodinterpolant
even for n to
= f60,
n = 10,000
as in the
plot above.
(x)let=alone
sin(10/x)
in a million
Chebyshev
points. mentioned under Myth 1, are just asofactor
of 2 larger for Chebyshev
accuracy
difference
Or how The
about
= 1,000,000?
Here30
is seconds
a plot of
polynomial
interpolant
tobetween Chebyshev interpolants and best approximations can never
plotnwas
obtained in about
on the
my laptop
by evalu∗
interpolants
as
for
best
approximations.
be large,
Ehlich
f (x) = sin(10/x)
in interpolant
a million Chebyshev
points.
The near
plot zero.
was obtained
in for
about
30 and Zeller proved in 1966 that kf − pn k cannot exceed kf − pn k
ating the
at 2,000 points
clustered
Secondly,
according
to
the
equioscillation
theorem
going
back
by more than
seconds on my laptop by evaluating the interpolant at 2000 points clustered
near the
zero.factor 2 + (2/π) log(n + 1). Usually the difference is less than that,
to the
Chebyshev
in the
1850s,
the maximum
error
of ahave
bestνapproxand in fact,
best known
error
bounds
for functions
that
derivatives or are
imation
is always attained
at least n under
+ 2 points
analytic, the
two smoothness
classesatmentioned
Mythin1,[−1,
are 1].
justFor
a factor of 2
larger for Chebyshev
interpolants
for best
approximations.
example, the
black curveasbelow
is the
error curve f (x) − p∗n (x),
Secondly,
to the
equioscillation
theorem
to Chebyshev
in the
x ∈according
[−1, 1], for
the best
approximant
to |x −going
1/4| back
of degree
100,
1850s, the equioscillating
maximum errorbetween
of a best
approximation
is always
attained
atred
at least n + 2
102
extreme values
≈ ±0.0027.
The
points in [−1,
1]. corresponds
For example,tothe
curve below
is the error
f (x)−p∗n (x), x ∈
curve
theblack
polynomial
interpolant
pn incurve
Chebyshev
[−1, 1], forpoints.
the best
approximant
tomost
|x − 1/4|
of of
degree
between 102
It is
clear that for
values
x, |f 100,
(x) −equioscillating
pn (x)| is much
extreme values
≈
±0.0027.
The
red
∗ curve corresponds to the polynomial interpolant pn
smaller than |f (x) − pn (x)|. Which approximation would be more
in Chebyshev points. It is clear that
for most values of x, |f (x)− pn (x)| is much smaller
useful in an application? I think the only reasonable answer
is, it
than |f (x) − p∗n (x)|. Which approximation would be more useful in an application?
depends.
Sometimes
one
really
does
need
a
guarantee
about
worstThe fast and stable algorithm that makes these calculations possible
comes
from reasonable
a
I think
the only
answer is, it depends. Sometimes one really does need
case
behaviour.
In other
situations,
wouldsituations,
be wasteful
sacrifice
The
and stable
algorithm
that as
makes
these calculations
representation of
thefast
Lagrange
interpolant
known
the barycentric
published
aformula,
guarantee
about
worst-case
behaviour.
In itother
it to
would
be wasteful to
so
much
accuracy
over
95%
of
the
range
just
to
gain
one
bit
of
the
Lagrange
interpolant
by Salzer inpossible
1972: comes from a representation
sacrifice so much accuracy over 95% of the range just to gain one bit of
of acaccuracy in a
,
n formula,
n
curacy in a small subinterval.
j
known as the barycentric
by Salzer
in 1972
[5]:
X
X
small
subinterval.
0 (−1)j fjpublished
0 (−1)
pn (x) =
,
x − xj j ,j=0
n x − xjj
n
X
X
j=0
0 (−1) fj
0 (−1)
pn (x) =
,
x
xon−the
xj summation signs signify
with the special case pn (x) = fj j=0
if x =xx−
.
The
primes
j
j
j=0
that the terms j = 0 and j = n are multiplied by 1/2. The work required is O(n) per
evaluation point,
andspecial
though
thepndivisions
look
dangerous
with the
case
(x) = fjbyif xx−=xjxjmay
. The
primes
on the for x ≈ xj ,
the formula summation
is numerically
wastheproved
in 2004.
signsstable,
signifyasthat
terms by
j =Nick
0 andHigham
j = n are
multiplied by 1/2. The work required is O(n) per evaluation point, and
the divisions by x − xj may look dangerous for x ≈ xj ,
Myth 3.though
Best
approximations are optimal
the formula is numerically stable, as was proved by Nick Higham
in 2004
[6].
This one sounds
true
by definition!
“Since the Remes algorithm, or indeed any other algorithm for producing genuine
best approximations, requires rather extensive computations, some interest attaches to
Mythapproxima4. Gauss quadrature has twice the order of
other more convenient procedures to give good, if not optimal, polynomial
tions.” (1968)
accuracy of Clenshaw–Curtis
“Ideally, we would want a best uniform approximation.” (1980)
Though the statement of this myth looks like a tautology, there isQuadrature
content in it.
The are usually derived from polynomials. We approximate an integral
formulae
“best approximation” is a common name for the unique polynomialby
p∗nathat
minimizes
finite
sum,
I=
Z
1
f (x)dx
≈
In =
n
X
wk f (xk ),
(1)
faug
July 12, 2011
10:51
Page 3
Myth 4. Gauss quadrature has twice the order of
accuracy of Clenshaw–Curtis
Quadrature formulae are usually derived from polynomials. We
approximate an integral by a finite sum,
Z 1
n
X
I=
f (x)dx ≈ In =
wk f (xk ),
(1)
−1
k=0
and the weights {wk } are determined by the principle of interpolating f by a polynomial at the points {xk } and integrating
the interpolant. Newton–Cotes quadrature corresponds to equispaced points, Clenshaw–Curtis quadrature to Chebyshev points,
and Gauss quadrature to Legendre points. Almost every textbook
first describes Newton–Cotes, which achieves In = I exactly if f is
a polynomial of degree n, and then shows that Gauss has twice this
order of exactness: In = I if f is a polynomial of degree 2n + 1.
Clenshaw–Curtis is occasionally mentioned, but it only has order
of exactness n, no better than Newton–Cotes.
A theorem of mine from 2008 makes this observation precise.
If f has a ν th derivative of bounded variation V , Gauss quadrature can be shown to converge at the rate O(V (2n)−ν ), the factor
of 2 reflecting its doubled order of exactness. The theorem asserts
that the same rate O(V (2n)−ν ), with the same factor of 2, is also
achieved by Clenshaw–Curtis. (Folkmar Bornemann (private communication) has pointed out that both of these rates can probably
be improved by one further power of n.)
The explanation for this surprising result goes back to O’Hara
and Smith in 1968 [13]. It is true that (n + 1)-point Gauss quadrature integrates the Chebyshev polynomials Tn+1 , Tn+2 , . . . exactly whereas Clenshaw–Curtis does not. However, the error that
Clenshaw–Curtis makes consists of aliasing them to the Chebyshev
polynomials Tn−1 , Tn−2 , . . . and integrating these correctly. As it
happens, the integrals of Tn+k and Tn−k differ by only O(n−3 ),
and that is why Clenshaw–Curtis is more accurate than its order of
exactness seems to suggest.
Myth 5. Gauss quadrature is optimal
‘However, the degree of accuracy for Clenshaw–
Curtis quadrature is only n − 1.’ (1997)
Gauss quadrature may not be much better than Clenshaw–Curtis,
but at least it would appear to be as accurate as possible, the gold
standard of quadrature formulae.
‘Clenshaw–Curtis rules are not optimal in that the degree of an n-point rule is only n − 1, which is well
below the maximum possible.’ (2002)
‘The precision is maximised when the quadrature is
This ubiquitous emphasis on order of exactness is misleading.
Gaussian.’ (1982)
Gauss quadrature
Legendre
points.
Almost Gauss
every textbook
first gives
describes
Newton–
Textbookstosuggest
that
the reason
quadrature
better
‘In fact, it can be shown that among all rules using
Cotes, which
In = I exactly is
if fitsishigher
a polynomial
of exactness,
degree n, and
resultsachieves
than Newton–Cotes
order of
butthen
thisshows
that Gauss has twice this order of accuracy: In = I if f is a polynomial of degree
n function evaluations, the n-point Gaussian rule is
is not correct. The problem with Newton–Cotes is that the sample
2n + 1. Clenshaw–Curtis is occasionally mentioned, but it only has order of exactness
likely to produce the most accurate estimate.’ (1989)
points are equally spaced: the Runge phenomenon. In fact, as Pólya
n, no better than Newton–Cotes.
provedthe
in 1933
Newton–Cotes
quadrature does
not converge
“However,
degree[8],
of accuracy
for Clenshaw–Curtis
quadrature
is only as
n − 1.”
There is another misconception here, quite different from the one
(1997)
n → ∞, in general, even if f is analytic.
just discussed as Myth 4. Gauss quadrature is optimal as measured
“Clenshaw–Curtis
rules are and
not optimal
in that the degree
of an
n-pointdifferrule is only
Clenshaw–Curtis
Gauss quadratures
behave
entirely
by polynomial order of exactness, but that is a skewed measure.
n − 1, which is well below the maximum possible.” (2002)
ently. Both schemes converge for all continuous integrands, and if
This ubiquitous emphasis on order of exactness is misleading. Textbooks suggest The power of polynomials is nonuniform: they have greater resintegrand
analytic,gives
the convergence
geometric.
Clenshaw–
that thethe
reason
Gauss is
quadrature
better resultsisthan
Newton–Cotes
is its higher
olution near the ends of an interval than in the middle. Suppose,
Curtis
is
easy
to
implement,
using
either
the
Fast
Fourier
Transform
order of exactness, but this is not correct. The problem with Newton–Cotes
is that the
for example, that f (x) is an analytic function on [−1, 1], which
sample or
points
arealgorithm
equally spaced:
the Runge
fact,reason
as Pólya
proved in
by an
of Waldvogel
inphenomenon.
2006 [9], andInone
it gets
means
it can be analytically continued into a neighbourhood of
1933, Newton–Cotes
quadrature
does Gauss
not converge
as n →
∞,ainbig
general,
is
little attention
may be that
quadrature
had
head even
start,if f [−1,
1] in the complex x-plane. Then polynomial approximants
analytic.
invented in 1814 [10] instead of 1960 [11]. Both Clenshaw–Curtis to f , whether Chebyshev or Legendre interpolants or best approxBoth Clenshaw–Curtis and Gauss quadrature behave entirely differently. Both
Gaussfor
quadrature
are practical
evenand
if nifisthe
in the
millions,
in the the
schemesand
converge
all continuous
integrands,
integrand
is analytic,
imants, will converge at a geometric rate O(ρ−n ) determined by
latteriscase
because
nodes and weights
can
calculated
by either
an algoconvergence
geometric.
Clenshaw–Curtis
is easy
to be
implement,
using
the Fast
how close any singularities of f in the plane are to [−1, 1]. To be
Fourier rithm
Transform
or by an algorithm
of Waldvogel
in 2006,
reasoninit gets
implemented
in Chebfun
due to Glaser,
Liu and
andone
Rokhlin
precise, ρ is the sum of the semiminor and semimajor axis lengths
little attention
may
be
that
Gauss
quadrature
had
a
big
head
start,
invented
in
1814
2007 [12]. (Indeed, since Glaser–Liu–Rokhlin, it is another myth to of the largest ellipse with foci ±1 inside which f is analytic and
instead of 1960. Both Clenshaw–Curtis and Gauss quadrature are practical even if n
Gauss
only practicable
for be
small
values by bounded.
is in theimagine
millions,that
in the
latterquadrature
case becauseisnodes
and weights can
calculated
an
But ellipses are narrower near the ends than in the midof
n.)
algorithm implemented in Chebfun due to Glaser, Liu and Rokhlin in 2007. (Indeed,
dle. If f has a singularity at x0 = iε for some small ε, then we
since Glaser–Liu–Rokhlin,
it isorder
another
to imagine
Gauss
quadrature
With twice the
ofmyth
exactness,
we that
would
expect
Gaussis only
get O(ρ−n ) convergence with ρ ≈ 1 + ε. If f has a singularity
at
√
practicable
for smallto
values
of n.) twice as fast as Clenshaw–Curtis. Yet it
quadrature
converge
2ε,
cor1
+
ε,
on
the
other
hand,
the
parameter
becomes
ρ
≈
1
+
Withdoes
twice
theUnless
order off exactness,
expect
Gauss
quadrature
to converge
not.
is analyticweinwould
a large
region
surrounding
[−1,
1] responding to much faster convergence. A function with a singutwice as fast as Clenshaw–Curtis. Yet it does not. Unless f is analytic in a large region
in
the
complex
plane,
one
typically
finds
that
Clenshaw–Curtis
surrounding [−1, 1] in the complex plane, one typically finds that Clenshaw–Curtis
larity at 1.01 converges 14 times faster than one with a singularity
quadrature
converges
rate asasGauss,
as illustrated
quadrature
converges
at about at
theabout
samethe
ratesame
as Gauss,
illustrated
by these curves
at 0.01i.
2
for f (x)by
= these
exp(−1/x
): for f (x) = exp(−1/x2 ):
curves
Quadrature rules generated by polynomials, including both
0
10
−5
error
10
Clenshaw−Curtis
−10
10
Gauss
−15
10
0
10
20
30
40
50
degree n
60
70
80
90
100
Gauss and Clenshaw–Curtis, show the same nonuniformity. This
might seem unavoidable, but in fact, there is no reason why a
quadrature formula (1) needs to be derived from polynomials. By
introducing a change of variables, one can generate alternative formulae based on interpolation by transplanted polynomials, which
may converge up to π/2 times faster than Gauss or Clenshaw–
Curtis quadrature for many functions. This idea was developed in a
paper of mine with Nick Hale in 2008 [14] and is related to earlier
work by Kosloff and Tal-Ezer in 1993 [15].
A theorem of mine from 2008 makes this observation precise. If f has a ν th derivative of bounded variation, Gauss quadrature can be shown to converge at the rate
O(n−2ν ), the factor of 2 reflecting its doubled order of accuracy. The theorem asserts
that the same rate O(n−2ν ), with the same factor of 2, is also achieved by Clenshaw–
Curtis.
The explanation for this surprising result goes back to O’Hara and Smith in 1968.
It is true that (n + 1)-point Gauss quadrature integrates the Chebyshev polynomi5
faug
July 12, 2011
10:51
Page 4
The following theorem applies to one of the transformed methods Hale and I proposed. Let f be a function that is analytic in an εneighborhood of [−1, 1] for ε ≤ 0.05. Then whereas Gauss quadrature converges at the rate In − I = O((1 + ε)−2n ), transformed
Gauss quadrature converges 50% faster, In − I = O((1 + ε)−3n ).
Here is an illustration for f (x) = 1/(1 + 25x2 ).
they can be computed accurately by solving an eigenvalue problem involving a ‘colleague matrix’. The details were worked out
by Specht in 1957 [18] and Good in 1961 [19].
Chebfun finds roots of a function f on [−1, 1] by approximating it by a polynomial expressed in Chebyshev form and then solving a colleague-matrix eigenvalue problem, and if the degree is
greater than 100, first subdividing the interval recursively to reduce it. These ideas originate with John Boyd in 2002 [20] and
10
are extraordinarily effective. Far from being exceptionally trouble10
some, polynomial root-finding when posed in this fashion begins
Gauss
to emerge as the most tractable of all root-finding problems, for we
10
can solve the problem globally with just O(n2 ) work to get all the
transformed Gauss
10
roots in an interval to high accuracy.
Far100from beingFor
exceptionally
troublesome,
polynomial
rootfinding
when
in this
example, the
function f (x)
= sin(1000πx)
on [−1,
1] posed
is
0
10
20
30
40
50
60
70
80
90
degree n
fashion begins
to
emerge
as
the
most
tractable
of
all
rootfinding
problems,
for
we
can
represented in Chebfun by a polynomial
of degree 4091. It takes
2
solve the problem
globally
with
just
O(n
)
work
to
get
all
the
roots
in
an
interval
to
2 seconds on my laptop to find all 2001 of its roots in [−1, 1], and
The fact
thatfact
some
quadrature
formulae converge
up to converge
π/2 times faster
Gauss
The
that
some quadrature
formulae
uphigh
tothan
π/2
accuracy.
−16
the maximum deviation from the exact values is 4.4 × 10 .
as n → ∞ is probably not of much practical importance. The importance is conceptual.
times faster than Gauss as n → ∞ is probably not of much practical
For example, the function f (x) = sin(1000πx) on [−1, 1] is represented in Chebfun
Here is another illustration of the robustness of polynomial
by a polynomial of degree 4091. It takes 2 seconds on my laptop to find all 2001 of its
importance. The importance is conceptual.
root-finding
on an interval. In Chebfun, we have plotted the funcMyth 6. Polynomial rootfinding is dangerous
roots in [−1, 1], and the maximum deviation from the exact values is 4.4 × 10−16 .
tion
f
(x)
=
exp(x/2)(sin(5x)
+ sin(101x))
and then
executedon an interHere is another illustration
of the robustness
of polynomial
rootfinding
Our final myth
with Jim Wilkinson
(1919–1986),
hero of mine who taught
Mythoriginates
6. Polynomial
root-finding
isadangerous
the
commands
r
=
roots(f-round(f)),
plot(r,f(r),’.’).
val.on In
we have plotted the function f (x) = exp(x/2)(sin(5x) + sin(101x))
me two courses in graduate school at Stanford. Working with Alan Turing
the Chebfun,
Pilot
This sequence
solves a collection
of hundreds of polynomial
rootand
thensome
executed
the commands
r = roots(f-round(f)),
plot(r,f(r),’.’).
This
Our finalinmyth
with that
Jim Wilkinson
hero
Ace computer
1950, originates
Wilkinson found
attempts to (1919–1986),
compute roots
ofa even
finding
problems
to
locate
all
the
points
where
f
takes
a
value
equal
low-degree
polynomials
failed
dramatically.
He
publicized
this
discovery
widely.
sequence
solves
a
collection
of
hundreds
of
polynomial
rootfinding
problems
to locate
of mine who taught me two courses in graduate school at Stanto an
integer
or a half-integer,
andtoplots
them asand
dots.plots
The them
compu“Our main object in this chapter has been to focus attention on the severe
theinherent
points
where
f takes
a value equal
an integer,
as dots. The
ford. Working with Alan Turing on the Pilot Ace computerall
in 1950,
limitations of all numerical methods for finding the zeros of polynomials.”
(1963) tation
took
2/3
of
a
second.
computation
took
2/3
of
a
second.
Wilkinson found that attempts to compute roots of even some low0
error
−5
−10
−15
“Beware: Some polynomials are ill-conditioned!” (1992)
degree
failed
dramatically.
He publicised
this discovThe
first ofpolynomials
these quotations
comes
from Wilkinson’s
book on rounding
errors, and
he also
coined
the memorable phrase “the perfidious polynomial” as the title of a 1984
ery
widely.
article that won the Chauvenet Prize for outstanding mathematical exposition.
What Wilkinson
discovered
the chapter
extreme has
ill-conditioning
of roots
‘Our main
object was
in this
been to focus
at- of certain
polynomials as functions of their coefficients. Specifically, suppose a polynomial pn is
tention on the severe inherent limitations of all nu
specified by its coefficients in the form a0 + a1 x + · · · + an xn . If pn has roots near the
methods
finding
zeros of polynomials.’
unit circle inmerical
the complex
plane,for
these
pose the
no difficulties:
they are well-conditioned
(1963)
functions of the
coefficients ak and can be computed accurately by Matlab’s “roots”
command, based on the calculation of eigenvalues of a companion matrix containing
‘Beware:
Somethepolynomials
the coefficients.
Roots far from
circle, however,are
suchill-conditioned!’
as roots on the interval [−1, 1],
(1992)
can be so ill-conditioned
as to be effectively uncomputable. The monomials xk form
exponentially bad bases for polynomials on [−1, 1].
The
of argument
these quotations
comes
fromabout
Wilkinson’s
book
on of
The
flawfirst
in the
is that it says
nothing
the condition
of roots
polynomials
as functions
of their
effective
[−1, 1] ‘the
based on
rounding
errors [16],
andvalues.
he alsoFor
coined
the rootfinding
memorableonphrase
pointwise samples, all one must do is fix the basis: replace the monomials xk , which are
perfidious polynomial’ as the title of a 1984 article that won the
orthogonal polynomials on the unit circle, by the Chebyshev polynomials Tk (x), which
Chauvenet
Prize
for outstanding
mathematical
are orthogonal
on the
interval.
Suppose a polynomial
pn is exposition
specified by [17].
its coefficients
What
Wilkinson
thepnextreme
in the form
a0 T0 (x)
+ a1 T1 (x)discovered
+ · · · + an Tnwas
(x). If
has rootsill-conditioning
near [−1, 1], these are
well-conditioned
the coefficientsasakfunctions
, and they of
cantheir
be computed
accurately
of roots offunctions
certainofpolynomials
coefficients.
Conclusion
by solving an eigenvalue problem involving a a “colleague matrix”. The details were
Specifically, suppose a polynomial pn is specified by its coefficients
worked out by Specht in 1957 and Good in 1961.
Perhaps
I might
close by mentioning
another perspective
on the
in the finds
formroots
a0 +ofaa1function
x + · · ·f+onan[−1,
xn1]
. If
the
unit I might
Perhaps
close
by mentioning
another perspective
on the misconceptions
that
n has roots near
Chebfun
bypapproximating
it by
a polynomial
misconceptions
that
have
affected
the
study
of
computation
withof variables
circle
the complex
these
posea no
difficulties:
are wellhave
affected the study of computation with polynomials. By the change
expressed
in in
Chebyshev
form plane,
and then
solving
colleague
matrix they
eigenvalue
problem,
and ifconditioned
the degree is greater
thanof
100,
subdividingathe
interval
reduce
x = tocos
θ, polynomials.
one can show
interpolation
by polynomials
in Chebyshev
Bythat
the change
of variables
x = cos θ, one
can show points is
functions
thefirst
coefficients
can recursively
be computed
k and
it. These
ideas
originate
with
John
Boyd
in
2002
and
are
extraordinarily
effective.
equivalent
to
interpolation
of
periodic
functions
by
series
of
sines
and
cosines in equisthat
interpolation
by
polynomials
in
Chebyshev
points
is
equivaaccurately by Matlab’s ‘roots’ command, based on the calculation
Conclusion
paced points.
latter is the subject
of discrete
Fourier
analysis,
and one
lentThe
to interpolation
of periodic
functions
by series
of sines
andcannot help
of eigenvalues of a companion matrix containing the coefficients.
noting
that
whereas
there
is
widespread
suspicion
that
it
is
not
safe
to
compute with
cosines
in
equispaced
points.
The
latter
is
the
subject
of
discrete
Roots far from the circle, however,
such
as
roots
on
the
interval
7
polynomials,
nobody
worriesand
about
the Fast
Fourier
Inthere
the end
Fourier
analysis,
one cannot
help
notingTransform!
that whereas
is this may
[−1, 1], can be so ill-conditioned as to be effectively uncomputable.
be the biggest
difference
between
Fourier
and
the difference
widespread
suspicion
that
it is not
safepolynomial
to computeinterpolants,
with polynomiThe monomials xk form exponentially bad bases for polynomials
in their reputations.
als, nobody worries about the Fast Fourier Transform! In the end
on [−1, 1].
bonus,
free
of charge.
this amay
be the
biggest
difference between Fourier and polynomial
The flaw in the argument is that it says nothing about theAnd
con- here’s
dition of roots of polynomials as functions of their values. For interpolants, the difference in their reputations.
here’s a bonus,
free of charge. Lagrange interpolation
effective root-finding on [−1, 1] based on pointwise samples,
all 7. And
Myth
Lagrange
discovered
one must do is fix the basis: replace the monomials xk , which
It was Waring, in Myth
the Philosophical
Transactions
of theLagrange
Royal Society in 1779. Euler
are orthogonal polynomials on the unit circle, by the Chebyshev
7. Lagrange
discovered
usedSupthe formula in 1783, and Lagrange in 1795.
polynomials Tk (x), which are orthogonal on the interval.
interpolation
pose a polynomial pn is specified by its coefficients in the form
a0 T0 (x) + a1 T1 (x) + · · · + an Tn (x). If pn has roots near [−1, 1], It was Waring in 1779 [21]. Euler used the formula in 1783, and
these are well-conditioned functions of the coefficients ak , and Lagrange in 1795. h
8
faug
July 12, 2011
10:51
Page 5
References
1 Weierstrass, K. (1885) Über die analytische Darstellbarkeit sogenannter willkürlicher Funktionen einer reellen Veränderlichen, Sitzungsberichte der Akademie zu Berlin, pp. 633–639 and 789–805.
2 Faber, G. (1914) Über die interpolatorische Darstellung stetiger Funktionen, Jahresber. Deutsch. Math. Verein., vol. 23, pp. 190–210.
3 Runge, C. (1901) Über empirische Funktionen und die Interpolation
zwischen äquidistanten Ordinaten, Z. Math. Phys., vol. 46, pp. 224–
243.
4 Trefethen, L.N. (2000) Spectral Methods in MATLAB, SIAM, Philadelphia.
5 Salzer, H.E. (1972) Lagrangian interpolation at the Chebyshev points
xn,v ≡ cos(vπ/n), v = 0(1)n; some unnoted advantages, Comp. J.
vol. 15, pp. 156–159.
6 Higham, N.J. (2004) The numerical stability of barycentric Lagrange
interpolation IMA J. Numer. Anal., vol. 24, no. 4, pp. 547–556.
7 Ehlich, H. and Zeller, K. (1966) Auswertung der Normen von Interpolationsoperatoren, Math. Ann., vol. 164, pp. 105–112.
8 Pólya, G. (1933) Über die Konvergenz von Quadraturverfahren, Math.
Z., vol. 37, pp. 264–286.
9 Waldvogel, J. (2006) Fast construction of the Fejér and Clenshaw–
Curtis quadrature rules, BIT Num. Math., vol. 46, no. 1. pp. 195–202.
10 Gauss, C. F. (1814) Methodus nova integralium valores per approximationem inveniendi, Comment. Soc. R. Scient. Göttingensis Rec., vol.
3, pp. 39–76.
11 Clenshaw, C. W. and Curtis, A. R. (1960) A method for numerical integration on an automatic computer, Numer. Math., vol. 2, pp. 197–205.
12 Glaser, A., Liu, X. and Rokhlin, V. (2007) A fast algorithm for the calculation of the roots of special functions, SIAM J. Sci. Comp., vol. 29,
no. 4, pp. 1420–1438.
13 O’Hara, H. and Smith, F. J. (1968) Error estimation in the Clenshaw–
Curtis quadrature formula, Comp. J., vol. 11, pp. 213–219.
14 Hale, N. and Trefethen, L. N. (2008) New quadrature formulas from
conformal maps, SIAM J. Numer. Anal., vol. 46, pp. 930–948.
15 Kosloff, D. and Tal-Ezer, H. (1993) A modified Chebyshev pseudospectral method with an O(N 1 ) time step restriction, J. Comp.
Phys., vol. 104, pp. 457–469.
16 Wilkinson, J.H. (1994) Rounding Errors in Algebraic Processes
(Prentice-Hall Series in Automatic Computation) Dover Publications.
17 Wilkinson, J.H. (1984) The perfidious polynomial, MAA Stud. Num.
Anal.
18 Specht, W. (1957) Die Lage der Nullstellen eines Polynoms. III, Math.
Nachr., vol. 16, pp. 363–389.
19 Good, I. J. (1961) The colleague matrix, a Chebyshev analogue of the
companion matrix, Quart. J. Math., vol. 12, pp. 61–68.
20 Boyd, J.P. (2002) Computing zeros on a real interval through Chebyshev expansion and polynomial rootfinding, SIAM J. Numer. Anal., vol.
40, no. 5, pp. 1666–1682.
21 Waring, E. (1779) Problems concerning interpolations, Phil. Trans.,
vol. 69, pp. 59–67.
Urban Maths: Virtual Unreality
A. Townie
riving through an unfamiliar city centre recently, with
its poorly signposted routes and confusing one-way systems, I became increasingly frustrated and completely lost.
Sometimes I went with the flow and took the direction being followed by the majority of the traffic; at other times I deliberately
avoided such directions. I began to feel like a particle randomly
diffusing through an incomprehensible network of roads! When
I eventually reached my destination and regained my equilibrium I
realised the process I’d followed through the streets reminded me of
a technique for solving linear networks that I’ve only come across
in the relatively recent past.
D
q
Imagine an electrical circuit comprising a number of interconnected electrical resistors. Focus on a particular interconnection
node and the nodes and resistors to which it is directly connected.
For example, let’s look at a segment of the circuit where node, m,
say, might be surrounded by nodes, q, r and s, to which it is connected via resistors R1 , R2 and R3 , as in Figure 1.
We start conventionally by analysing this part of the circuit using Kirchhoff’s current law and Ohm’s law. Kirchhoff’s current
law says that the sum of the electrical currents out of any interconnection node must be zero (this is basically a statement of the
conservation of electric charge). So, with currents i1 , i2 and i3
from Figure 1 we have:
r
R2
R1
i1
i2
i1 + i2 + i3 = 0.
(1)
Ohm’s law relates the voltage difference across a resistor to the
current flowing through it and the value of the resistance. Using Vx
to represent the voltage at node x, we have for the three branches
of Figure 1:
m
Vm − Vq = i1 R1 ,
Vm − Vr = i2 R2 ,
i3
R3
Vm − Vs = i3 R3 .
s
Figure 1: Segment of electrical network
(2)
Divide each of the equations in (2) by its value of resistance, sum
the resulting equations and make use of equation (1) to obtain, after