Download On the Roots of Digital Signal Processing

On the Roots of Digital Signal Processing
300 BC to 1770 AD
Copyright © 2007- Andreas Antoniou
Victoria, BC, Canada
Email: [email protected]
October 7, 2007
Frame # 1 Slide # 1
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Introduction
t Since the dawn of civilization, humans have found it useful
to assemble collections of numbers and to manipulate
them in certain ways in order to enhance their usefulness.
Frame # 2 Slide # 2
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Introduction
t Since the dawn of civilization, humans have found it useful
to assemble collections of numbers and to manipulate
them in certain ways in order to enhance their usefulness.
t By the 1500s, collections of numbers in the form of
numerical tables began to be published, which were used
to facilitate the calculations required in business and
commerce, in the emerging new sciences, and in
navigation.
Frame # 2 Slide # 3
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Introduction
t Since the dawn of civilization, humans have found it useful
to assemble collections of numbers and to manipulate
them in certain ways in order to enhance their usefulness.
t By the 1500s, collections of numbers in the form of
numerical tables began to be published, which were used
to facilitate the calculations required in business and
commerce, in the emerging new sciences, and in
navigation.
t Simultaneously, mathematical techniques began to evolve
that could be used to generated numerical tables or to
enhance their usefulness.
Frame # 2 Slide # 4
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Introduction Cont’d
t The everincreasing reliance of humans on numbers and
the evolution of related mathematical techniques that can
manipulate them have led to the evolution of what we refer
to today as digital signal processing.
Frame # 3 Slide # 5
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Introduction Cont’d
t The everincreasing reliance of humans on numbers and
the evolution of related mathematical techniques that can
manipulate them have led to the evolution of what we refer
to today as digital signal processing.
t Before we begin our search for the roots of DSP, we must
first decide what is DSP in today’s context.
Frame # 3 Slide # 6
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Introduction Cont’d
t The everincreasing reliance of humans on numbers and
the evolution of related mathematical techniques that can
manipulate them have led to the evolution of what we refer
to today as digital signal processing.
t Before we begin our search for the roots of DSP, we must
first decide what is DSP in today’s context.
t Nowadays, DSP relates to signals only part of the time but
let us consider the situation where we need to process a
continuous-time signal by digital means.
Frame # 3 Slide # 7
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Introduction Cont’d
t The everincreasing reliance of humans on numbers and
the evolution of related mathematical techniques that can
manipulate them have led to the evolution of what we refer
to today as digital signal processing.
t Before we begin our search for the roots of DSP, we must
first decide what is DSP in today’s context.
t Nowadays, DSP relates to signals only part of the time but
let us consider the situation where we need to process a
continuous-time signal by digital means.
Notes:
1. This presentation is based on an article published in the
IEEE Circuits and Systems Magazine [Antoniou, 2007].
2. References appear at the end of the slide presentation.
Frame # 3 Slide # 8
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
What is DSP?
In order to process a continuous-time signal, we need to
perform three operations:
t Sample and digitize the signal.
Processing
Sampling
x(nT)
Interpolation
y(nT)
y(t)
x(t)
Frame # 4 Slide # 9
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
What is DSP?
In order to process a continuous-time signal, we need to
perform three operations:
t Sample and digitize the signal.
t Process the digitized signal.
Processing
Sampling
x(nT)
Interpolation
y(nT)
y(t)
x(t)
Frame # 4 Slide # 10
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
What is DSP?
In order to process a continuous-time signal, we need to
perform three operations:
t Sample and digitize the signal.
t Process the digitized signal.
t Apply interpolation to the processed digitized signal to
generate a processed version of the continuous-time
signal.
Processing
Sampling
x(nT)
Interpolation
y(nT)
y(t)
x(t)
Frame # 4 Slide # 11
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Sampling and Digitization
x(t)
t
x(nT)
Frame # 5 Slide # 12
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Processing
x(nT)
nT
y(nT)
nT
Frame # 6 Slide # 13
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Interpolation
y(nT)
nT
y(t)
t
Frame # 7 Slide # 14
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
What is DSP? Cont’d
To trace the origins of DSP, we must, therefore, trace the origins
of the fundamental processes that make up DSP, namely,
t Sampling
t Processing
t Interpolation
Frame # 8 Slide # 15
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes of Syracuse
t Archimedes was born in Syracuse, Sicily, and lived during
the period 287-212 BC.
Frame # 9 Slide # 16
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes of Syracuse
t Archimedes was born in Syracuse, Sicily, and lived during
the period 287-212 BC.
t He is most famous for the the Archimedes principle which
gives the weight of a body immersed in a liquid.
Frame # 9 Slide # 17
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes of Syracuse Cont’d
Note: This image and some others to follow originate from The
MacTutor History of Mathematics Archive [Indexes of Biographies].
Frame # 10 Slide # 18
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes of Syracuse
t He was a great mathematician, developed fundamental
theories for mechanics, and is credited for many inventions,
like the Archimedes screw, and other things.
Note: This image originates from Wikipedia
[Archimedes Screw].
Frame # 11 Slide # 19
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes of Syracuse
t Archimedes was the first person to propose a formal
method for the calculation of π .
As will be demonstrated in the slides that follow,
Archimedes’ method entails both sampling as well as
interpolation.
Frame # 12 Slide # 20
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes’ Evaluation of π
t A lower bound for π can be readily obtained by inscribing a
hexagon inside a circle of radius 12 .
1
2
Frame # 13 Slide # 21
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t The regular hexagon can be broken down into 6 equilateral
triangles; hence the perimeter of the hexagon, denoted as
p6 , is 6 × 12 = 3, i.e, p6 = 3.
1
2
p6=3
Frame # 14 Slide # 22
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t The perimeter of the inscribed hexagon is obviously
smaller than the circumference of the circle, which is 2π ×
radius = π , i.e.,
3<π
1
2
p6=3
Frame # 15 Slide # 23
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t An upper bound for π can be readily obtained by
circumscribing a circle of radius 12 by a hexagon.
t Draw tangents at points A, B, C, D, E, and F as shown.
1
√
3
A
B
1
2
1
C 2
F
E
√
3
D
√
P6=2 3
Frame # 16 Slide # 24
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t The perimeter of the larger hexagon is given by
√
√
P6 = 6 × 1/ 3 = 2 3 = 3.4641.
1
√
3
A
B
1
2
1
C 2
F
E
√
3
D
√
P6=2 3
Frame # 17 Slide # 25
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t The circumference of the circle, π , is smaller that the
perimeter of the larger hexagon; hence we have
√
p6 = 3 < π < 2 3 = P6
1
√
3
A
B
1
2
1
C 2
F
E
√
3
D
√
P6=2 3
Frame # 18 Slide # 26
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Tighter lower and upper bounds on π can be readily
obtained by using 12-sided regular polygons (dodecagons)
instead of 6-sided ones, as shown below.
A
B
C
F
E
Frame # 19 Slide # 27
A. Antoniou
D
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t The inside dodecagon is obtained by drawing straight lines
that divide the arcs AB, BC, etc.
t The outside dodecagon is obtained by drawing tangents at
the 12 vertices of the inside dodecagon.
A
B
C
F
E
Frame # 20 Slide # 28
A. Antoniou
D
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Geometry will show that the perimeters of the larger and
smaller dodecagons are given by
2p6 P6
and p12 = p6 P12
P12 =
p6 + P6
respectively, or
2p6 P6
2 × 3 × 3.4641
= 3.2154
=
P2×6 =
p6 + P6
3 + 3.4641
and
√
p2×6 = p6 P2×6 = 3 × 3.2154 = 3.1058
Frame # 21 Slide # 29
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Geometry will show that the perimeters of the larger and
smaller dodecagons are given by
2p6 P6
and p12 = p6 P12
P12 =
p6 + P6
respectively, or
2p6 P6
2 × 3 × 3.4641
= 3.2154
=
P2×6 =
p6 + P6
3 + 3.4641
and
√
p2×6 = p6 P2×6 = 3 × 3.2154 = 3.1058
t Therefore, we have
3 < 3.1058 < π < 3.2154 < 3.4641
or
p6 < p2×6 < π < P2×6 < P6
Frame # 21 Slide # 30
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes found out that the same procedure can be
repeated with 24-sided, 48-sided, and 96-sided regular
polygons.
Frame # 22 Slide # 31
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes found out that the same procedure can be
repeated with 24-sided, 48-sided, and 96-sided regular
polygons.
t He also found out that the perimeters of successive outside
and inside polygons can be evaluated (in today’s
mathematical notation) as
p2n =
2pn Pn
pn + Pn
and P2n =
pn P2n
respectively (see [Burton, 2003] for details).
Frame # 22 Slide # 32
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes found out that the same procedure can be
repeated with 24-sided, 48-sided, and 96-sided regular
polygons.
t He also found out that the perimeters of successive outside
and inside polygons can be evaluated (in today’s
mathematical notation) as
p2n =
2pn Pn
pn + Pn
and P2n =
pn P2n
respectively (see [Burton, 2003] for details).
Note: Archimedes’ method was formulated in terms of
geometry. Algebra did not emerge as a subject of study
until the 800s AD when a man by the name of al-Khwarizmi
wrote two books on arithmetic and algebra.
Frame # 22 Slide # 33
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
Table 1 Bounds for π
No. of sides
6
12
24
48
96
Frame # 23 Slide # 34
Lower bound
3.0000
3.1058
3.1326
3.1394
3.1410
A. Antoniou
Upper bound
3.4641
3.2154
3.1597
3.1461
3.1427
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes repeated his procedure 5 times but stopped
with 96-sided polygons.
Frame # 24 Slide # 35
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes repeated his procedure 5 times but stopped
with 96-sided polygons.
t He concluded that the perimeter of the outside polygon is
larger than that of the circle whereas the perimeter of the
inner polygon is smaller than that of the circle in each case
by a grain of sand (ε in today’s terminology).
Frame # 24 Slide # 36
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes used his method to find the area of a circle.
Frame # 25 Slide # 37
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes used his method to find the area of a circle.
t In one of his propositions he states, in effect, that the area
of a circle is to the square of its diameter as 11 is to 14,
that is
22 2
11
Area
or Area =
r (See [Burton, 2003])
=
(2 × r )2
14
7
Frame # 25 Slide # 38
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes used his method to find the area of a circle.
t In one of his propositions he states, in effect, that the area
of a circle is to the square of its diameter as 11 is to 14,
that is
22 2
11
Area
or Area =
r (See [Burton, 2003])
=
(2 × r )2
14
7
t This implies that he actually interpolated his upper and
lower bounds of π to obtain the rational approximation
π≈
22
= 3.1429
7
which, fittingly enough, is known as the Archimedean π .
Frame # 25 Slide # 39
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes used his method to find the area of a circle.
t In one of his propositions he states, in effect, that the area
of a circle is to the square of its diameter as 11 is to 14,
that is
22 2
11
Area
or Area =
r (See [Burton, 2003])
=
(2 × r )2
14
7
t This implies that he actually interpolated his upper and
lower bounds of π to obtain the rational approximation
π≈
22
= 3.1429
7
which, fittingly enough, is known as the Archimedean π .
t This entails an error of 0.04%.
Frame # 25 Slide # 40
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t Archimedes used his method to find the area of a circle.
t In one of his propositions he states, in effect, that the area
of a circle is to the square of its diameter as 11 is to 14,
that is
22 2
11
Area
or Area =
r (See [Burton, 2003])
=
(2 × r )2
14
7
t This implies that he actually interpolated his upper and
lower bounds of π to obtain the rational approximation
π≈
22
= 3.1429
7
which, fittingly enough, is known as the Archimedean π .
t This entails an error of 0.04%.
t The average of the Archimedean upper and lower bounds
entails an error of 0.01%!
Frame # 25 Slide # 41
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t We now know that Archimedes’ method is valid for any
number of iterations.
It is commonly referred to, naturally enough, as the
Archimedean algorithm for the evaluation of π .
Frame # 26 Slide # 42
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t We now know that Archimedes’ method is valid for any
number of iterations.
It is commonly referred to, naturally enough, as the
Archimedean algorithm for the evaluation of π .
t It would yield π to a precision of 1 part 1010 in 17 iterations,
which would entail the use of 393,216-sided regular
polygons.
Frame # 26 Slide # 43
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t We now know that Archimedes’ method is valid for any
number of iterations.
It is commonly referred to, naturally enough, as the
Archimedean algorithm for the evaluation of π .
t It would yield π to a precision of 1 part 1010 in 17 iterations,
which would entail the use of 393,216-sided regular
polygons.
t Archimedes never used the symbol π in his writings. It
emerged in subsequent years and it is actually the first
letter of περιμετρoς, the Greek word for perimeter.
Frame # 26 Slide # 44
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Evaluation of π
Cont’d
t We now know that Archimedes’ method is valid for any
number of iterations.
It is commonly referred to, naturally enough, as the
Archimedean algorithm for the evaluation of π .
t It would yield π to a precision of 1 part 1010 in 17 iterations,
which would entail the use of 393,216-sided regular
polygons.
t Archimedes never used the symbol π in his writings. It
emerged in subsequent years and it is actually the first
letter of περιμετρoς, the Greek word for perimeter.
t Interest in π remained very strong through the ages. See
[History of Pi] for more information.
Frame # 26 Slide # 45
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes’ Contribution to the roots of DSP
t In his effort to calculate π , Archimedes was, in effect, the
first to apply sampling — the different polygons are
discrete approximations of the perimeter of the circle.
Frame # 27 Slide # 46
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes’ Contribution to the roots of DSP
t In his effort to calculate π , Archimedes was, in effect, the
first to apply sampling — the different polygons are
discrete approximations of the perimeter of the circle.
t In obtaining the Archimedean π , i.e., 22/7, he applied
interpolation for the first time.
Frame # 27 Slide # 47
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes’ Contribution to the roots of DSP
t In his effort to calculate π , Archimedes was, in effect, the
first to apply sampling — the different polygons are
discrete approximations of the perimeter of the circle.
t In obtaining the Archimedean π , i.e., 22/7, he applied
interpolation for the first time.
t The procedure he used to obtain progressively tighter
lower and upper bounds is in reality a recursive algorithm,
most probably the first recursive algorithm described in
Western literature.
Frame # 27 Slide # 48
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Archimedes’ Contribution to the Roots of DSP Cont’d
3.5
3.4
Estimate
3.3
3.2
π
3.1
3.0
2.9
Frame # 28 Slide # 49
1
2
3
Iteration
A. Antoniou
4
5
On the Roots of DSP: 300 BC to 1770 AD
Death of Archimedes
Frame # 29 Slide # 50
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Interpolation
t Interest in interpolation resurfaced in Europe during the
middle ages while the scientists of the time were trying to
fit curves to measured experimental data.
For example, in an attempt to characterize the orbits of
planets and other celestial objects.
Frame # 30 Slide # 51
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Interpolation
t Interest in interpolation resurfaced in Europe during the
middle ages while the scientists of the time were trying to
fit curves to measured experimental data.
For example, in an attempt to characterize the orbits of
planets and other celestial objects.
t Induction and interpolation techniques as we know them
today began to emerge during the 1600s and important
contributions were made by
– John Wallis (1616–1673)
– James Gregory (1638–1675)
– Isaac Newton (1643–1727)
Frame # 30 Slide # 52
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis
t Wallis was a clergyman but spent most of his life as an
accomplished mathematician.
Frame # 31 Slide # 53
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis
t Wallis was a clergyman but spent most of his life as an
accomplished mathematician.
t They say that he was the leading English mathematician
before Newton.
Frame # 31 Slide # 54
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t Wallis considered the area under the parabola
y = x2
to be made up of a series of elemental rectangles:
D
e
C
d
"
c
f
A
0
1
2
a
k
3
B
b
n
(a)
Frame # 32 Slide # 55
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
He noted that
Area abcfa ≈ (k ε)2 · ε = k 2 ε 3
Area abdea = (nε)2 · ε = n 2 ε 3
D
e
C
d
"
c
f
A
0
1
2
a
k
3
B
b
n
(a)
Frame # 33 Slide # 56
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t Therefore, the area under parabola AC, designated as AP ,
can be expressed in terms of the area of rectangle ABCDA,
AR , as
(02 + 12 + 22 + · · · + n 2 )ε 3
· AR
AP ≈ 2
(n + n 2 + n 2 + · · · + n 2 )ε 3
D
e
C
d
"
c
f
A
0
1
2
a
k
3
B
b
n
(a)
Frame # 34 Slide # 57
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
By applying induction, he deduced the following result:
1 1
1
02 + 12
= = +
2
2
1 +1
2
3 6
1
1
5
02 + 12 + 22
=
+
=
22 + 22 + 22
12
3 12
2
2
2
2
1
1
7
0 +1 +2 +3
= +
=
2
2
2
2
3 +3 +3 +3
18
3 18
..
.
1
1
02 + 12 + 22 + · · · + n 2
= +
2
2
2
2
n + n + n + ··· + n
3 6n
Frame # 35 Slide # 58
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t Then he did something that was never done before:
He made the base of each of the elemental rectangles
infinitesimally small and to compensate for that he made
the number of rectangles infinitely large, in today’s
language, and concluded that
(02 + 12 + 22 + · · · + n 2 )ε 3
· AR
n→∞ (n 2 + n 2 + n 2 + · · · + n 2 )ε 3
1
1
+
AR
= lim
n→∞ 3
6n
1
= AR
3
AP = lim
See [Burton, 2003] for details.
Frame # 36 Slide # 59
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t In effect, the area below the parabola is one-third the area
of the rectangle that contains the parabola.
D
e
C
d
"
c
f
A
0
1
2
a
k
3
B
b
n
(a)
Frame # 37 Slide # 60
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t Actually, the result turned out to be a trivial special case of
a result due to the great Archimedes himself, which states
that the area enclosed by parabola ABC and line DE shown
below is equal to four-thirds the area of triangle DEB.
C
A
E
D
B
See [Boyer, 1991] for details.
Frame # 38 Slide # 61
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t By finding the area under a parabola in a new way, Wallis
used the concept of infinity for the first time.
Frame # 39 Slide # 62
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t By finding the area under a parabola in a new way, Wallis
used the concept of infinity for the first time.
t He introduced the concept of the limit thereby resolving
Zeno’s Paradox once and for all.
Frame # 39 Slide # 63
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t By finding the area under a parabola in a new way, Wallis
used the concept of infinity for the first time.
t He introduced the concept of the limit thereby resolving
Zeno’s Paradox once and for all.
t He also coined the term interpolation and proposed the
symbol for infinity we use today (∞) according to historians.
Frame # 39 Slide # 64
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Zeno’s Paradoxes
t Zeno of Elea conceived many paradoxes and a typical
example is as follows.
t The arrow below must traverse half the distance to the
target before reaching the target and after that it must
traverse half of the remaining distance, and so on?
1
2
Frame # 40 Slide # 65
A. Antoniou
3
4
7
8
15
16
On the Roots of DSP: 300 BC to 1770 AD
Zeno’s Paradox Cont’d
t Therefore, the arrow will never hit the target because a
small distance to the target will always remain!
Frame # 41 Slide # 66
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Zeno’s Paradox Cont’d
t Therefore, the arrow will never hit the target because a
small distance to the target will always remain!
t For the same reason, Achilles, who was the fastest runner
in Greece, would never be able to catch up with a tortoise
that has been given a head start!
Frame # 41 Slide # 67
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Zeno’s Paradox Cont’d
t Therefore, the arrow will never hit the target because a
small distance to the target will always remain!
t For the same reason, Achilles, who was the fastest runner
in Greece, would never be able to catch up with a tortoise
that has been given a head start!
t The riddle is immediately solved by noting that an infinite
sum of numbers can have a finite value, for example
lim
n→∞
Frame # 41 Slide # 68
∞
1 n
2
= 1.0
n=1
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t What Wallis did, in effect, was to discretize the parabola by
circumscribing it in terms of a piecewise-constant function
in the same way as Archimedes had discretized the circle
by circumscribing it by an n-sided polygon.
D
e
C
d
"
c
f
A
0
1
2
a
k
3
B
b
n
(a)
Frame # 42 Slide # 69
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
John Wallis Cont’d
t He could have achieved the same result by inscribing a
piecewise-constant function in the parabola as shown
below, which is quite analogous to a continuous-time signal
that has been subjected to the sample-and-hold operation.
D
e
C
d
"
c
f
A
0
1
2
a
k
3
B
b
n
(b)
Frame # 43 Slide # 70
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
James Gregory
t James Gregory (1638–1675), a Scot mathematician,
extended the results of Archimedes on the area of the
circle to the area of the ellipse [Boyer, 1991].
Frame # 44 Slide # 71
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
James Gregory Cont’d
t He inscribed a triangle of area a0 in the ellipse and
circumscribed the ellipse by a quadrilateral of area A0 , as
shown.
Quadrilateral
Triangle
Frame # 45 Slide # 72
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
James Gregory Cont’d
t By successively doubling the number of sides of the
triangles and quadrilaterals, he generated the sequence
a0 , A0 , a1 , A1 . . . an , An , . . .
using the recursive relations
an =
Frame # 46 Slide # 73
2An−1 an
an−1 An−1 and An =
An−1 + an
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
James Gregory Cont’d
t By successively doubling the number of sides of the
triangles and quadrilaterals, he generated the sequence
a0 , A0 , a1 , A1 . . . an , An , . . .
using the recursive relations
an =
2An−1 an
an−1 An−1 and An =
An−1 + an
t Then he arranged the elements of the sequence obtained
into two sequences, as follows:
a0 , a1 , . . . an , . . .
Frame # 46 Slide # 74
A. Antoniou
and A0 , A1 , . . . An , . . .
On the Roots of DSP: 300 BC to 1770 AD
James Gregory Cont’d
t He concluded that each of the two sequences would, in his
words, converge to the area of the ellipse if n were made
infinitely large.
Frame # 47 Slide # 75
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
James Gregory Cont’d
t He concluded that each of the two sequences would, in his
words, converge to the area of the ellipse if n were made
infinitely large.
t Although he died is his thirties, he is known for several
other achievements:
– He is known for his work on series.
In fact,
x
0
x5
x7
1
x3
+
−
+ ···
dx = tan−1 x = x −
2
3
5
7
1+x
is known as the Gregory series [Boyer, 1991].
– He is known along with Newton for the Gregory-Newton
interpolation formula.
– Interestingly, he discovered the Taylor series, some 44
years before it was published by Brook Taylor (1685–1731).
Frame # 47 Slide # 76
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton
t The contributions of Newton to mathematics and science in
general are numerous, diverse, and well known.
Frame # 48 Slide # 77
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton
t The contributions of Newton to mathematics and science in
general are numerous, diverse, and well known.
t His most important contribution to the roots of DSP other
than calculus is the binomial theorem.
Frame # 48 Slide # 78
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton
t The contributions of Newton to mathematics and science in
general are numerous, diverse, and well known.
t His most important contribution to the roots of DSP other
than calculus is the binomial theorem.
t He started with the numerical values of the area
0
1
(1 − t 2 )n dt
(in today’s notation) for certain integer values of n, which
were estimated by Wallis a few years earlier using an
induction method (recall that there was no calculus at that
time).
Frame # 48 Slide # 79
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
···
0
1
(1 − t 2 )n dt
t By replacing the upper limit in the integration shown by x ,
he was able to obtain the following results:
x
(1 − t 2 ) dt = x − 13 x 3
0
x
(1 − t 2 )2 dt = x − 23 x 3 + 15 x 5
0 x
(1 − t 2 )3 dt = x − 33 x 3 + 35 x 5 − 17 x 7
0
Frame # 49 Slide # 80
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
···
x
0
x
0 x
0
(1 − t 2 ) dt = x − 13 x 3
(1 − t 2 )2 dt = x − 23 x 3 + 15 x 5
(1 − t 2 )3 dt = x − 33 x 3 + 35 x 5 − 17 x 7
Then through some laborious interpolation he found out that
x
0
Frame # 50 Slide # 81
1
(1 − t 2 ) 2 dt = x −
A. Antoniou
1
2
3
x3 −
1
8
5
x5 − · · ·
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
t The amazing regularity of his solutions led him to conclude
that
x
0
where
Frame # 51 Slide # 82
k 3 1 k 5
(1 − t ) dt = x −
x +5
x − ···
1
2
k 2n+1
1
x
− ···
+
2n + 1 n
2 k
1
3
k
k (k − 1) · · · (k − n + 1)
=
n
n
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
···
x
0
k 3 1 k 5
(1 − t ) dt = x −
x +5
x − ···
1
2
k 2n+1
1
x
− ···
+
2n + 1 n
2 k
1
3
Good as he was with the method of tangents (differentiation),
he differentiated both sides to obtain
k 2
k 4
2 k
x +
x − ···
(1 − x ) = 1 −
1
2
k 2n
+
x − ···
n
Frame # 52 Slide # 83
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
···
k 2
k 4
(1 − x ) = 1 −
x +
x − ···
1
2
k 2n
+
x − ···
n
2 k
Finally, if we replace −x 2 by x , the binomial series in its
standard form is revealed:
k
k 2
k
x+
x + ···
(1 + x ) = 1 +
1
2
k n
+
x + ···
n
See [Burton, 2003]
Frame # 53 Slide # 84
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
t The binomial series for integer values of n was known long
before Newton in terms of the Pascal triangle but it was not
discovered by Pascal.
Frame # 54 Slide # 85
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
t The binomial series for integer values of n was known long
before Newton in terms of the Pascal triangle but it was not
discovered by Pascal.
t It first appeared in a treatise written by a Chinese
mathematician by the name of Chu Shih-chieh (circa
1260-1320).
1
1
..
.
Frame # 54 Slide # 86
Pascal Triangle
1
1
1
1
2
1
3
3
4
6
5
10
10
..
.
A. Antoniou
1
1
4
1
5
1
..
.
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
t The binomial series was investigated by many others after
Newton.
Frame # 55 Slide # 87
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
t The binomial series was investigated by many others after
Newton.
t The great Gauss (1777-1855) generalized its application to
the case where n assumes arbitrary rational values.
Frame # 55 Slide # 88
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
t The binomial series was investigated by many others after
Newton.
t The great Gauss (1777-1855) generalized its application to
the case where n assumes arbitrary rational values.
t The work of Euler (1707-1783), Gauss, Cauchy
(1789-1857), and Laurent (1813-1854) on functions of a
complex variable has shown that the binomial theorem is
also applicable to the case where x is a complex variable.
Frame # 55 Slide # 89
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Newton Cont’d
t The binomial series was investigated by many others after
Newton.
t The great Gauss (1777-1855) generalized its application to
the case where n assumes arbitrary rational values.
t The work of Euler (1707-1783), Gauss, Cauchy
(1789-1857), and Laurent (1813-1854) on functions of a
complex variable has shown that the binomial theorem is
also applicable to the case where x is a complex variable.
Note: Some say that Gauss ‘discovered’ the binomial theorem at
the age of 15 without knowledge of Newton’s work.
Frame # 55 Slide # 90
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
The Binomial Theorem in DSP
t If we replace variable x in the binomial series by z −1 and
allow z to be a complex variable, then we get
k −1
k −2
−1 k
z +
z + ···
(1 + z ) = 1 +
1
2
k −n
+
z + ···
n
which is referred to in the DSP literature as the z transform
of right-sided signal
k
x (nT ) = u(nT )
n
where u(nT ) is the discrete-time unit-step function.
Frame # 56 Slide # 91
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
The Binomial Theorem in DSP
t Now if we expand the function
X (z) =
Kz m
(z − w )k
into a binomial series, where m and k are integers, and K
and w are real or complex constants, a whole table of z
transform pairs can be deduced [Antoniou, 2005].
Frame # 57 Slide # 92
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Table of Standard z Transforms
x (nT )
u(nT )
u(nT − kT )K
u(nT )Kw n
u(nT − kT )Kw n−1
u(nT )e −αnT
u(nT )nT
u(nT )nTe −αnT
Frame # 58 Slide # 93
A. Antoniou
X (z)
z
z −1
Kz −(k −1)
z −1
Kz
z −w
K (z/w )−(k −1)
z −w
z
z − e −αT
Tz
(z − 1)2
Te −αT z
(z − e −αT )2
On the Roots of DSP: 300 BC to 1770 AD
Interpolation
The interpolation process was explored by many since the time
of Newton:
t James Stirling (1692–1770) contributed to interpolation
and added to the work of Newton on cubic curves.
Frame # 59 Slide # 94
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Interpolation
The interpolation process was explored by many since the time
of Newton:
t James Stirling (1692–1770) contributed to interpolation
and added to the work of Newton on cubic curves.
t Joseph-Louis Lagrange (1736–1813) made important
contributions to astronomy, number theory, and calculus.
The so-called barycentric form of the Lagrange
interpolation formula is used to facilitate the application of
the Remez algorithm for the design of nonrecursive (FIR)
filters.
Frame # 59 Slide # 95
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Interpolation
The interpolation process was explored by many since the time
of Newton:
t James Stirling (1692–1770) contributed to interpolation
and added to the work of Newton on cubic curves.
t Joseph-Louis Lagrange (1736–1813) made important
contributions to astronomy, number theory, and calculus.
The so-called barycentric form of the Lagrange
interpolation formula is used to facilitate the application of
the Remez algorithm for the design of nonrecursive (FIR)
filters.
t Interpolation formulas were also proposed by Carl Fredrich
Gauss (1777–1855) and Wilhelm Bessel (1784–1846).
Frame # 59 Slide # 96
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolation Formula
If the values of a discrete-time signal x (nT ) are known at 0, T ,
2T , . . ., then the value of x (nT + pT ) for some value of p in the
range 0 < p < 1 can be determined as
p 2 2 p 2 (p 2 − 1) 4
δ + · · · x (nT )
x (nT + pT ) = 1 + δ +
2
4
p
+ δx (nT − 12 T ) + δx (nT + 12 T )
2
p(p 2 − 1) 3
δ x (nT − 12 T ) + δ 3 x (nT + 12 T )
+
2(3)
p(p 2 − 1)(p 2 − 22 ) 5
δ x (nT − 12 T )
+
2(5)
+δ 5 x (nT + 12 T ) + · · ·
where
δx (nT + 12 T ) = x (nT + T ) − x (nT )
is known as the central difference.
Frame # 60 Slide # 97
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolation Formula Cont’d
Neglecting differences of order 6 or higher, letting p = 1/2 in
the interpolation formula, and then eliminating the central
differences, we get (see [Antoniou, 2005] for details)
y (nT ) = x (nT + 12 T ) =
3
h(iT )x (nT − iT )
i =−3
Frame # 61 Slide # 98
i
h(iT )
−3
−2
−1
0
1
2
3
−5.859375E−3
4.687500E−2
−1.855469E−1
7.031250E−1
4.980469E−1
−6.250000E−2
5.859375E−3
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolation Formula Cont’d
The formula
y (nT ) = x (nT + 12 T ) =
3
h(iT )x (nT − iT )
i =−3
is a difference equation that represents a nonrecursive
discrete-time system which can perform interpolation:
x(nT)
Frame # 62 Slide # 99
Interpolation
System
A. Antoniou
y(nT)
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolation Formula Cont’d
x(t)
x(nT)
0 1 2 3
x(t)
y(nT)
0 1 2 3
Frame # 63 Slide # 100
nT
A. Antoniou
nT
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolation Formula Cont’d
t Interpolation is a process that will fit a smooth curve
through a number of sample points.
In effect, interpolation is essentially lowpass filtering.
y(nT)
nT
Frame # 64 Slide # 101
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolator
t To check this out, we obtain the transfer function of the
interpolator as
3
Y (z)
=
h(iT )z −k
H (z) =
X (z)
k =−3
by applying the z transform to the difference equation.
Frame # 65 Slide # 102
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolator
t To check this out, we obtain the transfer function of the
interpolator as
3
Y (z)
=
h(iT )z −k
H (z) =
X (z)
k =−3
by applying the z transform to the difference equation.
t Like other methods for the design of nonrecursive systems,
Stirling’s interpolation formula gives a noncausal system.
However, a causal interpolator can be obtained by
multiplying the transfer function by z −3 which amounts to
introducing a delay of 3 sampling periods.
Hence, we get
H (z) =
3
Y (z)
= z −3
h(iT )z −k
X (z)
k =−3
Frame # 65 Slide # 103
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolator
Cont’d
x(nT)
h(3T)
h(2T)
h(T)
h(0)
h(-T)
h(-2T)
h(-3T)
y(nT)
Frame # 66 Slide # 104
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Stirling Interpolatior Cont’d
t Evaluating the transfer function on the unit-circle of the z
plane, we get the amplitude response, phase response,
and group delay as
6
−jk ωT h(iT )e
Mc (ω) = i =0
θc (ω) = −3ω + arg
3
h(iT )e −jk ωT
i =−3
d θc (ω)
τc (ω) = −
dω
respectively.
Frame # 67 Slide # 105
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Amplitude Response of Stirling Interpolator
1.1
1.0
0.9
M(ω)
0.8
0.7
0.6
0.5
0.4
0
Frame # 68 Slide # 106
0.5
1.0
1.5
2.0
ω, rad/s
(a)
A. Antoniou
2.5
3.0
3.5
On the Roots of DSP: 300 BC to 1770 AD
Phase Response of Stirling Interpolator
0
θc(ω), rad
−2
−4
−6
−8
−10
0
Frame # 69 Slide # 107
0.5
1.0
1.5
2.0
ω, rad/s
(b)
A. Antoniou
2.5
3.0
3.5
On the Roots of DSP: 300 BC to 1770 AD
Group-Delay Characteristic of Stirling Interpolator
3.5
3.0
τc(ω), s
2.5
2.0
1.5
1.0
0.5
0
Frame # 70 Slide # 108
0.5
1.0
1.5
2.0
ω, rad/s
(c)
A. Antoniou
2.5
3.0
3.5
On the Roots of DSP: 300 BC to 1770 AD
Group-Delay Characteristic of Stirling Interpolator Cont’d
t As anticipated, the Stirling interpolator is a nonrecursive
lowpass digital filter.
t In fact it has nearly linear phase or constant group delay
with respect a fairly well-defined passband.
Frame # 71 Slide # 109
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Conclusions
t It has been demonstrated that the basic processes of DSP,
namely, discretization (or sampling) and interpolation have
been part of mathematics in one form or another since
classical times.
Frame # 72 Slide # 110
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Conclusions
t It has been demonstrated that the basic processes of DSP,
namely, discretization (or sampling) and interpolation have
been part of mathematics in one form or another since
classical times.
t By introducing the concepts of infinity, the limit, and
convergence and then extending the classical methods of
Archimedes, Wallis and Gregory rendered the emergence
of calculus almost inevitable.
Frame # 72 Slide # 111
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Conclusions
t It has been demonstrated that the basic processes of DSP,
namely, discretization (or sampling) and interpolation have
been part of mathematics in one form or another since
classical times.
t By introducing the concepts of infinity, the limit, and
convergence and then extending the classical methods of
Archimedes, Wallis and Gregory rendered the emergence
of calculus almost inevitable.
t In addition to consolidating the methods of tangents and
quadrature under the unified theory of calculus, Newton
discovered the binomial theorem which can be deemed to
be the z transform of a certain class of signals.
Frame # 72 Slide # 112
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Conclusions Cont’d
t Stirling discovered in the 1700s an interpolation method
that can be used to design nonrecursive filters which were
not invented until the 1960s.
The same method can be used to design differentiators
and integrators.
Frame # 73 Slide # 113
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
Conclusions Cont’d
t Stirling discovered in the 1700s an interpolation method
that can be used to design nonrecursive filters which were
not invented until the 1960s.
The same method can be used to design differentiators
and integrators.
t In short, mathematical discoveries made since the early
1600s are very much a part of the toolbox of a modern
DSP practitioner.
Frame # 73 Slide # 114
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
References
Antoniou, A. Digital Signal Processing: Signals, Systems, and
Filters, McGraw-Hill, 2005.
Antoniou, A. “On the roots of digital signal processing – Part I,”
IEEE Trans. Circuits and Syst., vol. 7, no. 1, pp. 8–18.
Archimedes Screw, http://en.wikipedia.org/wiki/
Archimedes’_screw
A History of Pi, http://www-history.mcs.st-and.ac.uk/HistTopics/
Pi_through_the_ages.html
Boyer, C. B. A History of Mathematics, Wiley, New York, 1991.
Burton, D. M. The History of Mathematics, An Introduction, Fifth
Edition, McGraw-Hill, New York, 2003.
Indexes of Biographies, The MacTutor History of Mathematics
Archive, School of Mathematics and Statistics, University of
St. Andrews, Scotland. http://turnbull.mcs.st-and.ac.uk/˜history/
BiogIndex.html
Frame # 74 Slide # 115
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD
This slide concludes the presentation.
Thank you for your attention.
Frame # 75 Slide # 116
A. Antoniou
On the Roots of DSP: 300 BC to 1770 AD