Download 1 Introduction to Fourier analysis 2 Periodic functions, continuous

1 Introduction to Fourier analysis
We consider a signal f (x) where x is time. In applications we may e.g. be given the function values f (x j ) for points x j = jh
with a step size h. Describing a signal in terms of function values is called a representation in the time domain.
However, we can also consider a signal f (x) as a superposition of terms with different frequencies ξ . The functions
sin(2πξ x)
cos(2πξ x),
have freqency ξ since we have cos 2πξ (x + ξ1 ) = cos(2πξ + 2π) = cos(2πξ ). We will use the functions u(ξ ) (x) defined
by
u(ξ ) (x) := e2π iξ x = cos(2πξ x) + i sin(2πξ x)
and use complex coefficients. This will make the formulas simpler, but we will get complex coefficients for a real signal. If
we can express the signal f (x) as a superposition of terms u(ξ ) (x) we call this a representation in the frequency domain.
The “Fourier transform” takes us from the time domain to the frequency domain. The “inverse Fourier transform” takes us
from the frequency domain to the time domain.
We will consider four different settings for the “Fourier transform”:
time domain
frequency domain
1-periodic function, continuous time
f (x), x ∈ [0, 1)
fˆk , k ∈ Z
1-periodic function, discrete time
f j for x j = Nj , j = 0, . . . , N − 1
fˆk , k = 0, . . . , N − 1
Nonperiodic function, discrete time
f j for x j = jh, j ∈ Z
Nonperiodic function, continuous time
f (x), x ∈ R
1 1
, 2h
fˆ(ξ ), ξ ∈ − 2h
fˆ(ξ ), ξ ∈ R
Warning: There are different conventions for defining “Fourier√transforms” in use in books and mathematical software:
Some people use eiξ x instead of e2π iξ x , and different factors 2π or 2π are used.
Mathematically speaking,
√ all these definitions are equivalent, as one can go from one convention to a different one by
inserting factors 2π or 2π in appropriate places.
In practice one has to be careful e.g. when using software like Matlab, Mathematica, Maple: You have to check the precise
definition of “Fourier transform” which the software uses.
In this class I will always represent f (x) as a superposition of terms fˆ(ξ ) · e2π iξ x or fˆk · e2π ikx . I will explain how to use
Matlab which uses different definitions.
We will first consider functions with period 1, i.e., f (x + 1) = f (x) for all x. Once we understand this case, it is easy to
generalize this to functions with period L, i.e., f (x + L) = f (x):
time domain
L-periodic function, continuous time
f (x), x ∈ [0, L)
L-periodic function, discrete time
f j for x j = Nj L, j = 0, . . . , N − 1
frequency domain
k
fˆk for ξk = , k ∈ Z
L
k
fˆk for ξk = , k = 0, . . . , N − 1
L
2 Periodic functions, continuous time: Fourier series
2.1 Vector spaces, span, inner product
“Vector space” means that we specify a set of vectors, and a set of scalars. Vectors can be added or multiplied by scalars.


u1


Example: Cn is a vector space: vectors have the form u =  ...  with u j ∈ C, and scalars are numbers in C.
un
For vectors u(1) , . . . , u(N) and scalars c1 , . . . , cN the linear combination v = c1 u(1) + · · · + cN u(N) is again a vector.
If c1 u(1) + · · · + cN u(N) can only be the zero vector for c1 = · · · = cN = 0 we say that the vectors u(1) , . . . , u(N) are linearly
independent.
We denote the set of all linear combinations by
o n
o
n
span u(1) , . . . , u(N) = c1 u(1) + · · · + cN u(N) | c j ∈ C
For the vector space Cn we introduce the inner product
n
(u, v) = ∑ ui vi
i=1
Here vi denotes the complex conjugate: For a complex number z = x + iy we have z = x − iy.
The norm of a vector is defined as
kuk = (u, u)1/2 .
q 2
2
For a vector u ∈ Cn this means that kuk = u1 + · · · + un .
We say that two vectors u, v are orthogonal if (u, v) = 0.
2.2 Orthogonal projection on a subspace V
If the vectors u(1) , . . . , u(N) are orthogonal on each other, i.e.,
u(k) , u(l) = 0
for k 6= l
we say that they form an orthogonal basis of the subspace
o
n
V = span u(1) , . . . , u(N) .
For a given vector w we now want to find the vector v ∈ V which is closest to w, i.e., kw − vk should be minimal. In this case
we must have that the difference vector w − v is orthogonal on the subspace V , i.e.,
for k = 1, . . . , N.
(1)
w − v, u(k) = 0
We want to find the coefficients c1 , . . . , cN of the vector v = c1 u(1) + · · · + cN u(N) . Plugging this into (1) gives using the
orthogonality of the vectors u(1) , . . . , u(N) that
w, u(k)
ck = (k) (k) .
(2)
u ,u
For a given vector w we can find the projection v as follows:
1. Compute the coefficients ck =
2. Let v = c1 u(1) + · · · + cN u(N) .
(w,u(k) )
for k = 1, . . . , N
(u(k) ,u(k) )
Claim: This vector v gives the minimal error kw − vk among all v ∈ V :
Proof: Assume that we use a vector v + d with some d ∈ V instead of v:
kw − (v + d)k2 = (w − v − d, w − v − d) = (w − v, w − v) − (w − v, d) − (d, w − v) +(d, d)
| {z } | {z }
0
0
as w − v is orthogonal on all vectors in V . Hence we have for nonzero d
kw − (v + d)k2 = kw − vk2 + kdk2 > kw − vk2 ,
|{z}
(3)
0
i.e., the error of v + d is strictly larger than the error of v.
Since w − v is orthogonal on v we have the Pythogoras equation
(w, w) = (v + (w − v), v + (w − v)) = (v, v) + (w − v, v) + (v, w − v) +(w − v, w − v)
| {z } | {z }
0
0
kwk2 = kvk2 + kw − vk2
Since v = c1 u(1) + · · · + cN u(N) with orthogonal vectors u(1) , . . . , u(N) we have
2
2
kvk2 = |c1 |2 u(1) + · · · + |cN |2 u(N) and hence
2
2
2 (N) kwk = |c1 | u + · · · + |cN | u + kw − vk2 .
2
2 (1) (4)
(5)
This equation can be used to compute kw − vk. Since kw − vk ≥ 0 we obviously have kwk ≥ kvk and
2
2
kwk2 ≥ |c1 |2 u(1) + · · · + |cN |2 u(N) .
2.3 Orthonormal basis u(1) , . . . , u(N)
If the orthogonal vectors u(1) , . . . , u(N) have all length 1, i.e.
u(k) , u(l) =
(
0
1
for k 6= l
for k = l
some of the formulas from the previous section become simpler:
The coefficients ck of the projection v = c1 u(1) + · · · + cN u(N) are given by
ck = w, u(k)
(6)
and equation (5) becomes
kwk2 = |c1 |2 + · · · + |cN |2 + kw − vk2 .
(7)
2.4 Functions as vectors
Functions also form a vector space: E.g., all continuous functions on the interval [−1, 1] form a vector space (since we can
add vectors and multiply vectors by scalars). We now consider complex valued functions, with complex scalars.
We can define an inner product as follows: For functions u(x), v(x) on [−1, 1] let
(u, v) =
Z 1
−1
u(x)v(x) dx.
Then we have the norm
kuk2 = (u, u) =
Z 1
−1
|u(x)|2 dx.
R
1
|u(x)|2 dx is finite. The
We can also allow more general functions u as vectors (e.g. piecewise continuous), as long as −1
vector space of these functions (where the integrals exist in the sense of Lebesgue) is called L2 ([−1, 1]), and the norm kuk is
called L2 -norm.
For functions u(1) , . . . , u(N) which are orthogonal on each other we consider again the subspace
n
o
V = span u(1) , . . . , u(N) .
For a given function w we want to find the best approximation v = c1 u(1) + · · · + cN u(N) ∈ V , in the sense that the error
kw − vk2 =
Z 1
−1
|w(x) − v(x)|2 dx
becomes minimal (“least squares approximation”). We can find v as follows:
1. Compute the coefficients ck =
(w,u(k) )
for k = 1, . . . , N
(u(k) ,u(k) )
2. Let v = c1 u(1) + · · · + cN u(N) .
(2)
(1) (2)
Example: Let u(1) (x) =
1 and u (x) = x. Then the subspace V = span{u , u } consists of all linear polynomials. Note
(1)
(2)
that we have u , u
= 0, i.e., we have an orthogonal basis.
For the given function w(x) = x2 find the best least-squares approximation with a function in V :
We first compute the coefficients
R1 2
w, u(1)
x dx 2/3 1
= ,
c1 = (1) (1) = R−1
=
1
2
3
u ,u
−1 1dx
and obtain
R1 3
w, u(2)
x dx
0
c2 = (2) (2) = R−1
=0
=
1
2
2/3
u ,u
−1 x dx
v(x) = c1 u(1) (x) + c2 u(2) (x) =
1
1
·1+0·x =
3
3
so the best approximation is the constant function v(x) = 13 . We now want to use (5) to compute the approximation error
kw − vk:
2
2
kw − vk2 = kwk2 − |c1 |2 u(1) − |c2 |2 u(2) 2
2
Z 1
2
1
· 2 − 02 ·
x2 · x2 dx −
=
3
3
−1
2 2
8
= − =
5 9 45
We obtain kw − vk =
q
8
45 .
2.5 The space TN of trigonometric functions
We are now interested in periodic functions: We say a function is periodic with period L if
f (x + L) = f (x)
for all x ∈ R.
We first consider the case of functions with period L = 1 (then the case with general period L will follow easily). We assume
we have a “signal” f (x) with
f (x + 1) = f (x)
for all x ∈ R.
In order to approximate this signal we want to use trigonometric functions with period 1: We want to use
v(x) =a0 · 1
+ a1 cos(2πx) + b1 sin(2πx)
+···
+ aN cos(2πNx) + bN cos(2πNx).
(8)
We denote the space of these functions by TN (trigonometric polynomials with frequency up to N). Note that v(x) is a sum
of terms with different frequencies:
• 1 has frequency 0
• cos(2πx) and sin(2πx) have frequency 1
.
• ..
• cos(2πNx) and sin(2πNx) have frequency N.
We consider the case of a complex valued signal, so we have ak , bk ∈ C.
Recall that
eix = cos x + i sin x
(9)
implying
eix + e−ix
2
eix − e−ix
.
sin x =
2i
cos x =
(10)
(11)
Using (10), (11) in (8) gives the function v(x) in the form
v(x) = fˆ0 · 1
+ fˆ−1 e−2π ix + fˆ1 e2π ix
+···
+ fˆ−N e−2π iNx + fˆN e2π iNx .
(12)
From this form of v(x) we can get to (8) by using (9).
Note that in (12) the function v(x) is written as a sum of terms with different frequencies:
• 1 has frequency 0
• e−2π ix and e2π ix have frequency 1
.
• ..
• e−2π iNx and e2π iNx have frequency N.
Note that for a real-valued function f (x) the coefficients fˆ−N , . . . , fˆN in (12) are in general complex, but the coefficients ak , bk
in (8) are real.
We use the form (12) for finding v(x). We can then easily convert this to the form (8). Therefore we use the functions
u(k) (x) = e2π ikx
for k ∈ Z
and have
TN = span{u(−N) , u(−N+1) , . . . , u(N) }.
Note that we have u(l) = cos(2πkx) − i sin(2πikx) = e−2π ilx and hence
(k)
u ,u
(l)
=
Z 1
0
e
2π ikx −2π ilx
e
dx =
(
0
1
for k 6= l
for k = l
This means that the functions u(−N) , u(−N+1) , . . . , u(N) form an orthonormal basis of the space TN .
For a given function f (x) we denote the best approximation in the space TN by f(N) (x) (instead of v(x)):
N
fˆk e2π ikx
∑
f(N) (x) =
k=−N
The coefficients fˆk are called the Fourier coefficients of the function f . Since we have an orthonormal basis we can use (6)
to compute them:
Z1
fˆk = f , u(k) =
f (x)e−2π ix dx.
0
Since the functions are 1-periodic we can also use
The equation (7) is here
R 1/2
−1/2 · · ·
(or any other interval of length 1).
2 2
2
k f k2 = fˆ−N + · · · + fˆN + f − f(N) which can be used to find f − f(N) . Because of f − f(N) ≥ 0 we have
k f k2 ≥
N
∑
k=−N
(13)
2
fˆk .
Since this is true for any N = 0, 1, 2, 3, . . . we have the Bessel inequality
∞
k f k2 ≥
∑
k=−∞
2
fˆk 2
2
ˆ
where the sum S = ∑∞
k=−∞ f k ≤ k f k converges since it is bounded from above. Hence we have
f − f(N) → k f k2 − S as N → ∞.
(14)
We have S ≤ k f k2 . Is S = k f k2 ? If yes, the Fourier series f(N) converges to f as N → ∞.
This also shows that we must have fˆk → 0 as k → ∞ or as k → −∞, i.e., the Fourier coefficients decay to zero as the frequency
|k| increases.
2.6 Example: Fourier series and convergence
Consider the 1-periodic function f (x) with
f (x) =
(
−1 for x ∈ [− 21 , 0)
1
for x ∈ [0, 21 )
(15)
and find the best approximation f(5) (x) in the space T5 .
We need to compute the Fourier coefficients fˆk = f , u(k) . We start with fˆ0 : Since u(0) (x) = 1 we get
fˆ0 =
Now we consider k 6= 0:
Z 1/2
−1/2
fˆk =
f (x) · 1dx =
Z 1/2
−1/2
f (x)dx =
Z 0
−1/2
Z 0
−1/2
(−1)dx +
Z 1/2
0
−2π ikx
(−1)e
1dx = − 21 + 21 = 0.
dx +
Z 1/2
0
1 · e−2π ikx dx
For the second integral we get
Z 1/2
0
e
−2π ikx
e−2π ikx
dx =
−2πik
1/2
x=0
=
−1 −2π ik/2
−1
e
2πik
(16)
We have for integer k that
e−2π ik/2 =
hence
Z 1/2
0
e−2π ikx dx =
(
1
−1
for even k
for odd k
(
0
−1
2π ik (−2)
=
− πik
for even k
for odd k
For the first integral in (16) we proceed in the same way and get exactly the same value. Therefore we obtain the Fourier
coefficients
(
0
for even k ∈ Z
fˆk =
2i
− π k for odd k ∈ Z
and the best approximation f(5) ∈ T5 is
1 6π ix 1 10π ix
1 −10π ix 1 −6π ix
2i
−2π ix
2π ix
f(5) (x) = −
− e
+e
+e
+ e
+ e
.
− e
π
5
3
3
5
Note that we have terms with frequency 1, 3 and 5. We now want to convert this to the form (8): E.g., for the terms of
frequency 3 we get with (9)
2i
1 −6π ix 1 6π ix
i
4
i
2i
−
− e
sin(6πx)
· sin(6πx) + sin(6πx) =
+ e
=− ·
π
3
3
π
3
3
3π
since the cosine terms cancel. Therefore we get
1
1
4
f(5) (x) =
sin(2πx) + sin(3 · 2πx) + sin(5 · 2πx)
π
3
5
For a general odd N we have
4
f(N) (x) =
π
1
1
1
1
sin(1 · 2πx) + sin(3 · 2πx) + sin(5 · 2πx) + · · · + sin(N · 2πx)
1
3
5
N
In order to compute this in Matlab we write an m-file fseries.m:
function y = fseries(x,N)
% Find f_(N)(x) where x is a vector
y = zeros(size(x));
for k=1:2:N
y = y + sin(k*2*pi*x)/k;
end
y = y*4/pi;
We can then plot the functions f (x), f(5) (x) and f(11) (x) on the interval [− 21 , 21 ] together:
x = -.5:.001:.5;
% points for plotting
plot(x,sign(x),x,fseries(x,5),x,fseries(x,11))
legend(’f’,’f_{(5)}’,’f_{(11)}’)
1.5
f
f(5)
1
f(11)
0.5
0
−0.5
−1
−1.5
−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
We now want to find the error f − f(N) by using (13). Here we have
k f k2 =
and for odd N
N
∑
k=−N
Hence
2
fˆk =
Z 1/2
−1/2
| f (x)|2 dx =
Z 1/2
−1/2
0.3
0.4
0.5
1dx = 1
2 1
1
1
1
2
2 2 + 2 + 2 +···+ 2 .
π
1
3
5
N
s
8
1
1
1
≈ 0.25874
1− 2
+ +
π
12 32 52
s
8
1
1
1
1
f − f(11) = 1 −
+ + + · · · + 2 ≈ 0.18357
π 2 12 32 52
11
f − f(5) =
What will happen as N tends to infinity? One can show that
toolbox for this:
1
12
+ 312 + 512 + · · · = π 2 /8, we can use e.g. the Matlab symbolic
>> syms k; symsum(1/(2*k-1)^2,k,1,Inf)
ans =
pi^2/8
Hence
∞
S=
so we get from (14)
2
8
∑ fˆk = π 2
k=−∞
1
1
1
=1
+
+
+
·
·
·
12 32 52
{z
}
|
2
π /8
f − f(N) → k f k2 − S = 1 − 1 = 0
as N → ∞
which means that the Fourier series
∞
4
sin(k · 2πx)
fˆk e2π ikx =
∑
π
k
k=−∞
k=1...∞
∑
(17)
k odd
converges to the function f in the mean square sense.
This does not mean that we have “pointwise convergence”: According to (15) we have f (0) = 1, but plugging in x = 0 in
(17) gives 0.
2.7 Convolutions
Convolutions can be used for signal processing. We can e.g. “smoothen” or “sharpen” a signal using convolutions.
For 1-periodic functions f , g we define the convolution q = f ∗ g by
q(x) =
Z 1
t=0
f (t)g(x − t)dt
(18)
Properties of the convolution:
• q is also 1-periodic
• f ∗ g = g ∗ f since using the change of variable s = x − t in (18) gives t = x − s , dt = −ds
q(x) =
Z x−1
s=x
f (x − s)g(s)(−ds) =
(as the integrand is 1-periodic in s, we have
Z x
s=x−1
Rx
s=x−1 (· · · )ds
=
f (x − s)g(s)ds =
Z 1
s=0
f (x − s)g(s)ds
R1
s=0 (· · · )ds.)
• ( f ∗ g)′ = f ′ ∗ g = f ∗ g′ : Assume that f ′ exists and is continuous, g is continuous. Then we can take the derivative
in (18) inside the integral:
′
q (x) =
Z 1
t=0
f (t)g′ (x − t)dt = f ∗ g′ .
• If g ∈ Tn then also q = f ∗ g ∈ Tn : For g(x) = ∑Nk=−N ĝk e2π ikx we get
#
"
Z
Z
N
1
q(x) =
f (t)
t=0
∑
ĝk e
2π ik(x−t)
N
dt =
k=−N
∑
k=−N |
with
d
dx
1
t=0
−2π ikt
f (t)e
{z
fˆk
N
dt ĝk e2π ikx = ∑ q̂k e2π ikx
k=−N
}
q̂k = fˆk ĝk
(19)
A simple example of a convolution is smoothing with a “moving average”: Define the 1-periodic function gn for x ∈ [− 21 , 21 )
by
(
1 1
, 2n )
n for x ∈ (− 2n
(20)
gn (x) =
0 otherwise
Note that
R 1/2
−1/2 gn (x)dx
= 1. For a 1-periodic function f the convolution qn = f ∗ gn gives
qn (x) = n
Z x+1/(2n)
f (t)dt
t=x−1/(2n)
1
1
which is the average of the function f over the interval [x − 2n
, x + 2n
].
If the function f is continuous this average will converge to f (x) as n → ∞.
Proof: For any ε > 0 there exists δ > 0 so that |y − x| < δ implies | f (y) − f (x)| < ε. Hence we have for n ≥ 1/δ that
R x+1/(2n)
1
1
| f (t) − f (x)| < ε for all t ∈ [x − 2n
, x + 2n
]. Then qn (x) − f (x) = n t=x−1/(2n) ( f (t) − f (x)) dt gives
|qn (x) − f (x)| ≤ n
Z x+1/(2n)
| f (t) − f (x)| ≤ ε
{z
}
≤ε
t=x−1/(2n) |
The functions gn become more and more concentrated near 0: We have for n = 1, 2, 3, . . .
1. gn (x) ≥ 0
2.
R 1/2
−1/2 gn (x)dx
=1
3. for every δ > 0: max[− 1 , 1 ]\[−δ ,δ ] gn (x) → 0 as n → ∞
2 2
The “limit” of gn for n → ∞ is something which is nonzero only for x = 0, but still has integral 1. This object is called δ . It is
not a function,
but a so-called generalized function. This means that pointwise evaluation δ (x) does not really make sense.
R
However, f (x)δ (x)dx still makes sense and gives f (0). For the 1-periodic function given by g(x) = δ (x) for x ∈ [− 12 , 21 ]
we can formally find the Fourier coefficients
ĝk =
Z 1/2
−1/2
Formally we have gn → δ as n → ∞ and
δ (x)e−2π ikx dx = e−2π i·0 = 1
f ∗ gn → f ∗ δ = f
for all k.
as n → ∞
So convolution with the Delta function gives the original function. Recall that four the Fourier coefficients we have for
q = f ∗ g that q̂k = fˆk ĝk , and we say that for the periodic delta function g we have ĝk = 1 for all k.
2.8 Completeness of the trigonometric basis u(k) , k ∈ Z
For a function f (x) with
R1
0
| f (x)|2 dx < ∞ we can define
fˆk : =
Z 1
f(N) (x) =
∑
0
N
f (x)e−2π ikx dx
fˆk e2π ikx
k=−N
The big question is: Do we have
i.e., does the infinite Fourier series
f − f(N) → 0
as N → ∞,
∞
∑
fˆk e2π ikx
k=−∞
converge in the sense of the norm k·k?
It turns out that the answer is yes, but this is not obvious.
We first consider a continuous function f .
We want to show: For any ε > 0 we can find some trigonometric function qn ∈ Tn such that
max | f (x) − qn (x)| ≤ ε.
(21)
This means that trigonometric functions can get arbitrarily close to any continuous function.
We need to construct a function fn ∈ Tn which is close to f . The idea is to use functions gn ∈ Tn which “become more and
more concentrated at 0”, i.e., they satisfy the properties (1.), (2.), (3.) from above. Then we use
qn = f ∗ gn
which gives a trigonometric polynomial qn ∈ Tn .
We first consider the functions
hn (x) :=
1 + cos(2πx)
2
n
for x ∈ [− 21 , 21 ]. Obviously hn (x) ≥ 0.
1
12
h1
0.9
g1
h10
g10
h40
0.8
g40
10
0.7
8
0.6
0.5
6
0.4
4
0.3
0.2
2
0.1
0
−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0
−0.5
0.5
−0.4
−0.3
−0.2
Using (10) we see that hn ∈ Tn . Note that hn (0) = 1. But for x 6= 0 we have 0 ≤
n → ∞.
−0.1
1+cos(2π x)
2
0
0.1
0.2
0.3
0.4
0.5
< 1 and hence hn (x) → 0 as
Since we need property (2.) from above, we define
gn (x) := cn hn (x)
1
with cn := R 1/2
(22)
−1/2 hn (x)dx
Since hn (−x) = hn (x) we have
Z 1/2
−1/2
hn (x)dx = 2
Z 1/2
0
hn (x) dx ≥ 2
| {z }
≥0
Z 1/2
0
 
= 2 −
implying
cn = R 1/2
1
−1/2 hn (x)dx
hn (x) sin(2πx) dx
| {z }
≤1
1+cos(2π x)
2
≤
n+1 1/2
π(n + 1)


=
2
π(kn + 1)
0
π(n + 1)
.
2
Now the functions gn satisfy all three properties (1.), (2.), (3.): (1.) holds since hn (x) ≥ 0. (2.) holds because of (22). For
π x)
πδ )
≤ 1+cos(2
=: q < 1
(3.) let δ > 0 and consider x ∈ [− 12 , 21 ] \ [−δ , δ ]: Then 1+cos(2
2
2
1 + cos(2πx) n π(n + 1) n
≤
q
gn (x) = cn
2
2
Obviously, the right hand side converges to zero for n → ∞.
Properties (1.), (2.), (3.) mean that we can prove ( f ∗ gn )(x) → f (x) in the same way as in Section 2.7:
We want to show (21) for a given ε > 0. Since f is continuous on [− 21 , 21 ] there exists δ > 0 such that
for all x, y with |x − y| < δ :
| f (x) − f (y)| < ε/2
(23)
We then have using qn = f ∗ gn and property (2.)
qn (x) − f (x) =
=
Because of (23) we have
Z 1/2
−1/2
[ f (x − t) − f (x)] gn (t)dt
Z
Z δ
[ f (x − t) − f (x)] gn (t)dt
[ f (x − t) − f (x)] gn (t)dt +
[− 21 , 12 ]\[−δ ,δ ]
−δ
|
{z
}
{z
}
|
➁
➀
➁<
Z δ
ε
−δ
2
gn (t)dt ≤
Z 1/2
ε
2
−1/2
(2.)
gn (t)dt =
ε
2
Let M = max[−1/2,1/2] | f (x)|. Then we can use property (3.) to find N such that
for n ≥ N:
max[− 1 , 1 ]\[−δ ,δ ] gn (x) ≤
2 2
which gives with | f (x − t) − f (x)| ≤ 2M that
Z
ε
➀≤
2M
dt ≤
1 1
4M
[− 2 , 2 ]\[−δ ,δ ]
Z
[− 21 , 21 ]
2M
ε
4M
ε
ε
dt =
4M
2
which proves (21). We have therefore proved
Theorem 1. Assume that f is 1-periodic and continuous. For any ε > 0 there exists a trigonometric polynomial qn ∈ Tn
such that
max | f (x) − qn (x)| ≤ ε.
x
Now we want to consider more general function.
For a piecewise continuous function f with finitely many jumps at points z1 , . . . , zk we can construct a continuous function
fε such that k f − fε k ≤ ε: This can e.g. be done by cutting out intervals [z j − δ , z j + δ ] and replacing the function f with a
straight line on these intervals (this is easy to show, but I’ll skip the details).
This means: For any given ε we can first construct a function fε with k f − fε k ≤ ε. Then we can use Theorem 1 to constract
for fε a function qn such that | fε (x) − qn (x)| ≤ ε and hence
2
k f ε − qn k =
Z 1
0
2
| fε (x) − qn (x)| dx ≤
Z 1
ε 2 dx = ε 2
0
so that we obtain
k f − qn k ≤ k f − fε k + k fε − qn k ≤ ε + ε = 2ε
i.e., we can find trigonometric functions qn such that the L2 -error k f − qn k is arbitrarily small.
The same argument works for more general functions: Assume we have a function f ∈ L2 , i.e.,
Z 1
0
| f (x)|2 dx < ∞
(in the sense of a Lebesgue integral). For any ε > 0 it is possible to construct a continous function fε with k f − fε k ≤ ε (see
e.g. Theorem 3.14 in [Rudin: Real and Complex Analysis]).
This implies that any square integrable function can be approximated by trigonometric polynomials:
R
Theorem 2. Assume that f satisfies 01 | f (x)|2 dx < ∞ (in the Lebesgue sense). For any ε > 0 there exists a trigonometric
polynomial qn ∈ Tn such that k f − qn k ≤ ε.
Corollary 3. Assume that f satisfies
R1
0
ˆ 2π ikx . Then we have
| f (x)|2 dx < ∞. Let f(N) (x) = ∑k=N
k=−N f k e
f − f(N) → 0
as N → ∞
Z 1
0
∞
| f (x)|2 dx =
∑
k=−∞
2
fˆk (24)
Proof. By Theorem 1 we can find for any given ε > 0 a function qN ∈ TN such that k f − qN k < ε. Since f(N) is the best
approximation of f in TN (see (3)) we have
f − f(N) ≤ k f − qN k ≤ ε
which gives (24). From (13) we have
2
k f k2 = f − f(N) +
N
∑
k=−N
2
fˆk 2
As N → ∞ we have f − f(N) → 0. Hence ∑Nk=−N fˆk converges to k f k2 for N → ∞.
2.9 Pointwise convergence of f(N) (x) to f (x)
Note that convergence f − f(N) → 0 as N → ∞ does NOT imply that fN (x) → f (x) for every point x. We saw an example
for this in section 2.6: At the point x = 0 the function jumped from −1 to 1, but we defined f (0) = 1. The Fourier series at
x = 0, however, converges to 0 which is the average of the left-sided limit and the right-sided limit.
It turns out that this problem does not only occur for discontinuous functions. It is possible (but complicated) to construct a
continuous function f (x) such that f(N) (0) does not converge to f (0) as N → ∞.
We call a 1-periodic function f (x) piecewise differentiable if f (x) is of the form



 f1 (x) for x ∈ (z0 , z1 )
.
f (x) = ..


 f (x) for x ∈ (z
,z )
m
m−1
ℓ
with 0 = z0 < z1 < · · · < zm = 1 and the function f j is continuous with a continuous derivative on [z j−1 , z j ] for j = 1, . . . , m.
Theorem 4. If the 1-periodic function f (x) is continuous and piecewise differentiable then the truncated Fourier series
N
f(N) (x) =
∑
fˆk e2π ikx
k=−N
converges pointwise to f (x): For every x ∈ R we have
lim f(N) (x) = f (x).
N→∞
[I will not give
the proof of this theorem here. This is a special case of the so-called Dirichlet theorem which states: If
Z x0 +ε f
(x)
− f (x0 ) and
dx < ∞ for some ε > 0, then the limits
x
− x0
x −ε
R1
0
| f (x)| dx < ∞
0
N
∑ fˆk e2πikx0 ,
N→∞
S+ = lim
k=0
S− = lim
N→∞
−1
∑
fˆk e2πikx0
k=−N
exist, and we have f (x0 ) = S− + S+ ]
Example: Consider the 1-periodic function f (x) with f (x) = |x| for x ∈ [− 21 , 12 ]. This function does not have a continuous
derivative f ′ (x) since f ′ (x) = −1 for x ∈ (− 12 , 0) and f ′ (x) = 1 for x ∈ ( 12 , 1). But this function is piecewise differentiable
for z0 = 0, z1 = 12 , z2 = 1 and it is continuous. Therefore it satisfies the assumptions of Theorem 4 and the Fourier series
converges for every x ∈ R:
∞
∑
fˆk e2π ikx = f (x).
k=−∞
We now consider a function f which is piecewise differentiable, but may have jumps at the points z j :
• If f is continuous at x, then we have pointwise convergence f(N) (x) → f (x) as N → ∞
• If f has a jump at x, then f(N) (x) will converge to the average
defined at x)
f (x − 0) + f (x + 0)
(no matter how f (x) happens to be
2
Theorem 5. Assume that the 1-periodic function f (x) is piecewise differentiable (but may have jumps). Then the truncated
Fourier series
N
fˆk e2π ikx
∑
f(N) (x) =
k=−N
converges pointwise to the average of the left-sided and right-sided limit: For every x ∈ R we have
f (x − 0) + f (x + 0)
.
2
lim f(N) (x) =
N→∞
Here the notation f (x − 0) and f (x + 0) denotes the one-sided limit from the left and right:
f (x + 0) := x→x
lim f (x)
f (x − 0) := x→x
lim f (x),
0
0
x>x0
x<x0
(
−1 x ∈ [− 12 , 0)
in section 2.6. This function is
1
x ∈ [0, 21 )
piecewise differentiable, with jumps at x = 0 and x = 12 . The Fourier series was
Example: We considered the 1-periodic function f (x) with f (x) =
∑
k∈Z
k odd
so this gives for x = 0 the series
−2i 2π ikx
e
πk
−2i
π
∑
k∈Z
k odd
1
.
k
Note that the series for only positive k does not converge, and the same for the series with only negative k:
1
= ∞,
k odd k
∑
k>0
1
= −∞
k odd k
∑
k<0
However, if we consider the sum from k = −N to N we have
f(N) (x) =
−2i 2π ikx
,
e
k=−N,...,N πk
∑
f(N) (0) =
k odd
−2i
=0
k=−N,...,N πk
∑
k odd
We will later investigate in more detail how f(N) (x) behaves near a point where f has a jump (“Gibbs phenomenon”).
2.10 Generalized functions, derivatives and the Delta ”function”
A function f is a rule which assigns to a given point x a function value y = f (x):
f
x 7→ y = f (x)
Instead of pointwise evaluation at an isolated point x we often consider “weighted averages” with respect to a smooth “test
function” g(x): For a given test function g we can compute a number a using
a=
Z ∞
−∞
f (x)g(x)dx
A generalized function f is something for which the pointwise value f (x) may not make sense. But the “integral” with a
smooth “test function”
Z ∞
f (x)g(x)dx
−∞
can still be meaningful.
Example: The Delta function δ (a.k.a. Dirac function) just returns the value g(0) of the test function, i.e., we define
Z ∞
−∞
f (x)δ (x)dx = f (0).
We can imagine the Delta function as the limit n → ∞ of functions gn with integral 1 such as
(
2n |x| ≤ 1n
gn (x) =
0 |x| > n1
(or functions gn which satisfy properties 1., 2., 3. in Section 2.7).
Similarly we define δb as the Delta function concentrated in x = b, i.e.,
Z ∞
−∞
f (x)δb (x)dx = f (b)
One advantage of the concept of generalized functions is that we can take derivatives of any function or generalized
function: If we test f ′ with a test function g which is zero outside of a bounded interval we get
Z ∞
−∞
f ′ (x)g(x)dx = [ f (x)g(x)]∞
−
|
{z −∞}
0
Z ∞
−∞
f ′ (x)g(x) = −
Z ∞
−∞
Z ∞
−∞
f (x)g′ (x)dx
f (x)g′ (x)dx
Note that the right hand side says: apply the generalized function f to the function −g′ (x).
Example 1: Let f (x) = δ3 (x). Then we have
Z ∞
−∞
′
f (x)g(x)dx = −
Z ∞
−∞
δ3 (x)g′ (x)dx = −g′ (3)
So the generalized function f ′ applied to a test function g yields −g′ (3).
(
0 for x < 0
. Then we have for a test function g which is zero
Example 2: Consider the Heaviside function H(x) =
1 for x > 0
outside of a bounded interval
Z ∞
−∞
f H ′ (x)g(x)dx = −
Z ∞
−∞
H(x)g′ (x)dx = −
Z ∞
0
g′ (x)dx = − [g(x)]∞
0 = g(0)
Hence we have that the derivative of the Heaviside function is the Delta function:
H ′ = δ0
Example 3: Consider a piecewise smooth function
f (x) =
(
f1 (x) for x < x0
f2 (x) for x ≥ x0
where f1 (x) has a continuous derivative on (−∞, x0 ] and f2 (x) has a continuous derivative on [x0 , ∞). Then
Z ∞
−∞
f ′ (x)g(x)dx = −
=
=
Z ∞
−∞
f (x)g′ (x)dx = −
0
[− f1 (x)g(x)]x−∞
+
|
{z
}
− f1 (x0 )g(x0 )
Z ∞
−∞
Z x0
−∞
Z x0
−∞
f1 (x)g′ (x)dx −
Z ∞
x0
f2 (x)g′ (x)dx
f1′ (x)g(x)dx + [− f2 (x)g(x)]∞
x0 +
|
{z
}
f2 (x0 )g(x0 )
Z ∞
x0
f2′ (x)g(x)dx
F(x)g(x)dx + ( f2 (x0 ) − f1 (x0 )) g(x0 )
where F denotes the “piecewise derivative”:
F(x) :=
(
f1′ (x) for x < x0
f2′ (x) for x ≥ x0
Therefore we obtain
f ′ (x) = F(x) + J · δx0 (x)
with J := f (x0 + 0) − f (x0 − 0)
where J = f2 (x0 ) − f1 (x0 ) denotes the size of the jump at x0 .
2.11 Properties of Fourier coefficients
Here we summarize the key properties:
fˆk =
Z 1
0
∞
f (x)e−2π ikx dx
Z 1
=
Z 1
0
f (t)g(x − t)dt
⇔
∑
f (x) =
fˆk e2π ikx Inversion
k=−∞
∞
| f (x)|2 dx
0
q(x) =
⇔
∑
k=−∞
2
fˆk q̂k = fˆk ĝk
Parseval
(25)
Convolution
If we apply certain operations on the function f , how is this reflected in the Fourier coefficients fˆk ?
Changing x to −x:
Let g(x) = f (−x). Then using ℓ = −k gives
∞
∞
∑
g(x) =
fˆk e2π ik(−x) =
k=−∞
hence ĝk = fˆ−k .
Shifting the function by a:
the change of variables
Let g(x) = f (x − a), i.e., we shift the graph of the function a distance a to the right. Then
∞
g(x) =
∑
k=−∞
Hence ĝk = e−2π ika fˆk .
fˆ−ℓ e2π iℓx
|{z}
ℓ=−∞
ĝℓ
∑
∞
fˆk e2π ik(x−a) =
fˆk e−2π ika e2π ikx
{z }
|
k=−∞
ĝk
∑
Let g(x) = f ′ (x). Then integration by parts gives
Taking the derivative:
ĝk =
Z 1
0
′
−2π ikx
f (x)e
h
−2π ikx
dx = f (x)e
|
{z
0
}
0
since f (x) and e−2π ikx are 1-periodic functions. Hence
g(x) = f ′ (x)
i1
−
⇐⇒
Z 1
0
f (x)(−2πik)e−2π ikx dx = (2πik) fˆk
ĝk = (2πik) fˆk
2
2
R
∞
ˆ
ˆ
Recall that functions with 01 | f (x)|2 dx < ∞ correspond to Fourier coefficients fˆk with ∑∞
k=−∞ f k < ∞. If the sum ∑k=−∞ f k
R1
does not have a finite value, then the corresponding function f (x) either has 0 | f (x)|2 dx = ∞, or f is only a “generalized
function” like the Delta function.
We consider the function f given by (15) from our earlier example. We found the Fourier coefficients
(
0
for even k ∈ Z
fˆk =
2i
− π k for odd k ∈ Z
Formally, the function g(x) = f ′ (x) should have the Fourier coefficients ĝk = (2πik) fˆk , i.e.,
(
0 for even k ∈ Z
ĝk =
4 for odd k ∈ Z
(26)
R
2
1
2
Obviously, ∑∞
k=−∞ |ĝk | = ∞, so g(x) is not a function where 0 |g(x)| dx has a finite value. What is the function g(x)? Note
that the function f (x) is constant, except for jumps of size 2 and −2 at x = 0 and x = 21 , respectively. The derivative of a
piecewise smooth function gives the derivative of each piece, plus Delta functions at each jump. Therefore we have here
g(x) = f ′ (x) = 2δ0 (x) − 2δ1/2 (x)
where δa denotes the Delta function concentrated at x = a. We know that for p(x) = δ0 (x) we obtain the Fourier coefficients
p̂k = 1 for all k. We have for q(x) = δ1/2 (x) = p(x − 12 ), hence
1
q̂k = e−2π ik 2 p̂k = e−π ik = (−1)k
|{z}
1
Therefore we have
ĝk = 2 · 1 − 2 · (−1)k
which exactly gives (26).
3 Periodic function, discrete time: discrete Fourier transform
3.1 Periodic functions for discrete time values, aliasing
So far we considered 1-periodic functions f (x) defined for x ∈ R. We could write this function in terms of the basis functions
u(k) (x) = e2π ikx for k ∈ Z.
Now we consider a step size 1/N and the discrete points x j = j/N for j ∈ Z. Now a signal is given by values f j at the points
x j , j ∈ Z. As the signal is periodic we have
f j+N = f j
for j ∈ Z.
Hence we only need to specify the values
~f = ( f0 , . . . , fN−1 ) ∈ CN .
We now want to use the values of the trigonometric functions u(k) (x) at the points x j : This function has the values
~u(k) = e2π ik·0/N , . . . , e2π ik· j/N
We have however so-called aliasing: The functions u(k) (x), u(k+N) (x), u(k+2N) (x), . . . have all exactly the same values at the
grid points x j (on all the other points in R they are different, though):
u(k+N) (x j ) = e2π i(k+N) j/N = e2π i jk/N |{z}
e2π i j = u(k) (x j )
1
This means that the
vectors ~u(k)
for all k ∈ Z are actually only N different vectors which keep repeating: Let e.g. N = 5:
(−1) (0) (1) (2) (3) (4)
. . . ,~u| (−2)
~u(5) , |{z}
~u(6) , . . .
{z }, ~u| {z }, ~u , ~u , ~u , ~u , ~u , |{z}
~u(3)
~u(4)
~u(0) ~u(1)
In order to represent the vector ~f = ( f0 , . . . , f4 ) ∈ C5 we need 5 basis functions. We pick the vectors ~u(0) ,~u(1) ,~u(2) ,~u(3) ,~u(4) .
We could also have picked the vectors ~u(−2) ,~u(−1) ,~u(0) ,~u(1) ,~u(2) , but these are exactly the same vectors, just in a different
order.
3.2 The discrete Fourier transform FN
We claim that the vectors ~u(0) , . . . ,~u(N−1) form an orthogonal basis of CN : We have
N−1
N−1
a := e2π i(k−l)/N
~u(k) ,~u(l) = ∑ e2π ik j/N e−2π il j/N = ∑ a j
j=0
j=0
For k = l we have a = 1 and hence
~u(k) ,~u(k) = N.
For k, l ∈ {0, . . . , N − 1} with k 6= l we have a 6= 1 and
N−1
aN − 1 e2π i(k−l) − 1 1 − 1
=
=
= 0.
~u(k) ,~u(l) = ∑ a j =
a−1
a−1
a−1
j=0
Now we can use (2) to write a vector ~f = ( f0 , . . . , fN−1 ) ∈ CN as a linear combination of the vectors ~u(0) , . . . ,~u(N−1) :
~f ,~u(k)
N−1
1 N−1
~f = ∑ fˆk~u(k)
fˆk = (k) (k) = ∑ f j e−2π ik j/N
N j=0
~u ,~u
k=0
(27)
Given a vector ~f ∈ CN in the “time domain” we obtain a vector ~fˆ ∈ CN in the “frequency domain”. This operation is called
the discrete Fourier transform FN , its inverse is called FN−1 :
~fˆ = F ~f ,
N
~f = F −1 ~fˆ
N
3.3 Interpretation of fˆ0 , . . . , fˆN−1 and the interpolating function f˜(x)
Case of odd N
Assume we have for N = 5 a vector ~f = ( f0 , . . . , f4 ) and we compute the discrete Fourier transform ~fˆ = ( fˆ0 , fˆ1 , fˆ2 , fˆ3 , fˆ4 ).
Somehow these coefficients fˆk tell us something about which frequencies are present in our signal. However, it is not quite
obvious how this works.
First consider the function
F̃(x) = fˆ0 u(0) (x) + fˆ1 u(1) (x) + fˆ2 u(2) (x) + fˆ3 u(3) (x) + fˆ4 u(4) (x)
This function satisfies F̃(x j ) = f j for j = 0, . . . , N, i.e., it passes through the points of our discrete signal (“the function F̃(x)
interpolates the given data points”). However, this is not very useful:
• Even if the values f0 , . . . , f4 are real, the values of F̃(x) will be complex and non-real for x between the
grid points.
π
ikx
π
ix
−2
π
ix
2
2
for k ≥ 0, so we cannot get terms like cos(2πx) = e
+e
.
The problem is that we use only e
• We use functions u(k) with freqencies up to k = 4. So we use functions with high frequencies to represent data on a
fairly coarse grid.
But remember that on the grid points x j = j/N the function u(3) coincides with u(−2) , and the function u(4) coincides with
u(−1) (for x 6= x j these functions are different though). Hence we can consider the function
f˜(x) = fˆ0 u(0) (x) + fˆ1 u(1) (x) + fˆ2 u(2) (x) + fˆ3 u(−2) (x) + fˆ4 u(−1) (x)
which still passes through the given data points. Now we have k = −2, −1, 0, 1, 2 and therefore f˜ ∈ T2 . Therefore we can
write f˜ in the form
f˜(x) = a0 + a1 cos(2πx) + b1 sin(2πx) + a2 cos(2π · 2x) + b2 sin(2π · 2x).
This function f˜(x) is much nicer than F̃(x):
• If the values f0 , . . . , f4 are real, the values of f˜(x) will be real for all x.
• We use a function in T2 with frequencies only up to k = 2.
For general odd N = 2n + 1 the Fourier vector ( fˆ0 , fˆ1 , . . . , fˆN−1 ) yields the interpolating function f˜ ∈ Fn given by
f˜(x) = fˆ0 u(0) (x) + fˆ1 u(1) (x) + · · · + fˆn u(n) (x) + fˆN−n u(−n) (x) + · · · + fˆN−1 u(−1) (x)
(28)
Therefore the entries of the Fourier vector should be interpreted as follows:
frequency
0
fˆ0
1
fˆ1
2 ···
fˆ2 · · ·
n
fˆn
n
···
fˆn+1 · · ·
2
fˆN−2
1
fˆN−1
So fˆ0 corresponds to frequency 0 with u(0) (x) = 1, it gives the mean value of our data. Then fˆ1 and fˆN−1 together tell us
about frequency 1. The highest frequency terms fˆn , fˆn+1 for frequency n are in the middle of the vector.
Case of even N
Consider for N = 6 a discrete signal a vector ~f = ( f0 , . . . , f5 ) with the discrete Fourier transform ~fˆ = ( fˆ0 , fˆ1 , fˆ2 , fˆ3 , fˆ4 , fˆ5 ).
Here the “bad” function F̃(x) would be
F̃(x) = fˆ0 u(0) (x) + fˆ1 u(1) (x) + fˆ2 u(2) (x) + fˆ3 u(3) (x) + fˆ4 u(4) (x) + fˆ5 u(5) (x).
On the grid points x j the function u(5) coincides with u(−1) , the function u(4) coincides with u(−2) and the function u(3)
coincides with u(−3) (x). So we replace fˆ4 u(4) (x) + fˆ5 u(5) (x) with fˆ4 u(−2) (x) + fˆ5 u(−1) (x). What should we do with fˆ3 u(3) (x)?
If we leave it as it is, or if we replace it with fˆ3 u(−3) (x) we would get something
“unbalanced”, causing a complex function
for real data. The answer is to replace fˆ3 u(3) (x) with fˆ3 12 u(3) (x) + u(−3) (x) , yielding the function
1 (3)
f˜(x) = fˆ0 u(0) (x) + fˆ1 u(1) (x) + fˆ2 u(2) (x) +
u (x) + u(−3) (x) + fˆ4 u(−2) (x) + fˆ5 u(−1) (x)
2
which still passes through the given data points. Note that
1
1 (3)
u (x) + u(−3) (x) = fˆ3 e2π i·3x + e−2π i·3x = fˆ3 cos(2π · 3x)
2
2
so that we can write f˜ in the form
f˜(x) = a0 + a1 cos(2πx) + b1 sin(2πx) + a2 cos(2π · 2x) + b2 sin(2π · 2x) + a3 cos(2π · 3x)
We denote these functions by T2,cos since we take the functions in T2 and add only the cosine term for frequency 3. Note
that it does not make sense to use sin(2π · 3x) since this function is zero at all grid points x j = j/6.
For general even N = 2n the Fourier vector ( fˆ0 , fˆ1 , . . . , fˆN−1 ) yields the interpolating function f˜ ∈ Tn−1,cos given by
1
f˜(x) = fˆ0 u(0) (x) + fˆ1 u(1) (x) + · · · + fˆn−1 u(n−1) (x) + fˆn u(n) (x) + u(−n) (x) + fˆN−n+1 u(−n+1) (x) + · · · + fˆN−1 u(−1) (x) (29)
2
Therefore the entries of the Fourier vector should be interpreted as follows:
frequency
0
fˆ0
1
fˆ1
2 ···
fˆ2 · · ·
n−1
fˆn−1
n n−1 ···
fˆn fˆn+1 · · ·
2
ˆfN−2
1
ˆfN−1
3.4 Discrete Fourier transform in Matlab
The discrete Fourier transform FN maps the vector ~f = ( f0 , . . . , fN−1 ) in the time domain to the vector ~fˆ = ( fˆ0 , . . . , fˆN−1 ) in
the frequency domain. The inverse discrete Fourier transform F −1 maps the vector ~fˆ back to the vector ~f :
N
~fˆ = F ~f ,
N
~f = F −1 ~fˆ
N
As an example consider the vector ~f = [1, 1, 1, 1] in the time domain. Since this corresponds to the function f˜(x) = 1 the
Fourier vector is ~fˆ = [1, 0, 0, 0]:
F4−1 [1, 0, 0, 0] = [1, 1, 1, 1].
F4 [1, 1, 1, 1] = [1, 0, 0, 0],
We use the m-files four.m
function fh = four(f)
fh = fft(f)/length(f);
and ifour.m
function f = ifour(fh)
f = ifft(fh)*length(fh);
Then we can compute the discrete Fourier transform in Matlab:
>> f = [1,1,1,1]
f =
1
1
>> fh = four(f)
fh =
1
0
>> ifour(fh)
ans =
1
1
1
1
0
1
0
1
In Matlab all indices are shifted by 1:
You can access components of a Matlab vector v using v(1), v(2), ... .
Note that Matlab begins counting with 1, whereas our data and Fourier vectors are indexed from 0 to N − 1. Therefore fˆj
corresponds to fh(j+1) in Matlab. In our example we have ~fˆ = ( fˆ0 , fˆ1 , fˆ2 , fˆ3 ) = (1, 0, 0, 0) and fˆ0 corresponds to fh(1) in
Matlab.
3.5 Properties of the discrete Fourier transform
Here we summarize the key properties:
1 N−1
fˆk = ∑ f j e−2π ik j/N
N j=0
qj =
1 N−1 2
∑ fj
N j=0
1 N−1
∑ fℓ g j−ℓ
N ℓ=0
N−1
⇔
=
fj =
fˆk e2π ik j/N
Inversion
k=0
N−1 2
∑ fˆk k=0
⇔
∑
q̂k = fˆk ĝk
Parseval
Convolution
(30)
Recall that
~u(k) ,~u(ℓ) =
(
N
0
for k = l
for k 6= l
This gave us the formula for fˆk in (27), and (4) gives
(~f , ~f ) =
yielding the Parseval formula.
N−1 2 (k) (k) ˆ
f
∑ k ~u ,~u
k=0
| {z }
N
Note that the sums on the left hand side in (30) correspond to the integrals on the left hand side in (25) if we apply a midpoint
rule to the integrals.
For the convolution we plug in the Fourier representation of g j−ℓ and switch the order of the sums:
qj =
1 N−1
1 N−1 N−1
fℓ g j−ℓ = ∑ fℓ ∑ ĝk e2π i( j−ℓ)k/N
∑
N ℓ=0
N ℓ=0 k=0
!
N−1
qj =
N−1
∑ ∑
fℓ e−2π iℓk/N ĝk e2π ik j/N
ℓ=0
k=0
|
{z
fˆk
}
2π ik j/N with q̂ = fˆ ĝ . Remember that the signals are periodic (wrap-around): g = g
Hence we have q j = ∑N−1
j
j+N . So
k
k k
k=0 q̂k e
g j−ℓ with j − ℓ < 0 has to be interpreted in this way.
3.6 Evaluating f˜(x) on a finer grid: “Extending the Fourier vector with zeros”
Recall that a Fourier vector ~fˆ = fˆ0 , . . . , fˆN−1 corresponds to a function
• f˜ ∈ Tn given by (28) for odd N = 2n + 1
• f˜ ∈ Tn−1,cos given by (29) for even N = 2n.
Applying FN−1 to ~fˆ gives back the values f0 , . . . , fN−1 on the grid points x j = j/N.
Often we would like to evaluate the function f˜(x) on a finer grid X j = j/M , j = 0, . . . , M − 1 with M > N. We can do this
in the following way: E.g., for a vector ~fˆ = fˆ0 , fˆ1 , fˆ2 , fˆ3 , fˆ4 of length N = 5 we have the function
f˜(x) = fˆ3 u(−2) (x) + fˆ4 (x)u(−1) (x) + fˆ0 u(0) (x) + fˆ1 u(1) (x) + fˆ2 (x)u(2) (x) ∈ T2
The function does not change if we add zero terms with u(−3) (x) and u(3) (x):
f˜(x) = |{z}
0 ·u(−3) (x) + fˆ3 u(−2) (x) + fˆ4 u(−1) (x) + fˆ0 u(0) (x) + fˆ1 u(1) (x) + fˆ2 u(2) (x) + |{z}
0 ·u(3) (x)
|{z}
|{z}
|{z}
|{z}
|{z}
ĝ4
ĝ3
ĝ5
ĝ6
ĝ0
ĝ1
ĝ2
which corresponds to the Fourier vector
~ĝ = fˆ0 , fˆ1 , fˆ2 , 0, 0, fˆ3 , fˆ4
which is obtained from ~fˆ by “sticking zeros into the middle of the vector”. Now applying the inverse Fourier transform
~g := F7−1~ĝ
gives the values g j = f˜( j/M) of the function f˜ on the finer grid.
If we start instead with a vector ~fˆ = fˆ0 , fˆ1 , fˆ2 , fˆ3 of length N = 4 we obtain the interpolating function
1
1
f˜(x) = fˆ2 u(−2) (x) + fˆ43 (x)u(−1) (x) + fˆ0 u(0) (x) + fˆ1 u(1) (x) + fˆ2 (x)u(2) (x) ∈ T1,cos
2
2
therefore the vector ~ĝ of length M = 7 which corresponds to the same interpolating function is
1
1
~ĝ = fˆ0 , fˆ1 , fˆ2 , 0, 0, fˆ2 , fˆ3
2
2
(
Tn
if N = 2n + 1 odd
. A
Tn−1,cos if N = 2n even
vector ~ĝ = (ĝ0 , . . . , ĝM−1 ) of length M > N which corresponds to the same interpolating function is given by
(
ˆ0 , . . . , fˆn , 0, . . . , 0, fˆn+1 , fˆN−1
f
if N = 2n + 1 odd
~ĝ =
1
1
fˆ0 , . . . , fˆn−1 , 2 fˆn , 0, . . . , 0, 2 fˆn , fˆn+1 , . . . , fˆN−1 f if N = 2n even
Summary: A vector ~fˆ = fˆ0 , . . . , fˆN−1 corresponds to an interpolating function f˜(x) ∈
This is implemented in the Matlab function fext (see m-file on course web page). We can use this to plot the interpolating
function on a finer grid with M points:
x = (0:5)/6;
X = (0:511)/512;
f = [0 0 0 1 1 1];
fh = four(f);
gh = fext(fh,512);
g = ifour(gh);
plot(x,f,’o’,X,g)
% grid of given signal
% fine grid for plotting
% given signal
1.2
1
0.8
0.6
0.4
0.2
0
−0.2
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
3.7 Least squares approximation with lower frequency functions: “Truncating the Fourier vector”
We are given the values f j at the points x j = Nj for j = 0, . . . , N − 1. We have seen that we can obtain an interpolating function
(
Tn
for odd N = 2n + 1
f˜ ∈
Tn−1,cos for even N = 2n
If we use a space Tm or Tm−1,cos with m < n we cannot expect to find a function g̃(x) which passes through all points. The
best we can do is to minimize the least squares error
N−1 2
∑ g̃(x j ) − f j j=0
.
(31)
Because of the orthogonality of the vectors ~u(k) (see section 3.2) we can obtain the least squares approximation by using only
terms which correspond to our subspace (see section 2.2): We have a vector ~f ∈ CN and using the discrete Fourier transform
we can write it as
~f =
N−1
∑
fˆk~u(k)
k=0
Case of odd M = 2m + 1: We want to approximate the values f j using functions Tm with M = 2m + 1 < N. For example
consider T2 where m = 2 and M = 2m+1 = 5. In this case we want to use the frequencies −2,−1, 0, 1, 2. Because of aliasing
~ ∈ span ~u(N−2) ,~u(N−1) ,~u(0) ,~u(1) ,~u(2)
the corresponding vectors are ~u(N−2) ,~u(N−1) ,~u(0) ,~u(1) ,~u(2) . The best approximation G
is obtained as
~ = fˆN−2~u(N−2) + fˆN−1~u(N−1) + fˆ0~u(0) + fˆ1~u(1) + fˆ2~u(2)
G
~ given by
and corresponds to the Fourier vector Ĝ
Ĝ0 , Ĝ1 , . . . , ĜN−1 = fˆ0 , fˆ1 , fˆ2 , 0, . . . , 0, fˆN−2 , fˆN−1
where we replace high frequency coefficients with zeros. The function g̃(x) corresponds to the Fourier vector ~ĝ of length
M = 2m + 1 = 5 given by
(ĝ0 , ĝ1 , ĝ2 , ĝ3 , ĝ4 ) = fˆ0 , fˆ1 , fˆ2 , fˆN−2 , fˆN−1
where we “remove the high frequency coefficients from the middle” of the vector ~fˆ.
To approximate the data values using Tm with M = 2m + 1 < N we use
(ĝ0 , . . . , ĝM−1 ) := fˆ0 , fˆ1 , . . . , fˆm , fˆN−m , . . . , fˆN−1
Case of even M = 2m: We want to approximate the values f j using functions Tm−1,cos with M = 2m < N. Here we
use for the highest frequency m only the cosine term. For example consider T1,cos = span {1, cos(2πx), sin(2πx), cos(4πx)}
~ given by
where m = 2 and M = 2m = 4. The best approximation corresponds to the Fourier vector Ĝ
Ĝ0 , Ĝ1 , . . . , ĜN−1 =
ˆ
ˆ
ˆ
ˆ
ˆf0 , fˆ1 , f2 + fN−2 , 0, . . . , 0, f2 + fN−2 , fˆN−1 .
2
2
The function g̃(x) corresponds to the Fourier vector ~ĝ of length M = 2m = 4 given by
(ĝ0 , ĝ1 , ĝ2 , ĝ3 ) = fˆ0 , fˆ1 , fˆ2 + fˆN−2 , fˆN−1 .
To approximate the data values using Tm−1,cos with M = 2m < N we use
(ĝ0 , . . . , ĝM−1 ) := fˆ0 , fˆ1 , . . . , fˆm−1 , fˆm + fˆN−m , fˆN−m+1 , . . . , fˆN−1
Summary: We can approximate a data vector ( f0 , . . . , fN−1 ) of length N using a trigonometric space of dimension M < N
as follows: define the vector ~g of length M by
(
fˆ0 , fˆ1 , . . . , fˆm , fˆN−m , . . . , fˆN−1
for odd M = 2m + 1
(ĝ0 , . . . , ĝM−1 ) :=
ˆf0 , fˆ1 , . . . , fˆm−1 , fˆm + fˆN−m , fˆN−m+1 , . . . , fˆN−1
for even M = 2m
Then the corresponding function
(
Tm
g̃(x) ∈
Tm−1,cos
for odd M = 2m + 1
for even M = 2m
is the best least squares approximation, i.e., it minizes (31).
This operation is implemented in the Matlab function ftrunc (see m-file on course web page).
Example: We consider a discrete signal which has value 0 for x ∈ [0, 21 ) and value 1 for x ∈ [ 12 , 1). We want to find the best
least squares approximation using a function g̃ ∈ T4 , i.e., M = 9.
x = (0:511)/512; %
f = (x>=.5);
%
fh = four(f);
gh = ftrunc(fh,9);
Gh = fext(gh,512);
G = ifour(Gh);
plot(x,f,x,G)
grid
signal: 1 for x>=.5, 0 else
% truncate with M=9
% extend for plotting
1.2
1
0.8
0.6
0.4
0.2
0
−0.2
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
3.8 Efficient computation of FN and FN−1 : Fast Fourier Transform
Assume we are given the vector ~fˆ = fˆ0 , . . . , fˆN−1 ∈ CN in the frequency domain. We want to find the vector ~f =
( f0 , . . . , fN−1 ) in the time domain. If we just use the formula
N−1
fj =
∑
fˆk e2π ik j/N
k=0
we have to compute N multiplications and N − 1 additions for each component, in addition to evaluating e2π ik j/N . This means
that the computation of ~f costs at least N 2 multiplications (plus a similar number of additions etc.). For a vector of length
N = 1000 this means that we have at least 106 operations. This means that it is not practical to compute discrete Fourier
transforms of long vectors.
Fortunately there is a way to reorganize the computation in such a way that only N · log2 N operations are required. E.g., for
N = 1024 we have
N 2 ≈ 106 ,
N · log2 N = 1024 · 10 ≈ 104
We first explain how to compute ~f if N is a power of 2:
[More details coming soon . . . ]
This algorithm is implemented in the Matlab functions fft and ifft and therefore used in our functions four and ifour
which were discussed in section 3.4.
3.9 Sampling a function
We can characterize a 1-periodic function F(x) by its Fourier coefficients F̂k , k ∈ Z.
Evaluating this function on grid points x j = j/N, j = 0, . . . , N − 1 gives values
f j = F( j/N)
with f j+N = f j . This process which takes us from the function F(x) to the vector ~f = ( f0 , . . . , fN−1 ) is called sampling.
Obviously, information about the behavior of f between the grid points gets lost during sampling.
By applying the discrete Fourier transform FN to ~f we obtain the Fourier coefficients fˆ0 , . . . , fˆN−1 .
How are Fourier coefficients fˆ0 , . . . , fˆN−1 related to the original Fourier coefficients F̂k , k ∈ Z?
(k)
Note that we have F(x) = ∑∞
k=−∞ F̂k u (x) and hence for x j = j/N
∞
F(x j ) =
∑
F̂k u(k) (x j )
(32)
k=−∞
Now remember the “aliasing property” of u(k) (x): On the grid points x j = j/N we have
· · · = u(k−N) (x j ) = u(k) (x j ) = u(k+N) (x j ) = u(k+2N) (x j ) = · · ·
Therefore we can simplify (32) by collecting the identical terms together:
N−1 ∞
F(x j ) = ∑ ∑ F̂k+ℓN u(k) (x j )
| {z } k=0 ℓ=−∞
|
{z
}
fj
ˆfk
ˆ (k)
We see that we obtain the discrete Fourier representation f j = ∑N−1
k=0 f k u (x j ). We have shown
∞
fˆk =
∑
F̂k+ℓN
ℓ=−∞
This is how we get from the coefficients F̂k , k ∈ Z of the original function to the coefficents fˆ0 , . . . , fˆN−1 of the sampled
version.
3.10 Sampling theorem, Nyquist frequency
[More details coming soon . . . ]
4 Nonperiodic function, discrete time
4.1 Introduction
Consider again the Fourier series: A signal f defined on [− 12 , 12 ] in the time domain corresponds to coefficients fˆk for k ∈ Z
in the frequency domain, and we have (25).
Now let us look at this result from a different perspective: Pretend that g j = fˆj is a signal defined at points j ∈ Z in the time
domain, then the function ĝ(ξ ) := f (ξ ) for ξ ∈ [− 12 , 21 ] should be the representation in the frequency domain. We get from
(25) (using −i instead of i)
gj =
1
2
Z
∞
ĝ(ξ )e2π iξ j dξ
1
−2
Z 1
0
⇔
|ĝ(ξ )| dξ
∑
=
The function
g̃(x) :=
g j e2π iξ j Inversion
j=−∞
∞
2
∑
ĝ(ξ ) =
j=−∞
Z 1/2
−1/2
(33)
2
g j Parseval
ĝ(ξ )e2π iξ x dξ
defined for x ∈ R passes through the data points, i.e., g̃( j) = g j .
Now we assume that we have a spacing of h instead of 1: We consider a signal with values f j at points jh for j ∈ Z. We
obtain this from (33) using the change of variables ξ = hξ̃ , dξ = hd ξ̃ :
gj = h
Z 1/(2h)
−1/(2h)
∞
ĝ(hξ̃ )e2π iξ̃ h j d ξ̃
⇔
ĝ(hξ̃ ) =
∑
g j e2π iξ̃ h j
j=−∞
We obtain with f j = g j /h and fˆ(ξ ) = ĝ(hξ ) the inversion formula below. The Parseval formula follows in the same way
from (33).
∞
fˆ(ξ ) = h
f j e−2π iξ jh
∑
j=−∞
∞
h
∑
j=−∞
∞
qj = h
∑
ℓ=−∞
2
f j
fℓ g j−ℓ
⇔
=
⇔
fj =
Z 1/(2h)
−1/(2h)
fˆ(ξ )e2π iξ jh dξ
Z 1/(2h) −1/(2h)
fˆ(ξ )2 dξ
q̂(ξ ) = fˆ(ξ )ĝ(ξ )
Inversion
Parseval
(34)
Convolution
2π iξ x with
(ξ )
We see that a signal
1 with
a spacing h in the time domain can be represented in terms of functions u (x) = e
1
frequencies ξ ∈ − 2h , 2h in the frequency domain. Note that
• for nonperiodic signals in the time domain we obtain real frequencies ξ (not only integers k as for periodic signals)
• the spacing h in the time domain corresponds to an interval ξ = [−M, M] in the frequency domain, with the Nyquist
frequency M = 1/(2h).
Proof of the convolution formula: we plug in the Fourier representation of g j−ℓ and then interchange the order of the
integral and the sum:
∞
∑
qj = h
ℓ=−∞
=
So we obtain q j =
Z 1/(2h)
In the formula f j =
function
−1/(2h)
∞
fℓ · g j−ℓ = h
Z 1/(2h) −1/(2h)
∞
∑
h
ℓ=−∞
∑
ℓ=−∞
fl ·
−1/(2h)
g(ξ )e2π iξ ( j−ℓ)h dξ
fl e−2π iξ ℓh ĝ(ξ )dξ
{z
fˆ(ξ )
|
Z 1/(2h)
}
q̂(ξ )e2π iξ jh dξ with q̂(ξ ) = fˆ(ξ )ĝ(ξ ).
Z 1/(2h)
−1/(2h)
fˆ(ξ )e2π iξ ( jh) dξ we can plug in arbitrary values x ∈ R instead of jh and obtain the interpolating
f˜(x) :=
Z 1/(2h)
−1/(2h)
fˆ(ξ )e2π iξ x dξ
Example: Consider the signal . . . , 0, 0, 0, 1, 0, 0, 0 . . . given by
(
1 for j = 0
fj =
0 for j 6= 0
We want to find the Fourier transform fˆ(ξ ):
1 1
ξ∈ − ,
2h 2h
∞
fˆ(ξ ) = h
∑
f je
−2π iξ jh
j=−∞
= h·1
Therefore the interpolating function f˜(x) is given by
f˜(x) =
Z 1/(2h)
−1/(2h)
h
sin
=
πx
2π iξ x
fˆ(ξ )e
πx h
dξ =
= sinc
with the sinc function defined by
Z 1/(2h)
x
2π iξ x
−1/(2h)
h·e
1 2π ixξ
e
dξ = h
2πix
1/(2h)
h

 sin(πx)
sinc x :=
πx
1
x 6= 0
x=0
This function is also available in Matlab.
Warning: Some math books define sinc x as sin x/x instead of sin(πx)/(πx).
We can use this function to express f˜(x) for an arbitrary signal f j :
∞
f˜(x) =
∑
f j sinc
j=−∞
4.2 Correlation function and autocorrelation
[More details coming soon . . . ]
4.3 Finding the Fourier transform using the FFT
[More details coming soon . . . ]
x
h
−j
ξ =−1/(2h)
=
h 1 π ix/h
e
·
− e−π ix/h
πx 2i
4.4 Bandlimited signals and sampling theorem
A signal of the form
f (x) =
Z M
−M
F(ξ )e2π iξ x dξ
is called bandlimited, since it only contains frequencies |ξ | ≤ M.
[More details coming soon . . . ]
5 Nonperiodic function, continuous time
Now we consider
a signal f (x) for x ∈ R which is not periodic and decays for x → ∞ and x → −∞. We assume that the
R∞
| f (x)|2 dx is finite. We claim that the following holds:
“energy “ −∞
fˆ(ξ ) =
q(x) =
Z ∞
−∞
Z ∞
−∞
f (x)e−2π iξ x dx
Z ∞
⇔
| f (x)|2 dx
=
f (t)g(x − t)dt
⇔
−∞
f (x) =
Z ∞
−∞
Z ∞
−∞
2
fˆ(ξ )e2π iξ x dξ
fˆ(ξ ) dξ
q̂(ξ ) = fˆ(ξ )ĝ(ξ )
Inversion
Parseval
(35)
Convolution
5.1 Symbolic Fourier transform in Matlab
Matlab has a symbolic toolbox which uses symbolic expressions like x*sin(x) instead of numbers. Here is an example:
>> syms x
%
>> f = x*sin(x)
%
f = x*sin(x)
>> g = diff(f,x)
%
g = sin(x) + x*cos(x)
>> int(g,x)
%
ans = x*sin(x)
define x as a symbolic variable
this is a symbolic expression
take derivative with respect to f
take indefinite integral with respect to x
Matlab
has built-in commands fourier and ifourier for the symbolic Fourier transform and its inverse. However, Matlab
R∞
uses −∞ f (x)e−ixξ dx for the Fourier transform instead of our definition.
Therefore we define commands Four and iFour for our definition of the Fourier transform and its inverse:
function F = Four(f,x,xi)
F = subs(fourier(f,x,xi),xi,2*pi*xi);
function f = iFour(F,xi,x)
f = 2*pi*subs(ifourier(F,xi,x),x,2*pi*x);
Note that for complicated functions Matlab is unable to find the Fourier transform. Here are some simple examples:
>> syms x xi
% define x, xi as symbolic variables
>> f = heaviside(x)*exp(-x); fh = Four(f,x,xi)
fh = 1/(pi*xi*2*i + 1)
% heaviside(x) is 1 for x>=0, 0 for x<0
>> gh = fh*fh; g = iFour(gh,xi,x)
g = x*exp(-x)*heaviside(x)
% g is convolution of f with f
>> f = exp(-abs(x)); fh = Four(f,x,xi)
fh = 2/(4*pi^2*xi^2 + 1)
>> f = sinc(x); fh = simplify(Four(f,x,xi))
fh = heaviside(xi + 1/2) - heaviside(xi - 1/2)
>> f = exp(-pi*x^2); fh = Four(f,x,xi)
fh = exp(-pi*xi^2)
>> f = 1; fh = Four(f,x,xi)
fh = dirac(xi)
% dirac(x) denotes the Delta-"function"