Download Discrete Fourier Transform

Random Functions
1) Random functions: an introduction
Def.1 Let T be some subset of Rn (n = 1,2,3,…) and  be an event space endowed with a
probability distribution P ;
a function u defined on T x  , i.e. u(t,) , (tT, ) , is a “random function”.
Interpretations: u(t,) can have two interpretations
A) for each sample value
 u(t,) is a function belonging to some space U ;
take any set V  U such that
AV  { u(t,)  V }
(1)
is an event, i.e. a set of which we know P(AV);
then put
PU(V) = P(AV) .
(2)
If we extend PU to the (minimal) -algebra containing all {V} of the form (1) we induce on
U a probability distribution and we can view {u(t,)} as a “stochastic variable” with
“values” in U.
Def 2: The particular u(t,  ) for a given   is called a realization of the random function.
This can be considered as a “sample” value in U of the random variable {u(t,)} .
This point of view becomes very simple when U is a finite dimensional space (not even necessarily
linear) but it becomes much more difficult to understand when U is infinite dimensional as the
following examples show
Example 1 : Let T R1 , U  {at+b ; a,b  R1} ,
i.e. U is the space of linear functions of t ;
then any distribution P on R2 is mapped onto U , e.g. starting from the sets
V = {at+b ; a  I1 , b  I2} , PU(V) =P(a  I1 , b  I2)
b
1
Q
1 

0
at+b

1
a
t
Note: assume that  
a
b
has mean and covariance
E  
a
b
C  
 a2  ab
 ab  b2
then we easily understand how to compute average and covariance of the variable { u(t,)} , i.e.
u t   Eut ,    Eat  b  a t  b



t '
Cuu t , t '  Eu t ,    u t u t ' ,    u t '  Ea  a t  b  b  a  a t 'b  b    t 1C  
1
Example 2 : this example is presented to show the complications that rise when we deal with
infinite dimensional cases.
Assume {a0 , a1 , b1 , ……an , bn……} are all independent random variables, normally distributed
and such that
a n  N 0,  n2 , bn  N 0,  n2 ;
this defines .





Then construct
u t ,    a 0  2 a n cos 2nt  bn sin 2nt  ; clearly here T  [0,1] .
n 1
In order that the above series define a function, the series itself has to converge in some sense.
This is related to the sequence n2 , but it is very difficult to make precise statements (necessary and
sufficient conditions) about his convergence.
For instance assume
n2 = 02 = const ;
then one can prove that the series is non convergent for almost all t  T with probability 1.
Otherwise assume


n 0
2
n
  ;
Then one can show the series is convergent in mean square sense for a set of  having
probability 1.
Finally assume that

n 
2
n 0
2
n
 
then the set of  for which the series is converging for all t  T has probability 1.
Since this interpretation of u(t,) requires a deeper knowledge of mathematics, we shall rather
prefer a different interpretation.
we consider for each value of the parameter t the function u(t,) as a random variable, so
that all together {u(t,)} represents a family of random variables.
When T is a finite set, we can always conventionally put T {t1 = 1 , t2 = 2, … tN = N} and then
we have a collection of N random variables u = [u1, u2, …uN] + which is completely described, in
a probabilistic sense, by a joint distribution function
B)
P ( u1a1 , u2a2 , …., uNaN ) = Fu(a1, a2 , …., aN)
(3).
When T is infinite we distinguish between two cases: T is discrete (e.g. T {1, 2, …}) or T has the
power of continuum (e.g. T [0,1] or T Rn…); in the former case we say that the random
function is discrete, in the latter that it is continuous.
Note that this has nothing to do with the regularity of the realizations u(t,  ), but only with the
cardinality of the set T.
In this case we have to identify a tool to represent the probability distribution of { u(t,)} .
This is usually done by specifying a sequence of distributions, called finite dimensional
distribution of the random function { u(t,)} , defined as
F(t1,.....tN ; a1 , …., aN) = P{ u(t1,)  a1 ...... u(tN,)  aN}
(4)
for  N > 0 and, given N, for every N-tuple (t1 , t2.....tN)
It is by means of a theorem by Kolmogorov that we know when we can use a sequence
FN(tN , aN)
(where tN = [t1,.....tN] +, aN = [a1 , …., aN] +)
to build a true probability distribution on the space of all functions defined on T, ℱT .
Theorem 1 : assume we are given a sequence FN(tN, , aN) ; then we can build on ℱT a probability
distribution, such that FN(tN, , aN) appear as its marginal distributions if compatibility conditions
are satisfied, namely
F(ti1,.....tiN ; ai1 , …., aiN) = P{ u(t1,)  ai1 ...... u(tN,)  aiN} =
= P{ u(t1,)  a1 ...... u(tN,)  aN} = F(t1,.....tN ; a1 , …., aN)
(5)
(i1, i2, ….iN) = permutation of (1,2, .... N) ,
F(t1,.....tN , tN+1 ; a1 , …., aN , + ) = F(t1,.....tN ; a1 , …., aN)
(6)
Note that (5) states an obvious necessary symmetry related to the meaning of F(t , a) while (6)
represents the condition that FN is a marginal distribution of FN+1.
The importance of Theorem 1 is in stating that such conditions are also sufficient to build a
probability distribution on ℱT .
Example 3: let us give at least an example where the full sequence on FN(tN , aN) is specified.
We do that by means of a so called gaussian family, i.e. a family of gaussian distributions, assuming
that
u (t1 ,  )
uN 
where
u (t 2 ,  )
 N u , Cu u

u (t N ,  )


(7)
u (t1 )
u (t 2 )
u

u (t N )
(8)
and
Cu u
C (t1 , t1 ) C (t1 , t 2 )  C (t1 , t N )

C (t 2 , t 2 )
C (t 2 , t N )



C (t N , t N )
(9)
In the above definition we must have indeed
u t   Eut, 
and
(10)
Ct, t '  Eut,   u t ut ' ,   u t '
(11)
Since we know that the marginal distribution of a gaussian variate is again gaussian, we have only
to be sure that the function C(t,t’) (also called the covariance function) we use to construct the
matrices (9) is such that Cuu is always symmetric and positive definite.
These properties are guaranteed if C(t,t’)= C(t’,t) and for instance

C t , t '   c n n t  n t ' ,
(12)
n 1
with
cn > 0 ,
(13)
and {n(t)} is any sequence of smooth functions.
In such a case in fact the quadratic form
2
N









C
t
,
t

c

t

t



c



i k
i k
n  n i
n k
i k
n   n t k k   0
i , k 1
n 1
i , k 1
n 1
 k 1

N


N
is always positive definite, as it should be.
 For instance if T = [0,L] (with L < +) we could take

t nt 'n
 e tt '
n 0 n!
C t , t '  
 or, for T = [l,L]
(with 0<l<L)

C t , t '   e  nt e nt ' 
n 0
 also the case T = [0,1] , cn > 0
1
1  e  ( t  t ')
;
,


n 0
n 1
C t , t '   c n cos 2nt  t '  c0   c n cos 2nt cos 2nt ' sin 2nt sin 2nt '
is an important example of the same kind.
Remark 1 : also the interpretation B) of { u(t,)} is not easy to follow.
Yet in the present note we will need only finite dimensional distributions, so that once we know that
we are working with correct entities, we can forget about the difficult theory which is behind.
Def.3 : when T is all of R1 , or a subset of it, and we think of t as a “time” variable, we say that
{u(t,)} is a stochastic process ; when T is R2 (or a subset) or a surface (e.g. a sphere), or R3
etc., we say that { u(t,)} is a random field.
Remark 2 : we can consider random fields on R1 too, if we do not use in any special way the
natural order of R1 , bringing to such concepts as past, present and future.
In fact all physical processes do obey to a causality principle in time, namely the future of a system
can depend on its past, but not viceversa. (*)
On the contrary when t is a 1-D coordinate in space, the value of { u(t,)} at any two points t1,t2
can have a perfectly symmetric relation.
/(*)Note : basically this means that since all processes of every day life are irreversible in time, there
can be no symmetry in our perception of past and future./
Remark 3 : there are cases in which it is desirable to use the more general theory of random
processes or random fields even if we have only a finite number of points in T.
For instance we can have a long series of time values T = {1, 2, …N} and the related variables
u1(), ….uN() , and we consider that as a “sample” from an otherwise unknown infinite sequence
{ u(t,) ; tZ} (*); or we could have T = {(i,j) ; 1  i , j  N} , i.e. a square grid in the plane of
N2 knots, placed at integer coordinates, and the corresponding variables { uij()} .
What makes this finite dimensional variables a field or a process, is the fact that we look at the
values attained at different points in relation to the position of such points in R1 , R2 , etc.
For instance we could expect that ui() and ui+1() are closer to one another than ui() and
ui+10() and so on .
/(*)Note : remember that Z is the field of integer numbers form –  to + . /
2) Moments
Def.4 : the average of first moment of the R.F. {u(t,)} is the function defined on T
u t   Eut, 
(14)
The R.F. {u(t,)} is said to be centred if u t   0 .
If u(t,) is not centred, however, as it is obvious, the R.F. u(t,) - u t  is .
Def.5: We call simple moments of order N of the R.F., the functions defined on {T}N ,
M N t1 t 2 .....t N   Eut1 ,   ut 2 ,   ......ut N ,  
(15)
We call central moments of order N of {u(t,)} the functions
M N t1 t 2 .....t N   Eu t1 ,    u t1  u t 2 ,    u t 2  ......u t N ,    u t N 
(16)
Among all central moments, of paramount importance is the central moment of order 2, also called
covariance function
C t1 , t 2   M 2 t1 , t 2   Eut1 ,    u t1  ut 2 ,    u t 2    Eut1 ,  ut 2 ,    u t1 u t 2  (17)
Remark 4 : we note, by taking t1 = t2 = t in (17), that the variance of the process is given by
 2 ut ,    C t , t  ;
this implies that C(t,t) has to be always positive.
The notion of covariance function is so important that one would like to know whether some given
C(t1,t2) can be considered as the covariance function of some R.F. ; i.e. we want to characterize the
family of all covariance functions.
To this aim the following Lemma is providing a first idea.
Lemma 1 : C(t1,t2) is a covariance function iff (=if and only if) it is a positive definite function,
i.e. iff, given any N-tuple (t1 , t2 , … tN ) and  N, the matrix
I,k = 1,2,….N
C N  C t i , t k 
is a covariance matrix (i.e. symmetric and positive definite)
Proof : it is enough to build a gaussian family with u t  =0 where CN is taken as the covariance
matrix of uN = [{u(t1,)}……. {u(tN,)}] , as in Example 3 .
More about covariance functions will be said in the next paragraph.
3) Homogeneous Random Fields
Def.4 : a random field {u(t,)} is said to be strongly homogeneous if all shifted fields
{v(t,)=u(t+,)} have the same probability distributions as {u(t,)}   T.
This essentially means that N,  {t1 , t2 , … tN } the vector [u(t1+,) …..u(tN+,)]+ has the
same distribution as the vector [u(t1,) …..u(tN,)]+, irrespectively of the value of .
We notice that such a concept can indeed be applied only on condition that the domain T is
invariant under translation, i.e., T is R1, R2, ..etc; we shall see at the end of the paragraph how to
generalize this definition to processes defined on the circle or on the 2-sphere (i.e. a sphere in R3).
Let us also remark that the concept of homogeneity translates the property that , whatever window
of size LxL we open on u(t,) , we expect to find in it a statistically similar behaviour.
Therefore if we take any particular function u0(t) and we ask ourselves if it can be considered as a
sample drawn form a homogeneous R.F., we have at first to see whether there are features in one
part of u0(t) which are significantly different from those contained into another part.
Just as an example, consider the two digital terrain models displayed in Fig.a) and Fig.b) in terms
of level lines.
It appears very obvious that a) can hardly (i.e. with very low probability) considered
homogeneous, whilst b) can be considered homogeneous at least to a visual inspection.
Fig. two different DTM with isolines with interdistance 20m
a) is probably non homogeneous
b) appears to be homogeneous
Let us now turn to the characterization of the moments of a homogeneous R.F.
Lemma 2 : if { u(t,)} is homogeneous then
u t   Eut,   u
(constant)
(19)
Ct1 , t 2   Eut1 ,   u  ut 2 ,   u    Ct1  t 2 
(20)
Proof.: Taking  = -t , since { u(t,)} and { u(t+,)}  { u(0,)} must have the same
distribution, we immediately see that
E{ u(t,)}=E { u(0,)} ,
 t,
i.e. (19) holds true.
In addition the vectors [{u(t1,)}, {u(t2,)}] + and [{u(t1+,)}, {u(t2+,)}] + must also have
the same distribution, and in particular, putting  = -t1, they must have the same distribution as
[u(0,) , u(t2-t1,)].
Accordingly
C t1 , t 2   Eu t1 ,    u  u t 2 ,    u   
 Eu 0,    u  u t 2  t1 ,    u    C t1  t 2 
which proves (20).
(21)
It is useful to notice that , by taking t2 = t1 = t , we can claim that


 2 u t ,    E u t ,    u 2  C 0
(22)
which we already know from the fact that all { u(t,)} have the same distribution,  t.
Remark 5 : if C () is a covariance function of a homogeneous R.F., then C () is an even
function
C(-) = C ()
(23)
In fact, since the vectors
[{u(t1,)}, {u(t2,)}] + , [u(0,) , u(t2-t1,)] + [u(t2-t1,), u(0,) ]
must all possess the same distribution, we must have, assuming for the sake of simplicity that u  0 ,
Ct 2  t1   Eu0, ut 2  t1 ,    Eut1  t 2 , u0,    Ct1  t 2 
Def 5: a R.F. { u(t,)} is said to be weakly homogeneous, or homogeneous up to the second
order, if (19), (20) hold, i.e.
Eut ,   u  constant
Eut1 ,   u  ut 2 ,   u    Ct1  t 2 
It is important to stress that indeed (19), (20) are not enough to say that { u(t,)} is homogeneous ,
in fact they do not even imply that the distributions of the first and second order F(t1 ; a1) ,
F(t1, t2 ; a1 , a2) are shift-invariant.
So strong homogeneity does imply weak homogeneity, while the viceversa is not true in general.
Nevertheless, due to the particular form of their distribution functions, for gaussian R.F.. it is true
that (19) (20) imply strong homogeneity.
Def.6 : a random process, i.e. a R.F. on R1, which is homogeneous (weakly homogeneous) is said
to be stationary (weakly stationary).
One important piece of work that has been done to understand weakly homogeneous R.F. regards
the characterization of their covariance functions C().
We have already seen two necessary conditions for C() to be a covariance function, i.e.
C(0) > 0 , C(-) = C()
(24)
Here is another necessary condition.
Remark 6 : C() is a covariance function (of a homogeneous R.F.) only if
| C() |  C(0) (25)
In fact, let us first observe that the function
R  
C  
C 0
(26)
represents as a matter of fact the correlation between u(t+,) and u(t,) because C() is just
the covariance between these two variables and  ut   ,     ut ,    C0 , as (22)
implies.
Since, as it is well known, |R()|  1 , (25) follows .
Equality in (25) can only be verified when R()=1 , i.e. when u(t+,) and u(t,) are fully
linearly dependent, a hypothesis that we shall exclude for the future.
In particular (25) says to us that a covariance function is always bounded outside the origin and that
it can only be unbounded if the R.F. has not a finite variance.
To be specific, in what follows, we shall always consider R.F. which have finite variances
everywhere, so C(0) is always finite and C() is always absolutely bounded.
We will see in par.4) a complete characterization of covariance functions, C().