Download Ch.3 slides

3. DISCRETE-TIME RANDOM PROCESSES
Outline
Random variables
Random processes
Filtering random processes
Spectral factorization
Special types of random processes
• Autoregressive moving average processes
• Autoregressive processes
• Moving average processes
1
3 Random processes
September 12, 2016
Random variables
Definitions
A random variable
is a function that assigns a number to each outcome of a
random experiment.
Probability distribution function:
Pr
Probability density function:
Mean or expected value:
Z
Variance:
Var
Z
2
3 Random processes
September 12, 2016
Random variables
Definitions
Joint probability distribution function:
Pr
Joint probability density function:
Correlation:
Covariance:
Cov
Correlation coefficient
3
3 Random processes
September 12, 2016
Random variables
and
strongly correlated
uncorrelated
and
(small )
Linearly dependent
4
3 Random processes
September 12, 2016
Random variables
Definitions
and
Two random variables
and
are independent if
Two random variables
or
are orthogonal if
and
or
Two random variables
are uncorrelated if
Orthogonal random variables are not necessarily uncorrelated
Zero-mean uncorrelated random variables are orthogonal
5
3 Random processes
September 12, 2016
Random processes
Definitions
is an indexed sequence of random variables (a “signal”)
A random process
Mean and variance:
Autocorrelation and autocovariance:
Cross-correlation and cross-covariance
September 12, 2016
3 Random processes
6
Uncorrelated and orthogonal processes are defined as for variables but now
Random processes
Stationarity
. Implies
First-order stationarity if
.
Second-order stationarity if
Implies
and
Properties of WSS processes:
maximum value:
mean-square value:
symmetry:
3 Random processes
September 12, 2016
7
periodic with period
mean-square periodicity:
are wide-sense stationary and ii)
, and iii)
; ii)
jointly wide-sense stationary if i) both
and
Two processes
Wide-sense stationarity, if i)
Stationarity in the strict sense, if the process is stationary for all orders
Random processes
Autocorrelation and autocovariance matrices
samples in a vector
and collect
We consider a WSS process




 ..
 .


..
.
..
.







Autocovariance matrix:
Autocorrelation matrix:

where
8
nonnegative definite; hence the eigenvalues of
is Toeplitz, Hermitian, and
The autocorrelation matrix of a WSS process
are nonnegative
3 Random processes
September 12, 2016
Random processes
Sample mean:
X
Realization 1
Realization 5
Ensemble mean:
When is the sample mean equal to the ensemble mean (expectation)?
9
3 Random processes
September 12, 2016
Realization 4
Realization 3
Realization 2
Random processes
Ergodicity
Sample mean:
X
A WSS process is ergodic in the mean if (mean-square convergence)
or
Var
and
(unbiased)
Necessary and sufficient condition:
X
Sufficient condition:
and
10
3 Random processes
September 12, 2016
Random processes
White noise
with autocovariance:
White noise is a discrete-time random process
for
i.e.
.
(probability density not important)
All variables are uncorrelated with variance
The power spectrum of zero-mean white noise is constant:
X
11
3 Random processes
September 12, 2016
Random processes
Power spectrum
The power spectrum of a WSS process is the DTFT of the autocorrelation:
X
Also:
X
Since the autocorrelation is conjugate symmetric, the power spectrum is real:
If the stochastic process is real, the power spectrum is even:
The power spectrum is nonnegative:
The total power is proportional to the area under the power spectrum:
Z
(use inverse DTFT, take
12
3 Random processes
September 12, 2016
)
Random processes
Power spectrum
autocorrelation matrix are upper and lower
of the
The eigenvalues
bounded by the maximum and minimum value, respectively, of the power spectrum:
as
The power spectrum is related to the mean of

 X

has a nonzero mean or a periodicity, the power spectrum contains im-
If



pulses
13
3 Random processes
September 12, 2016
Filtering random processes
that is filtered
and correlation
is a WSS process with mean
Suppose
is also
; then the output
by a stable LSI filter with unit sample response
WSS with
:
is the “(deterministic) autocorrelation” of
where
X
is given by
The power of
September 12, 2016
3 Random processes
14
and
zero outside
where we assume
X
X
Filtering random processes
In terms of the power spectrum, this means that
has a
, if
also has a pole (zero) at
, then
and
pole (zero) at
So assuming no pole/zero cancelations between
and another at
the conjugate reciprocal location
is a narrow-band bandpass filter with center frequency
, bandwidth
If
, and magnitude 1, then the output power is
Z
so the power spectrum describes how the power is distributed over frequency
15
3 Random processes
September 12, 2016
Spectral factorization
may be factored as
then
of a WSS process is a continuous function of
If the power spectrum
X
Proof:
then we can write
is analytic in
If
September 12, 2016
3 Random processes
16
is real,
, and since
is the IDTFT of
so
X
and
X
,
Spectral factorization
Proof (continued):
Now we can write
)
X
(
)
then we can express the third exponential as
(
)
(
X
X
)
If we now define the second exponential as
(
X
)
X
(
and so we obtain
3 Random processes
17
with
September 12, 2016
Spectral factorization
is causal, stable, and minimum phase; moreover it is monic:
The filter
A process that can be factorized as described earlier is a regular process
Properties of a regular process
driven by
white noise with variance
• A regular process can be realized as the output of a filter
, then the output is white
(whitening)
noise with variance
• If the process is filtered by the inverse filter
• The process and the white noise contain the same information (compression)
18
3 Random processes
September 12, 2016
Spectral factorization
Suppose the power spectrum is a rational function
then the spectral factorization tells us we can factor this as
where
whose roots are all inside the unit circle
; so the poles and zeros occur in
is real, we have
Since
conjugate reciprocal pairs and we simply relate the zeros inside the unit circle to
and the poles inside the unit circle to the zeros of
19
3 Random processes
the zeros of
September 12, 2016
Special types of random processes
Autoregressive moving average processes
with the filter
of variance
P
P
Suppose we filter white noise
can then be written as
The power spectrum of the output
Such a process is known as an autoregressive moving average process of order
, or ARMA
poles and
process has
The power spectrum of an ARMA
zeros with
conjugate reciprocal symmetry
20
3 Random processes
September 12, 2016
Special types of random processes
Autoregressive moving average processes
:
and
From the LCCDE between
X
X
and take the expectation:
we can multiply both sides with
can further be expressed as
and
The crosscorrelation between
X
X
X
X
For
, this leads to the Yule-Walker equations

X



X
21
3 Random processes



September 12, 2016
Special types of random processes
Autoregressive moving average processes

..
.
..
.

















..
.
, it gives a recursion for the autocor
and







..
.


















Given the filter coefficients


















..
.
..
.
..
.
:

..
.
..
.
..
.

















The Yule-Walker equations can be stacked for

relation
September 12, 2016
3 Random processes
22
and
Given the autocorrelation, we may compute the filter coefficients
Special types of random processes
Autoregressive processes
process is an autoregressive process, or AR
An ARMA
:
The Yule-Walker equations are given by
 
 
 
 
 .. 
.
 
..
.







 
..
.

..
.








..
.
:
from the Yule-Walker equations is easy (linear)
Estimating




 ..
 .

Stacking the Yule-Walker equations for

X
23
3 Random processes
September 12, 2016
Special types of random processes
Moving average processes
process is a moving average process, or MA
An ARMA
:
The Yule-Walker equations are given by
X
The autocorrelation function is zero outside
from the Yule-Walker equations is not easy (nonlinear)
Estimating
24
3 Random processes
September 12, 2016