Download WORD

LECTURE#19
11/22/04
Stochastic (or random) processes
Description
A stochastic process is a collection, or ensemble, of functions of time, any one of which
might be observed on any trial of a random experiment. It can also be thought of as a
family of random variables (or vectors) indexed by a parameter set: {x t, t  T}. A
stochastic process is a function of two variables: t (time parameter) and  (probability
parameter). In mathematical notation:
{xt, tT}  {xt(), tT, }
For each t, xt() is a random variable (vector). For each , x() is a realization of the
process (sample function, sample sequence).
Example: rainfall or flow time series at a point as an observed member of the ensemble of
possible rain or flow occurrences.
If the random variables xt are discrete the stochastic process has a discrete state space. If
the random variables xt are continuous, the stochastic process has a continuous state
space.
If T (parameter set) is discrete, the stochastic process is a discrete parameter process. If T
is continuous then the stochastic process is a continuous parameter process.
Classification of Stochastic Processes
Parameter Set
Discrete
Continuous
Continuous
Random Sequence
Stochastic Process
Random Function
Discrete
Discrete Parameter
Chain
Continuous Parameter
Chain
State - Space
Random Walk Process
Toss a coin at times 0, 1, 2, …
If heads (h) comes up, we take a step +x forward.
If tails (t) comes up, we take a step -x backward (x 0)
All steps are executed instantaneously. The probability space for this case becomes
={h,t}.
Let P(h)=p and P(t)-1-p=q (for a fair coin p=q=1/2).
Define the random variable (for each time):
 x   h
 x   t
Wn() = 
Then,
P{Wn()=+x} = p;
P{Wn()=-x} = q
Denote by xn the position at time instant n, before the execution of a step at that instant.
Assuming that we start from the origin, x0=0, our position at time n is given by
n 1
x n   wi ;
n=1, 2, …
i 0
{xn, n=1, 2, …} is a discrete parameter chain.
For each n, xn() is a discrete random variable given by the sum of wi.
Each realization of the chain is a sequence of real numbers of the form kDx, k=0, 1, …
The probability space on which the random walk is defined, is the space of sequences of
the form:
' = htthhht …
The difference equation: xn+1=xn+wn; where n=0,1,…; and x0=0 (Random Difference
n 1
Equation) generates the solution: xn =
 wi
i 0
Ergodic Hypothesis
Any statistic calculated by averaging over all members of an ergodic ensemble at a fixed
time can also be calculated by averaging over all time on a single representative member
of the ensemble.
Representative means that the realization must display at various points in time the full
range of amplitude and rate of change of amplitude which are to be found among all the
members of the ensemble.
For ergodic processes one computes statistics from a single realization; for example,
1
E[X] = lim
T   2T
T
 x(t )dt
T
T
1
lim
 x(t ) x(t   )dt
T   2T T
In practice most stationary function results are computed assuming ergodicity. Gelb
(1974), pages 38-39, presents an example of an ergodic process.
x(t,t+) =
Why Ergodicity?
All previously defined/discussed probability density functions and their respective
moments are defined over an ensemble of the stochastic (random) process – the
collection of all possible realizations. Unfortunately, in hydrology, we are able to
observe, for most of the time, only one realization of the random processes of interest.
Take for example, having rainfall record from a network comprising many stations, but
of only one storm. How then can the distributions, of for that matter, the moments of the
random process be estimated from one realization? How do hydrologists bypass
ensemble averaging? The answer to the above questions is ergodicity. Ergodicity states
that averaging over the ensemble is equivalent to averaging over a realization. For the
rainfall network example, one may average over all the stations for that one storm to get
an idea of the mean and the variance of the rainfall climatology.
But why do we need to learn all this on stochastic systems
Let’s remind ourselves of the ubiquitous presence of uncertainty that essentially makes
all our ‘deterministic’ efforts look random/stochastic.
Why Stochastic Models, estimation and control?
When considering system analysis or controller design, the engineer has at his disposal a
wealth of knowledge derived from deterministic system and classical theories. One would
then naturally ask, why do we have to go beyond these results and propose stochastic
system models (worry about stochastic processes) ? To answer this we should relook at
the figure we provided in the very first class.
Remember the following:
1. No mathematical model (deterministic) is perfect
2. No measurement is perfect
Stochastic Models and the theories on optimal estimation (to be covered next) have one
of the most unique applications as an optimal recursive data processing algorithm in
hydrology. One such method is also known as Filtering or data assimilation As an
example, consider, that you have data being received on stream flow measurements on a
regular basis at a hydrologic center that is responsible for issuing forecasts. As part of this
system, the center needs to re-estimate (or update) the hydrologic model parameters in an
online (or real-time) fashion as every new observation of what the model is predicting
becomes available. In real life both the measurements and the model predictions are
random processes. Filtering theory such as Kalman Filtering can handle such realistic
situations in a recursive fashion.
NEXT CLASS: Introduction to Filtering