Download Prediction_1

Probability Spaces
A probability space is a triple (, M , P)
 Sample Space (any nonempty set),
M Set of Events
a sigma-algebra over  (closed under
complementation and countable unions)
P Probability Measure (a countably
additive function P : M  [0,1]
such that P()  1 )
http://en.wikipedia.org/wiki/Probability_space
Random Variables
A (complex valued) random variable on a
probability space (, M , P) is a function
1
X :   C that is measureable ( X (O)  M
for every open O  C )
The expectation E ( X ) 
X dP


The distribution of a random variable X :   C
is the measure  X on C that satisfies
1
 X (O)  P( X (O)) for every open O  C )
z d X
Problem 1. Show that E ( X ) 
C
where z : C  C is the identity function.

Examples
  {1,2,3,4,5,6}, M  2 , P({n})  , n  

1
6
The identity function X :     C is a
random variable that models the number of
a randomly thrown dice.
  R, M  Borel , P(( a, b)) 
1
2
2

b
a
e
( x   ) 2 /( 2 2 )
dx
The identity function X :     C is a real
random variable whose distribution is Gaussian
2

with mean
and variance  .
2
2
Problem 2. Compute E ( X ) and R x d X

Products
The product of probability spaces (i , Mi , Pi ), i  1,2
is the probability space (, M , P) where:
  1  2 , M is the sigma-algebra over 
generated by the sets in M1  M2 , and
P : M  [0,1] is the unique countably additive
whose restriction to M1  M2 satisfies
P( A1  A2 )  P1 ( A1 ) P2 ( A2 ), A1  M1 , A2  M2 .
Andrei Nikolajevich Kolmogorov (1950) The
modern measure-theoretic foundation of
probability theory; the original German version
(Grundbegriffe der Wahrscheinlichkeitrechnung)
appeared in 1933 (countable products also exist)
Independence
Random variables X 1 , X 2 on a probability space
(, M , P) are independent if for all M1 , M 2  M
P( X 11 (M1 )  X 21 (M 2 ))  P( X 11 (M1 )) P( X 21 (M 2 ))
Example If (, M , P) is the prod. of prob. spaces
(i , M i , Pi ), i  1,2, and i :   i , i  1,2 denotes
coordinate projections, and Yi : i  C , i  1,2
are random variables, then the random variables
X i  Yi  i :   C, i  1,2 are independent.
Correlation
Let X 1 , X 2 be random variables on a prob. space
(, M , P ). The relation X1  X 2  E ( | X1  X 2 | )
2
is an equivalence relation and L (, M , P)
denotes the set of equivalence classes of
2
random variables X satisfying E ( | X | )  .
Henceforth we identify random variables with
their equivalence class. The correlation of X1 with
X 2 is denoted by C ( X 1 , X 2 )  E ( X 1 X 2 ). It gives
a scalar product and therefore a Hilbert space
structure on L2 (, M , P).
2
Correlation Properties
Problem 3. Show that if X 1 , X 2  L (, M , P)
2
are independent then C ( X 1 X 2 )  E ( X 1 ) E ( X 2 ).
Problem 4. Show that if X  L (, M , P)
2
then | E ( X ) |  E ( | X | ) .
2
Henceforth we identify random variables with
their equivalence class.
Correlation Properties
The Gramm matrix for X 1 , X 2 ,..., X n  L (, M , P)
2
is G(i, j )  E ( X i X j ), i, j  1,..., n
Thm X 1 ,..., X n are linearly dependent iff det G  0.
Proof Let v  [a1 ,..., an ]  C and define
X  a1 X 1    an X n . Then X  0 
T
0  E( | X | )  E
2




n

n

a
a
X
X

v
Gv
i
j
i
j
i , j 1
a
a
E
X
X

a
a
G
(
i
,
j
)

v
Gv
.

i
j
i
j
i
j
i , j 1
i , j 1
n
n

Random Processes
A (discrete) random process is a sequence of
random variables X i , i  Z on a prob. space
(, M , P ).
The process is wide sense stationary if
X i  L2 (, M , P), i  Z and the correlations
E ( X i X j )  ci  j (they depend only on i-j)
Problem 5. Prove that the correlation sequence c
is positive definite.
Spectral Measure
Henceforth we consider a wide sense stationary
random process X i  L (, M , P), i  Z with
2
correlation sequence c.
Herglotz’s theorem implies that there exists a
measure  on the circle group T  R /( 2Z )
such that cn   (en ), n  Z .
This is the spectral measure of the process. It
encodes all of the correlation properties of the
processes such as predictability.
Spectral Process
Theorem 3.1.1 There exists a unique isometry
 : L (T , d )  span { X i }  L (, M , P)
2
2
such that  (en )  X n , n  Z .
Proof For any trigonometric polynomial
P  L (T , d )
2
|| P ||
2
L2 (T , d )
  ( | P | )  E ( |  ( P) | )  ||  ( P) ||
2
2
2
L2 (  ,M , P )
So the result follows since trigonometric
polynomials are dense in C (T ) and hence
dense in L2 (T , d ).
Denegerate Processes
Definition 3.1.1 A process X i is degenerate if
span { X i } is finite dimensional.
2
Problem 6 Show this holds iff dim L (T , d )  .
N
 k  where
Problem 7 Show that if  

k 1
k
{1 ,..., N } are distinct points in T and  k  0
then {1, e1 ,..., eN 1} is a basis for L (T , d ).
2
Suggestion Show that L (T , d ) is isomorphic to
N
N
C with scalar product (u, v)   k 1  k uk vk . Then
use the fundamental theorem of algebra to show
{1, e1 ,..., eN 1} are lin. ind. in L2 (T , d ).
2
Denegerate Processes
Lemma 3.1.1 A measure  has the form
N
N


det
(

(
e
))
.
iff
k

m

n
m
,
n

0
k
k 1

Proof The n-th column of the matrix is 
1 
 e ( ) 
N
vn   k 1  k en ( k ) wk where wk   1 k 
  

so rank of matrix is < N+1 so det = 0. 
eN ( k )
If det = 0 then since the matrix is a
2
Gram matrix for the Hilbert space L (T , d )
{1, e1 ,..., eN } are linearly dependent.
2
Problem 8. Show this implies dim L (T , d )  N .
Denegerate Processes
Definition Define  : L (T , d )  L (T , d )
2
by ( h)( )  e1 ( )h( ), h  L (T , d ),   T .
2
2
Problem 9. Show that  is unitary.
Continuation of Proof. Assuming that  is unitary
2
and dim L (T , d )  N there exists an orthonormal
2
basis v1 ,..., vK , K  N for L (T , d ) such that
 vi  e vi , 1  i  K . If e0   k 1 ak vk then
K
i i
cn   (en e0 )   (( e0 ) e0 )   [( j 1 e
K
n
  j 1 | a j | e
K
2
in j
so  

in j
a j v j )( j 1 a j v j )]
K
|
a
|

.
k

k
k 1
N
2
Examples of Stationary Processes
White Noise is a stationary random process Wi
with E (Wn )  0 and correlation sequence
ci  j  E (Wi W j ) 
1 if i  j
0 if i  j
A stationary random process is white noise iff
its spectral measure equals d  21 d .
2
B

b

b
e

b
e



H
(T ) then X  b W
If
0
1 1
2 2
given by X n  b0Wn  b1Wn1  b2Wn2  
is a stationary random process.
Problem 10. Compute the spectral measure of X .