Download STA 348 Introduction to Stochastic Processes Lecture 20

STA 348
Introduction to
Stochastic Processes
Lecture 20
1
Continuous MC Transition
Probability Function

Probability of going from i to j after time t,
called the transition probability function Pij(t)
Pij (t )  P  X (t )  j | X (0)  i 

Probability Transition Functions satisfy:

Chapman-Kolmogorov eqations:
Pij (t  s )   k Pik (t )  Pkj ( s )

Kolmogorov’s Backward Equations:
Pij (t )   k i qij Pkj (t )  vi Pij (t )
2
Kolmogorov’s Forward
Equations

For all states i, j and times t ≥ 0, we have
Pij (t )   k  j Pik (t )qkj  Pij (t )v j


Note: Forward equations don’t hold for all CMC’s, but do
hold for all Birth & Death and finite state-space CMC’s
Proof:
3
Example

Find Pij(t) for machine with Exp(λ) work time
& Exp(µ) repair time using forward eqn’s
4
Example

Find forward eqn’s of Birth & Death process
5
Example

Find forward eqn’s for pure birth process,
and show that
 Pii (t )  e  it , i  0


 jt t  j s
 Pij (t )   j 1e 0 e Pi , j 1 ( s )ds , j  i  1
6
Example (cont’d)
7
CMC Limiting Probabilities

If CMC is ergodic (all states communicate &
are +ve recurrent), probabilities of being in
state j after time t→∞ converge to limiting
values Pj  lim Pij (t ) (indep. of starting state i)
t 



Pj is long run-proportion CMC is in state j
If CMC starts from stationary initial distribution Pj,
then P[X(t)=j|X(0)~{Pi}] = Pj
Pj’s satisfy: v j Pj   k  j Pk qkj , j &

j
Pj  1
8
CMC Limiting Probabilities

Stationary probs Pj satisfy v j Pj   k  j Pk qkj ,
because of forward eqn’s:
9
Example

Find stationary Pj for machine with Exp(λ)
work time & Exp(µ) repair time
10
Example

Stationary distr. of Birth & Death process
11
Example (cont’d)
12