Download .pdf

MTHE/STAT455, STAT855
Fall 2014
Stochastic Processes
Final Exam, Solutions
1. (15 marks)
(a) (6 marks) We have M0 = 0. If B is initially at position 1, then the time for
1
. For k > 1,
A to find B is Geometric with parameter 1 − α. Thus, M1 = 1−α
conditioning on B’s first move, we obtain by the law of total expectation that
1
Mk = α(1 + Mk ) + (1 − α)(1 + Mk−1 ), or Mk = 1−α
+ Mk−1 . Recursing this gives
k−1
k
Mk = 1−α + M1 = 1−α .
(b) (9 marks)
(i) For a given k, the state space is S = {0, 1, . . . , k}.
(ii) The transition matrix is given by

1
0
···

 1−α
α
0
···

 0
1−α α
0
···

P= .
.
.
.
...
..
..
..
 ..

 0
···
0 1−α
α

0
···
0
1−α
0
0
0
..
.









0 

α
where the rows going down (and columns going right) correspond to the states
0, 1, . . . , k, in that order.
(iii) All states have period 1. State 0 is recurrent and states 1, . . . , k are transient.
(iv) The chain is not irreducible because state 0 does not communicate with any
other state.
2. (15 marks)
(a) (4 marks) False. The Galton-Watson branching process or the simple random
walk as discussed in class are both time-homogeneous, but neither is stationary.
MTHE/STAT455, STAT855 -- Final Exam Solutions, 2014
Page 2 of 5
(b) (4 marks) True. We have for m < n and i, j ∈ S, and any k such that n + k and
m + k are both nonnegative,
P (Xn = j, Xm = i)
P (Xn = j Xm = i) =
P (Xm = i)
P (Xn+k = j, Xm+k = i)
(by stationarity)
=
P (Xm+k = i)
= P (Xn+k = j Xm+k = i).
Hence X is time-homogeneous.
(c) (4 marks) True. If X is time reversible then the stationary distribution π (which
under the given conditions exists, is unique, and has all positive components)
satisfies the local balance equations πi pij = πj pji for all i, j ∈ S. For these to be
satisfied it is necessary that if pij > 0 then pji > 0 and if pij = 0 then pji = 0.
(d) (3 marks) False. A counterexample is given by the transition matrix in Problem
4.
3. (15 marks)
(a) (7 marks) Following the hint, we follow just {Xn : n ≥ 0}, which keeps track
of just the unmutated individuals. This process is itself a branching process in
which each individual has 0 offspring with probability 1/3 and 2 offspring with
probability 2/3 (“offspring” only count here if they are unmutated). We wish to
find the probability of ultimate extinction for this unmutated population. The
generating function of the family size distribution is G(s) = 13 + 23 s2 . The solution
to s = G(s) is the smallest root in [0, 1] of the quadratic equation 32 s2 − s + 13 = 0.
The roots are
p
1 ± 1 − 8/9
1 ± 1/3
1
=
= or 1.
4/3
4/3
2
Thus, the probability of ultimate extinction of the unmutated population is 1/2.
(b) (8 marks) Since {Xn : n ≥ 0} is a branching process we have that E[Xn ] =
µnX , where µX is the mean family size. From part(a), the mean family size (of
unmutated offspring) is 2× 23 = 43 . Thus, E[Xn ] = (4/3)n . For E[Yn ] we follow the
hint and condition on Xn−1 and Yn−1 (note that {Yn : n ≥ 0} is not a branching
process of the type discussed in class). Suppose we are given that Xn = j and
1
= 16
Yn = k. Then each of the j unmutated individuals will have on average 2 × 12
mutated offspring, and each of the k mutated individuals will have on average 2 ×
MTHE/STAT455, STAT855 -- Final Exam Solutions, 2014
Page 3 of 5
= 32 mutated offspring. That is, E[Yn Xn−1 = j, Yn−1 = k] = 6j + 3k
. Then by
2
3
1 4 n−1
1
+ 32 E[Yn−1 ].
the law of total expectation, E[Yn ] = 6 E[Xn−1 ] + 2 E[Yn−1 ] = 6 ( 3 )
Recursing, and using the fact that E[Y0 ] = 0, we have
3
4
n−1
4
3
+ E[Yn−1 ]
3
2
" #
n−1
n−2
3 1 4
3
1 4
+
+ E[Yn−2 ]
6 3
2 6 3
2
n−1 n−2 2
1 4
3 1 4
3
+
+
E[Yn−2 ]
6 3
2 6 3
2
#
n−1 n−2 2 " n−3
3 1 4
3
1 4
3
1 4
+
+
+ E[Yn−3 ]
6 3
2 6 3
2
6 3
2
n−1 n−2 2 n−3 3
1 4
3 1 4
3 1 4
3
+
+
+
E[Yn−3 ]
6 3
2 6 3
2 6 3
2
1
E[Yn ] =
6
=
=
=
=
..
.
=
=
=
=
n−1 k n−k−1
4
1X 3
6 k=0 2
3
n−1 X
n−1 k
9
1 4
6 3
8
k=0
n−1
1 4
(9/8)n − 1
6 3
9/8 − 1
n n
n n
4
9
3
4
−1 =
−
.
3
8
2
3
4. (15 marks)
(a) (9 marks) Note that local balance is not satisfied because the local balance equations are π1 = π2 , π1 = π3 , and 2π2 = π3 , but the first two equations imply
π2 = π3 , which contradicts the third equation. The first two global balance equations are
1
π2 +
3
1
=
π1 +
3
π1 =
π2
2
π3
3
1
π3 ,
3
(1)
(2)
which together with the normalization constraint π1 + π2 + π3 = 1 are enough to
determine π1 , π2 , and π3 . Plugging (1) into (2) gives π2 = 13 ( 13 π2 + 32 π3 ) + 31 π3 ,
MTHE/STAT455, STAT855 -- Final Exam Solutions, 2014
Page 4 of 5
or π2 = 58 π3 . Plugging this back into (1) gives π1 = 13 ( 85 )π3 + 23 π3 = 87 π3 . The
7
and
normalization constraint gives 78 π3 + 85 π3 + π3 = 1, or π3 = 25 . Then π1 = 20
1
π2 = 4 .
(b) (6 marks) A continuous time Markov chain for which {Xn : n ≥ 0} is the embedded discrete time chain has infinitesimal rates qij = vi pij for some positive
values v1 , v2 , v3 , and the transition probabilities pij are the entries in the transition matrix P from part(a). If this continuous time chain were time reversible
then the local balance equations πi qij = πj qji would be satisfied by the stationary
distribution π (note that this π is not the same as the one in part(a)). Writing
these out gives
π1 v1 = π2 v2 ;
π1 v1 = π3 v3 ;
π2 v2
1
2
= π3 v3 .
3
3
But the first two equations imply π2 v2 = π3 v3 , which contradicts the third equation. Thus the local balance equations cannot be satisfied and so the continuous
time chain cannot be time reversible.
5. (15 marks)
(a) (9 marks) Let the transition probabilities of the embedded jump chain be denoted
by pij . Conditioning on the first jump we have that
ηi = P (TA < ∞ X(0) = i)
X
=
P (TA < ∞ X(0) = i, first jump is to j)pij
j∈S
=
X
=
X
j∈A
j∈A
pij +
X
P (TA < ∞ X(0) = i, first jump is to j)pij
j∈Ac
ηj pij +
X
j ∈A
/
ηj pij =
X
ηj pij ,
j∈S
noting that ηj = 1 for j ∈ A. But since pij = qij /vi for i 6= j and pii = 0, we see
that
X
X
ηi =
ηj qij /vi or vi ηi =
ηj qij .
j6=i
j6=i
(b) (6 marks) Letting Ti denote the holding time in state i and conditioning on the
MTHE/STAT455, STAT855 -- Final Exam Solutions, 2014
Page 5 of 5
first jump of the chain, we have that
X
µi = E[TA X(0) = i] =
E[TA X(0) = i, first jump is to j)pij
j∈S
=
X
E[Ti ]pij +
j∈A
= E[Ti ]
X
(E[Ti ] + µj )pij
j ∈A
/
X
pij +
j∈S
= E[Ti ] +
X
µj pij
j ∈A
/
X
µj pij
j ∈A
/
Since E[Ti ] = 1/vi , pij = qij /vi , and pii = 0 we obtain
µi =
1 X qij
+
µj
vi
vi
or
vi µi = 1 +
j ∈A
/
X
j ∈A
/
Finally, since µj = 0 for j ∈ A, we can also write this as
v i µi = 1 +
X
j6=i
qij µj .
qij µj .