Download HOMEWORK 2: SOLUTIONS 1. Consider three events A+,A−,A0 in

HOMEWORK 2: SOLUTIONS
1.
Consider three events A+ , A− , A0 in the same probability space Ω with a probability P on it. Suppose that A+ and A− are conditionally independent given
A0 . Show that P (A+ |A0 ∩ A− ) = P (A+ |A0 ).
Solution.
By definition, A+ and A− are conditionally independent given A0 , if
P (A+ ∩ A− |A0 ) = P (A+ |A0 ) P (A− |A0 )
We thus have
P (A+ |A0 ∩ A− ) =
P (A+ ∩ A0 ∩ A− )
P (A+ ∩ A0 ∩ A− )/P (A0)
=
P (A0 ∩ A− )
P (A0 ∩ A− )/P (A0 )
P (A+ ∩ A− |A0 )
=
P (A− |A0 )
= P (A+ |A0 ).
2.
Suppose that the sequence X0 , X1 , X2 , . . . of random variables with values in Z
have the Markov property.
(i) Show that the sequence X03 , X13 , X23 , . . . also has the Markov property.
(ii) Given an example where the sequence X02 , X12 , X22 , . . . does not have the
Markov property.
Hint: If x3 = y then xp= y 1/3 . For
example, x3 = −8 means that x = −2. But if
p
x2 = y then x equals |y| or − |y|. For example, x2 = 100 means that x = 10
or x = −10.
Solution.
(i) We have
3
3
= i1 , . . . X03 = i0 )
P (Xn+1
= j|Xn3 = i, Xn−1
1/3
1/3
= P (Xn+1 = j 1/3 |Xn = i1/3 , Xn−1 = i1 , . . . , X0 = i0 )
= P (Xn+1 = j 1/3 |Xn = i1/3 )
3
= P (Xn+1
= j|Xn3 = i)
(ii) Consider the MC (X0 , X1 , X2 , . . .) with transition diagram:
1
−1
1/2
1
1/2
1
0
Let us assume that
P (X0 = 0) = P (X0 = −1) = 1/2.
1
We have
2
P (X22 = 1|X12 = 1, X02 = 0) = P (Xn+1
= 1|Xn = 1, Xn−1 = 0) = 1.
On the other hand,
P (X22 = 1|X12 = 1) = 3/4.
Since the two are not equal, the process is not Markovian.
To see the computation of the last probability, do this:
P (X22 = 1|X12 = 1) =
P (X22 = 1, X12 = 1)
P (X12 = 1)
Now,
1
1
P (X22 = 1, X12 = 1) = P (X22 = 1, X12 = 1|X0 = 0) + P (X22 = 1, X12 = 1|X0 = −1)
2
2
1
1
= P (X2 = 1, X1 = −1|X0 = 0) + P (X2 = 1, X1 = −1|X0 = −1)
2
2
1
1 1
3
= ·1+ · = .
2
2 2
4
On the other hand, if X0 = 0 then X1 = 1. If X0 = 1 then X1 = ±1. In either
case X12 = 1 for sure. So P (X12 = 1) = 1.
3.
In the Dark Ages, Harvard, Dartmouth, and Yale admitted only male students.
Assume that, at that time, 80 percent of the sons of Harvard men went to
Harvard and the rest went to Yale, 40 percent of the sons of Yale men went to
Yale, and the rest split evenly between Harvard and Dartmouth; and of the sons
of Dartmouth men, 70 percent went to Dartmouth, 20 percent to Harvard, and
10 percent to Yale.
(i) Find the probability that the grandson of a man from Harvard went to
Harvard.
(ii) Modify the above by assuming that the son of a Harvard man always went to
Harvard. Again, find the probability that the grandson of a man from Harvard
went to Harvard.
(iii) Find the stationary distribution(s) of the Markov chain.
Solution. (i) The probability that the grandson of a man from Harvard went
to Harvard equals
0.82 + 0.2 × 0.3 = 0.7.
(ii) Now the probability that the grandson of a man from Harvard went to
Harvard equals 1.
(iii) We let Xn (n = 0, 1, 2, . . .) be the university attended by the the n-th
generation son. This is, by assumption, a Markov chain with values in S =
{H, D, Y }.
2
In case (i) the transition probability matrix (with this order of states (H, D, Y ))
is


0.8 0 0.2
P = 0.2 0.7 0.1
0.3 0.3 0.4
Any stationary distribution π satisfies πP = π, i.e.
π(H) = 0.8π(H) + 0.2π(D) + 0.3π(Y )
π(D) = 0.7π(D) + 0.3π(Y )
π(Y ) = 0.2π(H) + 0.1π(D) + 0.4π(Y )
These equations are linearly dependent, so I can always omit one of them: I
choose to omit the last one. The second equation gives π(D) = π(Y ). The
first one gives 0.2π(H) = (0.2 + 0.3)π(D) = 0.5π(D), so π(H) = (5/2)π(D). In
addition,
1 = π(H) + π(D) + π(Y ) = (5/2 + 1 + 1)π(D) = 9/2π(D)
whence π(D) = 2/9. So the stationary distribution π = (π(H), π(D), π(Y )) is
given by
π(H) = 5/9, π(D) = 2/9, π(Y ) = 2/9.
In case (ii) the transition probability matrix is


1
0
0
P = 0.2 0.7 0.1 .
0.3 0.3 0.4
Any stationary distribution π satisfies πP = π, i.e.
π(H) = π(H) + 0.2π(D) + 0.3π(Y )
π(D) = 0.7π(D) + 0.3π(Y )
π(Y ) = 0.1π(D) + 0.4π(Y )
These equations are linearly dependent, so I can always omit one of them: I
choose to omit the last one. But observe that π(H) cancels from the first one,
which means that it can be chosen arbitrarily: set π(H) = p, where p is any
number, such that 0 ≤ p ≤ 1. The second one gives π(D) = π(Y ). In addition,
1 = π(H) + π(D) + π(Y ) = p + 2π(D),
whence π(D) = (1 − p)/2. We then have
π(H) = p,
π(D) = (1 − p)/2,
π(Y ) = (1 − p)/2.
Since p is arbitrary, we have an infinite number of stationary distributions.
3
4.
Consider an experiment of mating rabbits. We watch the evolution of a particular gene that appears in two types, G or g. A rabbit has a pair of genes, either
GG (dominant), Gg (hybrid–the order is irrelevant, so gG is the same as Gg)
or gg (recessive). In mating two rabbits, the offspring inherits a gene from each
of its parents with equal probability. Thus, if we mate a dominant (GG) with
a hybrid (Gg), the offspring is dominant with probability 1/2 or hybrid with
probability 1/2.
Start with a rabbit of given character (GG, Gg, or gg) and mate it with a hybrid. The offspring produced is again mated with a hybrid, and the process is
repeated through a number of generations, always mating with a hybrid.
(i) Write down the transition probabilities of the Markov chain thus defined.
(ii) Assume that we start with a hybrid rabbit. Let µn be the probability distribution of the character of the rabbit of the n-th generation. In other words,
µn (GG), µn (Gg), µn(gg) are the probabilities that the n-th generation rabbit is
GG, Gg, or gg, respectively. Compute µ1 , µ2 , µ3. Can you do the same for µn
for general n?
(iii) Find the stationary distribution(s) of the Markov chain.
Solution. (i) We consider a Markov chain with values in S = {GG = 1, Gg =
2, gg = 3} and transition probability matrix
1 1 
0
2
2
1
1

P = 4 2 14  .
0 21 12
(ii) We start with X0 being distributed as µ0 = (µ0 (1), µ0(2), µ0 (3)) = (0, 1, 0),
and, letting mun = (µ0 (n), µ0 (n), µ0 (n)) be the distribution of Xn , we have
µn = µ0 Pn .
So
1
2
µ1 = µ0 P = (0, 1, 0)  14
0
µ2 = µ0 P2 = (µ0 P)P = µ1 P =
1
2
1
2
1
2
0

1
4
1
2
1
2
( 14 , 21 , 41 )  14
0
So we see that µ2 = µ1 . Therefore
= ( 41 , 12 , 41 ).
1
2
1
2
1
2
µ3 = µ2 P = µ1 P = µ2 = µ1 .
And so, for all n ≥ 1,
µn = µ1 = ( 14 , 21 , 14 ).
(iii) Since µ2 = µ1 we have
µ1 = µ1 P
4
0

1
4
1
2
= ( 41 , 12 , 41 ).
and so µ1 is a stationary distribution. We can see that there are no other
stationary distributions by seeing, for example, that the nullity of the system of
equations π = πP, π 1 = 1 is 1.
5.
A certain calculating machine uses only the digits 0 and 1. It is supposed to
transmit one of these digits through several stages. However, at every stage,
there is a probability p that the digit that enters this stage will be changed
when it leaves and a probability q = 1 − p that it won’t. Form a Markov chain
to represent the process of transmission by taking as states the digits 0 and 1.
(i) What is the matrix of transition probabilities?
(ii) For n = 2, 3 find the probability that the machine, after n stages of transmission produces the digit 0 (i.e., the correct digit).
(iii) Find the stationary distribution(s) of the Markov chain.
Solution. (i) The picture we have here is like this:
X
X
X
X
0
1
2
3
−→
···
STAGE 1 −→
STAGE 2 −→
STAGE 3 −→
where the Xn take values in S = {0, 1} and where each stage either flips the
entering digit or keeps it the same, independently of the others. So
q p
P=
p q
(ii) The probability that that the machine will produce 0 in two stages if it
starts with 0 is p2 + q 2 .
(iii) Any stationary distribution π = (π(0), π(1)) satisfies
π(0)q = π(1)p,
π(0) + π(1) = 1,
whence the stationary distribution is unique with
π(0) = π(1) = 1/2.
6.
Mr Baxter’s bank account (see Example in lecture notes) evolves, from month
to month, according to the rule
Xn+1 = max(Xn + Dn − Sn , 0),
n = 0, 1, 2, . . .
where X0 , D0 , S0 , D1 , S1 , D2 , S2 , . . . are independent random variables. Suppose
that the Sn have the common distribution
P (Sn = 0) = p,
P (Sn = 1) = q = 1 − p.
Suppose that the Dn have the common distribution
P (Dn = k) = αk (1 − α),
5
k = 0, 1, 2, . . . ,
where 0 < p, α < 1.
(i) Compute the one-step transition probabilities
(ii) (Attempt to) draw the transition diagram.
(iii) What is the form of the transition probability matrix?
(iv) Write down the equation that you need to solve in order to compute the
stationary distribution (if it exists!) of the Markov chain.
Solution.
(i) If Xn = x ≥ 1 then Xn+1 takes values in the set {x − 1, x, x + 1, x + 2, . . .}.
We have
px,x−1 = P (Xn+1 = x − 1|Xn = x)
= P (Dn − Sn = 1) = P (Dn = 0, Sn = 1) = (1 − α)q.
For m ≥ 0,
px,x+m = P (Dn − Sn = m)
= P (Dn = m, Sn = 0) + P (Dn = m + 1, Sn = 1)
= αm (1 − α)p + αm+1 (1 − α)q.
P
P
(It is always good to check that y px,y = px,x−1 + m≥0 px,x+m = 1.)
If Xn = 0 then Xn+1 takes values in the set {0, 1, 2, . . .}. For m ≥ 1, e have
p0,m = αm (1 − α)p + αm+1 (1 − α)q,
as before. But
p0,0 = P ({Dn = 0, Sn = 0} or {Dn = 1, Sn = 1} or {Dn = 0, Sn = 1})
= (1 − α)p + α(1 − α)q + (1 − α)q
= (1 − α)(1 + αq).
(ii)-(iii) In class.
(iv) For all x ≥ 1 we have that any stationary distribution satisfies
X
π(x) =
π(y)py,x
y
= π(x + 1)px+1,x + π(x)px,x +
x
X
m=1
π(x − m)πx−m,x ,
one equation for each x ≥ 1, together with
∞
X
π(x) = 1.
x=0
(We need not consider an equation for π(0), because there is always a redundancy amongst the equations π = πP owing to, precisely, the last normalisation
condition.)
6
7.
Definition: A Markov chain is called an actuarial chain if:
(a) It has a finite number of states (typically less than 10).
(b) It is an absorbing chain.
(c) It has time-varying one-step transition probabilities. 1
Consider an urn with n balls, out of which m are red and n − m blue. (See
exercise of earlier homework.) Assign the value 1 to a red and 0 to a blue ball.
Start picking the balls, one by one (without replacement), and let St be the sum
of the values of the balls you have picked up to the t-th step. Start with S0 = 0
and observe that, obviously, Sn = m. Define St = n for all t > n.
(i) Show that (St , t = 0, 1, . . .) is a Markov chain.
(ii) Compute P (St+1 = j|St = i) for all values of states i, j and ‘times’ t. Observe
that the result depends on t.
(iii) Conclude that the chain is an actuarial one.
Solution.
(i)+(ii)+(iii) Recall that St denotes the number of red balls picked by time t.
So at time t (i.e. right after the t-th selection), we have an urn that contains
n − t balls out of which m − St are red. Hence, assuming that S0 = 0, we have
P (St+1 = x + 1|St = x, St−1 , St−2 , . . .) = P (St+1 = x + 1|St = x) =
m−x
,
n−t
as long as m − x ≤ n − t, 0 ≤ x < m. On the other hand,
P (St+1 = x|St = x, St−1 , St−2 , . . .) = P (St+1 = x|St = x) = 1 −
m−x
.
n−t
8.
Consider the Ehrenfest chain with N molecules.
(i) Compute its stationary distribution π.
(ii) Suppose that N = 1020 molecules. Show that
π(N/2) ≈ 10−10 = 0.0000000001
but
π(N/2.0001) ≈ 10−60,000,000,000
which is the number 0.0.......01 (i.e. 60, 000, 000, 000 zeros after the decimal
point).
√
Hint: You may use Stirling’s approximation, i.e. n! ≈ nn e−n 2πn for large n.
(iii) If it takes about 1 millimetre to write the symbol 0 show that you need
about 5 Earths to write the last number explicitly.
Hint: The diameter of the Earth is about 12 thousand km.
(iv) If you wanted a faithful plot of π(i) as a function of i, what shape would
the plot have? Sketch it.
1
Strictly speaking, an actuarial chain is a continuous-time chain, but we shall waive this requirement for the purposes of this part of the course.
7
Solution. (i) Letting x be the number of molecules in the left room, we have,
for 1 ≤ x ≤ N − 1,
N −x+1
N
x
px,x−1 =
N
Fix a state 1 ≤ x ≤ N and Partition the state space S = {0, 1, . . . , N} into
A = {0, 1, . . . , x−1} and Ac = {x, x+1, . . . , N}. If π is a stationary distribution,
we have that the flow
XX
N −x+1
π(i)pi,j = π(x − 1)px−1,x = π(x − 1)
F (A, Ac ) =
N
i∈A j∈Ac
px−1,x =
from A to Ac must be equal to the flow
XX
x
π(j)pj,i = π(x)px,x−1 = π(x)
F (Ac , A) =
N
j∈Ac i∈A
from Ac to A. Hence
N −x+1
π(x − 1),
x
−x+2
, π(x − 2) =
for all 1 ≤ x ≤ N. Therefore π(x − 1) = N x−1
π(x) =
N −x+3
,
x−2
N −x+1N −x+2
N
· · · π(0)
x
x−1
1
N!/(N − x)!
=
π(0)
x!
N
π(0).
=
x
π(x) =
But
N
X
N X
N
π(0) = 2N π(0),
1=
π(x) =
x
x=0
x=0
whence π(0) = 2−N and so
N −N
2 ,
π(x) =
x
i.e. the binomial distribution with parameter 1/2.
(ii) If N is large then
√
N N e−N 2πN
N
p
√
≈
x
xx e−x 2πx(N − x)N −x e−(N −x) 2π(N − x)
s
2πN
e−x−N +x
N x N N −x
= x
x (N − x)N −x e−x e−(N −x)
2πx2π(N − x)
s
x N −x
1
N
N
N
√
=
x
N −x
x(N − x)
2π
8
etc, and so
If x = N/2, we have
N
N/2
≈
N
N/2
= 2N
Therefore
N/2 1
√
2π
r
N
N/2
N/2
1
√
2π
r
2
.
πN
4
= 2N
N
s
N
(N/2)(N/2)
r
r
r
2
2
2 −10
π(N/2) ≈
=
=
10 .
πN
π1020
π
On the other hand, we find, by taking logarithms, that π(N/2.0001) is millions
of orders of magnitude smaller.
(iii) Easy.
(iv)
π (x)
0
N
N/2
x
9