Download Homework #1

Homework #1
(due Wednesday, April 12, in class)
1. Suppose (Xn )∞
n=0 is a discrete-time Markov chain with state space S = {1, 2} and transition
matrix
0.8 0.2
P=
.
0.3 0.7
(a) Calculate P (X2 = 1, X3 = 1|X0 = 1, X1 = 1).
(b) Calculate P (X2 = 1|X0 = 1).
(c) Calculate P (X1 = 1|X0 = 1, X2 = 1).
∞
2. Suppose (Xn )∞
n=0 and (Yn )n=1 are time-homogeneous Markov chains with state space S
and transition probabilities p(i, j) and q(i, j) respectively. Suppose (Xn )∞
n=0 is independent
of (Yn )∞
.
For
all
n,
let
Z
=
(X
,
Y
),
meaning
that
if
X
=
i
and
Y
n
n n
n
n = j, then Zn is
n=0
∞
the ordered pair (i, j). Show that (Zn )n=0 is a time-homogeneous Markov chain with state
space S × S, and give a formula for the transition probabilities.
3. Suppose a coin is tossed repeatedly. Let X0 = 0, and for n ≥ 1, let Xn be the number of
heads in the first n tosses. Let Y0 = 0, let Y1 = X1 , and for n ≥ 2, let Yn be the number of
heads in the (n − 1)st and nth tosses of the coin.
(a) Is (Xn )∞
n=0 a Markov chain? Show that your answer is correct.
(b) Is (Yn )∞
n=0 a Markov chain? Show that your answer is correct.
4. A gambler starts out with $1 and wishes to increase his fortune to $5. Each time the
gambler makes a bet, he wins the amount of the bet with probability 1/2 and loses the
amount of the bet with probability 1/2. The gambler decides to follow a strategy called
“bold play”. If he has $2 or less, he bets everything he has. If he has $3, then he bets $2,
and if he has $4, then he bets $1, so that he gets to $5 if he wins. He keeps placing bets
until either his fortune reaches $5 or he loses all of his money.
(a) Model this process by a Markov chain. Give the state space and transition matrix.
(b) Find the probability that the gambler succeeds in accumulating $5 before going broke.
5. In the board game Parcheesi, a player rolls two 6-sided dice on each turn. If a player rolls
doubles (that is, the same number appears on both dice) three times in a row, then she has
to move a token back to the starting position. On average, how many times will she have
to roll the dice before rolling doubles three times in a row?