Download .pdf

MTHE/STAT455, STAT855
Fall 2013
MTHE/STAT455, STAT855, Stochastic Processes
Midterm Exam
Instructions:
(a) The exam is closed book. No books are allowed. You may use one 8.5 × 11 inch sheet
of notes and a calculator.
(b) There are 3 questions. Stat 855 students must do all of the problems. Stat 455 students
must do 2 of questions 1,2 and 3, and if you answer all three of questions 1, 2 and
3, you must specify which two you want graded (the default if you do not specify is
questions 1 and 2).
(c) Each question is worth 15 marks for a total of 30 marks (Mthe/Stat455) or 45 marks
(Stat855)
(d) Show all your work. Partial credit is given.
(e) You have 120 minutes. Good luck!
MTHE/STAT455, STAT855 -- Midterm Exam, 2013
Page 2 of 3
1. (15 marks) Answer the following questions by conditioning on the first move, and doing
further conditioning as necessary. Define appropriate quantities.
(a) (7 marks) A particle moves among the vertices of a hexagon. At each step it
chooses a vertex at random from its two neighbouring vertices and moves to the
chosen vertex. Starting at a given vertex find the expected number of steps until
the particle reaches the opposite vertex (“opposite” meaning the one that is three
edges away).
(b) (8 marks) A particle moves among the vertices of a cube. At each step it chooses
a vertex at random from its three neighbouring vertices and moves to the chosen
vertex. Starting at a given vertex find the expected number of steps until the
particle reaches the opposite vertex (“opposite” meaning the one that is three
edges away).
2 (15 marks) Let X = {Xn : n ≥ 0} be a discrete-time, time-homogeneous Markov chain.
Suppose that X is irreducible and has period 4.
(k)
(a) (9 marks) Let Yn = Xnk for n = 0, 1, . . .. For k = 2, 3, 4 consider the Markov
(k)
chains {Yn : n ≥ 0}, and for each of these three Markov chains, give
(i) the period of the Markov chain (you may assume all states have the same
period);
(ii) the number of communicating classes.
(You don’t need to prove your answers, but can if you want to and an incorrect
proof, if reasonable, will still give you partial marks).
(b) (4 marks) Let Y = {Yn : n ≥ 0} be an independent copy of X. Show that the
bivariate chain {(Xn , Yn ) : n ≥ 0} cannot be irreducible.
(c) (2 marks) Let Z1 , Z2 , . . . be independent and identically distributed random variables on
P the postive integers with P (Zi = k) > 0 for all k ≥ 1. Let T0 = 0 and
Tn = ni=1 Zi for n ≥ 1, and consider the Markov chain Wn = XTn for n ≥ 0
(this is an irreducible Markov chain but you don’t need to show this). Prove that
{Wn : n ≥ 0} has period 1. (855 students may want to work on this problem
last).
MTHE/STAT455, STAT855 -- Midterm Exam, 2013
Page 3 of 3
3. (15 marks) Let {Xn : n ≥ 0} be a discrete-time, time-homogeneous Markov chain. Let
fij (n) be the probability that the first time the chain visits j is at time n, starting in
state i; i.e.,
fij (n) = P (Xn = j, Xn−1 6= j, . . . , X1 6= j X0 = i),
and let pij (n) be the n-step transition probability from state i to state j. Assume i 6= j.
(a) (5 marks) For n ≥ 1, show that
pij (n) =
n
X
fij (r)pjj (n − r).
r=1
(b) (5 marks) Show that
Pij (s) = Fij (s)Pjj (s)
where
Pij (s) =
∞
X
n=0
sn pij (n)
and
Fij (s) =
∞
X
sn fij (n)
n=0
are the generating functions of {pij (n)}n and {fij (n)}n , respectively.
P
(c) (5 marks) Use part(b) to show that if state j is transient then ∞
n=0 pij (n) < ∞
for all i.