Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
MTHE/STAT455, STAT855 Fall 2013 MTHE/STAT455, STAT855, Stochastic Processes Midterm Exam Instructions: (a) The exam is closed book. No books are allowed. You may use one 8.5 × 11 inch sheet of notes and a calculator. (b) There are 3 questions. Stat 855 students must do all of the problems. Stat 455 students must do 2 of questions 1,2 and 3, and if you answer all three of questions 1, 2 and 3, you must specify which two you want graded (the default if you do not specify is questions 1 and 2). (c) Each question is worth 15 marks for a total of 30 marks (Mthe/Stat455) or 45 marks (Stat855) (d) Show all your work. Partial credit is given. (e) You have 120 minutes. Good luck! MTHE/STAT455, STAT855 -- Midterm Exam, 2013 Page 2 of 3 1. (15 marks) Answer the following questions by conditioning on the first move, and doing further conditioning as necessary. Define appropriate quantities. (a) (7 marks) A particle moves among the vertices of a hexagon. At each step it chooses a vertex at random from its two neighbouring vertices and moves to the chosen vertex. Starting at a given vertex find the expected number of steps until the particle reaches the opposite vertex (“opposite” meaning the one that is three edges away). (b) (8 marks) A particle moves among the vertices of a cube. At each step it chooses a vertex at random from its three neighbouring vertices and moves to the chosen vertex. Starting at a given vertex find the expected number of steps until the particle reaches the opposite vertex (“opposite” meaning the one that is three edges away). 2 (15 marks) Let X = {Xn : n ≥ 0} be a discrete-time, time-homogeneous Markov chain. Suppose that X is irreducible and has period 4. (k) (a) (9 marks) Let Yn = Xnk for n = 0, 1, . . .. For k = 2, 3, 4 consider the Markov (k) chains {Yn : n ≥ 0}, and for each of these three Markov chains, give (i) the period of the Markov chain (you may assume all states have the same period); (ii) the number of communicating classes. (You don’t need to prove your answers, but can if you want to and an incorrect proof, if reasonable, will still give you partial marks). (b) (4 marks) Let Y = {Yn : n ≥ 0} be an independent copy of X. Show that the bivariate chain {(Xn , Yn ) : n ≥ 0} cannot be irreducible. (c) (2 marks) Let Z1 , Z2 , . . . be independent and identically distributed random variables on P the postive integers with P (Zi = k) > 0 for all k ≥ 1. Let T0 = 0 and Tn = ni=1 Zi for n ≥ 1, and consider the Markov chain Wn = XTn for n ≥ 0 (this is an irreducible Markov chain but you don’t need to show this). Prove that {Wn : n ≥ 0} has period 1. (855 students may want to work on this problem last). MTHE/STAT455, STAT855 -- Midterm Exam, 2013 Page 3 of 3 3. (15 marks) Let {Xn : n ≥ 0} be a discrete-time, time-homogeneous Markov chain. Let fij (n) be the probability that the first time the chain visits j is at time n, starting in state i; i.e., fij (n) = P (Xn = j, Xn−1 6= j, . . . , X1 6= j X0 = i), and let pij (n) be the n-step transition probability from state i to state j. Assume i 6= j. (a) (5 marks) For n ≥ 1, show that pij (n) = n X fij (r)pjj (n − r). r=1 (b) (5 marks) Show that Pij (s) = Fij (s)Pjj (s) where Pij (s) = ∞ X n=0 sn pij (n) and Fij (s) = ∞ X sn fij (n) n=0 are the generating functions of {pij (n)}n and {fij (n)}n , respectively. P (c) (5 marks) Use part(b) to show that if state j is transient then ∞ n=0 pij (n) < ∞ for all i.