Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

no text concepts found

Transcript

Fourth Assignment 1. (4 points) Let (Xn , n = 0, 1, . . .) be a Markov chain with values in a countable set S. (i) Show that if i ∈ S is recurrent and i communicates with some j ∈ S then j is recurrent. (ii) Show that if i ∈ S is positive recurrent and i communicates with some j ∈ S then j is positive recurrent. 2. (4 points) Consider a Markov chain with values in the set of nonnegative integers and transition probabilities pi,i+1 = pi , pi,i−1 = qi with pi + qi = 1 for all i and p0 = 1. Give a necessary and sufficient condition so that there is a unique stationary distribution. 3. (5 points) Let f (x) = 2 min(x, 1 − x), 0 ≤ x ≤ 1. Let X0 be a random variable with values in [0, 1] and define, recursively, Xn+1 = f (Xn ), n ≥ 0. Then (Xn , n = 0, 1, . . .) is a Markov chain with values in the uncountable set [0, 1]. (i) Show that there is a way to choose the law of X0 so that each Xn has the same law. (ii) Can you find this law? 4. (5 points) Let (Xn , n = 0, 1, . . .) be a Markov chain with values in a countable set S. Consider a sequence 0 := T0 < T1 < T2 < · · · of strictly increasing random variables and define Y (t), t ≥ 0, by Y (t) := Xk , Tk ≤ t < Tk+1 , k = 0, 1, . . . Show that a necessary condition for Y to have the Markov property is that the differences Tk+1 − Tk , k = 0, 1, . . ., are independent random variables, each having an exponential distribution. 5. (6 points) Consider a connected graph G = (V, E) and, to each P edge e ∈ E, assign weight Re interpreting it as electric resistance. For a vertex a, define Ra = ( (1/Re ))−1 , where the sum is eff be the effective over all edges e which have a as an endpoint. Pick two vertices a, b and let Rab resistance; that is, if a unit voltage source is applied at terminals a, b then a total current of size Rab is induced. Now let (Xn , n ≥ 0) be a random walk on the graph whose law is determined −1 by the law of X0 and the transition probabilities pij = P (Xn+1 = j | Xn = i) = Rij /Ri−1 . Let Ta := inf{n ≥ 0 : Xn = a}. eff . (i) Show that Pa (Tb < Ta ) = Ra /Rab (ii) Use this for the simple random walk1 on the hypercube {0, 1}N to compute the probability that, starting from a = (0, 0, . . . , 0), the random walk hits b = (1, 1, . . . , 1) before returning to a. 6. (8 points) Let g(x, y) be the Green’s kernel for a random walk with values in Z3 , that is, g(x, y) is the expected total number of visits to y ∈ Z3 , when the random walk starts from x ∈ Z3 . (i) Show that Z eiθ(y−x) 3 dθ, g(x, y) = (2π)3 B 3 − cos θ1 − cos θ2 − cos θ3 where B = [−π, π]3 . (ii) (Hard) Use this to show that |x − y|g(x, y) → 3 , 2π as |x − y| → ∞. (In other words, g(x, y) behaves like the classical Newtonian potential–it is proportional to the inverse of the distance–when x is far from y.) A simple random walk on the hypercube {0, 1}N moves from state x to state x + ei with probability 1/N , where ei ∈ {0, 1}N has 1 in the i-th position and 0 everywhere else, and + is interpreted as modulo 2. In other words, at each time, a coordinate is picked at random and flipped. 1 1