Download Assignment 4

Fourth Assignment
1. (4 points) Let (Xn , n = 0, 1, . . .) be a Markov chain with values in a countable set S. (i) Show
that if i ∈ S is recurrent and i communicates with some j ∈ S then j is recurrent. (ii) Show that
if i ∈ S is positive recurrent and i communicates with some j ∈ S then j is positive recurrent.
2. (4 points) Consider a Markov chain with values in the set of nonnegative integers and transition
probabilities pi,i+1 = pi , pi,i−1 = qi with pi + qi = 1 for all i and p0 = 1. Give a necessary and
sufficient condition so that there is a unique stationary distribution.
3. (5 points) Let f (x) = 2 min(x, 1 − x), 0 ≤ x ≤ 1. Let X0 be a random variable with values in
[0, 1] and define, recursively,
Xn+1 = f (Xn ), n ≥ 0.
Then (Xn , n = 0, 1, . . .) is a Markov chain with values in the uncountable set [0, 1]. (i) Show
that there is a way to choose the law of X0 so that each Xn has the same law. (ii) Can you find
this law?
4. (5 points) Let (Xn , n = 0, 1, . . .) be a Markov chain with values in a countable set S. Consider a
sequence 0 := T0 < T1 < T2 < · · · of strictly increasing random variables and define Y (t), t ≥ 0,
by Y (t) := Xk , Tk ≤ t < Tk+1 , k = 0, 1, . . . Show that a necessary condition for Y to have
the Markov property is that the differences Tk+1 − Tk , k = 0, 1, . . ., are independent random
variables, each having an exponential distribution.
5. (6 points) Consider a connected graph G = (V, E) and, to each P
edge e ∈ E, assign weight Re
interpreting it as electric resistance. For a vertex a, define Ra = ( (1/Re ))−1 , where the sum is
eff be the effective
over all edges e which have a as an endpoint. Pick two vertices a, b and let Rab
resistance; that is, if a unit voltage source is applied at terminals a, b then a total current of size
Rab is induced. Now let (Xn , n ≥ 0) be a random walk on the graph whose law is determined
−1
by the law of X0 and the transition probabilities pij = P (Xn+1 = j | Xn = i) = Rij
/Ri−1 . Let
Ta := inf{n ≥ 0 : Xn = a}.
eff .
(i) Show that Pa (Tb < Ta ) = Ra /Rab
(ii) Use this for the simple random walk1 on the hypercube {0, 1}N to compute the probability
that, starting from a = (0, 0, . . . , 0), the random walk hits b = (1, 1, . . . , 1) before returning to a.
6. (8 points) Let g(x, y) be the Green’s kernel for a random walk with values in Z3 , that is, g(x, y)
is the expected total number of visits to y ∈ Z3 , when the random walk starts from x ∈ Z3 . (i)
Show that
Z
eiθ(y−x)
3
dθ,
g(x, y) =
(2π)3 B 3 − cos θ1 − cos θ2 − cos θ3
where B = [−π, π]3 . (ii) (Hard) Use this to show that
|x − y|g(x, y) →
3
,
2π
as |x − y| → ∞.
(In other words, g(x, y) behaves like the classical Newtonian potential–it is proportional to the
inverse of the distance–when x is far from y.)
A simple random walk on the hypercube {0, 1}N moves from state x to state x + ei with probability 1/N , where
ei ∈ {0, 1}N has 1 in the i-th position and 0 everywhere else, and + is interpreted as modulo 2. In other words, at each
time, a coordinate is picked at random and flipped.
1
1