Download Assignment 4

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts
no text concepts found
Transcript
Fourth Assignment
1. (4 points) Let (Xn , n = 0, 1, . . .) be a Markov chain with values in a countable set S. (i) Show
that if i ∈ S is recurrent and i communicates with some j ∈ S then j is recurrent. (ii) Show that
if i ∈ S is positive recurrent and i communicates with some j ∈ S then j is positive recurrent.
2. (4 points) Consider a Markov chain with values in the set of nonnegative integers and transition
probabilities pi,i+1 = pi , pi,i−1 = qi with pi + qi = 1 for all i and p0 = 1. Give a necessary and
sufficient condition so that there is a unique stationary distribution.
3. (5 points) Let f (x) = 2 min(x, 1 − x), 0 ≤ x ≤ 1. Let X0 be a random variable with values in
[0, 1] and define, recursively,
Xn+1 = f (Xn ), n ≥ 0.
Then (Xn , n = 0, 1, . . .) is a Markov chain with values in the uncountable set [0, 1]. (i) Show
that there is a way to choose the law of X0 so that each Xn has the same law. (ii) Can you find
this law?
4. (5 points) Let (Xn , n = 0, 1, . . .) be a Markov chain with values in a countable set S. Consider a
sequence 0 := T0 < T1 < T2 < · · · of strictly increasing random variables and define Y (t), t ≥ 0,
by Y (t) := Xk , Tk ≤ t < Tk+1 , k = 0, 1, . . . Show that a necessary condition for Y to have
the Markov property is that the differences Tk+1 − Tk , k = 0, 1, . . ., are independent random
variables, each having an exponential distribution.
5. (6 points) Consider a connected graph G = (V, E) and, to each P
edge e ∈ E, assign weight Re
interpreting it as electric resistance. For a vertex a, define Ra = ( (1/Re ))−1 , where the sum is
eff be the effective
over all edges e which have a as an endpoint. Pick two vertices a, b and let Rab
resistance; that is, if a unit voltage source is applied at terminals a, b then a total current of size
Rab is induced. Now let (Xn , n ≥ 0) be a random walk on the graph whose law is determined
−1
by the law of X0 and the transition probabilities pij = P (Xn+1 = j | Xn = i) = Rij
/Ri−1 . Let
Ta := inf{n ≥ 0 : Xn = a}.
eff .
(i) Show that Pa (Tb < Ta ) = Ra /Rab
(ii) Use this for the simple random walk1 on the hypercube {0, 1}N to compute the probability
that, starting from a = (0, 0, . . . , 0), the random walk hits b = (1, 1, . . . , 1) before returning to a.
6. (8 points) Let g(x, y) be the Green’s kernel for a random walk with values in Z3 , that is, g(x, y)
is the expected total number of visits to y ∈ Z3 , when the random walk starts from x ∈ Z3 . (i)
Show that
Z
eiθ(y−x)
3
dθ,
g(x, y) =
(2π)3 B 3 − cos θ1 − cos θ2 − cos θ3
where B = [−π, π]3 . (ii) (Hard) Use this to show that
|x − y|g(x, y) →
3
,
2π
as |x − y| → ∞.
(In other words, g(x, y) behaves like the classical Newtonian potential–it is proportional to the
inverse of the distance–when x is far from y.)
A simple random walk on the hypercube {0, 1}N moves from state x to state x + ei with probability 1/N , where
ei ∈ {0, 1}N has 1 in the i-th position and 0 everywhere else, and + is interpreted as modulo 2. In other words, at each
time, a coordinate is picked at random and flipped.
1
1