Download CS5314 Randomized Algorithms [

CS5314 Randomized Algorithms
Homework 2 (Suggested Solution)
1. A fixed point of a permutation π : [1, n] → [1, n] is a value for which π(x) = x. Find the
variance in the number of fixed points of a permutation chosen uniformly at random from
all permutations.
P
Hint: Let Xi be an indicator such that Xi = 1 if π(i) P
= i. Then, ni=1 Xi is the number
of fixed points. You cannot use linearity to find Var[ ni=1 Xi ], but you can calculate it
directly.
Ans. For each i, we have
E[Xi ] = Pr(Xi = 1) = 1/n and
E[Xi2 ] = Pr(Xi2 = 1) = 1/n.
For any i, j with i 6= j, we have
E[Xi Xj ] = Pr(Xi Xj = 1) = Pr(Xi = 1 ∩ Xj = 1)
= Pr(Xi = 1)Pr(Xj = 1 | Xi = 1) =
1
.
n(n − 1)
P
P
Let X be the number of fixed points. So, X = ni=1 Xi , and E[X] = ni=1 E[Xi ] = 1.
Then, we have
Ã
!2 
n
X
Var[X] = E[X 2 ] − (E[X])2 = E 
Xi  − 1
i=1
=
n
X
i=1
E[Xi2 ] +
X
E[Xi Xj ] − 1 = n ·
i6=j
1
1
+ n(n − 1) ·
− 1 = 1.
n
n(n − 1)
2. Generalize the median-finding algorithm to find the kth largest item in a set of n items for
any given value of k. Prove that your resulting algorithm is correct, and bound its running
time.
Ans. We first consider the case where k ≤ n/2. For this case, we scan the n items and
obtain the minimum value, say, t. Then, we add n − 2k + 1 items to the set of n items,
each item having a value equal to t. It is easy to check that the kth largest item among
the original n items will be the median of the new set of 2n − 2k + 1 items. Therefore, we
can apply the median-finding algorithm for the new set to obtain the desired kth largest
item of the original set. The running time is O(n).
For the case where k > n/2, we proceed by adding items with maximum value instead.
The running time for this case is also O(n).
3. The weak law of large numbers state that, if X1 , X2 , X3 , . . . are independent and identically
distributed random variables with finite mean µ and finite standard deviation σ, then for
any constant ε > 0 we have
¯
¶
µ¯
¯ X1 + X2 + · · · + Xn
¯
lim Pr ¯¯
− µ¯¯ > ε = 0.
n→∞
n
1
Use Chebyshev’s inequality to prove the weak law of large numbers.
Ans. Firstly,
¯
µ¯
¶
¯ X1 + X2 + · · · + Xn
¯
Pr ¯¯
− µ¯¯ > ε = Pr (|(X1 + X2 + · · · + Xn ) − nµ| > nε) .
n
By Chebyshev’s inequality and the independence of Xi ’s, we have:
Pr (|(X1 + X2 + · · · + Xn ) − nµ| > nε) ≤
Pn
=
Var[X1 + X2 + · · · + Xn ]
(n²)2
Var[Xi ]
nσ 2
σ2
=
=
.
n2 ²2
n2 ²2
n²2
i=1
Combining, we have
¯
¶
µ¯
¯
¯ X1 + X2 + · · · + Xn
σ2
− µ¯¯ > ε ≤ lim 2 = 0,
0 ≤ lim Pr ¯¯
n→∞ n²
n→∞
n
which completes the proof of the weak law of large numbers.
4. Consider a collection X1 , . . . , Xn P
of n independent integers chosen uniformly at random
from the set {0, 1, 2}. Let X = ni=1 Xi and 0 < δ < 1. Derive a Chernoff bound for
Pr(X ≥ (1 + δ)n) and Pr(X ≤ (1 − δ)n).
Ans. Let Yi = Xi − 1, so that Yi ’s are independent, and we have
Pr(Yi = 1) = Pr(Yi = 0) = Pr(Yi = −1) = 1/3.
Then, we have for any i,
t Yi
E[e
1
1
2
] = et + e−t =
3
3
3
µ
1 t 1 −t
e + e
2
2
¶
2 2
≤ et /2 ,
3
where the last inequality is proven in class.
P
Let Y = ni=1 Yi . Based on the above inequality and the independence of Yi ’s, we have
µ ¶n
n
Y
2
2
t Yi
t Yi
E[e ] =
E[e ] ≤
et n/2 .
3
i=1
Then, for any t > 0, by Markov inequality,
Pr(Y ≥ a) = Pr(e
tY
E[et Y ]
≥e )≤
≤
eta
ta
µ ¶n
2
2
et n/2−ta .
3
By setting t = a/n, we obtain
µ ¶n
2
2
Pr(Y ≥ a) ≤
e−a /2n .
3
Pn
Pn
+ 1) = Y + n. So we have:
µ ¶n
2
2
Pr(X ≥ (1 + δ)n) = Pr(Y ≥ δn) ≤
e−δ n/2 .
3
On the other hand, X =
i=1
Xi =
i=1 (Yi
2
Next, by symmetry, we notice that
µ ¶n
2
2
Pr(Y ≤ −a) ≤
e−a /2n ,
3
so we have:
µ ¶n
2
2
Pr(X ≤ (1 − δ)n) = Pr(Y ≤ −δn) ≤
e−δ n/2 .
3
5. Let X1 , X2 , . . . , Xn be independent Poisson trials such that Pr(Xi ) = pi . Let X =
and µ = E[X]. During the class, we have learnt that for any δ > 0,
µ
¶µ
eδ
Pr(X ≥ (1 + δ)µ) <
.
(1 + δ)(1+δ)
Pn
i=1
Xi
In fact, the above inequality holds for the weighted
Pn sum of Poisson trials. Precisely, let
a1 , . . . , an be real numbers in [0, 1]. Let W = i=1 ai Xi and ν = E[W ]. Then, for any
δ > 0,
µ
¶ν
eδ
Pr(W ≥ (1 + δ)ν) <
.
(1 + δ)(1+δ)
(a) Show that the above bound is correct.
P
Ans. Since W = ni=1 ai Xi , we have
ν = E[W ] =
n
X
ai E[Xi ] =
i=1
For any i,
n
X
ai pi .
i=1
£
¤
ta
E etai Xi = pi etai + (1 − pi ) = 1 + pi (etai − 1) ≤ epi (e i −1) .
Claim 1. For any x ∈ [0, 1], etx − 1 ≤ x(et − 1).
Proof. Let f (x) = x(et − 1) − etx + 1. Then, f 0 (x) = (et − 1) − tetx , so that f 0 (x) = 0
only when x = x∗ = (ln(et − 1) − ln t)/t. Also, f 00 (x) = −t2 etx ≤ 0, which implies
that x∗ is a unique maximum point.
In other words, for x ∈ [0, 1], f (x) achieves minimum value either at f (0) or f (1).
Thus, f (x) ≥ min{f (0), f (1)} = 0 for all x ∈ [0, 1], and the claim follows.
Hence,
£
¤
ta
t
E etai Xi ≤ epi (e i −1) ≤ eai pi (e −1) ,
and by the independence of Xi ’s and property of MGF,
n
n
£
¤ Y
£
¤ Y
t
t
E etW =
E etai Xi ≤
eai pi (e −1) = eν(e −1) .
i=1
i=1
For any t > 0, we have
t
tW
Pr(W ≥ (1 + δ)ν) = Pr(e
t(1+δ)ν
≥e
E[etW ]
eν(e −1)
≤ t(1+δ)ν ≤ t(1+δ)ν .
e
e
Then, for any δ > 0, we can set t = ln(1 + δ) > 0 and obtain:
µ
¶ν
eδ
.
Pr(W ≥ (1 + δ)ν) <
(1 + δ)(1+δ)
3
(b) Prove a similar bound for the probability Pr(W ≤ (1 − δ)ν) for any 0 < δ < 1.
Ans. For any t < 0, we have
t
tW
Pr(W ≤ (1 − δ)ν) = Pr(e
≥e
E[etW ]
eν(e −1)
≤ t(1−δ)ν ≤ t(1−δ)ν .
e
e
t(1−δ)ν
Then, for any 0 < δ < 1, we can set t = ln(1 − δ) < 0 and obtain:
µ
¶ν
e−δ
Pr(W ≤ (1 − δ)ν) <
.
(1 − δ)(1−δ)
6. Let X1 , . . . , Xn be independent random variables such that
Pr(Xi = 1 − pi ) = pi
Let X =
Pn
i=1
and
Pr(Xi = −pi ) = 1 − pi .
Xi . Prove
2 /n
Pr(|X| ≥ a) ≤ 2e−2a
.
Hint: You may assume the inequality (no need to prove)
pi eλ(1−pi ) + (1 − pi )e−λpi ≤ eλ
2 /8
.
This inequality is difficult to prove directly.1
Ans. For any i, the MGF of Xi is:
MXi (t) = E[etXi ] = pi et(1−pi ) + (1 − pi )e−tpi ≤ et
2 /8
.
Since Xi ’s are independent, we have
E[etX ] =
n
Y
2 n/8
MXi (t) ≤ et
.
i=1
For any t > 0, we have
tX
Pr(X ≥ a) = Pr(e
E[etX ]
2
≥e )≤
≤ et n/8−ta .
ta
e
ta
Then, for any a > 0, by setting t = 4a/n > 0, we have
Pr(X ≥ a) ≤ e−2a
2 /n
.
Similarly, for any t < 0, we have
tX
Pr(X ≤ −a) = Pr(e
−ta
≥e
E[etX ]
2
) ≤ −ta ≤ et n/8+ta .
e
Then, for any a > 0, by setting t = −4a/n < 0, we have
Pr(X ≤ −a) ≤ e−2a
Combining, we have:
2 /n
2 /n
Pr(|X| ≥ a) ≤ 2e−2a
1
.
.
I do not know how to prove this either. If somehow you know how to prove it, please let us know!
4
7. (Further studies: No marks) Previously, we have shown the expected time for the Randomized Quicksort to sort n numbers is O(n log n). Here, we want to prove further that
the algorithm runs in O(n log n) time with high probability. Consider the following view of
Randomized Quicksort. Every point in the algorithm where it decides on a pivot element
is called a node. Suppose the size of the set to be sorted at a particular node is s. The node
is called good if the pivot element divides the set into two parts, each of size not exceeding
2s/3. Otherwise, the node is called bad.
The nodes can be thought of as forming a tree in which the root node has the whole set
to be sorted and its children have the two sets formed after the first pivot step and so on.
(a) Show that the number of good nodes in any path from the root to a leaf in this tree
is not greater than c log2 n, where c is some positive constant.
Ans. Number of good nodes is at most log n/ log(3/2) = c log n.
(b) Show that, with high probability (greater than 1 − 1/n2 ), the number of nodes in a
given root to a leaf path of the tree is not greater than c0 log2 n, where c0 is another
constant. (Hint: If c0 is much larger than c, there must be many bad nodes on the
path... Will that be likely to occur?)
Ans. Consider any path P from the root to a leaf. Let ` be its length and B be the
number of bad nodes in the path. From (a), we know that B ≥ ` − c log n. Also, each
node is bad with probability 2/3.
For large enough c0 , say c0 = 100c, we have
Pr(` > c0 log n) ≤ Pr(B > c0 log n − c log n) ≤ (2/3)c
0
log n−c log n
< n−2 .
(c) Show that, with high probability (greater than 1 − 1/n), the number of nodes in the
longest root to leaf path is not greater than c0 log2 n. (Hint: How many nodes are
there in the tree?)
Ans. Number of leaves in the tree is n. The result follows from (b) by union bound.
(d) Use your answers to show that the running time of Randomized Quicksort is O(n log n)
with probability at least 1 − 1/n.
Ans. From part (c), with probability at least 1 − 1/n, the height of the tree is
O(log n). On the other hand, each element is compared at most once in a particular
level in the tree, so that each level requires O(n) time. Thus, with probability at least
1 − 1/n, the running time is O(n log n).
5