Download Lecture No. 10(A) : Method of Conditional Probabilities 1 - CSE-IITM

IITM-CS6845: Theory Toolkit
March 9, 2011
Lecture No. 10(A) : Method of Conditional Probabilities
Lecturer: N. S. Narayanaswamy
Scribe: Sajin Koroth
Theme: Explicit Constructions
Lecture Plan: Today we will be seeing the method of conditional probabilities. It is more
of an application than an elaborate theory like d-wise independent sample spaces which
we saw in the last lecture. We will motivate the method by applying
it to a non-trivial
problem, i.e to obtain 2-coloring of a complete graph to get at most nr 2r monochromatic
2(2)
r-cliques. We will see one more application, which is a maximization problem(unlike the
earlier one which tries to minimize cliqes), k-SAT. We will end the lecture by generalizing
the method of conditional probabilities to a framework which incorporates all the above
mentioned applications.
1
Explicit Construction of 2-coloring of a complete graph
with bounded monochromatic r-cliques using method of
Conditional Probabilities
Recall that in the very first lecture to obtain a lowerbound on the Ramsey number R(r, r)
we used the following probabilistic method argument. Let us consider a random 2-coloring
of the edge set of complete graph, Kn . The expected number of monochromatic r-cliques
are
r
n 2 E [#r − cliques] =
∵ Two of the 2(2) colorings are monochromatic
r
r 2(2)
r
Hence if nr 21−(2) < 1, then there is at least one coloring where there are no monochromatic
r-clique, hence for all such n, r, R(r, r) > n. The above analysis also guarantees that for
r
any complete graph Kn , there is a 2-coloring of its edge set such there are at most nr 21−(2)
monochromatic r-cliques. We are interested in constructing such a 2-coloring. Note that
this task is not obvious, no naive approaches like color everything red or color everything
blue would not work, as you need to control both red r-cliques and blue r-cliques. But note
that if we analyze the coloring step by step, then we can write expression for conditional
probability of introducing an r-monochromatic clique, by coloring the ei+1 th edge given a
coloring of first i-edges in some fixed ordering of the edges. Let Wi denote the number of
No. 10(A)-1
monochromatic r-cliques in the partially colored graph Gi where only the edges, e1 , . . . , ei
are colored. Hence the we can write E[Wi+1 ] as
E[Wi+1 ] = E[Wi+1 |ei+1 = red]Pr[ei+1 = red] + E[Wi+1 |ei+1 = blue]Pr[ei+1 = blue]
E[Wi+1 |ei+1 = red] + E[Wi+1 |ei+1 = blue]
=
2
Hence we get that,
E[Wi+1 ] ≥ min {E[Wi+1 |ei+1 = red], E[Wi+1 |ei+1 = blue]}
So if we color the ei+1 ’th edge to the one color which achieves the minimum in the above
expression, we are guaranteeing that the number of monochromatic cliques in our graph
is at most the expected number of monochromatic cliques in a random partial coloring.
Hence after coloring all the edges we get to a coloring where the number of monochromatic
r
r-cliques is at most nr 21−(2) . The costliest step in our explicit construction is checking the
number of monochromatic r-cliques in the partially colored graph after fixing the color of
ith edge to a specific color, for both the colors. The naive algorithm for checking r-cliques
takes time nr and hence if r is a constant our explicit construction algorithm runs in time
polynomial.
2
Explicit construction for a maximization problem
Consider the following problem, MAX-SAT, which given a 3 − CN F formula on n variables
and m clauses which is such that in any clause the literals appearing are distinct, asks for
an assignment to the variables x1 , . . . , xn which maximizes the number of satisfied clauses.
Let Xj , 1 ≤ j ≤ m be the indicator random variable denoting whether the jth clause is
satisfied by the random assignment or not. Since the three literals appearing in the jth
clause are distinct, out of the 23 = 8 possible values of these literals only one (0, 0, 0) is
bad. P
Hence E[Xj ] = 87 . Let X denote the total number of
P satisfied clauses, by definition
X = j Xj . Hence using linearity of expectation E[X] = j E[Xj ] = 7m
8 .
We are interested in, given a MAX-SAT instance φ construct an assignment which satisfies
at least 7m
8 clauses. The construction is very similar to the construction of above. But
the subtle difference is in the fact that unlike the earlier problem this is a maximization
problem.
Let us analyze a random assignment of truth values to x1 , . . . , xn step by step. Let Wi
denote the expected number of satisfied clauses in a partial assignment which fixes only
x1 , . . . , xi . Similar to the above problem we can write Wi+1 as
E[Wi+1 ] = E[Wi+1 |ei+1 = 0]Pr[ei+1 = 0] + E[Wi+1 |ei+1 = 1]Pr[ei+1 = 1]
E[Wi+1 |ei+1 = 0] + E[Wi+1 |ei+1 = 1]
=
2
No. 10(A)-2
Hence,
E[Wi+1 ] ≤ max {E[Wi+1 |ei+1 = 0], E[Wi+1 |ei+1 = 1]}
So if we assign xi+1 such that it maximizes the above expression we guarantee that the
number of satisfied clauses are at least that of the expected number of satisfied clauses.
Note that to find the number of satisfied clauses is easy as you just have to go through each
clause and see if at least one of the three literals is set to 1, which a naive approach solves
in time O(nm). Hence the construction is a polynomial time construction.
3
Generalized Conditional Probability Method
Consider the uniform distribution on {0, 1}l . Let A1 , . . . , As be a set of events such that
s
X
Pr [Ai ] = k
i=1
If we define indicator random variable Xi for the event Ai then the above condition tells us
that
" s
#
X
E [X] = E
Xi
i=1
=
=
s
X
i=1
s
X
E [Xi ] (∵ Linearity of Expectation)
Pr [Ai ]
i=1
= k
That is the expected number of events which happen from {A1 , . . . , As } is k. Hence this
also tells us that there is at least one point in {0, 1}l where the number of events which
happen from {A1 , . . . , As } is at most k. To obtain that point algorithmically we can use
the method of conditional probabilities as follows. Consider the fixation of xi , 1 ≤ i ≤ l to
either 1 or 0 based on whether Pr[Ai |x1 = b1 , . . . , xi−1 = bi−1 , xi = 0] is the minimum or
Pr[Ai |x1 = b1 , . . . , xi−1 = bi−1 , xi = 1] is the minimum (where b1 , . . . , bi−2 are the values
fixed by the algorithm in stages 1, . . . , i − 1). This will give us a point in {0, 1}l in which
No. 10(A)-3
at most k events from A1 , . . . , As will happen because,
1
Pr[Ai |x1 · · · xi−1 = b1 · · · bi−1 , xi = 0] +
2
1
Pr[Ai |x1 · · · xi−1 = b1 · · · bi−1 , xi = 1]
2
≥ min{Pr[Ai |x1 · · · xi−1 = b1 · · · bi−1 , xi = 0],
Pr[Ai |x1 · · · xi−1 = b1 · · · bi−1 ] =
Pr[Ai |x1 · · · xi−1 = b1 · · · bi−1 , xi = 1]}
Also note that the algorithm maintains that ∀i, 1 ≤ i ≤ l,
Pr[Ai |x1 · · · xi−1 = b1 · · · bi−1 ] ≥ Pr[Ai |x1 · · · xi = b1 · · · bi ]
Hence,
Pr[Ai ] ≥ Pr[Ai |x1 · · · xl = b1 · · · bl ]
So we are guaranteed that at x1 · · · xl = b1 · · · bl at most k events occur. Also note that we
could have ensured at least k events occur by choosing max instead of min in our algorithm.
No. 10(A)-4