Download Heuristics for finding large independent sets, with applications to

Heuristics for finding large independent sets,
with applications to coloring semi-random graphs
(Extended abstract)
Uriel Feige
Abstract
We study a semi-random graph model for finding independent sets. For > 0, an n-vertex graph with an independent set S of size n is constructed by blending random
and adversarial decisions. Randomly and independently
with probability p, each pair of vertices, such that one is
in S and the other is not, is connected by an edge. An adversary can then add edges arbitrarily (provided that S remains an independent set). The smaller p is, the larger the
control the adversary has over the semi-random graph. We
design heuristics that with high probability recover S when
p > (1 + ) ln n=jS j, for any constant > 0. We show that
when p < (1 , ) ln n=jS j, an independent set of size jS j
cannot be recovered, unless NP BPP .
We use our results to obtain greatly improved coloring algorithms for the model of k -colorable semi-random
graphs introduced by Blum and Spencer.
1. Introduction
An independent set in a graph is a set of vertices no
two of which are connected by an edge. Finding a maximum size independent set (MIS) is a fundamental problem
in combinatorial optimization, and is related to many other
problems such as vertex cover (the complement of an independent set), clique (an independent set in the complement
of the graph), and coloring (covering the vertices of a graph
by the minimum number of independent sets). As is well
known, MIS is NP-hard.
Much work was devoted to developing heuristics (we use
the term heuristic to denote algorithms that are not always
guaranteed to return the optimal solution) for finding large
Department of Applied Math and Computer Science, the Weizmann
Institute, Rehovot 76100, Israel. [email protected].
Incumbent of the Joseph and Celia Reskin Career Development Chair. Part
of this work done while the author was visiting the NEC Research Institute.
y NEC Research Institute, 4 Independence Way, Princeton, New Jersey,
08540. [email protected]
Joe Kiliany
independent sets (see [8], for example). However, it is not
easy to evaluate the performance of heuristics. The empirical approach is to run the heuristic on a collection of input
graphs (“benchmarks”), and record the sizes of independent
sets returned by the heuristic. If one heuristic consistently
outperforms another, then we have empirical evidence of
it being a better heuristic. Though running heuristics on
benchmarks is sometimes informative, we seek a more rigorous measure for evaluating heuristics.
One such measure is the approximation ratio. An algorithm is said to approximate MIS within a ratio > 1 if
for every input graph, the size of the maximum independent set is a factor of at most larger than the size of the
independent set returned by the algorithm. For NP-hard optimization problems in general, one can evaluate heuristics
based on their approximation ratios. However, it is known
(through work culminating in [13]) that for any constant
> 0, MIS cannot be approximated within ratio of n1,
(where n is the number of vertices in the input graph) unless
NP has randomized polynomial time algorithms. Hence, if
we were to evaluate heuristics for MIS based on their approximation ratios, all heuristics would perform badly. The
best approximation ratio known to be achievable for MIS is
O(n=(log n)2 ) [7].
As very little can be done with the MIS problem on worst
case instances (unless P=NP), one would like to compare
the performance of heuristics on average instances, or those
that typically occur in practice. But how does one model
such instances? One possible model is that of a random
graph (see the survey in [10]). The question then arises of
how well random graphs model inputs that interest us in
“real life” applications. But even regardless of this question, the random graph model does not seem to provide a
good way of distinguishing between good and bad heuristics in the case of MIS. For example, if each edge is chosen
independently at random with probability 1/2, then the size
of the maximum independent set is almost surely roughly
2 log n. An elementary greedy heuristic almost surely finds
an independent set of size log n. No heuristic, not even the
most sophisticated one, is known to find independent sets
significantly larger than log n. Hence, most heuristics have
roughly the same performance in this random graph model,
making it an inadequate framework for comparing between
them. Another random model that was suggested is similar to the random graph model, but with an independent set
of size K planted in the graph. The larger Kpis, the easier it is to find the independent set. If K > c n log n for
sufficiently large c, the vertices of the independent set are
easily recognized to be the K vertices of lowest degree in
G. Hence a trivial heuristic will solve MIS in this case.
But even the most sophisticated heuristics guarantee only a
marginal improvement in the size of K . The lowest value of
K that can be provably handled bypknown heuristics (spectral techniques, in this case) is ( n) [4]. Hence also the
planted independent set model does not provide sharp distinctions between good and bad heuristics.
In this paper we study a semi-random model for MIS,
which has some similarities with semi-random models studied for the coloring problem in [5]. In our model there is an
independent set S of size n planted in a graph, for some
> 0. Edges connecting S to the rest of the graph are
placed at random, with probability p = O( logn n ), creating a
random graph Gmin . An adversary can then add (but not remove) arbitrary edges to Gmin (as long as S remains an independent set), thus obtaining the input graph G. The goal
of the heuristic is to recover an independent set of size n.
The semirandom model has a random component Gmin ,
and when p is large enough, hardness results for worst case
instances no longer apply. It also has an adversarial (worst
case) component, which can be used to make G more similar to inputs that may occur in practice – with the restriction
that Gmin must remain a subgraph. Technically, the adversarial component can be used to alter degrees of vertices
(and foil heuristics based on vertex degrees), to create independent sets that are “local maxima” (and foil heuristics
based on local search), and to modify the spectrum of the
graph. We are not aware of any previously published algorithm that can handle our semirandom model.
We design an algorithm that recovers an independent set
of size n in the semirandom graph model. Our algorithm
requires a value of p slightly above ln n=n. We also show
that this value of p is best possible up to low order terms,
unless NP has randomized polynomial time algorithms.
Our algorithm for finding independent sets can also be
used for coloring. In the random k -colorable graph model,
a graph is partitioned into k color classes, and edges are
placed at random with probability p between color classes.
In this model Alon and Kahale [2] show that when p = c=n
and c > 0 is large enough, a k -coloring can be recovered.
Blum and Spencer [5] introduced the framework of semirandom k -colorable graphs, where in addition to the random edges, an adversary can add arbitrary edges between
color classes. (Several variations on this model are presented in [5], and the results discussed here apply to all of
them. This model was further studied in [17].) Algorithms
designed for the random graph model often do not work in
the semi-random model. For the semirandom model, when
color classes are of the same size (the balanced case), Blum
and Spencer [5] design algorithms that handle values of p
above n =n, for some > 0 that depends on k . No better results were known, and the algorithm of [2] does not
seem to apply to this model. Our new algorithm for finding independent sets can be used also for k -coloring (in
the balanced case), by extracting the color classes one by
one. It offers major improvement over the results of [5]
on semirandom graphs, as it handles values of p as low
as (1 + )k ln n=n. This answers an open question of [5]
and [10]. Moreover, the value of p is almost best possible,
as coloring semirandom k -colorable graphs with values of
p below (1 , ) ln n=n is NP-hard. (Note that in the random
model values of p = O(1=n) can be handled [2].)
1.1. The semi-random graph model
The graph G has n vertices. Its edge set E is generated
partly at random and partly by an adversary.
Let 0 < < 1 be a constant, and p = (c ln n)=n, where
c is a large enough constant that depends on .
1. An independent set S of size n is chosen at random.
Let S denote the vertices of G not belonging to S .
2. Random component: for any pair of vertices u; v such
that u 2 S and v 2 S, the edge (u; v ) is placed in E
with probability p, independently of all other events.
This gives the random graph Gmin .
3. Adversarial component: having complete knowledge
of Gmin , an adversary may add to E any number of
edges, provided that S remains an independent set.
This gives the graph G.
Let Gmax be the graph obtained by planting the independent set S of size n in an otherwise complete graph
on n vertices. Then G is an arbitrary graph “sandwiched”
between Gmin and Gmax . This fixes S as an independent
set, allows the adversary complete control for placing edges
both of whose endpoints are in S, and partial control over
those edges with one endpoint in S and the other in S (the
adversary controls a fraction of (1 , p) of these edges).
The lower p is, the more control the adversary has over the
graph.
Our algorithm is required to recover an independent set
of size at least n. We wish the algorithm to succeed with
high probability, where probability is computed over the
choice of Gmin , and over the random coin tosses of the
algorithm, regardless of the adversarial component of the
graph.
We note that G may contain several independent sets of
size n. In this case the algorithm is required to output just
one of them, not necessarily the original S .
Observe that the algorithm is in essence required to output Gmax . The random graph Gmin differs from Gmax in
that it misses some of the edges contained in Gmax . The
only changes that the adversary makes to Gmin is to put
back some of these missing edges. Hence, it may appear
that the adversary only makes the problem easier. Nevertheless, many algorithms that would recover a large independent set in the random graph Gmin would fail on G.
A major motivation for the semi-random graph model is to
identify those algorithms that work on random graphs, and
are also robust enough to withstand adversarial “help”.
Notation: throughout, , p and c will be used only as
above. That is, jS j = n, and the edge probability of the
random component is p = c ln n=n. For a set T of vertices,
N (T ) denotes the set of their neighbors in graph G. We
denote set subtraction by A n X (that is, those elements in
A that are not in X ).
1.2. Our results
The properties of our main algorithm are summarized in
the next theorem.
Theorem 1 For c = (1 + )=, where > 0, there is a
random polynomial time algorithm that with overwhelming
probability recovers an independent set of size n in the
semi-random graph model.
“Overwhelming probability” means with probability 1 ,
o(1) for n sufficiently large. Using the adversarial component of the graph, it can be shown that the value of c in
Theorem 1 is best possible, up to low order terms.
Theorem 2 In the semi-random model, if c = (1 , )= for
some > 0, then unless NP BPP , every random polynomial time algorithm will with overwhelming probability
fail to find an independent set of size n in G (against an
optimal adversary).
The proof of Theorem 2 is sketched in the appendix.
Theorem 1 implies new results about coloring k colorable graphs in the semi-random model of Blum and
Spencer [5]. In this model, a graph is partitioned into k
color classes. Each edge ei between different color classes
is included with probability pi , where pi is controlled by an
adversary, subject to pi p, for some p < 1. Clearly, the
smaller p, the stronger the adversary. Blum and Spencer
give algorithms to color semi-random 3-colorable graphs
when p > n2=3 =n, and to color k -colorable semi-random
graphs in which the sizes of the color classes are balanced,
whenever p > n =n, for > (k 2 , k , 2)=(k 2 + k , 2). We
significantly lower the values of p for which a coloring can
be found. Our bounds on p are best possible up to constant
multiplicative factors (via an argument similar to the proof
of Theorem 2).
Theorem 3 For any constant k , there is a polynomial time
algorithm that with overwhelming probability recovers the
largest independent set in k -colorable graphs in the semirandom graph model of [5], whenever p > (1+)nk ln n .
It is not hard to show that in semirandom k -colorable
graphs with p as in Theorem 3, the largest independent set
corresponds to a color class. Hence after the largest independent set is recovered, it can be removed from the graph,
which then remains (k , 1)-colorable. When k = 3, the
remaining graph is bipartite and can be two-colored in polynomial time. For k > 3, the other color classes can be recovered by repeatedly applying Theorem 3, but only if the
color classes are large enough. If they are too small, then
the number of vertices n0 remaining in the graph may be
ln n0
too low, causing p < (k,1)
n0 , and Theorem 3 may not
apply. In fact, in [5] it is shown that coloring semi-random
4-colorable graphs is NP-hard even for much larger values
of p, when the color classes are highly unbalanced.
We remark that our algorithm for finding independent
sets works also in models that have less randomness (and
hence, are more adversarial) than our semirandom graph
model. One such model is the d-neighbors model, where
in Gmin , each vertex of S has d random neighbors in S .
The graph G is then an arbitrary graph sandwiched between
Gmin and Gmax . It can be shown that a simple modification
of our algorithm recovers in this model independent sets of
size n, when d is a large enough constant that depends
only on . Observe that in this model Gmin has only O(n)
edges, whereas in our original semirandom model Gmin has
(n log n) edges.
1.3. Techniques and related work
Lovasz introduced the theta function as an upper bound
on the size of the maximum independent set [15]. The theta
function can be approximated within arbitrary precision in
polynomial time, using semidefinite programming. Goemans and Williamson [12] showed how semidefinite programming can be used in order to approximate problems
such as max-cut. Inspired by their work, Karger, Motwani
and Sudan [14] used semidefinite programming to obtain
improved coloring algorithms. Alon and Kahale [3] used
the work of [14] to show that the theta function can be used
to find medium size (n vertex) independent sets in graphs
that have linear size independent sets (improving the values
of previously obtained in [7]).
In terms of approximation ratio, the theta function (and
similar semidefinite programs) appear to have little to offer.
In [9] it is shown that for every > 0 there are graphs with
multiplicative gaps of n1, between the size of the maximum independent set and the value of the theta function.
Indeed, Håstad’s result [13] implies that, unless NP is easy,
no easily computable function will give better than a n1,
approximation in the worst case.
However, our current work singles out semidefinite programming as an approach that can cope with the semirandom graph model, unlike other heuristics for MIS. In more
detail, our algorithm has two phases. In the first phase (Sections 2.1 and 2.2) G is partitioned into a small number of
parts, such that some of these parts are composed mostly of
vertices of S . This first phase uses semidefinite programming. Its analysis is based only on Gmin , and goes through
regardless of what the adversary does. This illustrates the
robustness of (some) algorithms based on semidefinite programming.
In the second phase (Sections 2.3, 2.4 and 2.5) we “clean
up” the output of the first phase, and extract S (or a different
independent set of the same size). Many of the difficulties
introduced by the adversary manifest themselves in the second phase. In particular, there is the problem of getting out
of local maxima. To illustrate this problem, assume that the
algorithm already found a maximal independent set I composed mostly of vertices of S (though not containing all of
S ). One may then hope that local heuristics such as k-opt
(exchanging a constant k number of vertices of I with V n I
so as to hopefully get a new independent set that is not maximal and hence can be expanded) would allow one to eventually extract S . However, in our semi-random model, the adversary is strong enough so as to make no k -exchange possible, even when I is almost as large as n (details omitted).
Our method of improving over local maxima is based on
global computations (finding maximum matchings) rather
than local ones, and may be of independent interest.
As pointed out above, our work shows that algorithms
based on semidefinite programming perform well on random instances of MIS, and are robust enough to withstand
adversaries that add edges to the graph. We remark that
a similar phenomenon occurs for graph bisection. Boppana [6] considers a model of random graphs in which the
edge probability for edges crossing the intended bisection
is slightly smaller than that of edges outside the bisection.
In this model, he shows how to find the planted bisection.
We propose an algorithm similar to that of Boppana, formulated as a semidefinite program. Based on Boppana’s techniques, we show that this algorithm finds the bisection in
a semirandom model in which an adversary adds arbitrary
edges within each side of the bisection, and removes arbitrary edges connecting the two sides of the bisection. This
gives another example of the robustness of algorithms based
on semidefinite programming. More details are sketched in
the appendix.
1.4. Useful properties of semi-random graphs
Finding independent sets is NP-hard. Hence our algorithm will have to use some special property of G, inherited from Gmin . The property we use is expansion, as formulated in the following lemmata. Recall that overwhelming probability means with probability 1 , o(1) for n sufficiently large.
Lemma 4 Let c > 0, and let t = n(log n), , for some ,
0 < < 1. Then when n is large enough, with overwhelming probability over the choice of Gmin , for every T S
S 0 S , each of cardinality t, there is some edge in
Gmin joining T and S 0 .
and
, 2
Proof: (Sketch) There are at most nt ways of choosing
T and S 0 . For each such choice, the probability of the2 bad
event that there is no edge joining T and S 0 is (1 , p)t . For
large enough n, the lemma follows from the union bound
on the probabilities of the bad events. 2
Similarly, we can show:
Lemma 5 Let c = (1 + )=, let d > 0 be an arbitrary
constant, and assume that n is large enough. Then with
overwhelming probability over the choice of Gmin ,T
for every T S of cardinality at most 31n=32d, jN (T ) S j djT j.
Throughout we shall assume that Gmin has the above
expansion properties. As G contains all edges of Gmin ,
these properties are preserved in G.
In Section 2.2, we shall also use the following property
of Gmin , namely:
Lemma 6 Let c = (1 + )=, with 0 < < 1. Then
with overwhelming probability over the choice of Gmin , the
number of edges in Gmin is at most 2n ln n=.
Of course, G may contain many more edges than Gmin .
We shall use Lemma 6 in our analysis, but shall NOT assume that G is sparse.
2. Our algorithm and its analysis
Our algorithm has five phases, described and analyzed
in the following subsections. Many of the constants involved are arbitrary and are specified only for concreteness.
Throughout our analysis, we ignore divisibility issues, eschewing careful roundoff analyses. Such considerations do
not materially affect our argument.
2.1. A coarse partition using semidefinite programming
The following lemma is implicit in [3, 14].
Lemma 7 Let G(V; E ) be a graph on n vertices that contains an independent set of size K . Then one can find in
polynomial time a set Q of K=2 vertices (not necessarily
belonging to the independent set), and a set of K=2 unit
vectors in Rn associated with these vertices, such that for
any two vertices vi ; vj 2 Q, if (vi ; vj ) 2 E , then the inner
product of the associated vectors satisfies
hzi ; zj i < ,K=(2n , K )
(i.e., the angle between vi and vj is large).
For completeness, the proof of Lemma 7 is presented in
the appendix.
Using Lemma 7, we extract a small number of large sets
V1 ; V2 ; : : : as follows. We define G0 = G and inductively,
S
we let Gi be the subgraph of G induced on V n j i Vj .
We generate Vi+1 by applying the algorithm of Lemma 7 to
Gi , setting the value of K to be n=4. Hence the set Vi+1
contains K=2 = n=8 vertices, and we also have vectors
associated with these vertices (zj denotes the vector associated with vertex vj ) such that whenever (vj ; vk ) 2 E ,
hzj ; zk i < ,(n=4)=(2n , n=4) < ,=8:
We stop producing new Vi when the algorithm fails to find
such a set. In this case, the set of remaining vertices contains less than n=4 vertices of S (or the algorithm would
succeed).
The subgraph
T of G induced on Vi will be denoted by Qi .
Let Si = Vi S .
Definition 1 A set Vi is good if jSi j 2 n=32.
Proposition 8 The process described above gives a partition for which
X
fijVi is goodg
jSi j n=2:
Proof: Since Vi = n=8, by construction, there are at
most 8= sets. Less than n=4 vertices of S are not contained in any Vi (i.e., those discarded at the end). The number of vertices of S contained in sets Vi that are not good is
2
at most 32n 8 n
. As jS j = n, the proof follows. 2
4
2.2. Refining the partition using random hyperplanes
We further partition each Vi . The desired outcome of this
phase is summarized in Lemma 10.
Let Si = Vi nSi , and let Ei be the set of edges connecting
Si and Si in Gmin . (Note, Ei does not contain the edges
added by the adversary.)
By our construction, for each vertex vj 2 Vi we can
associate a unit vector zj 2 Rn such that for any two vertices vj ; vk 2 Vi , if (vj ; vk ) 2 E (and as a special case,
if (vj ; vk ) 2 Ei ), then the inner product of the associated
vectors satisfies hzj ; zk i < ,=8. This inequality implies
that the angle between the vectors zj and zk is at least some
constant > =2. (One can take = cos,1 (,=8). The
exact value of is irrelevant to our analysis.)
We partition Qi using a technique developed by [12, 14].
We pass a random hyperplane through the origin, and separate the vertices Vi into two sets, depending on the side
of the hyperplane on which they lie. As shown in [12], if
(vj ; vk ) 2 E , a random hyperplane will separate zj and zk
with probability at least = = > 1=2.1
Choosing h random hyperplanes partitions the vertices
Vi into m 2h sets of vertices, Vi1 ; : : : ; Vim . This randomized partition procedure, which we denote partition(Qi ; h),
tends to shatter Qi in the following sense:
Definition 2 Let E (Vi1 ; : : : ; Vim ) denote all pairs
(vj ; vk ) 2 Ei such that vj ; vk 2 Viq for some q.
Lemma 9 For a good Vi and for Ei as defined above, the
expectation of jE (Vi1 ; : : : ; Vim )j is at most jEi j(1 , )h ,
where the expectation is taken over the coin tosses of
partition(Qi ; h).
Proof: Each pair (vj ; vk ) 2 Ei has probability (1 , )h
of not being separated by at least one of the hyperplanes.
The proof follows from the linearity of the expectation. 2
Modifying an approach of [14], our algorithm computes
(Vi1 ; : : : ; Vim ) = partition(Qi ; h) for each Qi , and then
removes a maximal matching from each set Vij . For each
Vi , this gives independent sets Ii1 ; : : : ; Iim (some of which
might be empty), and some left over vertices (removed by
the matching) Mi . These independent sets and the sets of
leftover vertices form the refined partition of G.
Definition 3 An independent set Iij is useful if jIij
3jIij j=4.
T
Si j Recall that = = > 1=2 and let h be the least integer
satisfying (1 , )h 2 =16 ln n. The following lemma is
of central importance to our analysis. Its proof is given in
the appendix.
Lemma 10 Let h be as above and let fIij g be the independent sets obtained by running partition(Qi ; h) on each of
the subgraphs Qi , and removing a maximal independent set
1 Better partitioning techniques are also suggested in [14], but are not
needed for our results.
from each Vij . Then with probability at least 1=2 (over the
random choices of partition(Qi ; h)), at least n=8 vertices
of S are in useful independent sets Iij .
2.3. Creating a linear size independent set I
In this section we find in
(n).
G an independent set of size
The number of independent sets constructed in Section 2.2 is at most 8m= log n (for large enough n).
Lemma 10 describes an event that happens with probability
at least 1/2, and we assume that this event holds. (The randomized algorithm can be repeated several times with independent coin tosses so as to make the probability of failure
arbitrarily small.) Hence n=8 vertices of S are in useful
independent sets.
We now guess which are the useful independent sets.
The number of possible combinations here is less than n,
and we can just try out all possibilities. (More efficiently,
one may guess just one useful independent set, and deduce
the rest via matching techniques. Details are omitted from
this preliminary version.) Combine the useful independent
sets to obtain a set J , and remove a maximal matching from
J to obtain an independent set I . At most one fourth of the
vertices of J do not belong to S , and for each edge removed
from JTat least one of its endpoints is not in S . It follows
that jI S j jJ j , 2jJ j=4 jJ j=2 jS j=16.
2.4. Purifying I .
T
be a an independent set in G with jI S j jS j=16. Observe that by Lemma 4, almost all vertices of
I belong to S , and only a small number of vertices from I
might belong to S. In this section we extract from I a subset
I 0 , all of which is contained in S .
Denote the vertices of G by v1 ; : : : ; vn . Let l be the
smallest integer greater than 32=.
Based on G(V; E ) and I S
, we describe a bipartite graph
G0 (V 0 ; E 0 ), where V 0 = R L. The right hand side R of
G0 contains the n , jI j vertices V n I . The left hand side
L contains l copies of I . Namely, each vertex vi 2 I is
represented l times in L, as vi1 ; vi2 ; : : : vil . The edges E 0
are obtained from G in a natural way: (vij ; vk ) 2 E 0 if
vij 2 L, vk 2 R, and (vi ; vk ) 2 E .
We now describe the algorithm purify(I ).
Let
I
purify(I )
1. Construct G0 as described above.
2. Find a maximum matching M in G0 .
3. Return I 0 , where I 0 I contains those vertices vi 2 I
for which there is some 1 j l for which the vertex
vij was left unmatched by M .
Clearly, the algorithm runs in polynomial time.
Lemma 11 shows that in semi-random graphs, it indeed purifies large independent sets.
Lemma 11 Let G be a semi-random graph satisfying
Lemma 5, and assume that n is large enough so that the parameter d in Lemma 5 can be chosen to be larger
than l. Let
T
I be an independent set in G such that jI S j > jS j=16.
Then purify(I ) returns an independent set I 0 S , with
I 0 > jS j=32.
Proof: To see that jI 0 j > jS j=32, observe that the size of
M is at most jRj < n. For every vertex vi 2 I n I 0 , there
are l vertices vi1 ; : : : ; vil 2 L matched with vertices in R.
Hence jI n I 0 j < n=l, and jI 0 j > jI j , n=l = n=16 ,
n=32 = n=32.
It remains to show T
that I 0 S . WeTfirst analyze the
graph G. Let A = I S and B
T = I S . By assump
tion, jAT
j > jS j=16. As N (B ) S I , it follows that
jN (B ) S j < 15jS j=16. By Lemma 5, it follows that
0
jB j < (15
=16)=d; it then follows that for every
T jS jT
T BT B ,
0
0
0
jN (B )T (S I )j > ljB j (noteTthat TN (B ) (S I) =
N (B 0 )T S ). Observe that N (B ) (S I) is disjoint from
N (A) I.
We now analyze G0 . Let B 0 denote those vertices of L
that originate from B . (That is, vij 2 B 0 if vi 2 B .) We
claimTthat there is a matching M 0 from B 0 to the vertices
of R S . SinceTno other vertex from L can be matched
to vertices of R S , the existence of M 0 implies that any
maximal matching must match every vertex of B 0 . Consider
an arbitrary B 00 B 0 . B 00 must contain at least jB 00 j=l
distinct representatives from B . Then jN (B 00 )j > (jB 00 j=l)
l = jB 00 j. Hence by Hall’s theorem, all vertices of B 0 will
be matched. It follows that B I n I 0 , implying I 0 A S. 2
2.5. Expanding I 0 .
To expand I 0 , ultimately recovering S , we perform the
following procedure.
expand(I 0 )
1. Set V 0
= V , N (I 0 ). Let G0 be G induced on V 0 .
2. Compute a maximum matching on G0 .
3. Return I 00 , the set of unmatched vertices of G0 .
Lemma 12 Let I 0 S and jS j=32 < jI 0 j
Lemma 5 holds, then I 00 S and jI 00 j > jI 0 j.
< jS j.
If
Proof: Since I 0 S , V 0 will contain S ; define Q =
0
V n S . Since S is an independent set, the maximum size of
a matching on G0 is jQj, and this size can be achieved only if
every vertex in Q is contained in the matching. By construction, S \ N (Q) (the neighborhood of Q, restricted to S ) is
contained in S , I 0 . Noting that jS , I 0 j < 31jS j=32, it follows from the expansion properties of G that jQj < jS j=32,
or S \ N (Q) would have at least 31jS j=32 vertices. It then
follows from the expansion properties that jS \ N (Q)j >
jQj, unless Q is empty (in which case expand(I 0 ) = S ),
and that jS \ N (Q0 )j > jQ0 j for every nonempty Q0 Q.
Hence there exists a complete matching from Q to S . This
implies that any maximum matching found will include every vertex from Q and that at least one vertex from S , I 0
will be left over, since jS , I 0 j jS \ N (Q)j > jQj.
expand(I 0 ) will be contained in S and properly contain I .
2
[7] R. Boppana and M. Halldorsson. “Approximating
maximum independent sets by excluding subgraphs”.
BIT, 32 (1992), 180–196.
[8] David S. Johnson and Michael A. Trick (editors).
“Cliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge, 1993”, DIMACS
Series in Discrete Mathematics and Theoretical Computer Science, 26, American Mathematical Society,
1996.
[9] U. Feige. Randomized graph products, chromatic
numbers, and the Lovasz #-function. Combinatorica
17 (1) (1997) 79–90.
Lemma 12 implies that repeated applications of
expand(I 0 ) (each time with I 0 being the previous I 00 ) will
recover S . (In fact, the maximum matching found in the
first application of expand(I 0 ) can be reused, making repeated applications of expand(I 0 ) unnecessary. Details are
omitted.)
[10] A. Frieze and C. McDiarmid. “Algorithmic theory of
random graphs”. Random Structures and Algorithms
10 (1997), 5–42.
Acknowledgements
[12] Michel X. Goemans and David P. Williamson. “Improved approximation algorithms for maximum cut
and satisfiability problems using semidefinite programming”, Journal of the ACM, 42 (6) (1995), 1115–
1145.
Allan Frieze gave useful comments on an earlier version
of this manuscript.
References
[1] F. Alizadeh. “Interior point methods in semidefinite
programming with applications to combinatorial optimization”, SIAM J. Optimization, 5(1), 13–51, 1995.
[2] N. Alon and N. Kahale. “A spectral technique for coloring random 3-colorable graphs”. SIAM J. Comput.,
26(6), 1733–1748, 1997.
[3] N. Alon and N. Kahale. “Approximating the independence number via the #-function”. Math. Programming, to appear.
[4] N. Alon, M. Krivelevich and B. Sudakov. “Finding a
large hidden clique in a random graph”. In Proc. Ninth
SODA, 594–598, 1998.
[5] A. Blum and J. Spencer. “Coloring random and semirandom k -colorable graphs”. Journal of Algorithms,
19(2):204–234, September 1995.
[6] R. Boppana. “Eigenvalues and graph bisection: An
average-case analysis”. In Proceedings of the 28th Annual Symposium on Foundations of Computer Science,
pages 280–285, Los Angeles, CA, October 1987.
IEEE Computer Society Press.
[11] Z. Furedi and J. Komlos. “The eigenvalues of random
symmetric matrices”. Combinatorica, 1(3), 233–241,
1981.
[13] J. Håstad. Clique is hard to approximate within n1, .
Proc. 37th Annual Symp. on Foundations of Computer
Science, pages 627–636, 1996.
[14] D. Karger, R. Motwani, and M. Sudan. “Approximate
graph coloring by semidefinite programming.” Journal of the ACM, 45(2) , 246–265, 1998.
[15] Lovász. On the Shannon capacity of a graph. IEEE
Transactions on Information Theory IT-25, pp. 1-7,
1979.
[16] S. Poljak, F. Rendl. “Nonpolyhedral relaxations of
graph-bisection problems”. SIAM J. Optimization,
5(3), 467–487, 1995.
[17] C. Subramanian. “Minimum coloring random and
semi-random graphs in polynomial expected time”.
Proc. 36th Annual Symposium on Foundations of
Computer Science, 1995, 463–472.
A. Proofs
n
Proof of Theorem 2: (Sketch) Recall that p = c ln
n and
consider the random component of G. With overwhelming
probability, S will have a set T of n isolated vertices not
connected to S , where is some constant that depends on
. Using T , an adversary can embed a hard instance of independent set in the semi-random graph.
Let G0 be a graph on 3n =2 vertices, V 0 , in which we
seek to find an independent set of size n =2 (this problem is
NP-hard). Let V1 consist of n , n =2 vertices, V2 consist
of n , n , n vertices. Construct the graph G with vertices
V 0 [ V1 [ V2 . The edges of G consist of those of G0 (those
connect only vertices in V 0 ), and all edges between every
vertex in V2 and any other vertex (hence vertices in V2 have
degree n , 1, and vertices in V1 have degree jV2 j). Then,
permute the vertex labels of G at random. It follows from
the construction that G has an independent set of size n iff
G0 has an independent set of size n =2. Observe that given
G0 , the graph G is constructed in random polynomial time.
We next argue that when constructing a semirandom
graph with an independent set of size n, the adversary can
create a graph isomorphic to G whenever Gmin has n isolated vertices (which happens with overwhelming probability) and G0 has an independent set of size n =2. Hence, if
there existed an algorithm that didn’t fail with overwhelming probability, it must succeed on G with some constant
probability, revealing that G0 has such an independent set.
Theorem 2 would then follow. Note that the adversary need
not run in polynomial time.
Our adversary computes an independent set I of G0 ,
where jI j = n =2. Given Gmin , with a set T of n isolated vertices in S, the adversary maps the n =2 vertices of
I arbitrarily to distinct vertices of S and maps the n vertices of V 0 n I (distinctly) to T . Given this mapping, it then
adds the edges corresponding to those in G0 . By the construction, it is allowed to add these edges, and furthermore,
every nonedge in G0 corresponds to a nonedge in Gmin . Finally, the adversary connects every vertex of S n T to every
other vertex. It can be verified that this graph is isomorphic to G. V1 corresponds to those vertices in S that do not
correspond to I and V2 corresponds to S n T . 2
Proof of Lemma 7: Consider the following semidefinite
program (which can be solved up to arbitrary precision in
polynomial time, using the Ellipsoid algorithm). Given the
graph G, we find an order n matrix X = fxij g satisfying
the following constraints:
1.
2.
3.
4.
8i; j; 0 xij 1,
8(i; j ) 2 E; xij = 0,
P
i xii = K ,
8i; Pj xij = Kxii ,
5. The matrix X is positive semidefinite.
As G has an independent set S of size K , the above
semidefinite program is feasible. Setting xij = 1 whenever vi ; vj 2 S , and xij = 0 otherwise, gives a matrix X
satisfying the above constraints. To see that X is semidefinite, let qi be 1 if vi 2 S , and 0 otherwise, and observe that
X = Y Y T , where Y = [q1 q2 qn ]T and Y T denotes the
transpose of Y .
A positive semidefinite matrix X can be decomposed in
polynomial time into X = Y Y T , where Y is a matrix with
n rows, and Y T is its transpose. We denote the row vectors of Y by y1 ; : : : ; yP
n . The entry xij is the inner product
hyi ; yj i. Let y0 = i yi =K . From constraints 3 and 4
it follows that hy0 ; y0 i = K 2 =K 2 = 1. Hence y0 is a
unit vector. Moreover, constraint 4 implies that for every
i, hyi ; yi i = hy0 ; yi i. (Geometrically, this means that the
points yi , 1 i n, all lie on an n-dimensional sphere of
radius 1=2, and the points 0 and y0 are antipodal.)
Assume w.l.o.g. that the vectors yi are sorted by their
lengths, and consider now only the vectors y1 ; : : : ; yK=2 .
For every such vector, K=2n < hyi ; yi i 1 (from
constraints 1 and 3). Associate now with the vertices
v1 ; : : : ; vK=2 unit vectors z1 ; : : : ; zK=2 , where vector zi is
in direction yi , hy0 ; yi iy0 (i.e., we project out direction
y0 ).
Proposition 13 For 1 i < j
then hzi ; zj i < ,K=(2n , K ).
Proof: Let
K=2, if (vi ; vj ) 2 E ,
wi = yi , hy0 ; yi iy0 .
hyi ; yj i = 0. This gives
If
(vi ; vj ) 2 E then
hwi ; wj i = 0 , 2hy ; yi ihy ; yj i + hy ; y ihy ; yi ihy ; yj i
= ,hy ; yi ihy ; yj i:
This implies that hzi ; zj i < 0, but does not bound its
0
0
0
0
0
0
0
0
magnitude, since the wi are not unit vectors. By using the
interpretation of the yi as lying on a sphere of radius 1=2 we
2
2
obtain that their lengths satisfy
p(jwi j) +(1=2 ,hy0 ; yi i) =
1=4, implying that zi = wi = hy0 ; yi i , (hy0 ; yi i)2 . It follows that
hzi ; zj i =
, p hy ; y i i
p hy ; yj i
:
hy ; yj i , (hy ; yj i)
hy ; yi i , (hy ; yi i)
By elementary
p calculus it can be shown that the function
f (x) = x= x , x is positive and monotone increasing over (0; 1); hence hzi ; zj i is minimized when hy ; yi i
and hy ; yj i are minimized. As these minimums are in
both cases more than K=2n, it follows that hzi ; zj i <
,(K=2n) =(K=2n , (K=2n) ) = ,K=(2n , K ). 2
This completes the proof of Lemma 7. 2
0
0
0
0
2
0
2
0
2
0
0
2
2
Proof of Lemma 10: Consider an arbitrary set Vij that
is the outcome of partitionT(Qi ; h). WeTdefine the surplus
of Vij as sur(Vij ) = jS Vij j , jS Vij j. From Vij ,
an arbitrary maximal matching is removed so as to obtain an independent set Iij (that may possibly be empty).
jIij T S j. Recall that Iij is useful if
jIij S j 3jIijTj=4. From Lemma 4 it can easily be de
duced that
Tif jIij S j > n=(log n) for some 0 < < 1,
then jIij S j = o(jIij j). Hence Iij is useful. From the
Clearly,
T sur(Vij )
above it follows that if sur(Vij ) > n=(log n) then necessarily Iij will be useful. We now show that many sets Vij
have a large surplus, which then implies Lemma 10.
By Proposition 8, at least n=2 vertices from S belong to
good Vi . A good Vi is partitioned by h random hyperplanes
into m parts fVij g. From Lemma 9, the expectation of
jE (Vi1 ; : P
: : ; Vim )j is at most jEi j(1 , )h . P
Hence the expectation of i jE (Vi1 ; : : : ; Vim )P
j is at most i jEi j(1 , )h .
From Lemma 6 it follows that i jEi j 2n ln n=. Hence
with probability at least 1=2 we have that
X
i
jE (Vi ; : : : ; Vim )j 4n ln n(1 , )h =:
1
By our choice of h, (1 , )h
X
i
=16 ln n, implying
2
jE (Vi ; : : : ; Vim )j n=4:
1
Recall that our algorithm removes a maximal matching
from each Vij . For the sake of analysis, we as a thought
experiment remove a maximal matching containing only
edges from Ei , giving a set Uij . (The algorithm itself
does not have the luxury of knowing which edges of Qi
belong to Ei ). Then over all good Vi , the total number
of edges removed is at most n=4, meaning that at least
n=2 , n=4 = n=4 vertices from
S remain in the sets
fUij g. Call a set Uij large if jUij T S j 2 n=128m. As
there are at most 8m= sets Uij , those that are not large can
contain in total at most n=16 vertices from S . Hence at
least n=4 , n=16 = 3n=16 vertices from S are contained in large Uij .
By our choice of h and the fact that > 1=2, large Uij
contain (n=(log n) ) vertices of S for some < 1. As
they contain no edges
from Gmin , Lemma 4 implies that
T
for large Uij , jUij Sj o(jUij j). ToTobtain Uij from Vij ,
exactly (jVij j , jUij j)=2 vertices of S Vij were removed.
Hence the surplus of Vij must be (1 , o(1))jUij j. So if
we consider only those Vij that had large Uij in the above
experiment, they give rise to useful Iij containing a total of
at least
(1 , o(1))
X
large Uij
vertices from S .
jUij j (1 , o(1))3n=16 > n=8
2
B. Graph bisection
The bisection size b(G) of a graph G is the minimum
number of edges in a balanced cut (each side contains n=2
vertices). This problem is NP-hard. Boppana [6] develops
a heuristic for this problem, and analyses its performance
on random graphs with planted bisections. Specifically, he
considers random graphs whose vertex set is partitioned into
two equal size sets S and S, and two vertices are connected
by an edge with probability p if they belong to the same set,
and with probability q if they belong to different sets. If q
is sufficiently smaller than p, then w.h.p. S; S is a unique
minimum bisection. Boppana shows that when
r
p , q 10 (p + q) logn n
then with high probability (over the choice of input graph)
his heuristic recovers the minimum bisection.
We consider a semirandom model for graph bisection.
In this model, a graph Grand is chosen at random as above.
Then an adversary removes edges of his choice from the cut
S; S, and adds edges of his choice outside the cut (i.e., connecting pairs of vertices in S , or pairs of vertices in S). This
gives the semirandom graph G. Clearly, if S; S is the minimum bisection in Grand , then it is a minimum bisection
also in G.
We show that w.h.p. over the choice of Grand , an algorithm similar to that of Boppana outputs b(G). To also
output the bisection itself, determine which pairs of vertices
belong to the same side of the bisection by adding/removing
an edge between them to obtain a graph G0 , and checking
whether b(G0 ) = b(G). (We use here the fact that G0 is
sufficiently random for Boppana’s algorithm to work.)
Let h(G) be an arbitrary function on graphs which has
the following properties:
1. Lower bound. For every graph G, h(G) b(G).
2. Bounded monotonicity. Let G+ be the graph G with
one additional edge. Then h(G) h(G+ ) h(G) +
1.
3. Probably good. With high probability over the choice
of Grand ,
h(Grand) = b(Grand):
For any function h satisfying the above properties, we
have that with high probability over the choice of Grand ,
h(G) = b(G) also in the semirandom model. This follows from the fact that whenever h(Grand ) = b(Grand ),
then necessarily h(G) = h(Grand ) , r, where r is the
number of edges removed from the cut by the adversary.
(When h(Grand ) = b(Grand), each edge removed from
the cut decreases b(G) by one and h(G) by at most one
(bounded monotonicity), and hence h(G) by exactly one
(lower bound). Each edge added outside the cut does not
increase b(G), and hence can neither increase h(G) (lower
bound), nor decrease h(G) (monotonicity).)
Boppana [6] describes a polynomial time computable
function h and shows that it satisfies properties 1 and 3
above. It is not easy (for us) to verify that Boppana’s function satisfies property 2. Instead, we propose the following semidefinite relaxation of bisection, for which proving properties 1 and 2 is straightforward, and property 3
is proved using the techniques of [6]. For a graph G(V; E )
find an order n matrix X = fxij g satisfying:
1.
2.
8i, xii = 1,
P
ij xij = 0,
3. The matrix X is positive semidefinite,
and let h(G) be given by
h(G) = min
X
X
(i; j ) 2 E
1 , xij ;
2
i<j
where X ranges over all matrices satisfying the above constraints. Using semidefinite programming, h(G) can be
computed in polynomial time within arbitrary precision
limits. To see that h(G) b(G), consider the indicator vector s 2 f+1; ,1gn having entry +1 for vertices in S and ,1
for vertices in S, and let X = ssT . To see that h is bounded
1,x
monotone use the fact that jxij j 1 and 0 2 ij 1. To
see that h is probably good, consider the dual of the above
semidefinite minimization problem (see [1] for rules how to
obtain the dual).
Pn
Maximize m=2 + ( i=1 yi )=4 s.t.
,A , y0J , Y is p.s.d.
Here m is the number of edges in the graph, A is its adjacency matrix, J is the all 1 matrix, y0 is an auxiliary variable
which affects feasibility but not the objective function, and
Y is a diagonal matrix with y1 ; : : : ; yn along its diagonal.
For every 1 i n, let yi be the difference between the
number of neighbors that vertex i has on the other side of
the bisection and the number of neighbors that vertex i has
on the same side of the bisection. The value of the maximization semidefinite program is then exactly b(G) (though
we have not yet shown that the solution is feasible). Furthermore, regardless of the value of y0 , the vector s described
above is an eigenvector of the matrix M = ,A , y0 J , Y
with eigenvalue 0. It remains to show that w.h.p. (over the
choice of Grand ), it is possible to choose y0 such that s
is the eigenvector of smallest eigenvalue. For this, choose
y0 = ,1, giving ,A + J = Ac + I , where Ac is the
adjacency matrix of the complement of the graph Grand ,
and I is the identity matrix. The complement graph can
be viewed as a sum of two random graphs – one with
edge probability 1 , p, and the other a bipartite graph with
edge probability p , q . From [11], with high probability,
,O(ppn) lower bounds hold for the smallest eigenvalue
of the first graph and for the second smallest eigenvalue
of the second graph.
p These bounds can be combined with
(p , q)n=2 , O( pn log n) lower bounds on the entries of
the matrix (,Y ), that hold with high probability, showing
that M is almost surely positive semidefinite when p , q is
large enough. Details omitted.
Remark: Possibly, for every graph G, our function h
and Boppana’s function give the same value. See also [16]
for other semidefinite formulation for graph bisection.