Download On Horn envelopes and hypergraph transversals

ON HORN ENVELOPES AND HYPERGRAPH
(Extended Abstract)
TRANSVERSALS
Dimitris Kavvadias 1'4 Christos H. Papadimitriou 2'4'5 Martha Sideri 3'4
A b s t r a c t : We study the problem of bounding from above and below a given set of bit
vectors by the set of satisfying truth assignments of a Horn formula. We point out
a rather unexpected connection between the upper bounding problem and the problem
of generating all transversals of a hypergraph, and settle several related complexity
questions.
1. I N T R O D U C T I O N
Recently there has been much interest in the model theory of Boolean Logic, that is,
the relationship between a Boolean formula and the corresponding set of models (in
this paper by model we shall mean "satisfying truth assignment"). There are at least
three distinct motivations for this problem: Identifying plausible Boolean formulae
that describe a set of 0 - 1 vectors is one form of "discovering structure" in raw
data [DP, KKS1]. Besides, simplifications and approximations of a Boolean formula
via its models may represent a plausible vivid form of the formula, ready to make
complicated approximate inferences very rapidly [SK, Pal. Finally, characterizing
the sets of models of various subclasses of Boolean Logic and related formalisms
is an important part of understanding the semantic power and usefulness of such
formalisms. Therefore, the intricate algorithmic problems involved in going back
and forth between a Boolean formula and its set of models have been investigated
in the recent literature. This paper is a contribution to this line of research.
Horn formulae (Boolean formulae in conjunctive normal form with at most
one positive literal per clause) comprise an important subclass because of their
many positive algorithmic properties (for example, their satisfiability problem is Pcomplete), and they represent a natural style for expressing real-life conditions (logic
programming is one manifestation of this aspect of Horn formulae). Accordingly,
there has been considerable interest recently in inferring Horn formulae from a given
set of models. In [DP] an algorithm is presented for telling whether a given set
of vectors in {0, 1} '~ is precisely the set of all models of some Horn formula (in
this paper we slightly improve on the performance of this algorithm, Corollary 1
to Proposition 1). If the answer is negative, "and there is no Horn formula with
precisely the given set of models, then the following natural question arises: What
is the Horn formula whose set of models best approximates the given set of vectors?
1Department of Mathematics, Univ. of Patras, Patras, Greece.
2University of California at San Diego, La Jolla, California 92093-0114, U.S.A.
3Athens University of Economics and Business, Athens 10434 Greece.
4Research partially supported by the Esprit project ALCOM.
5Research partially supported by the National Sience Foundation.
400
Set inclusion is perhaps the most natural concept of approximation here; that is,
"we seek a Horn formula whose set of models includes the given set, and is as small
as possible. Such a Horn formula is called the Horn envelope of the given set of
models. It is easy to see (Corollary 2 to Proposition 1) that this set of models is
unique and easy to compute. By the latter we mean that there is an algorithm that
outputs this smallest superset in time which is polynomial in both the input and
the output. This concept of efficiency is the appropriate one here, since the required
output may be exponentially larger than the input% The difficult part is to compute
the corresponding Horn formula. The algorithm should be polynomial in the given
data and the output, that is, the number of Horn clauses produced.
Several recent papers have addressed this problem. [DP] give an algorithm that
does output such a Horn formula in output-polynomial time; the problem is that the
Horn formula produced by their algorithm may be highly redundant, and therefore
have exponentially more clauses than the correct, non-redundant output. [KKS1]
present a polynomial algorithm that outputs a Horn formula whose set of models
has with high probability a small (as a percentage of 2") symmetric difference from
the desired minimum enveloping set of models. However, no polynomial algorithm
for finding the "Horn envelope" had been known --neither was there any evidence
that this algorithmic task is impossible. One of our main results (Theorem 1) is
evidence that computing the Horn envelope is a very hard problem indeed, since its
solution would yie]d an output-polynomial algorithm for generating all transversals
of a hypergraph. We introduce this interesting problem next.
A hypergraph is a set H of subsets of { 1 , 2 , . . . , n } . A hilling set of H is a
subset t of { 1, 2 , . . . , n} such that for all h E H t intersects h. A transversalof H is a
minimal hitting set of H. We let tr(H) be the set of all transversals of H. It turns
out that tr(tr(tr(H))) = tr(H) [Be]. Generating in output-polynomial time tr(H),
given H, is a well-studied algorithmic problem, first proposed in [JYP], which has
been open. In a recent paper, [EG] point out that this problem is equivalent to a
number of other important problems in Boolean Logic, database theory, switching
theory, distributed systems, and artificial intelligence. In this paper we present
a rather unexpected connection between the transversal problem and the problem
of generating the Horn envelope of a given set of models (Lemma 1 and Theorem
1). Our result strongly suggests that there is no output polynomial algorithm for
generating the Horn envelope of a set of models.
We also study the opposite problem of computing the Horn core (as opposed to
envelope) of a given set of models. That is, we are now seeking a Horn formula whose
set of models is included in the given set, and is as large as possible. Unlike the
case of the Horn envelope, the maximal Horn core is not unique, but a maximal core
can be computed in polynomial time; however, computing --even approximating-6 See [JYP, EG] for extensive discussions of the algorithmic issues involved in
output-efficient enumeration of combinatorial configurations. Incidentally, in many
computational problems considered in this paper the input is a set of models, subsets,
or 0 - 1 vectors; if the size of the input is 2 ~
then these problems can be solved
trivially in polynomial time by exhaustion. As is commonplace in analyzing the
complexity of hypergraph problems, our results are interesting when the input set
is considerably smaller than this.
401
the core with maximum cardinality is NP-eomplete (Theorem 2). Interestingly, for
the polynomial upper bound we reduce the problem to a polynomially solvable one
about Horn clauses.
In the next section we introduce some basic facts about the model theory of Horn
formulae. In Section 3 we introduce the transversal problem and its relationship with
Horn envelopes. In Section 4 we discuss the problem of Horn cores, and in Section
5 the problems that are left open by this work.
2. M O D E L S
OF HORN
CLAUSES
Let { x l , . . . , x ~ } be a set of Boolean variables. A literal is either a variable or a
negation, and a clause is the disjunction (or) of literals. A Horn clause is a clause
with at most one (unnegated) variable. Examples of Horn clauses are (-~XlVX3V-~x4),
(x2), and (-~Xl V -~x3 Y -~x4). Clauses that do have a positive literal are usually
denoted in implicational form (for the first two Horn clauses above: (Xl A x4 -+ x3)
and (--+ x2))). A Horn formula is a set of Horn clauses.
A model t is a vector in {0, 1} '~ --intuitively, a truth assignment to the Boolean
variables. A model satisfies a clause if for some literal of the clause either the literal
is xi and ti = 1, or the literal is --xi and ti -- 0. If t and g are models, we say that
t ~ g ifti = 1 implies t~ --= 1. We denote by t a g the vector which is 1 exactly where
both t and t ~ are 1. If r is a Boolean formula in conjunctive normal form, sat(C) is
the set of all models that satisfy all clauses of r
A basic question is, given a set of models M _C {0, 1} '~, is there a Horn formula C such that M -= sat(b)? The answer is quite straightforward, and given
below (although researchers in the area have been aware of this characterization,
this particular statement and proof have not appeared, to our knowledge, in the
literature).
P r o p o s i t i o n 1. Let M C_ {0, 1} ~. The following are equivalent:
(a) There is a Horn formula C such that sat(b) = M.
(b) For each t ~ M either there is no t ~ E M with t ~ g, or there is a unique
minimal t ~ E M such that t < g.
(c) I f t , t ~ E M , t h e n a l s o t A g
EM.
S k e t c h : T h a t (a) implies (c) is easy. To establish (b) from (e), take t ~ to be the A
of all t" E M such that t < t ' . Finally, if we have property (b), we can construct
the following set of Horn clauses: For each t ~ M let t ~ be the model guaranteed by
(b); create a Horn clause ((A~,=I x~)
xj) for each j such that tj -- 0 and tj
It is easy to see that the set of all these Horn clauses comprise the desired C. [ ]
C o r o l l a r y 1: Given M, we can test whether there is a Horn formula C with sat(r ) -M in O(m2n) time, where m = [M[. Furthermore, if the answer is positive, then we
can generate the clauses of C in polynomial time.
S k e t c h : We can test part (c) of the Proposition as follows: For any t , g E M
compute ~ A g and add it to M. Sort by bucket sorting, delete duplicates, and test
whether the resulting set contains the same number of elements as M.
To generate the clauses of r for each g E M find the minimal t ~ M such that
ff is the model guaranteed by (b), and add the clauses described in the proof of the
402
implication from (b) to (a). O(rn2n 2) such clauses are added, and it is easy to check
that all other clauses generated in the proof of the proposition are redundant. []
Incidentally, the O(m2n 2) bound in the proof of the corollary is the best possible.
Also, notice that the corollary improves slightly on the O(m2nlog m) algorithm in
[DP].
But suppose that M does not satisfy the condition of the proposition; what is
the smallest superset of M that does? This corresponds to asking, what is the most
restrictive Horn formula which implies the Boolean formula with models precisely
those in M ?
C o r o l l a r y 2: Given M, there is a unique minimal M D M such that sat(r ) =
for some Horn formula r This set can be generated in output polynomial time.
S k e t c h : The algorithm is this: While the test in Corollary 1 fails, repeat with the
new M. This yields the closure of M under A, obviously a unique set. []
Notice that, although we can produce easily the Horn formula r such that
sat(b ) = M, if it exists, we have not said how, if it does not exist, to produce r
such that sat(b ) = M. Define the Horn envelope of M to be the non-redundant set
of Horn clauses r such that sat(b) = M. The main algorithmic problem studied in
this paper is thus, given a set of models M, to generate its Horn envelope. This
is an intriguing and well-known problem, attacked for example in [DP], [KKS1], as
well as the next section.
3. T R A N S V E R S A L S
AND ENVELOPES
We defined the transversal problem in the introduction. An immediate but loose
connection between this problem and the Horn envelope problem is given by the
concept of the standard clauses, introduced next.
We can identify a set M of models with a hypergraph H, where each model
t E M contributes a hyperedge in H consisting of all positions at which t is zero
(notice the departure from the usual correspondence). For each i E { 1 , 2 , . . . , n} now
omit from H all sets in H that do not contain i, and delete i from the remaining sets;
call the resulting hyperhraph Hi. Define now the following Horn clauses: First, for
each t E tr(H) we have the clause (Vi~t -.xi) Finally, for each i and each t E tr(Hi)
we have the clause ((Aje, zj) .--, xi). These are called the standard clauses associated
with M.
The intuition behind the standard clauses is this: The sets in M (via the correspondence "set of zeros") are precisely all sets that can be expressed as unions of
the hyperedges of H. How can we describe these sets in terms of "constraints?" The
standard clauses simply state that, for examp:% if such a set fails to contain any
element in a transversal of Hi, then it cannot contain i. Similarly for the transversals
of H. The next lemma (whose technical proof we omit here) states essentially that
these constraints capture M.
L e m m a 1: The formula consisting of all standard clauses associated with M is
logically equivalent to the Horn envelope of M. []
E x a m p l e : As the following examples show, unfortunately the standard clauses may
be redundant. Suppose that M consists of the models 01010, 01100, and 00111. The
403
corresponding hyperhraph H~ has transversals {2, 3, 4} and {2, 5}, while {3, 4} is one
of the transversals of Hh. Thus the following are standard clauses: (x3 A x4 --, xs),
(x2 Ax5 --~ xl), and (x2 A x 3 A x 4 ~ xl). It is easy to see that the first two
logically imply the third. If we omit the third model, then a more benign form of
redundancy results: The standard clause (~x3 V -~x4) logically implies the standard
clause (x3Ax4 ~ xl). Also, if in this last example we add the model 10111, then there
is no unique Horn envelope, since either one of (x3 A x4 --~ xl) and (X 3 A x 4 ~ Xh)
can be omitted (in the light of the standard clauses (xl ---*as) and (as ~ al)). [ ]
Despite this loose connection, the standard clauses are useful for establishing a more
intriguing relationshp between the two problems:
T h e o r e m 1: If there is an output-polynomial (or polynomial- delay) algorithm for
generating the Horn envelope of a given a set of models M, then there is an outputpolynomial (respectively, polynomial-delay) algorithm for generating tr(H), given a
hypergraph H.
S k e t c h : Let H be a hypergraph. Define the following hypergraph H I = H U { h + j :
j ~ h E H} (where + abbreviates union with a singleton). Define now It(H) to be
the set of models whose sets of zeros coincide with the hyperedges of H ~. We can
establish the following:
L e m m a 2: The Horn envelope of It(H) contains precisely the clauses (Viet -~x~) for
all t E tr(H).
The proof of Lemma 2 establishes that the standard clauses of # ( H ) are precisely
(Viet ~xi) for all t E tr(H); since these clauses can be shown non-redundant, the
result would follow from Lemma 1. By the definition of standard clauses, this is
tantamount to the following claim, whose proof follows from the definition of H I
and completes the proof of Theorem 1.
C l a i m : tr(H) = tr(Y'). Furthermore, for all j tr(H~) = {t E tr(H~): j ~ t).
4. H O R N
CORES
Another natural way of approximating a set M of models in terms of Horn formulae
is by finding not the most restrictive Horn formula r with sat(r ) D M (this would
be the Horn envelope we studied so far), but the least restrictive r with sat(e) C M.
Such a formula is called a Horn core of M, and its set of models is denoted __MM.In
contrast to the case of Horn envelopes which are unique, a set of models M may
have several Horn cores r all with maximal sat(e) = M C M. The following result
settles the computational complexity of the corresponding algorithmic problems:
T h e o r e m 2. (a) There is a polynomial-time algorithm which, given M C_ {0, 1} n,
generates one of the maximal subsets M and the corresponding Horn core. In fact,
all such maximal sets and formulae can be generated with polynomial delay.
(b) However, it is NP-complete to find the A/l with maximum cardinality.
(c) Furthermore, this latter problem is NP-hard to approximate within any constant
factor.
404
S k e t c h : The proof of Part (a) is interestingly "incestuous," in that it relies on an
algorithmic fact about Horn clauses. Given M, we create a Horn formula with the
vectors in M as variables. Intuitively, for t E M t = true means that t E M. For
any t, t / C M there are two cases: If t A t ~ ~ M , then obviously not both models can
be in M_M_;we write the clause (--t V "~t~). If however t A t' = t" E M , then we add the
clause ((t A t') ~ t"). The maximal M ' s then correspond to the maximal models of
the resulting Horn formula.
For (b) we reduce the NP-complete problem C L I Q U E to the problem of maximum eardinality ._M_M.Given a graph G = (V, E) and an integer k we construct a
set of models M C_ {0, 1}IEI (equivalently, subsets of E), as follows: M contains all
singletons {e}, and, for each vertex v E V, the set {e C E : v E e}. It then follows
that there is a M of size IEI + k if and only if there is a clique of size k in G. For
part (c) we amplify the above construction. [ ]
5. D I S C U S S I O N
This paper is a contribution to the study of the algorithmic problems related to
knowledge representation. It also adds a new m e m b e r to the intriguing class of
configuration enumeration problems that are related to the transversal problem. It
is at present open whether the Horn envelope problem is equivalent to the transversal
problem - - t h a t is, whether the reduction in Theorem 1 also goes the other way. We
conjecture that it does.
There are several interesting related open complexity questions, which can be
seen as generalizations of Corollary 1. Given a set of models M , is it the case that
M = sat(e) for a r which is (a) Krom (that is, 2-SAT, in conjunctive normal form
with at most two literals per clause); (b) 3-SAT; (c) with at most m (a given number)
clauses'? We know that (a) above is in P. We conjecture that (b) is coNP-complete
and that (c) is EP2-complete.
Another interesting open problem is this: Given a Horn formula, how hard is it
to generate its characteristic models, that is, a minimal set of models M such that
sat(C) = M . Characteristic models were shown in [KKS2] to be important alternative representations of a Horn formula. As was pointed out by Bart Selman (private
communication), our Theorem 1 and the approximation algorithm in [KKS1] imply
that generating characteristic models is also related to transversal enumeration.
Finally, it would be interesting if the insights into Horn formulae presented
in this paper could lead to improved algorithms for learning Horn formulae from
equivalence and membership queries [AFP]; such a result would also lead to improved
approximations of the Horn envelope using the ideas in [KKS1].
REFERENCES
[AFP] D. Angluin, M. Frazier, L. Pitt "Learning conjunctions of Horn clauses," 1990
FOCS pp. 186-192.
[Be] C. Berge Graphes et IIypergraphes, Dunod, 1980.
[DP] R. Dechter and J. Pearl "Structure identification in relational data," Artificial
Intelligence, 1993.
[EG] T. Eiter, G. Gottlob "Identifying the minimal transversals of a hypergraph and
related problems," S I A M J. Comp., to appear.
405
[JYP]
[KKS1]
:KKS2]
[Pal
[SK]
D. S. Johnson, M. Yannakakis, C. H. Papadimitriou "On generating all maximal
independent sets," IPL 27, 119-123, 1988.
H. A. Kautz, M. J. Kearns, B. Selman "Horn approximations of empirical data,"
to appear in Artificial Inlelligence, 1993.
H. A. Kautz, M. J. Kearns, B. Selman "Reasoning with characteristic models,"
to appear in AAAI, 1993.
C. H. Papadimitriou "On selecting a satsfying truth assignment," Proc. 1991
FOCS.
B. Selman, H. A. Kautz "Knowledge compilation using Horn approximation,"
Proc. A A A I 1991.