Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
ON HORN ENVELOPES AND HYPERGRAPH (Extended Abstract) TRANSVERSALS Dimitris Kavvadias 1'4 Christos H. Papadimitriou 2'4'5 Martha Sideri 3'4 A b s t r a c t : We study the problem of bounding from above and below a given set of bit vectors by the set of satisfying truth assignments of a Horn formula. We point out a rather unexpected connection between the upper bounding problem and the problem of generating all transversals of a hypergraph, and settle several related complexity questions. 1. I N T R O D U C T I O N Recently there has been much interest in the model theory of Boolean Logic, that is, the relationship between a Boolean formula and the corresponding set of models (in this paper by model we shall mean "satisfying truth assignment"). There are at least three distinct motivations for this problem: Identifying plausible Boolean formulae that describe a set of 0 - 1 vectors is one form of "discovering structure" in raw data [DP, KKS1]. Besides, simplifications and approximations of a Boolean formula via its models may represent a plausible vivid form of the formula, ready to make complicated approximate inferences very rapidly [SK, Pal. Finally, characterizing the sets of models of various subclasses of Boolean Logic and related formalisms is an important part of understanding the semantic power and usefulness of such formalisms. Therefore, the intricate algorithmic problems involved in going back and forth between a Boolean formula and its set of models have been investigated in the recent literature. This paper is a contribution to this line of research. Horn formulae (Boolean formulae in conjunctive normal form with at most one positive literal per clause) comprise an important subclass because of their many positive algorithmic properties (for example, their satisfiability problem is Pcomplete), and they represent a natural style for expressing real-life conditions (logic programming is one manifestation of this aspect of Horn formulae). Accordingly, there has been considerable interest recently in inferring Horn formulae from a given set of models. In [DP] an algorithm is presented for telling whether a given set of vectors in {0, 1} '~ is precisely the set of all models of some Horn formula (in this paper we slightly improve on the performance of this algorithm, Corollary 1 to Proposition 1). If the answer is negative, "and there is no Horn formula with precisely the given set of models, then the following natural question arises: What is the Horn formula whose set of models best approximates the given set of vectors? 1Department of Mathematics, Univ. of Patras, Patras, Greece. 2University of California at San Diego, La Jolla, California 92093-0114, U.S.A. 3Athens University of Economics and Business, Athens 10434 Greece. 4Research partially supported by the Esprit project ALCOM. 5Research partially supported by the National Sience Foundation. 400 Set inclusion is perhaps the most natural concept of approximation here; that is, "we seek a Horn formula whose set of models includes the given set, and is as small as possible. Such a Horn formula is called the Horn envelope of the given set of models. It is easy to see (Corollary 2 to Proposition 1) that this set of models is unique and easy to compute. By the latter we mean that there is an algorithm that outputs this smallest superset in time which is polynomial in both the input and the output. This concept of efficiency is the appropriate one here, since the required output may be exponentially larger than the input% The difficult part is to compute the corresponding Horn formula. The algorithm should be polynomial in the given data and the output, that is, the number of Horn clauses produced. Several recent papers have addressed this problem. [DP] give an algorithm that does output such a Horn formula in output-polynomial time; the problem is that the Horn formula produced by their algorithm may be highly redundant, and therefore have exponentially more clauses than the correct, non-redundant output. [KKS1] present a polynomial algorithm that outputs a Horn formula whose set of models has with high probability a small (as a percentage of 2") symmetric difference from the desired minimum enveloping set of models. However, no polynomial algorithm for finding the "Horn envelope" had been known --neither was there any evidence that this algorithmic task is impossible. One of our main results (Theorem 1) is evidence that computing the Horn envelope is a very hard problem indeed, since its solution would yie]d an output-polynomial algorithm for generating all transversals of a hypergraph. We introduce this interesting problem next. A hypergraph is a set H of subsets of { 1 , 2 , . . . , n } . A hilling set of H is a subset t of { 1, 2 , . . . , n} such that for all h E H t intersects h. A transversalof H is a minimal hitting set of H. We let tr(H) be the set of all transversals of H. It turns out that tr(tr(tr(H))) = tr(H) [Be]. Generating in output-polynomial time tr(H), given H, is a well-studied algorithmic problem, first proposed in [JYP], which has been open. In a recent paper, [EG] point out that this problem is equivalent to a number of other important problems in Boolean Logic, database theory, switching theory, distributed systems, and artificial intelligence. In this paper we present a rather unexpected connection between the transversal problem and the problem of generating the Horn envelope of a given set of models (Lemma 1 and Theorem 1). Our result strongly suggests that there is no output polynomial algorithm for generating the Horn envelope of a set of models. We also study the opposite problem of computing the Horn core (as opposed to envelope) of a given set of models. That is, we are now seeking a Horn formula whose set of models is included in the given set, and is as large as possible. Unlike the case of the Horn envelope, the maximal Horn core is not unique, but a maximal core can be computed in polynomial time; however, computing --even approximating-6 See [JYP, EG] for extensive discussions of the algorithmic issues involved in output-efficient enumeration of combinatorial configurations. Incidentally, in many computational problems considered in this paper the input is a set of models, subsets, or 0 - 1 vectors; if the size of the input is 2 ~ then these problems can be solved trivially in polynomial time by exhaustion. As is commonplace in analyzing the complexity of hypergraph problems, our results are interesting when the input set is considerably smaller than this. 401 the core with maximum cardinality is NP-eomplete (Theorem 2). Interestingly, for the polynomial upper bound we reduce the problem to a polynomially solvable one about Horn clauses. In the next section we introduce some basic facts about the model theory of Horn formulae. In Section 3 we introduce the transversal problem and its relationship with Horn envelopes. In Section 4 we discuss the problem of Horn cores, and in Section 5 the problems that are left open by this work. 2. M O D E L S OF HORN CLAUSES Let { x l , . . . , x ~ } be a set of Boolean variables. A literal is either a variable or a negation, and a clause is the disjunction (or) of literals. A Horn clause is a clause with at most one (unnegated) variable. Examples of Horn clauses are (-~XlVX3V-~x4), (x2), and (-~Xl V -~x3 Y -~x4). Clauses that do have a positive literal are usually denoted in implicational form (for the first two Horn clauses above: (Xl A x4 -+ x3) and (--+ x2))). A Horn formula is a set of Horn clauses. A model t is a vector in {0, 1} '~ --intuitively, a truth assignment to the Boolean variables. A model satisfies a clause if for some literal of the clause either the literal is xi and ti = 1, or the literal is --xi and ti -- 0. If t and g are models, we say that t ~ g ifti = 1 implies t~ --= 1. We denote by t a g the vector which is 1 exactly where both t and t ~ are 1. If r is a Boolean formula in conjunctive normal form, sat(C) is the set of all models that satisfy all clauses of r A basic question is, given a set of models M _C {0, 1} '~, is there a Horn formula C such that M -= sat(b)? The answer is quite straightforward, and given below (although researchers in the area have been aware of this characterization, this particular statement and proof have not appeared, to our knowledge, in the literature). P r o p o s i t i o n 1. Let M C_ {0, 1} ~. The following are equivalent: (a) There is a Horn formula C such that sat(b) = M. (b) For each t ~ M either there is no t ~ E M with t ~ g, or there is a unique minimal t ~ E M such that t < g. (c) I f t , t ~ E M , t h e n a l s o t A g EM. S k e t c h : T h a t (a) implies (c) is easy. To establish (b) from (e), take t ~ to be the A of all t" E M such that t < t ' . Finally, if we have property (b), we can construct the following set of Horn clauses: For each t ~ M let t ~ be the model guaranteed by (b); create a Horn clause ((A~,=I x~) xj) for each j such that tj -- 0 and tj It is easy to see that the set of all these Horn clauses comprise the desired C. [ ] C o r o l l a r y 1: Given M, we can test whether there is a Horn formula C with sat(r ) -M in O(m2n) time, where m = [M[. Furthermore, if the answer is positive, then we can generate the clauses of C in polynomial time. S k e t c h : We can test part (c) of the Proposition as follows: For any t , g E M compute ~ A g and add it to M. Sort by bucket sorting, delete duplicates, and test whether the resulting set contains the same number of elements as M. To generate the clauses of r for each g E M find the minimal t ~ M such that ff is the model guaranteed by (b), and add the clauses described in the proof of the 402 implication from (b) to (a). O(rn2n 2) such clauses are added, and it is easy to check that all other clauses generated in the proof of the proposition are redundant. [] Incidentally, the O(m2n 2) bound in the proof of the corollary is the best possible. Also, notice that the corollary improves slightly on the O(m2nlog m) algorithm in [DP]. But suppose that M does not satisfy the condition of the proposition; what is the smallest superset of M that does? This corresponds to asking, what is the most restrictive Horn formula which implies the Boolean formula with models precisely those in M ? C o r o l l a r y 2: Given M, there is a unique minimal M D M such that sat(r ) = for some Horn formula r This set can be generated in output polynomial time. S k e t c h : The algorithm is this: While the test in Corollary 1 fails, repeat with the new M. This yields the closure of M under A, obviously a unique set. [] Notice that, although we can produce easily the Horn formula r such that sat(b ) = M, if it exists, we have not said how, if it does not exist, to produce r such that sat(b ) = M. Define the Horn envelope of M to be the non-redundant set of Horn clauses r such that sat(b) = M. The main algorithmic problem studied in this paper is thus, given a set of models M, to generate its Horn envelope. This is an intriguing and well-known problem, attacked for example in [DP], [KKS1], as well as the next section. 3. T R A N S V E R S A L S AND ENVELOPES We defined the transversal problem in the introduction. An immediate but loose connection between this problem and the Horn envelope problem is given by the concept of the standard clauses, introduced next. We can identify a set M of models with a hypergraph H, where each model t E M contributes a hyperedge in H consisting of all positions at which t is zero (notice the departure from the usual correspondence). For each i E { 1 , 2 , . . . , n} now omit from H all sets in H that do not contain i, and delete i from the remaining sets; call the resulting hyperhraph Hi. Define now the following Horn clauses: First, for each t E tr(H) we have the clause (Vi~t -.xi) Finally, for each i and each t E tr(Hi) we have the clause ((Aje, zj) .--, xi). These are called the standard clauses associated with M. The intuition behind the standard clauses is this: The sets in M (via the correspondence "set of zeros") are precisely all sets that can be expressed as unions of the hyperedges of H. How can we describe these sets in terms of "constraints?" The standard clauses simply state that, for examp:% if such a set fails to contain any element in a transversal of Hi, then it cannot contain i. Similarly for the transversals of H. The next lemma (whose technical proof we omit here) states essentially that these constraints capture M. L e m m a 1: The formula consisting of all standard clauses associated with M is logically equivalent to the Horn envelope of M. [] E x a m p l e : As the following examples show, unfortunately the standard clauses may be redundant. Suppose that M consists of the models 01010, 01100, and 00111. The 403 corresponding hyperhraph H~ has transversals {2, 3, 4} and {2, 5}, while {3, 4} is one of the transversals of Hh. Thus the following are standard clauses: (x3 A x4 --, xs), (x2 Ax5 --~ xl), and (x2 A x 3 A x 4 ~ xl). It is easy to see that the first two logically imply the third. If we omit the third model, then a more benign form of redundancy results: The standard clause (~x3 V -~x4) logically implies the standard clause (x3Ax4 ~ xl). Also, if in this last example we add the model 10111, then there is no unique Horn envelope, since either one of (x3 A x4 --~ xl) and (X 3 A x 4 ~ Xh) can be omitted (in the light of the standard clauses (xl ---*as) and (as ~ al)). [ ] Despite this loose connection, the standard clauses are useful for establishing a more intriguing relationshp between the two problems: T h e o r e m 1: If there is an output-polynomial (or polynomial- delay) algorithm for generating the Horn envelope of a given a set of models M, then there is an outputpolynomial (respectively, polynomial-delay) algorithm for generating tr(H), given a hypergraph H. S k e t c h : Let H be a hypergraph. Define the following hypergraph H I = H U { h + j : j ~ h E H} (where + abbreviates union with a singleton). Define now It(H) to be the set of models whose sets of zeros coincide with the hyperedges of H ~. We can establish the following: L e m m a 2: The Horn envelope of It(H) contains precisely the clauses (Viet -~x~) for all t E tr(H). The proof of Lemma 2 establishes that the standard clauses of # ( H ) are precisely (Viet ~xi) for all t E tr(H); since these clauses can be shown non-redundant, the result would follow from Lemma 1. By the definition of standard clauses, this is tantamount to the following claim, whose proof follows from the definition of H I and completes the proof of Theorem 1. C l a i m : tr(H) = tr(Y'). Furthermore, for all j tr(H~) = {t E tr(H~): j ~ t). 4. H O R N CORES Another natural way of approximating a set M of models in terms of Horn formulae is by finding not the most restrictive Horn formula r with sat(r ) D M (this would be the Horn envelope we studied so far), but the least restrictive r with sat(e) C M. Such a formula is called a Horn core of M, and its set of models is denoted __MM.In contrast to the case of Horn envelopes which are unique, a set of models M may have several Horn cores r all with maximal sat(e) = M C M. The following result settles the computational complexity of the corresponding algorithmic problems: T h e o r e m 2. (a) There is a polynomial-time algorithm which, given M C_ {0, 1} n, generates one of the maximal subsets M and the corresponding Horn core. In fact, all such maximal sets and formulae can be generated with polynomial delay. (b) However, it is NP-complete to find the A/l with maximum cardinality. (c) Furthermore, this latter problem is NP-hard to approximate within any constant factor. 404 S k e t c h : The proof of Part (a) is interestingly "incestuous," in that it relies on an algorithmic fact about Horn clauses. Given M, we create a Horn formula with the vectors in M as variables. Intuitively, for t E M t = true means that t E M. For any t, t / C M there are two cases: If t A t ~ ~ M , then obviously not both models can be in M_M_;we write the clause (--t V "~t~). If however t A t' = t" E M , then we add the clause ((t A t') ~ t"). The maximal M ' s then correspond to the maximal models of the resulting Horn formula. For (b) we reduce the NP-complete problem C L I Q U E to the problem of maximum eardinality ._M_M.Given a graph G = (V, E) and an integer k we construct a set of models M C_ {0, 1}IEI (equivalently, subsets of E), as follows: M contains all singletons {e}, and, for each vertex v E V, the set {e C E : v E e}. It then follows that there is a M of size IEI + k if and only if there is a clique of size k in G. For part (c) we amplify the above construction. [ ] 5. D I S C U S S I O N This paper is a contribution to the study of the algorithmic problems related to knowledge representation. It also adds a new m e m b e r to the intriguing class of configuration enumeration problems that are related to the transversal problem. It is at present open whether the Horn envelope problem is equivalent to the transversal problem - - t h a t is, whether the reduction in Theorem 1 also goes the other way. We conjecture that it does. There are several interesting related open complexity questions, which can be seen as generalizations of Corollary 1. Given a set of models M , is it the case that M = sat(e) for a r which is (a) Krom (that is, 2-SAT, in conjunctive normal form with at most two literals per clause); (b) 3-SAT; (c) with at most m (a given number) clauses'? We know that (a) above is in P. We conjecture that (b) is coNP-complete and that (c) is EP2-complete. Another interesting open problem is this: Given a Horn formula, how hard is it to generate its characteristic models, that is, a minimal set of models M such that sat(C) = M . Characteristic models were shown in [KKS2] to be important alternative representations of a Horn formula. As was pointed out by Bart Selman (private communication), our Theorem 1 and the approximation algorithm in [KKS1] imply that generating characteristic models is also related to transversal enumeration. Finally, it would be interesting if the insights into Horn formulae presented in this paper could lead to improved algorithms for learning Horn formulae from equivalence and membership queries [AFP]; such a result would also lead to improved approximations of the Horn envelope using the ideas in [KKS1]. REFERENCES [AFP] D. Angluin, M. Frazier, L. Pitt "Learning conjunctions of Horn clauses," 1990 FOCS pp. 186-192. [Be] C. Berge Graphes et IIypergraphes, Dunod, 1980. [DP] R. Dechter and J. Pearl "Structure identification in relational data," Artificial Intelligence, 1993. [EG] T. Eiter, G. Gottlob "Identifying the minimal transversals of a hypergraph and related problems," S I A M J. Comp., to appear. 405 [JYP] [KKS1] :KKS2] [Pal [SK] D. S. Johnson, M. Yannakakis, C. H. Papadimitriou "On generating all maximal independent sets," IPL 27, 119-123, 1988. H. A. Kautz, M. J. Kearns, B. Selman "Horn approximations of empirical data," to appear in Artificial Inlelligence, 1993. H. A. Kautz, M. J. Kearns, B. Selman "Reasoning with characteristic models," to appear in AAAI, 1993. C. H. Papadimitriou "On selecting a satsfying truth assignment," Proc. 1991 FOCS. B. Selman, H. A. Kautz "Knowledge compilation using Horn approximation," Proc. A A A I 1991.