* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Using Hopfield Networks to Solve Assignment Problem and N
Birthday problem wikipedia , lookup
Renormalization group wikipedia , lookup
Lateral computing wikipedia , lookup
Exact cover wikipedia , lookup
Pattern recognition wikipedia , lookup
Natural computing wikipedia , lookup
Perturbation theory wikipedia , lookup
Artificial neural network wikipedia , lookup
Dynamic programming wikipedia , lookup
Computational electromagnetics wikipedia , lookup
Genetic algorithm wikipedia , lookup
Least squares wikipedia , lookup
Simulated annealing wikipedia , lookup
Travelling salesman problem wikipedia , lookup
Computational complexity theory wikipedia , lookup
Knapsack problem wikipedia , lookup
Multi-objective optimization wikipedia , lookup
Simplex algorithm wikipedia , lookup
Inverse problem wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
Multiple-criteria decision analysis wikipedia , lookup
Using Hopfield Networks to Solve Assignment Problem and N-Queen Problem: An Application of Guided Trial and Error Technique Christos Douligeris1 and Gang Feng2 1 2 Department of Informatics, University of Piraeus, Piraeus, 18534, Greece. Telecommunications and Information Technology Institute, Florida International University, Miami, FL 33174. Abstract. In the use of Hopfield networks to solve optimization problems, a critical problem is the determination of appropriate values of the parameters in the energy function so that the network can converge to the best valid solution. In this paper, we first investigate the relationship between the parameters in a typical class of energy functions, and consequently propose a “guided trial-and-error" technique to determine the parameter values. The effectiveness of this technique is demonstrated by a large number of computer simulations on the assignment problem and the N-queen problem of different sizes. 1 Introduction The continuous Hopfield neural network (CHNN) [1] can be used to solve an optimization problem in such a way that the cost function and constraints are first mapped to an energy function (if possible) and then a solution is obtained as the network stabilizes. Ever since Hopfield and Tank applied this network to solve the traveling salesman problem (TSP) [2], it has been employed to solve a variety of combinatorial optimization problems. However, a critical problem arising in the use of the HNN to solve optimization problem is how to choose the best parameters in the energy function so that the network can converge to valid solutions of high quality. In this paper, we will propose a relatively general method to determine the parameters. In the past decade, the most extensively used method is the trial-and-error technique, and to our point of view, this technique (at most with more constraints) will still be used in the future, especially for those problems that are NP-hard or NP-complete. This is based on the observation that given an energy function for a specific problem, it seems that we can at most determine a range for each parameter that might result in better solutions. Therefore, what we need to do is to find as many constraints for these parameters as possible, and thus considerably reduce the number of trials before good parameters are found. This method for determining parameters, slightly different from the original trial-and-error technique, can be called “guided trail and error" method. Previous related works include Aiyer’s eigenvalue analysis [4], Abe’s suppressing spurious states [5], and Gee’s application of polytope concept [3]. All of these works, however, are solely based This was partially supported by the Univ. of Piraeus Research Center. I.P. Vlahavas and C.D. Spyropoulos (Eds.): SETN 2002, LNAI 2308, pp. 325–336, 2002. c Springer-Verlag Berlin Heidelberg 2002 326 C. Douligeris and G. Feng on the analysis of the TSP. Matsuda published a series of papers [6]-[7] to study this problem. His basic idea is to analyze the stability of feasible and infeasible solutions and thus obtain some constraints for the parameters. However, we notice that in his most recent publication [7] even using the “optimal" network that he claimed can distinguish optimal solutions most sharply among all the networks to solve the 20 × 20 assignment problem, the percentage of experiments that the network converges to optimal solutions is only 58%, which leaves much to be desired considering that assignment problems of such small size are rather easy to be solved [8]-[9]. Moreover, the stability analysis in [7] is based on the assumption that any neuron in the network can exactly converge to “0" or “1", which is definitely not the case for a continuous HNN. Therefore there is a need for a methodology that will draw upon this experience and present more practical and efficient results. The rest of the paper is organized as follows. We first discuss the relation of the parameters for a specific class of Hopfield energy functions in Section 2. In Section 3, the effectiveness of the convergence theorem and parameter selection method is demonstrated through a large number of tests on two combinatorial optimization problems. Finally, Section 4 concludes this paper. 2 The Relation of Parameters for a Class of Energy Functions In this Section, we first provide a general form for a class of Hopfield energy functions, and then analyze what values should be chosen for the parameters so that the CHNN can converge to valid solutions of high quality. As a result, a “guided trial-and-error" technique is presented to determine the parameter values. 2.1 The General Form of a Class of Energy Functions There exists a class of optimization problems [2], [7], [16], [17] that can be described as follows: n n minimize fxi Vxi x i n subject to Vxj = k for any x (1) j n Vyi = k for any i y Vxi ∈ {0, 1} for any x, i where V = (Vxi ) ∈ {0, 1}n×n is a two dimensional variable matrix with each entry being the output of a neuron, k(≤ n) is an integer representing the number of nonzero variables in each row and each column, and generally k ∈ {1, 2}. Function fxi is a linear combination of the variables in S = V \{Vxi } and is in the following form: fxi (V ) = cxi + Vyj ∈S cxi yj Vyj (2) Using Hopfield Networks to Solve Assignment Problem and N-Queen Problem 327 where cxi and cxi yj are real numbers. Moreover, we assume that for any x = 1, 2, · · · , n, i = 1, 2, · · · , n, fxi is positive at any point in the hypercube. More clearly, for any x and i, fxi (Ṽ ) ≥ 0, where Ṽ is a n × n matrix with Ṽyj ∈ [0, 1] for any y and j. When using a CHNN to solve the problem defined by (1), one can construct the following energy function: 2 n 2 n n n n n C E= Vxj − k + Vyi − k + D fxi Vxi 2 x y x j i i (3) where C and D, the Lagrange multipliers, are positive real numbers. The dynamics of the corresponding CHNN can be obtained in terms of a relation duxi /dt = −∂E/∂Vxi , given by n n duxi = −C Vxj + Vyi − 2k − Dfxi , (4) dt y j in which uxi denotes the input of neuron xi. In the rest of this Section, we will investigate ways to determine the values of parameters C and D. To ensure the effectiveness of our approach, we assume that the D term describes the objective function of a specific optimization problem, rather than a constraint. More clearly, there is an additional assumption for fxi : given a valid solution Ṽ to (1), assume that for at least some x, i, y and j, fxi (Ṽ ) = fyj (Ṽ ). Throughout this paper the notation V̂ is used to denote the output matrix when the CHNN converges to a stable state. 2.2 The Relation between Parameters C and D Let us first consider what values for C and D could possibly make the network converge to a valid solution. It is clear that a valid solution Ṽ to problem (1) must have exactly k “1"s in each row and each column of Ṽ . When the CHNN converges to a stable state, however, the output of each neuron may not be exactly “1" or “0". Therefore, we cannot expect the network to directly converge to a valid solution; instead, we should try to “extract" a valid solution from the stable state if it is possible. For this reason, we start from the following lemma. Lemma 1. A necessary condition to guarantee that a valid solution can be extracted from a stable state is that there are at least k nonzero entries in each row and each column in the matrix V̂ . It is clear that we can not extract a solution if the number of nonzero entries is less than k in some row or column of V̂ . 328 C. Douligeris and G. Feng Lemma 2. A necessary condition to guarantee that there are at least k nonzero entries in each row and each column in the matrix V̂ can be given by n V̂xj + j n V̂yi > 0, for any x, i. (5) y n n Theorem 1. A necessary condition for j V̂xj + y V̂yi > 2αk, for any x, i, and α ∈ [0, 1) can be given by Dfmin < 2(1 − α)kC (6) where fmin = min{fxi |x, i = 1, 2, · · · , n}. ✷ Proof: Omitted for saving space. Corollary 1. A necessary condition to guarantee that a valid solution can be extracted from the stable state can be given by Dfmin < 2kC. (7) Proof: In theorem 1, let α = 0. From lemma 2 and lemma 1, we get that the corollary holds. ✷ To better understand the significance of theorem 1, let us assume that at a stable state, there are exactly k nonzero entries in each row and each column of V̂ , and thus it is clear that the value of α denotes to what degree a nonzero entry approaches “1". On the other hand, given a specific value for C, D must be small enough to ensure that the summation of a row and a column to be greater than 2αk. Theorem 1 is useful in such cases where the obtained solution becomes satisfactory only if each nonzero entry very closely approaches “1". Nonetheless, one should notice that (6) is neither a necessary nor a sufficient condition to guarantee that a valid solution can be obtained. Another two necessary conditions stronger than (6) and (7) are given in the following theorem and the subsequent corollary. Theorem 2. A necessary condition to guarantee that (a) at a stable state there are in total at least nk (product of n and k) nonzero entries in V̂ , and n n (b) for any x, i, and α ∈ [0, 1), j V̂xj + y V̂yi > 2αk is the following: Df˜min < 2(1 − α)kC ˜ where fmin is the nkth minimum number among all fxi ’s. (8) ✷ Proof: Omitted for saving space. Corollary 2. A necessary condition to guarantee that a valid solution can be extracted from the stable state can be given by Df˜min < 2kC. (9) Proof: Omitted for saving space. ✷ Since f˜min ≥ fmin (due to the fact that fmin is the minimum number among all fxi ’s), the necessary conditions (8) and (9) are stronger than (6) and (7), respectively, in the sense that given specific values for C, k and α, the range of D becomes smaller. Using Hopfield Networks to Solve Assignment Problem and N-Queen Problem 329 2.3 The Initial Value of D In a Hopfield energy function, each energy term is usually assigned a parameter. For instance, in (3) the two parameters C and D are used to combine two energy terms. In fact it is obvious that in optimization their relative values are important, and one of them is redundant. Thus, any one of the parameters can be arbitrarily assigned a value as a reference to determine the values of other parameters. For this reason, from now on we assume that parameter C has already been assigned a specific value, and our goal is to find an appropriate value for D. We notice, however, that given a specific value for C, the range of D determined by the strongest necessary condition in the last subsection, i.e. (9), is still very large. To find a good initial value for D, we need to consider what values of these parameters could possibly result in a good solution since our final goal is to find a solution which is not only valid, but of high quality as well. The difficulty in considering this point lies in that one can hardly find any conditions that can guarantee that better solutions are available. However, researchers in the area have obtained some practical experience [4], [7]. It is well known, for example, that when the network converges to a stable state, the closer the nonzero entry in V̂ approaches “1", possibly the better the solution. For this reason, theorem 2 can be used to find a good initial value for D since only if the summation of each row and each column approaches 2k is it possible that each nonzero entry approaches “1". Thus the initial value of D can be given as follows: Dinitial = 2(1 − α)k C f˜min (10) with α being a value close to 1. For a specific problem, the value of f˜min can be obtained by estimation. If the expression of fxi does not contain the second term in (2), namely the problem is an assignment problem, then it is possible to find the exact value of f˜min by greedy search, but it might be very time consuming. 2.4 The Fine-Tuning of D Once an initial value for D is obtained, by gradually increasing or decreasing its current value, it is not hard to find a good value of D that may result in good valid solutions. To make a fine-tuning of D so that the quality of solutions can be further improved, we find that there is a basic rule that may be helpful. Theorem 3. Assume that at any stable state of the CHNN defined by (3) and (4), there are exactly k nonzero entries in each row and each column of V̂ , and each nonzero is approximately equal to the same value, then the ratio of the second energy term in (3) to the first one is approximately in reverse proportion to the value of D. Proof: Omitted for saving space. ✷ Theorem 3 indicates that a smaller value for D could possibly increase the proportion of the energy in the second term of (3) to the total energy. Since the second term of (3) is contributed from the objective function in (1), it is possible that a smaller value of D 330 C. Douligeris and G. Feng could lead to a better solution. This is because the CHNN will continuously reduce the energy function until it reaches a stable state. Thus, among the total decreased energy, the more the energy coming from the objective term, most possibly the better the obtained solution. However, the dilemma lies in that if the value of D is too small, the nonzero entries can hardly approach “1". Therefore, one has to make many trials to find a balanced value for D. 2.5 The Value of the Parameter Associated with the Binary Constraint The CHNN treats any problem it solves as a linear or nonlinear programming problem since the network only tries to find a local minimum of the energy function without caring whether the output of each neuron has reached an integral value. Therefore, if the output of each neuron can not approach a binary value when we try to solve an integer programming problem, another energy term due to the binary constraint should be added to the energy function. For a specific problem that can be formulated as (1), it is well known that we have two ways to describe the binary constraint as an energy term: n n x or E0 = n n x i Vxi (1 − Vxi ) (11a) i n n n n Vxi Vxj − (k − 1) + Vxi Vyi − (k − 1) . (11b) j=i x i y=x The first expression is effective because it becomes zero only when Vxi has a binary value. The second expression becomes zero if Vxi = 0 or in the case where Vxi = 1 and the bracket part equals to zero. Therefore, it still has the effect to help the output reach a binary value. We prefer the latter expression since the former one includes self-interaction terms which may make our following analysis more complicated. By assuming a positive parameter A is associated with E0 , the energy function (3) is now modified as 2 n 2 n n n n n C E= Vxj − k + Vyi − k + D fxi Vxi + AE0 . 2 x y x j i i (12) There are three parameters in (12). Although at first glance it seems difficult to analyze the mutual relations between these parameters, we find that (12) can be rewritten to be the same form as (3): 2 n 2 n n n n n C E= Vxj − k + Vyi − k + D fˆxi Vxi (13) 2 x y x j i i in which Using Hopfield Networks to Solve Assignment Problem and N-Queen Problem 331 n n Vxj + Vyi − 2k + 2 fˆxi = fxi + γ (14) j=i y=x and γ = A/D. Now, it is clear that our former analysis on the choice of the values of C and D still applies to this case. Similar to the analysis in the last subsection, in order to let more of the total decreased energy come from the objective term, from (14) we know A/D should be as small as possible. In fact, it has been noted that the binary constraint term will considerably increase the number of local minimums, therefore in most cases A is set to 0. However, we also find that in some cases this term can not be omitted, even if the value of A is very small [10]. 2.6 The Guided Trial-and-Error Technique By summarizing the above analysis, we give a formal statement on the “guided trialand-error" technique. When using a CHNN to solve a specific problem formulated as (1), and assuming that its energy function is given by (13), the “guided trial-and-error" technique can be described as follows: 1. Initialization. (1) Arbitrarily choose a value for C. (2) Choose a value for ∆t, which satisfies ∆t < 1/(βmax C)1 (3) Compute the initial value of D by (10) when α is set to a value close to 1, e.g. α = 0.99. (4) Let γ = 0. 2. Tuning. Keep tuning the parameters according to the following rules until satisfactory results are found: (1) Decrease (increase) ∆t if the network converges too fast (slow) to a stable state with unsatisfactory solutions. (2) Increase D if the outputs of neurons do not approach binary values; if this has no effect, then increase γ. (3) Decrease γ if the outputs of neurons approach binary values, but with solutions of low quality; if γ can not be decreased any more, then try to decrease D. During the increase of D, inequality (9) should be strictly satisfied. On the other hand, when the values of D and γ are small, the number of iterations should be large enough to ensure that the network can reach a stable state. 3 Experimental Results In this section, the “guided trial-and-error" technique is applied to solve the assignment problem and the N-queen problem to test its effectiveness. In our following experiments, the sigmoid function is used as the neuron’s I/O function: uxi 1 1 + tanh Vxi = . (15) 2 2u0 The maximum slope of this function is βmax = 1/(4u0 ). 1 ∆t is the discretized dt in the dynamic equation, and βmax is the maximum slope of the input-output equation of the neuron. For more details regarding this constraint, please see [11]. 332 C. Douligeris and G. Feng 3.1 The Linear Assignment Problem The linear assignment problem (AP) needs to assign n jobs to n persons in a one-to-one way such that the total cost is minimized. This problem can be solved by converting it to a minimum cost flow problem [8] or using the Hungarian method [9] in time O(n3 ), where n is the problem size. In recent years, instead of using these ordinary linear programming methods, many researchers try to develop parallel algorithms based on, for example, Hopfield-type neural networks [12]-[14] and statistical physics [15] so as to considerably reduce the time complexity. In this paper, we use the original CHNN to solve the assignment problem by formulating it as (1), in which k = 1 and fxi equals to a deterministic value, namely (2) becomes fxi = cxi . Now let us show how to choose the parameter values using the “guided trial-anderror technique". First, we let C = 50, γ = 0, and the parameter u0 in the sigmoid function be 0.01. From the convergence theorem proposed in [11] we know that the timestep ∆t should be less than 0.0008 to ensure that the network continuously converges to a stable state. Thus, we let ∆t = 0.00075. To determine an initial value for the parameter D, we assume that each cxi is a randomly generated real number and is uniformly distributed in (0, 1]. Thus, from equation (10), the initial value of D is given by Dinitial = (n2 + 1)/n ≈ n when α = 0.99 (we only consider the cases where n is big enough such that 1/n is negligible). We have tested a number of AP instances when the problem size varies from 20 to 200. In our simulations, all of the above parameter values are kept unchanged except that we let D = n when n ≤ 100, and D = 100 when n > 100. For a specific problem size n, ten problem instances are randomly generated such that cxi ’s are integers uniformly distributed in [1, 1000]. For each problem instance, ten simulations are made when the initial values of neurons are changed. Thus, we obtain 100 solutions for problem instances with the size of n. Note that this is same method used in [7]. The initial value of a neuron is given by uxi (t = 0) = −u0 ln(n − 1) + 0.01Rand (16) where Rand is a random number in [−1, 1]. Using this initialization method, the output of each neuron approximately equals to 1/n when t = 0, and thus the summation of the entries in a row or a column of the matrix V (t = 0) is approximately 1. The following rule is used to terminate a simulation. For a specific problem instance, the Hungarian method is used to obtain its optimal solution. In every 100 iterations of a specific simulation, the total decrease of energy ∆E in one iteration is computed as follows: ∆E = n x,i ∆Exi = n x,i 2 C∆Vxi − 1 ∆uxi Vxi . ∆t (17) A simulation is terminated if |∆E| < !, or if the temporary solution is equal to the optimal solution, whichever condition occurs first. ! is set to a value such that when |∆E| < !, the network approximately approaches a stable state and the solution can hardly be changed. In our experiments we let ! = n × 10−8 . The statistical results are shown in Table 1. The first row denotes the problem size, the second row is the number of simulations when optimal solutions are obtained, the Using Hopfield Networks to Solve Assignment Problem and N-Queen Problem 333 third row is the average error, and the last row is the average iteration number for a single AP instance. The average error is computed as follows: 10 10 1 Solij − Opti × 100% 100 i j Opti where Solij is the solution obtained in the jth simulation for problem instance i, and Opti is the optimal solution of problem instance i. From Table 1, we may find that when the problem size is relatively small, the CHNN can obtain an optimal solution with a very high probability, even though we have not put much effort to tune parameter D. To show the effectiveness of our theoretical analysis, one may make a comparison of our results with those shown in [7]. When using the same problem formulation, the author of [7] can obtain an optimal solution with a probability of only 1% even when the problem size is 20, and the probability was improved to 58% after the energy function was reformulated. We can further improve the performance of the CHNN by fine-tuning the value of D. As stated in theorem 3, under certain assumptions the ratio of the objective energy term to the constraint energy term is in reverse proportion to the value of D. Therefore, a smaller value of D could possibly improve the quality of the solution. We have done the same experiments as described above when D is set to a fixed value 15. The corresponding results are shown in Table 2. To ensure that the results are comparable, the initial values of the neurons for a specific simulation of Table 2 are set to the same as those for the corresponding simulation of Table 1. From Table 2, one may notice that the probability that an optimal solution can be obtained has been considerably increased. The expense is that the number of iterations also increases. Table 1. Results of the Assignment Problem when D = n (n ≥ 100) and D = 100 (n > 100) problem size n optimal solution convergence (%) average error(%) average iterations 20 100 0 166 40 100 0 255 60 90 2.59 646 80 85 5.79 657 100 52 17.63 932 200 78 8.91 1523 100 99 0.16 1898 200 86 4.41 5334 Table 2. Results of the Assignment Problem when D = 15 problem size n optimal solution convergence (%) average error(%) average iterations 20 100 0 193 40 100 0 447 60 100 0 1577 80 95 2.60 1640 334 C. Douligeris and G. Feng 3.2 The N-Queen Problem The N-queen problem can be stated as follows: given a n by n chessboard, we need to place n queens on the chessboard such that any two queens can not attack each other. The constraints in this problem can be described more precisely as follows: in each row or each column, there is exactly one queen; in each positive or negative diagonal, there is also exactly one queen. The N-queen problem can also be formulated as (1) when k = 1 and fxi are defined as follows: fxi = Vx+j,i+j + Vx−j,i−j . (18) j=0 1≤x+j≤n 1≤i+j≤n j=0 1≤x−j≤n 1≤i−j≤n However, one should note that the above expression violates the second assumption we impose on fxi in(1), because for a valid solution Ṽ to this problem, we always have fxi Ṽ = fyj Ṽ = 0 for any x, i, y and j. In fact, (18) is the expression of a constraint, rather than an objective term in the general sense. For this reason, we can not determine the value of parameter D by means of the theorems in Section 3. Instead, we should choose a value so that the importance of the two constraint-terms (C term and D term) can be balanced. Although we can only arbitrarily choose an initial value for D, the values of other parameters can be readily determined in accordance with the convergence theorem in [11]. Similarly, we let γ = 0, u0 = 0.01, C = 50, ∆t = 0.00075. After a number of experiments, we find that the network can obtain valid solutions with high probability when D is around 15. Note that for this problem, a valid solution is also an “optimal" solution. The experimental results when D = 10 and D = 15 are shown in Tables 3 and 4, respectively. The problem size varies from 30 to 200. For a specific problem size n, we have done 100 simulations, each with different initial values for the neurons. The initialization method is same with the one used in the assignment problem except that the amplitude of the random number is set to 0.005. Also, in every 100 iterations of a specific simulation, the total decrease of the energy is computed according to (17), and the temporary output matrix is processed using the recommended post-processing method described in the previous Section. If |∆E| < 10−6 or a valid solution is obtained then the simulation is terminated. From Tables 3 and 4, one may notice that the CHNN can obtain valid solutions with very high probability even when the problem size becomes large. Now let us make a comparison with the results reported by other researchers. In recent years, many works have been done on the use of neural networks (not necessarily the Hopfield networks) to solve the N-queen problem. In [16], the highest probability that a valid solution can be obtained is 73% when n = 30, and 90% when n = 50, and no results were given when n > 50. In [18] and [19], the sizes of the studied problems are restricted to be 10 and 8, respectively. In [20], a dual-mode dynamics neural network is employed to solve this problem and this method is demonstrated to outperform many other methods when n ≤ 40, but no results are reported for problems of larger size. In [21], the self-feedback controlled chaotic neural network is used to solve this problem of large size, and after Using Hopfield Networks to Solve Assignment Problem and N-Queen Problem 335 the self-feedback factor is finely tuned, the success rate to solve the 200-queen problem is around 98.8%. Thus, our result is comparable with the best result in the literature. Table 3. Results of the N-queen Problem when D = 10 problem size n valid solution convergence (%) average iterations 30 98 131 50 100 119 80 98 172 100 99 204 200 92 358 100 100 133 200 97 252 Table 4. Results of the N-queen Problem when D = 15 problem size n valid solution convergence (%) average iterations 4 30 95 170 50 100 112 80 100 122 Conclusions In this paper, we have addressed the dynamics of the continuous Hopfield neural networks. In particular, we investigated the mutual relation between the parameters in a typical class of Hopfield energy functions, and thus proposed a “guided trial and error" technique for determining the parameters. The effectiveness of this technique has been demonstrated by a large number of computer simulations when the HNN is used to solve the assignment problem and the N-queen problem. Compared with previous works, the performance of the HNN has been considerably improved. References 1. J. J. Hopfield, “Neurons with graded response have collective computational properties like those of two-state neurons,” Proc. National Academy of Sciences USA, vol.81, pp.3088-3092, May 1984. 2. J. J. Hopfield and D. W. Tank, “Neural computation of decisions in optimization problems,” Biological Cybernetics, vol.52, pp.141-152, 1985. 3. A. H. Gee, “Problem solving with optimization networks,” Ph.D. dissertation, Univ. Cambridge, July 1993. 4. S. V. Aiyer, M. Niranjan and F. Fallside, “A theoretical investigation into the performance of the Hopfield model,” IEEE Trans. Neural Networks, vol.1, no.2, pp.204-215, June 1990. 5. S. Abe, “Global convergence and suppression of spurious states of the Hopfield neural networks,” IEEE Trans. Circuits Syst. I, vol.40, no.4, pp.246-257, April 1993. 6. S. Matsuda, “The stability of the solution in Hopfield neural network,” Proc. Int. Joint Conf. Neural networks,pp. 1524-1527, 1993. 7. S. Matsuda, “Optimal” Hopfield network for combinatorial optimization with linear cost function,” IEEE Trans. Neural Networks , vol.9, no.6, pp.1319-1330, November 1998. 336 C. Douligeris and G. Feng 8. E.L. Lawler, Combinatorial Optimization: Networks and Matroids, Holt, Rinehart and Winston, 1976. 9. H.W. Kuhn, “The Hungarian method for the assignment problem,” Naval Res. Logist. Quart., pp.83-97, 1955. 10. G. Feng and C. Douligeris,“Using Hopfield networks to solve traveling salesman problems based on stable state analysis technique,” Proc. Int. Joint Conf. Neural networks,Vol.6, pp. 521-526, Jul 24-Jul 27 2000. 11. G. Feng and C. Douligeris,“On the Convergence and Parameter Relation of Discrete-Time Continuous-State Hopfield Networks with Self-Interaction Neurons,” IEICE Trans. Fundamentals, Vol.E84-A No.12, pp.3162-3173, Dec. 2001. 12. S.P. Eberhardt, T. Duad, DA.Kerns, T.X.Brown and A.P. Thakoor, “Competitive neural architecture for hardware solution to the assignment problem,” Neural networks,Vol.4, no.4, pp.431-442,1991. 13. J. Wang, “Primal and dual assignment networks,” IEEE Trans. Neural Networks, vol.8, no.3, pp.784-790, May 1997. 14. W.J. Wolfe, J.M. MacMillan, G. Brady, R. Mathews, J.A. Rothman, M.D. Orosz, C. Anderson and G. Alaghand, “Inhibitory grids and the assignment problem,” IEEE Trans. Neural Networks, vol.4, no.2, pp.319-331, March 1993. 15. A.L. Yuille and J.J. Kosowsky, “Statistical physics algorithms that converge,” Neural computation, pp.341-356, 1994. 16. S. Bharitkar and J.M., Mendel,“Hysteretic Hopfield neural network,” IEEE Trans. Neural Networks, vol.11, no.4, pp. 879-888, Jul 2000. 17. Y. Takefuji and K.C. Lee,“Artificial neural networks for four-coloring map problems and K-colorability problems,” IEEE Trans. Circuits Syst., vol.38, no.3, pp.326-333, March 1991. 18. T. Kwok, K. Smith and L. Wang, “Solving combinatorial optimization problem by chaotic neural networks,” Proc. Of the Artificial Neural networks in Eng., vol.8, pp.317-322, Nov. 1998. 19. I. N. Silva, A. N. Souza and M. E. Bordon, “A modified Hopfield model for solving the N-queen problem,” IJCNN’2000, Como, Italy, pp.509-514, 2000. 20. S. Lee and J. Park, “Dual-mode dynamics neural networks for non-attacking N-queen problem,” IEEE Intel. Symp. Intelligent Control, pp.588-593, August 1993. 21. M. Ohta, “On the self-feedback controlled chaotic neural networks and its application to N -queen problem,” IJCNN’99, Washington DC, pp.713-716, July 1999.