Download Bilinear Programming

Bilinear Programming
Artyom G. Nahapetyan
Center for Applied Optimization
Industrial and Systems Engineering Department
University of Florida
Gainesville, Florida 32611-6595
Email address: [email protected]
1
Introduction
A function f (x, y) is called bilinear if it reduces to a linear one by fixing the vector x or y
to a particular value. In general, a bilinear function can be represented as follows:
f (x, y) = aT x + xT Qy + bT y,
where a, x ∈ Rn , b, y ∈ Rm , and Q is a matrix of dimension n × m. It is easy to see that
bilinear functions compose a subclass of quadratic functions. We refer to optimization
problems with bilinear objective and/or constraints as bilinear problems, and they can be
viewed as a subclass of quadratic programming.
Bilinear programming has various applications in constrained bimatrix games, Markovian assignment and complementarity problems. Many 0-1 integer programs can be formulated as bilinear problems. An extensive discussion of different applications can be found
in [3]. Concave piecewise linear and fixed charge network flow problems, which are very
common in the supply chain management, can be also solved using bilinear formulations
(see, e.g., [5] and [6]).
2
Formulation
Despite a variety of different bilinear problems, most of the practical problems involve a
bilinear objective function and linear constraints, and theoretical results are derived for
those cases. In our discussion we consider the following bilinear problem, which we refer
to as BP.
min f (x, y) = aT x + xT Qy + bT y,
x∈X,y∈Y
1
where X and Y are nonempty polyhedra. The BP formulation is also known as a bilinear
problem with a disjoint feasible region because the feasibility of x (y) is independent form
the choice of the vector y (x).
2.1
Equivalence to Other Problems
Below we discuss some theoretical results, which reveal the equivalence between bilinear
problems and some of concave minimization problems.
Let V (x) and V (y) denote the set of vertices of X and Y , respectively, and g(x) =
miny∈Y f (x, y) = aT x + miny∈Y {xT Qy + bT y}. Note that miny∈Y f (x, y) is a linear programm. Because the solution of a linear problem attains on a vertex of the feasible region,
g(x) = miny∈Y f (x, y) = miny∈V (Y ) f (x, y). Using those notations, the BP problem can be
restated as
min f (x, y) = min{min f (x, y)} = min{ min f (x, y)} = min g(x).
x∈X,y∈Y
x∈X
y∈Y
x∈X y∈V (Y )
x∈X
(1)
Observe that the set of vertices of Y is finite, and for each y ∈ Y , f (x, y) is a linear function
of x; therefore, function g(x) is a piecewise linear concave function of x. From the later
it follows that BP is equivalent to a piecewise linear concave minimization problem with
linear constraints.
It also can be shown that any concave minimization problem with a piecewise linear separable objective function can be reduced to a bilinear problem. To establish this
relationship consider the following optimization problem:
X
min
φi (xi ),
(2)
x∈X
i
where X is an arbitrary nonempty set of feasible vectors, and φi (xi ) is a concave piecewise
linear function of only one component xi , i.e.,
 1
c xi + s1i (= φ1i (xi ))
xi ∈ [λ0i , λ1i )


 2i
2
2
ci xi + si (= φi (xi ))
xi ∈ [λ1i , λ2i )
φi (xi ) =
,
···
···


 ni
ci xi + sni i (= φni i (xi )) xi ∈ [λni i −1 , λni i ]
with c1i > c2i > · · · > cni i . Let Ki = {1, 2, . . . , ni }. Because of the concavity of φi (xi ), the
function can be written in the following alternative form
φi (xi ) = min {φki (xi )} = min {cki xi + ski }.
k∈Ki
k∈Ki
Construct the following bilinear problem:
XX
XX
min f (x, y) =
φki (xi )yik =
(cki xi + ski )yik
x∈X,y∈Y
i
i
k∈Ki
P
(3)
(4)
k∈Ki
where Y = [0, 1] i |Ki | . The proof of the following theorem follows directly from equation
(3), and for details we refer to the paper [5].
2
Theorem 1 If (x∗ , y ∗ ) is a solution of the problem (4) then x∗ is a solution of the problem
(2).
Observe that X is not required to be a polytop. If X is a polytop then the structure of
the problem (4) is similar to BP.
Furthermore, it can be shown that any quadratic concave minimization problem can be
reduced to a bilinear problem. Specifically, consider the following optimization problem:
min φ(x) = 2aT x + xT Qx,
x∈X
(5)
where Q is a symmetric negative semi-definite matrix. Construct the following bilinear
problem
min f (x, y) = aT x + aT y + xT Qy,
(6)
x∈X,y∈Y
where Y = X.
Theorem 2 (see [2]) If x∗ is a solution of the problem (5) then (x∗ , x∗ ) is a solution of
the problem (6). If (x̂, ŷ) is a solution of the problem (6) then x̂ and ŷ solve the problem
(5).
2.2
Properties of a Solution
In the previous section we have shown that BP is equivalent to a piecewise linear concave
minimization problem. On the other hand it is well known that a concave minimization
problem over a polytop attains its solution on a vertex (see, for instance, [1]). The following
theorem follows from this observation.
Theorem 3 (see [2] and [1]) If X and Y are bounded then there is an optimal solution of
BP, (x∗ , y ∗ ), such that x∗ ∈ V (X) and y ∗ ∈ V (Y ).
Let (x∗ , y ∗ ) denote a solution of BP. By fixing the vector x to the value of the vector
x∗ , the BP problem reduces to a linear one, and y ∗ should be a solution of the resulting
problem. From the symmetry of the problem, a similar result holds by fixing the vector y
to the value of the vector y ∗ . The following theorem is a necessary optimality condition,
and it is a direct consequence of the above discussion.
Theorem 4 (see [2] and [1]) If (x∗ , y ∗ ) is a solution of the BP problem, then
minx∈X f (x, y ∗ ) = f (x∗ , y ∗ ) = miny∈Y f (x∗ , y)
(7)
However, (7) is not a sufficient condition. In fact it can only guarantee a local optimality
of (x∗ , y ∗ ) under some additional requirements. In particular, y ∗ has to be the unique
solution of miny∈Y f (x∗ , y) problem. From the later it follows that f (x∗ , y ∗ ) < f (x∗ , y),
∀y ∈ V (Y ), y 6= y ∗ . Because of the continuity of the function f (x, y), for any
T y ∈ V (y), y 6=
y ∗ , f (x∗ , y ∗ ) < f (x, y) in a small neighborhood Uy of the point x∗ . Let U = y∈V (Y ),y6=y∗ Uy .
3
Then f (x∗ , y ∗ ) < f (x, y), ∀x ∈ U , y ∈ V (Y ), y 6= y ∗ . At last observe that Y is a polytop,
and any point of the set can be expressed through a convex combination of its vertices.
From the later it follows that f (x∗ , y ∗ ) ≤ f (x, y), ∀x ∈ U , y ∈ Y , which completes the
proof of the following theorem.
Theorem 5 If (x∗ , y ∗ ) satisfies the condition (7) and y ∗ is the unique solution of the
problem miny∈Y f (x∗ , y) then (x∗ , y ∗ ) is a local optimum of BP.
Recall that BP is equivalent to a piecewise concave minimization problem. Under the
assumptions of the theorem, it is easy to show that x∗ is a local minimum of the function
g(x) as well (see [2]).
3
Methods
In this section we discuss methods to find a solution of a bilinear problem. Because BP is
equivalent to a piecewise linear concave minimization problem, any solution algorithm for
the later can be used to solve the former. In particular, one can employ a cutting plain
algorithms developed for those problems. However, the symmetric structure of the BP
problem allows constructing more efficient cuts. In the paper [4], the author discusses an
algorithm, which converges to a solution that satisfies condition (7), and then proposes a
cutting plain algorithm to find the global minimum of the problem.
Assume that X and Y are bounded. Algorithm 1, which is also known as the “mountain
climbing” procedure, starts from an initial feasible vector y 0 and iteratively solves two linear
problems. The first LP is obtained by fixing the vector y to the value of the vector y m−1 .
The solution of the problem is used to fix the value of the vector x and construct the
second LP. If f (xm , y m−1 ) 6= f (xm , y m ), then we continue solving the linear problems by
fixing the vector y to the value of y m . If the stopping criteria is satisfied, then it is easy to
show that the vector (xm , y m ) satisfies the condition (7). In addition, observe that V (X)
and V (Y ) are finite. From the later and the fact that f (xm , y m−1 ) ≥ f (xm , y m ) it follows
that the algorithm converges in a finite number of iterations.
Let (x∗ , y ∗ ) denote the solution obtained by the Algorithm 1. Assuming that the vertex
∗
x is not degenerate, denote by D the set of directions dj along the ages emanating from the
point x∗ . Recall that g(x) = miny∈Y f (x, y) is a concave function. To construct a valid cut,
for each direction dj find the maximum value of θj such that g(x∗ + θj dj ) ≥ f (x∗ , y ∗ ) − ε,
i.e.,
θj = argmax{θj |g(x∗ + θj dj ) ≥ f (x∗ , y ∗ ) − ε},
Algorithm 1 : Mountain Climbing Procedure
Step 1: Let y 0 ∈ Y denote an initial feasible solution, and m ← 1.
Step 2: Let xm = argminx∈X {f (x, y m−1 )}, and y m = argminy∈Y {f (xm , y)}.
Step 3: If f (xm , y m−1 ) = f (xm , y m ) then stop. Otherwise, m ← m + 1 and go to Step
2.
4
Algorithm 2 : Cutting Plane Algorithm
Step 1: Apply Algorithm 1 to find a vector (x∗ , y ∗ ) that satisfies the relationship (7).
Step 2: Based on the solution (x∗ , y ∗ ), compute the appropriate cuts and construct the
sets X1 and Y1 .
Step 3: If X1 = ∅ or Y1 = ∅, then stop; (x∗ , y ∗ ) is a global ε-optimal solution. Otherwise,
X ← X1 , Y ← Y1 , and go to Step 1.
where ε is a small positive number. Let C = (d1 , . . . , dn ),
( µ
)
¶T
1
1
∆1x = x|
,...,
C −1 (x − x∗ ) ≥ 1 ,
θ1
θn
and X1 = X
T
∆1x . If X1 = ∅ then
min f (x, y) ≥ f (x∗ , y ∗ ) − ε,
x∈X,y∈Y
and (x∗ , y ∗ ) is a global ε-optimum of the problem. If X1 6= ∅ then one can replace X by
the set X1 , i.e, consider the optimization problem
min
x∈X1 ,y∈Y
f (x, y),
and run Algorithm 1 to find a better solution. However, because of the symmetric structure
of the problem, a similar procedure can be applied to Tconstruct a cut for the set Y . Let
∆1y denote the corresponding half-space, and Y1 = Y ∆1y . By updating both sets, i.e.,
considering the optimization problem
min
x∈X1 ,y∈Y1
f (x, y),
the cutting plane algorithm (see Algorithm 2) might find a global solution of the problem
using less number of iterations.
References
[1] Horst R, Pardalos P, Thoai N. Introduction to global optimization, 2-nd Edition,
Boston: Springer, 2000.
[2] Horst R, Tuy H. Global Optimization, 3-rd Edition, New York: Springer, 1996.
[3] Konno H. A Bilinear Programming: Part II. Applications of Bilinear Programming.
Technical Report 71-10, Operations Research House, Stanford University, Stanford,
CA.
5
[4] Konno H. A Cutting Plane Algorithm for Solving Bilinear Programs. Mathematical
Programming 1976;11:14-27.
[5] Nahapetyan A, Pardalos P. A Bilinear Relaxation Based Algorithm for Concave Piecewise Linear Network Flow Problems. Journal of Industrial and Management Optimization 3(1), pp. 71-85, 2007.
[6] Nahapetyan A, Pardalos P. Adaptive Dynamic Cost Updating Procedure for Solving
Fixed Charge Network Flow Problems. Computational Optimization and Applications, submitted for publication.
6