Download - INFORMS PubsOnline

OPERATIONS RESEARCH
Vol. 60, No. 1, January–February 2012, pp. 138–149
ISSN 0030-364X (print) — ISSN 1526-5463 (online)
http://dx.doi.org/10.1287/opre.1110.1005
© 2012 INFORMS
On the Complexity of Nonoverlapping Multivariate
Marginal Bounds for Probabilistic Combinatorial
Optimization Problems
Xuan Vinh Doan
DIMAP and ORMS Group, Warwick Business School, University of Warwick, Coventry CV4 7AL,
United Kingdom, [email protected]
Karthik Natarajan
Department of Management Sciences, College of Business, City University of Hong Kong,
Hong Kong, [email protected]
Given a combinatorial optimization problem with an arbitrary partition of the set of random objective coefficients, we evaluate the tightest-possible bound on the expected optimal value for joint distributions consistent with the given multivariate
marginals of the subsets in the partition. For univariate marginals, this bound was first proposed by Meilijson and Nadas
[Meilijson, I., A. Nadas. 1979. Convex majorization with an application to the length of critical path. J. Appl. Probab. 16(3)
671–677]. We generalize the bound to nonoverlapping multivariate marginals using multiple-choice integer programming.
New instances of polynomial-time computable bounds are identified for discrete distributions. For the problem of selecting
up to M items out of a set of N items of maximum total weight, the multivariate marginal bound is shown to be computable
in polynomial time, when the size of each subset in the partition is O4log N 5. For an activity-on-arc PERT network, the
partition is naturally defined by subsets of incoming arcs into nodes. The multivariate marginal bound on expected project
duration is shown to be computable in time polynomial in the maximum number of scenarios for any subset and the size of
the network. As an application, a polynomial-time solvable two-stage stochastic program for project crashing is identified.
An important feature of the bound developed in this paper is that it is exactly achievable by a joint distribution, unlike
many of the existing bounds.
Subject classifications: probability bounds; integer programming; PERT.
Area of review: Optimization.
History: Received October 2010; revisions received March 2011, May 2011; accepted June 2011.
1. Introduction
tools such as the IBM EinsStat, the Altos Variety, and the
Cadence Encounter Timing System. The basic model in
SSTA is motivated from the Project Evaluation and Review
Technique (PERT) in project management. A PERT network is a directed acyclic graph representation of a project
that consists of several activities with partially specified
precedence relationships among the activities. The project
completion time is given by the length of the longest
path on this graph between a fixed start and sink node.
The goal is to estimate the probability distribution and
moments of the project completion time, given probability distribution information on the random activity durations. In SSTA, the network is a timing graph with nodes
representing input and output pins of gates and arcs representing input-output delays of gates. Using the probability distributions of the individual input-output delays of
gates, SSTA aims to find the distribution and expected
latest arrival time at the sink node of the timing graph.
However, in contrast with deterministic analysis, it is well
known that the probabilistic analysis of PERT networks
is much more difficult (see Hagstrom 1988). The timing
The analysis of combinatorial optimization problems with
random objective coefficients is an important but challenging problem. As motivation, we consider a timing analysis application arising in the design and analysis of digital
integrated circuits. The timing characteristic of a circuit is
significantly affected by variations in fabrication process
parameters and variations in operating environmental factors such as temperature and supply voltages. A detailed
description on the different sources of variations that
influence circuit behavior can be found in the book of
Sapatnekar (2004). In conjunction with shrinking device
sizes, there has been an increasing use of statistical methods
in the design and analysis of digital circuits (Agarwal
et al. 2003, Visweswariah et al. 2006, Blaauw et al. 2008).
This is known as statistical static timing analysis (SSTA).
Although early research in this area dates back to the
1960s–1970s (Kirkpatrick and Clark 1966, Nadas 1979),
the past decade has seen a widespread adoption of SSTA
by the electronic design automation (EDA) community.
SSTA has now been integrated into several EDA software
138
Doan and Natarajan: Multivariate Marginal Bounds
139
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
graphs of industrial application-specific integrated circuits
are also very large (often with millions of gates) with gate
delays that are possibly dependent on each other. Developing tractable methods that can perform accurate probabilistic analysis of these networks remains a significant
challenge. Our goal in this paper is to propose a new
approach that provides efficiently computable bounds on
such networks while capturing partial dependence information among the random variables. The approach is inspired
by a bound proposed by Meilijson and Nadas (1979) and
Nadas (1979), which we review next.
Consider a generic linear combinatorial optimization
problem in maximization form with objective coefficients
c = 4c1 1 c2 1 0 0 0 1 cN 5:
Z4c5 = max c0 x
s0t0 x ∈ X ⊆ 801 19N 0
(1)
Suppose c̃ = 4c̃1 1 c̃2 1 0 0 0 1 c̃N 5 is random. Then, the optimal value Z4c̃5 is random and is the object of our
interest. To avoid trivialities, we assume that the sets
8x ∈ X — xi = 09 and 8x ∈ X — xi = 19 are nonempty for
each index i = 11 0 0 0 1 N . Meilijson and Nadas (1979) proposed an upper bound on the expected optimal objective
value in (1) using only the univariate marginal distributions of c̃. Their problem was motivated in the context of
finding a worst-case upper bound on the expected project
duration that is valid over all joint distributions of the activity durations with the given marginals. The upper bound
was obtained through the solution of the following convex minimization problem over the decision variables d =
4d1 1 d2 1 0 0 0 1 dN 5 ∈ N :
Ɛ6Z4c̃57 ¶ inf Z4d5 +
d
N
X
Ɛ6c̃i − di 7
+
1
(2)
Ɛ max c̃i ¶ inf d +
i=110001N
Ɛ
N
X
i=1
+
c̃i − T
¶ Pinf
i
di =T
N
X
Ɛ6c̃i − d7 0
+
(4)
i=1
McNeil et al. (2005) have discussed the relevance of these
bounds to the actuarial sciences and portfolio risk management community. Weiss (1986) evaluated the bound for
combinatorial optimization problems such as the shortestpath, maximum flow, and reliability problem. Extensions
to incompletely specified univariate marginal distributions
with moment information have been proposed in Klein
Haneveld (1986), Birge and Maddox (1995), and Bertsimas
et al. (2004, 2006). Meilijson (1991), and Natarajan et al.
(2009) have extended the univariate marginal bound to integer programs using a binary reformulation.
In this paper, we generalize the result for probabilistic combinatorial optimization problems by assuming that
information on nonoverlapping multivariate marginals are
available. A popular tool to construct multivariate distributions from univariate distributions is the copula that
helps distinguish the dependencies from the marginals. Formally, an N -dimensional copula is defined as a distribution
function on the unit hypercube 601 17N with standard uniform marginal distributions (see McNeil et al. 2005). Sklar
(1959) showed that for all multivariate distributions F with
marginal distributions F1 1 F2 1 0 0 0 1 FN , there exists a copula
C2 601 17N → 601 17 such that
F 4c1 1 c2 1 0 0 0 1 cn 5 = C F1 4c1 51 F2 4c2 51 0 0 0 1 FN 4cN 5
for all 4c1 1 c2 1 0 0 0 1 cn 5 ∈ 6−ˆ1 ˆ7N 0
For a multivariate distribution with continuous marginals,
the copula is uniquely defined as
i=1
where 6y7+ = max4y1 05. Furthermore, they showed that
the bound in (2) was tight by constructing a joint distribution for c̃ with the correct marginals that attained the
upper bound exactly. The bound can thus be interpreted
as being robust against dependence. For PERT networks,
Klein Haneveld (1986) interpreted the formulation on the
right-hand side of (2) as finding reference values d for the
durations of the activities, such that the project completion
time based on d is balanced with the sum of the expected
delays of the activity durations beyond d. For Z4c5 =
maxi ci , this bound reduces to the maximally dependent
bound of Lai and Robbins (1976):
P
given T . For Z4c5 = i ci , this reduces to the comonotonic
upper bound discussed in Rüschendorf (1983):
d
N
X
Ɛ6c̃i − d7
+
0
(3)
i=1
The result extends to the increasing convex order bound,
which provides a tight upper bound on Ɛ6Z4c̃5 − T 7+ for a
C4u1 1 u2 1 0 0 0 1 un 5 = F F1−1 4u1 51 F2−1 4u2 51 0 0 0 1 FN−1 4uN 5
for all 4u1 1 u2 1 0 0 0 1 un 5 ∈ 601 17N 0
The copula can be used for constructing multivariate discrete distributions too. However, the copula might no longer
be unique. As compared to univariate marginals, analysis
under multivariate marginals is far more challenging. The
concept of a copula is known to be inadequate in this setting
(see Scarsini 1989). Genest et al. (1995) showed that the
only copula consistent with all nonoverlapping multivariate marginals is the independence copula. Li et al. (1996b)
proposed a linkage function to characterize distributions
with nonoverlapping multivariate marginals by emphasizing the separate roles of the dependence structure between
the marginals, and the dependence structure within each
of the marginals. Difficulties in constructing distributions
with prescribed multivariate marginals has also resulted in
fewer known bounds. The reader is referred to Li et al.
(1996a), Rüschendorf (2004), and Embrechts and Pucetti
Doan and Natarajan: Multivariate Marginal Bounds
140
(2006) for some of the known bounds. However, none of
these bounds are directly applicable to the combinatorial
optimization problem.
Our interest in multivariate marginal bounds are motivated by realistic assumptions on dependence information
among random parameters in applications from different
areas such as risk management and PERT networks discussed earlier:
(a) Consider an N -dimensional vector of nonnegative random losses that can be partitioned into subvectors representing losses for policies within specific risk
categories. The goal is to compute the worst-case expected
aggregate loss of a financial position, given subvector
loss distributions but allowing for arbitrary dependencies between subvectors. These subvectors could represent
losses from companies in industry sectors such as health
care, energy, and the Internet, or from countries in different geographical locations. Our focus is on aggregate loss
defined by the sum of the M highest losses from the set
of N losses. For M = 1, this reduces to maximum loss,
whereas for M = N , this reduces to sum of the losses.
(b) Consider the estimation of the expected project duration in an activity-on-arc PERT network with random
activity durations. A simplifying assumption often made
in the analysis of PERT networks is statistical independence among the activity durations. Ball et al. (1995) and
Möhring and Radermacher (1989) review methods that
compute, bound, or approximate the expected project duration with independent activity durations. Ringer (1971) and
van Dorp and Duffey (1999), however, argue that in construction projects the activity durations are often correlated
due to dependence on factors such as weather, manpower
skills, site conditions, and supervision quality. Fulkerson
(1962) proposed a lower bound by using dependence information among activity durations incoming into each node.
For his result to be a valid lower bound, an explicit assumption of independence among the activity durations entering different nodes needed to be made. Kleindorfer (1971)
and Shogan (1983) developed both upper and lower bounds
using dependencies among durations of activities entering a node and independence among activities entering
different nodes. On the other hand, we are interested in
developing a worst-case upper bound on the expected completion time that uses dependency information for activities entering a node, but does not assume independence
among activities entering different nodes. For the SSTA
application discussed earlier, a common assumption is that
the input-output delays of gates that are nearby are highly
correlated due to spatial proximity (see Sapatnekar 2004,
Visweswariah et al. 2006). Our worst-case estimate on the
expected timing of this circuit would then be consistent
with the correlation information of nearby gates and allow
for factors that might make the delays of gates that are far
away arbitrarily correlated.
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
1.1. Problem Description
The formal description of the problem is provided next.
Consider a partition of the index set N = 811 21 0 0 0 1 N 9 into
subsets N1 1 0 0 0 1 NR such that
N=
R
[
Nr
and
Nr ∩ Ns = ™
for all r 6= s0
r=1
Given a vector c ∈ N , let c r ∈ Nr denote the subvector
formed with the elements in the rth subset Nr where
Nr = —Nr — is the size of the subset. The probability measures Pr for the subvectors c̃r are assumed to be known.
Let P4P1 1 0 0 0 1 PR 5 denote the set of joint probability measures for the random vector c̃ consistent with the prescribed
probability measures for the subvectors c̃r . No assumption
on the dependencies between random variables in distinct
subsets are made. The independence measure among the
subvectors thus forms one feasible distribution. For R > 1,
the joint distribution is incompletely specified. For R = N ,
only the univariate marginals are specified. Our goal is to
compute the supremum of the expected optimal objective
value in (1) consistent with the nonoverlapping multivariate
marginals:
Z
Z∗ =
sup
Z4c5 dP 4c50
(5)
P ∈P4P1 10001PR 5
For ease of exposition, we restrict our attention in the paper
to discrete multivariate marginals with bounded support.
It is useful to note that Theorems 1 and 3 and Propositions 1 and 2 are also applicable to continuous multivariate
marginals with finite second moments.
Assumption. The discrete probability distribution for the
subvectors c̃r are defined by the scenarios
P crk for k =
11 0 0 0 1 Kr with probabilities prk satisfying k prk = 1:
Pr 4c̃r = crk 5 = prk
for all k = 11 0 0 0 1 Kr 1 r = 11 0 0 0 1 R0
We obtain the following key results:
(a) In §2, we generalize the Meilijson and Nadas (1979)
bound to nonoverlapping multivariate marginals. Using an
expanded set of decision variables, the computation of the
tight bound Z ∗ is shown to be related to solving a multiplechoice integer program. This leads to a polynomial-time
computable bound for the problem selecting up to M items
out a set of N items of maximum total weight when
the size of each subset in the partition is O4log N 5. This
extends the polynomial complexity result of Meilijson and
Nadas (1979), where the size of each subset in the partition
is O415.
(b) In §3, we identify a weaker upper bound based
on a reduced-integer program. A condition is identified
under which the bound is tight. This leads to polynomialtime computable bounds for worst-case expected project
duration in PERT networks with the partition defined by
subsets of incoming arcs into nodes. A two-stage stochastic program in project crashing is identified for which a
polynomial-time algorithm is provided.
Doan and Natarajan: Multivariate Marginal Bounds
141
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
2. A Multivariate Marginal Formulation
Let Xr denote the projection of X onto the space of the
decision variables in the rth subset:
Xr = projr 4X5 = 8xr — x ∈ X9 ⊆ 801 19 0
= min
d1 10001dR
max
x∈X
+
R
X
dr 4xr 5
Then Z
R
X
ƐPr
h
max 4c̃0r xr
xr ∈Xr
− dr 4xr 55
i
0
(6)
c0 x =
R
X
dr 4xr 5 +
r=1
R
X
x∈X
r=1
dr 4xr 5 +
r=1
x∈X
R
X
R
X
r=1
max 4c0r xr − dr 4xr 550
xr ∈Xr
dr 4xr 5 +
R
X
r=1
r=1
max 4c0r xr − dr 4xr 550
xr ∈Xr
Taking expectations with respect to probability measures
P ∈ P4P1 1 0 0 0 1 PR 5 and minimum with respect to all the dr
variables, we get
ƐP 6Z4c̃57
R
R
h
i
X
X
0
¶ min max dr 4xr 5 + ƐPr max 4c̃r xr − dr 4xr 55
x∈X
d1 10001dR
r=1
r=1
xr ∈Xr
for all P ∈ P4P1 1 0 0 0 1 PR 50
∗
Using strong duality for linear programming, Ẑu∗ is also
the optimal objective value to the dual linear program with
decision variables 4‹4x51 ƒrk 4xr 55r1 k1 x1 xr :
Ẑu∗ = max
Kr
R X
X
X
prk c0rk xr ƒrk 4xr 5
r=1 k=1 xr ∈Xr
X
‹4x5 = 11
X
ƒrk 4xr 5 = 11
r = 11 0 0 0 1 R1
X
‹4v5 −
Kr
X
(8)
prk ƒrk 4xr 5 = 01
k=1
∀ xr ∈ Xr 1 r = 11 0 0 0 1 R1
∀ x ∈ X1
ƒrk 4xr 5 ¾ 01
∀ xr ∈ Xr 1 k = 11 0 0 0 1 Kr 1
r = 11 0 0 0 1 R0
∗
Consider a set of optimal solutions 4dr∗ 4xr 51 t ∗ 1 yrk
5r1 k1 xr
∗
∗
and 4‹ 4x51 ƒrk 4xr 55r1 k1 x1 xr to the primal and dual linear
programs, respectively. Construct a mixture distribution P̄
as follows:
(a) Pick a random feasible solution x ∈ X with probability ‹∗ 4x5.
(b) For each r, the random subvector c̃r is given by the
∗
scenarios crk with probabilities qrk
4xr 5 defined as
p ƒ ∗ 4x 5
∗
for k = 110001Kr 0
qrk
4xr 5 = P̄r1xr 4c̃r = crk 5 = PKrrk rk ∗ r
l=1 prl ƒrl 4x r 5
PK r ∗
∗
Clearly, k=1
qrk 4xr 5 = 1 with qrk
4xr 5 ¾ 0. For P̄ , the
marginal probabilities can be evaluated as
P̄r 4c̃r = crk 5 =
X
∗
‹∗ 4x5qrk
4xr 5
x∈X
=
X
X
xr ∈Xr v∈X2 vr =xr
¶ Ẑu∗ .
Hence, Z
Step 2. Prove that Z ∗ ¾ Ẑu∗ . We provide an explicit
construction of a distribution P̄ ∈ P4P1 1 0 0 0 1 PR 5 such that
ƐP̄ 6Z4c̃57 ¾ Ẑu∗ . The upper bound Ẑu∗ can be computed as
∀ k = 11 0 0 0 1 Kr 1
xr ∈Xr
‹4x5 ¾ 01
P
Because the right-hand side is independent of any particular
feasible solution, the following inequality holds:
Z4c5 ¶ max
∀ x r ∈ Xr 1
k = 11 0 0 0 1 Kr 1 r = 11 0 0 0 1 R0
4c0r xr − dr 4xr 550
Upper bounding
r dr 4x r 5 by maxx∈X
r dr 4x r 5 and
c0r xr − dr 4xr 5 by maxxr ∈Xr 4c0r xr − dr 4xr 55, we obtain:
R
X
(7)
yrk ¾ prk 4c0rk xr − dr 4xr 551
v∈X2 vr =xr
P
c0 x ¶ max
∀ x ∈ X1
dr 4xr 51
x∈X
c0r xr =
r=1
R
X
r=1
s0t0
= Ẑu∗ .
Proof.
Step 1. Prove that Z ∗ ¶ Ẑu∗ . For any feasible solution x ∈ X and a collection of vectors d1 1 0 0 0 1 dR with
dr 405 = 0,
R
X
s0t0 t ¾
r=1
r=1
∗
yrk
r=1 k=1
For example, the projection onto the space of a single variable is the set 801 19. Our first theorem provides the generalization of the Meilijson and Nadas bound in (2) using an
expanded set of decision variables.
Theorem 1. Let dr = dr 4xr 5 x ∈X be a decision vector
r
r
for r = 11 0 0 0 1 R with dr 405 = 0. Define
Kr
R X
X
Ẑu∗ = min t +
Nr
Ẑu∗
the optimal objective value to a linear program with decision variables 4dr 4xr 51 t1 yrk 5r1 k1 xr :
=
X
xr ∈Xr
= prk 0
∗
∗
prk ƒrk
4xr 5
‹ 4v5 PKr
∗
prk ƒrk
4xr 5
∗
l=1 prl ƒrl 4x r 5
Doan and Natarajan: Multivariate Marginal Bounds
142
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
Hence, P̄ ∈ P4P1 1 0 0 0 1 PR 5. The expected optimal value
under the distribution P̄ satisfies
ƐP̄ 6Z4c̃57 ¾
X
‹∗ 4x5
r=1
x∈X
=
X
R
X
‹∗ 4x5
ƐP̄r1xr 6c̃0r xr 7
Kr
R X
X
R
h
i+ X
min d + Ɛ max c̃i − d
0
∗
qrk
4xr 5c0rk xr
d
r=1 k=1
x∈X
PKr
R X
∗
0
X
X
k=1 prk ƒrk 4x r 5crk x r
=
‹∗ 4v5
PK r
∗
r=1 xr ∈Xr v∈X2vr =xr
l=1 prl ƒrl 4x r 5
=
Kr
R X
X
X
For any i 6= 1, by increasing di up to d1 , the first term
maxi∈N di remains unaffected whereas the second term does
not increase, but possibly decreases. Hence, there exists an
optimal solution with all the di values equal. This leads to
the single-variable optimization problem:
∗
prk ƒrk
4xr 5c0rk xr
r=1 k=1 xr ∈Xr
i∈Nr
r=1
(ii) Let X = 8e4N +15 1 09, where e4N +15 is the vector in
with all ones. Then we have
+
X
X
max
ci xi − TxN +1 x ∈ X =
ci − T 0
N +1
i∈N
= Ẑu∗ 0
The first inequality is obtained by evaluating the objective
function value at the feasible solution x ∈ X chosen at step
(a) of the distribution instead of the corresponding optimal
solution. The remaining equalities follow from dual feasibility and strong duality. Hence, Z ∗ ¾ ƐP̄ 6Z4c̃57 ¾ Ẑu∗ . From
steps 1 and 2, Z ∗ = Ẑu∗ . ƒ
i∈N
Consider the modified problem in N + 1 dimensions
with R + 1 partitions, of which the last partition is
NR+1 = 8N + 19 corresponding to the variable xN +1 . The
projection of the feasible region is Xr = 8e4Nr 5 1 09 for all
r = 11 0 0 0 1 R + 1. Using Theorem 1, the tight upper bound
is given as
min
R
X
+
dr + dR+1
+
R
X
Ɛ
X
+
c̃i − dr
For the special case of the maximum of random variables and the sum of random variables, we use the result
in Theorem 1 to extend the univariate marginal bounds in
(3) and (4) to multivariate marginal bounds.
d1 10001dR 1dR+1
Proposition 1. (i) For Z4c5 = maxi ci , the following
inequality holds:
R
i
i+ h
h
X
ƐP max c̃i ¶ min d + ƐPr max c̃i − d
We have x+ + y + ¾ 6x + y7+ for all x1 y ∈ and the equality can happen when x = 0 or y = 0. Thus, we can claim
that by setting dR+1 = −T , the tight upper bound is still
obtained by solving:
d
i∈N
and the bound is tight.
P
(ii) For Z4c5 = i ci and T ∈ , the following inequality
holds:
+
R
+ X
X
X
ƐP
c̃i − T ¶ Pmin
ƐPr
c̃i − dr
dr =T
r=1
i∈Nr
for all P ∈ P4P1 1 0 0 0 1 PR 51
and the bound is tight.
4N 5
4N 5
4N 5
min
d1 10001dN
R
h
i+ X
0
max di + Ɛ max4c̃i − di 5
i∈N
r=1
+ 6−T − dR+1 7 0
min
d1 10001dR
i∈Nr
It is easy to check that there exists an optimal solution
such that all the di values are equal. Let d1 = maxi∈N di .
R
X
+
dr − T
+
r=1
R
X
r=1
Ɛ
X
+ c̃i − dr
0
i∈Nr
P
The term 6 r dr − T 7+ is nondecreasing in dr . If the term
P
r dr − T = … > 0, we can decrease at least one of the dr s
by … such that the first term decreases by … whereas one of
the expectation terms would increase by at most …. Using a
similar argument for a negative …, we canPverify that there
exists an optimal solution that satisfies r dr = T . Thus,
the tight upper bound is hence the optimal value of the
optimization problem:
4N 5
Proof. (i) Let X = 8e1 1 e2 1 0 0 0 1 eN 9 where ei is the
unit vector in N with 1 in the ith position and 0 otherwise.
Then, max8c0 x — x ∈ X9 = maxi∈N ci . If R > 1, the projec4N 5
4N 5
4N 5
tion of the feasible region is Xr = 8e1 r 1 e2 r 1 0 0 0 1 eNr r 1 09.
Using Theorem 1 and noting that 0 ∈ Xr with dr 405 = 0,
the tight upper bound is given as
i∈Nr
+
for all P ∈ P4P1 1 0 0 0 1 PR 51
r
r=1
i∈Nr
r=1
i∈N
r=1
min
d1 10001dR
s0t0
R
X
r=1
R
X
Ɛ
X
+
c̃i − dr
i∈Nr
dr = T 0
ƒ
r=1
From Theorem 1, given a set of vectors d1 1 0 0 0 1 dR , evaluating the upper bound reduces to
1. Computing the optimal P
value to the deterministic
maximization problem maxx∈X r dr 4xr 5 and
2. Computing expectations of the random terms
maxxr ∈Xr 4c̃0r xr − dr 4xr 55 for r = 11 0 0 0 1 R.
Doan and Natarajan: Multivariate Marginal Bounds
143
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
The feasible region X for combinatorial optimization
problems can be represented using binary variables and linear inequalities as
X
R
Ar x r ¶ b 0
X = x ∈ 801 19N r=1
The next proposition uses this representation of the feasible
region to reformulate the deterministic term in the upper
bound as a multiple-choice integer program. A multiplechoice integer program is a linear binary optimization problem in which the variables are partitioned and precisely one
variable from each subset in the partition is selected (see
Bean 1984).
Proposition 2. The tight upper bound Z ∗ in Theorem 1 is
computable as
R
i
h
X
0
∗
Z = min Ẑ4d1 10001dR 5+ ƐPr max 4c̃r xr −dr 4xr 55 1
d1 10001dR
xr ∈Xr
r=1
where Ẑ4d1 1 0 0 0 1 dR 5 is the optimal objective value to a
multiple-choice
integer program over the decision vectors
zr = zr 4xr 5 x ∈X for r = 11 0 0 0 1 R:
r
Ar xr z∗r 4xr 5 =
r=1 xr ∈Xr
R
X
Ar x∗r
r=1
¶ b0
The objective value satisfies
max
x∈X
R
X
dr 4xr 5 =
r=1
R
X
dr 4x∗r 5
r=1
=
R X
X
dr 4xr 5z∗r 4xr 5
r=1 xr ∈Xr
¶ Ẑ4d1 1 0 0 0 1 dR 50
From steps 1 and 2, Ẑ4d1 1 0 0 0 1 dR 5 = maxx∈X
proving the desired result. ƒ
R X
X
dr 4xr 5zr 4xr 5
R
X
X
Ar xr zr 4xr 5 ¶ b1
P
r
dr 4xr 5,
2.1. Application to Subset Selection
r=1 xr ∈Xr
s0t0
R X
X
r
Ẑ4d1 1 0 0 0 1 dR 5
= maxz1 1 0001 zR
P
Step 2. Prove that Ẑ4d1 1 0 0 0 1 dR 5 ¾ maxx∈X r dr 4xr 5.
Given vectors d1 1 0 0 0 1 dR , consider an optimal solution
P
x∗ to maxx∈X r dr 4xr 5. For each r, set z∗r 4x∗r 5 = 1.
Set z∗r 4xr 5 = 0 for all xr ∈ Xr 1 xr 6= x∗r . Clearly,
P
∗
∗
∗
xr ∈Xr zr 4x r 5 = 1. Thus, z1 1 0 0 0 1 zR ∈ Z because
(9)
Consider the problem of selecting up to M items out of a
total of N items of maximum total weight:
r=1 xr ∈Xr
X
zr 4xr 5 = 11
∀ r = 11 0 0 0 1 R1
xr ∈Xr
zr 4xr 5 ∈ 801 191
∀ xr ∈ Xr 1 r = 11 0 0 0 1 R0
Proof. Let Z denote the feasible region to the multiplechoice integer program (9). Thus,
Ẑ4d1 1 0 0 0 1 dR 5 =
R X
X
max
4z1 1 0001 zR 5∈Z
dr 4xr 5zr 4xr 50
r=1 xr ∈Xr
P
Step 1. Prove that Ẑ4d1 1 0 0 0 1 dR 5 ¶ maxx∈X r dr 4xr 5.
Given vectors
d1 1 0 0 0 1 dR , consider an optimal
solution
P
z∗1 1 0 0 0 1 z∗R to formulation (9). Set x∗r = xr ∈Xr xr z∗r 4xr 5.
Then, we check for the feasibility of x∗ :
R
X
r=1
Ar x∗r
=
R X
X
Ar xr z∗r 4xr 5 ¶ b0
r=1 xr ∈Xr
Furthermore, x∗r ∈ Xr ⊆ 801 19Nr implies that x∗ ∈ X ⊆
801 19N . The objective value satisfies
Ẑ4d1 1 0 0 0 1 dR 5 =
R X
X
dr 4xr 5z∗r 4xr 5
r=1 xr ∈Xr
=
R
X
dr 4x∗r 5
r=1
¶ max
x∈X
R
X
r=1
dr 4xr 50
Z4c5 = max c0 x
0
s0t0 e4N 5 x ¶ M1
x ∈ 801 19N 0
(10)
In a risk management context, c denotes a nonnegative loss
vector and Z4c5 defines the sum of the M highest losses in
this set. Our next theorem provides an instance of a polynomial computable bound on the expected value of Z4c̃5
in (10) using multivariate marginal distribution information of c̃.
Theorem 2. Given a scenario representation for the random weights, the tight multivariate marginal upper bound
Z ∗ on the expected optimal value in (10) is computable
in time polynomial in the maximum number of scenarios
in any subset and N when the size of each subset in the
partition is O4log N 5.
Proof. Consider a partition of the index set N =
811 0 0 0 1 N 9. Assume that the size of each subset Nr in the
partition is Nr = O4log N 5. The projection of the feasible
region of (10) on the space of decision variables in the rth
subset is
0
Xr = xr ∈ 801 19Nr e4Nr 5 xr ¶ M 0
Doan and Natarajan: Multivariate Marginal Bounds
144
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
Using Proposition 2, Z ∗ can be computed as
Z ∗ = min t +
Kr
R X
X
restricted to the first S subsets with a knapsack capacity
restricted to B:
yrk
r=1 k=1
ZS 4B5 = max
s0t0 t ¾ Ẑ4d1 1 0 0 0 1 dR 51
S X
X
yrk ¾ prk c0rk xr − prk dr 4xr 5 ∀ xr ∈ Xr 1
s0t0
S X
X
X
Let —Xr — denote the size of the set Xr and Kmax = maxr Kr
denote the maximum number of scenarios in any subset.
Let poly4N 5 = N O415 denote a polynomial in N . The total
number of decision variables in formulation (11) is polynomial in the size of the input because
number of decision variables in 4115
t
R
X
—Xr — +
r=1
| {z }
| {z }
=1+
2O4log N 5 +
r=1
(14)
Kr
where ZS 4B5 is −ˆ if the set is infeasible. The optimal objective value Ẑ4d1 1 0 0 0 1 dR 5 is equal to ZR 4M5. The
overall time complexity of implementing this method is
P
O4M r —Xr —5. Since —Xr — is polynomially bounded in the
N , Ẑ4d1 1 0 0 0 1 dR 5 can be computed in time polynomial in
the size of the input. From the equivalence on separation and optimization (Grötschel et al. 1988), the result
follows. ƒ
3. A Reduced Formulation
If not, find a violated inequality.
From Proposition 2, Ẑ4d1 1 0 0 0 1 dR 5 is the optimal objective value to the multiple choice knapsack problem:
Ẑ4d1 1 0 0 0 1 dR 5
dr 4xr 5zr 4xr 5
r=1 xr ∈Xr
0
e4Nr 5 xr zr 4xr 5 ¶ M1
r=1 xr ∈Xr
X
Define Z0 4B5 = 0 for B = 01 11 0 0 0 1 M. To compute ZS 4B5
for S = 11 0 0 0 1 R, the dynamic programming recursion is
set up as
0
t ¾ Ẑ4d1 1 0 0 0 1 dR 50
R X
X
∀ xr ∈ Xr 1 r = 11 0 0 0 1 S0
e4NS 5 xS ¶ B1
To analyze the complexity of the optimization problem (11), consider the separation version of the problem
(see Grötschel et al. 1988). Since testing for the feasibility
of the second set of inequalities in (11) is easy, we restrict
our attention to the following:
Given t and d, check if
s0t0
(13)
s0t0 xS ∈ XS 1
r=1
R X
X
∀ r = 11 0 0 0 1 S1
0
= 1 + poly4N 5 + O4RKmax 50
= max
zr 4xr 5 ∈ 801 191
yrk
R
X
zr 4xr 5 = 11
xr ∈Xr
ZS 4B5 = max ZS−1 4B − e4NS 5 xS 5 + dS 4xS 5
Kr
r=1
dr 4xr 5
R
X
R
X
0
e4Nr 5 xr zr 4xr 5 ¶ B1
r=1 xr ∈Xr
k = 11 0 0 0 1 Kr 1 r = 11 0 0 0 1 R0
= |{z}
1 +
dr 4xr 5zr 4xr 5
r=1 xr ∈Xr
(11)
zr 4xr 5 = 11
(12)
∀ r = 11 0 0 0 1 R1
xr ∈Xr
zr 4xr 5 ∈ 801 191
∀ xr ∈ Xr 1
r = 11 0 0 0 1 R0
Dudzinski and Walukiewicz (1987) provide a pseudopolynomial time dynamic programming algorithm for solving multiple choice knapsack problems. We outline the
algorithm in our context. Let ZS 4B5 be the optimal solution to the multiple choice knapsack problem (12) when
The total number of decision variables in Theorem 1 could
be very large compared to the number of decision variables
in the original deterministic optimization problem. In this
section, we provide a weaker upper bound wherein the
number of decision variables in the formulation does not
grow exponentially in size of the problem. For univariate
marginals, this bound reduces to the Meilijson and Nadas
(1979) result. An advantage of the bound is that the computation relates directly to the complexity of solving the
original combinatorial optimization problem. This property
has been exploited by Meilijson and Nadas (1979), Weiss
(1986), Klein Haneveld (1986), and Bertsimas et al. (2004,
2006) in proposing algorithms for univariate marginals.
Theorem 3. (i) Let d = 4d1 1 0 0 0 1 dN 5 ∈ N be a decision
vector. An upper bound on Z ∗ is computable as
Z
∗
¶ Zu∗ = min
d
Z4d5+
R
X
r=1
ƐP r
h
max 4c̃0r xr −d0r xr 5
xr ∈Xr
i
0
(15)
(ii) Assume that for each r = 11 0 0 0 1 R, the set of subvectors xr ∈ Xr /809 are linearly independent. Then, Z ∗ = Zu∗ .
Doan and Natarajan: Multivariate Marginal Bounds
145
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
Proof. (i) Consider an optimal solution d∗ to formulation (15). For the upper bound defined in Theorem 1, set
0
dr∗ 4xr 5 = d∗r xr . Then the tight upper bound satisfies
∗
Z ¶ max
x∈X
R
X
dr∗ 4xr 5 +
r=1
R
X
= max d x +
x∈X
R
X
ƐPr
h
∗
ƐP r
= Z4d 5 +
R
X
ƐPr
h
r=1
max 4c̃0r xr
xr ∈Xr
max 4c̃0r xr
xr ∈Xr
r=1
max 4c̃0r xr
xr ∈Xr
− dr∗ 4xr 55
0
− d∗r xr 5
R
X
X
i
0
− d∗r xr 5
∀ x ∈ X1
(16)
r=1
yrk ¾ prk 4c0rk xr − d0r xr 51
∀ x r ∈ Xr 1
k = 11 0 0 0 1 Kr 1 r = 11 0 0 0 1 R0
Zu∗
Using strong duality for linear programming,
is the optimal objective value to the dual linear program with decision
variables 4‹4x51 ƒrk 4xr 55r1 k1 x1 xr :
Zu∗ = max
Kr
R X
X
X
prk c0rk xr ƒrk 4xr 5
X
‹4x5 = 11
X
ƒrk 4xr 5 = 11
xr ∈Xr
xr ‹4x5 −
x∈X
Kr
X
X
prk xr ƒrk 4xr 5 = 01
k=1 xr ∈Xr
‹4x5 ¾ 01
∀ xr ∈ Xr 1 k = 11 0 0 0 1 Kr 1
r = 11 0 0 0 1 R0
∗
Consider a set of optimal solutions 4d∗r 1 t ∗ 1 yrk
5r1 k and
∗
∗
4‹ 4x51 ƒrk 4xr 55r1 k1 x1 xr to the primal and dual linear programs, respectively. From the dual feasibility conditions,
we have
xr ∈Xr v∈X2 vr =xr
∗
prk ƒrk
4xr 51
∀ xr ∈ Xr 1 r = 11 0 0 0 1 R0
k=1
With this equality, it is easy to verify that the distribution P̄
constructed in Theorem 1 provides the tight upper bound as
before. Because the construction is identical to Theorem 1,
we omit it for brevity. Hence, Z ∗ = Zu∗ . ƒ
The next example shows that the bound in Theorem 3
might not be the tightest bound in general. Consider an
instance of the subset selection problem with N = 4,
M = 2, and R = 2. The subsets are N1 = 811 29 and N2 =
831 49. The feasible set X contains a total of 11 feasible solutions and X1 = X2 = 801 192 . Define a total of 8
decision variables dr 4xr 5 for xr ∈ 801 192 , r = 11 2, with
d1 401 05 = d2 401 05 = 0. The tight bound Z ∗ in Theorem 1
can be calculated as
Z ∗ = min t +
Kr
R X
X
yrk
r=1 k=1
s0t0 t ¾ 01
t ¾ max d1 401151d1 411051d1 41115 1
t ¾ max d2 401151d2 411051d2 41115 1
∗
∀xr ∈ 801192 1 k = 110001Kr 1 r = 1120
This can be reformulated as linear optimization problem
and solved by a linear programming solver. To compute the
upper bound Zu∗ in Theorem 3, we define 4d1 1 d2 1 d3 1 d4 5
as decision variables:
∀ x ∈ X1
ƒrk 4xr 5 ¾ 01
X
Kr
X
∀ k = 11 0 0 0 1 Kr 1
r = 11 0 0 0 1 R1
∀ r = 11 0 0 0 1 R1
X
k=1
yrk ¾ prk c0rk xr −prk dr 4xr 51
x∈X
X
xr ∈Xr
t ¾ max d1 40115+d2 401151d1 40115+d2 41105 1
t ¾ max d1 41105+d2 401151d1 41105+d2 41105 1
r=1 k=1 xr ∈Xr
s0t0
‹∗ 4v5 =
v∈X2 vr =xr
yrk
d0r xr 1
v∈X2 vr =xr
Given that the subvectors xr ∈ Xr /809 are linearly independent, this implies
i
r=1 k=1
s0t0 t ¾
Kr
X
X
∗
‹∗ 4v5 =
xr
prk ƒrk
4xr 5 1
X
∀ r = 11 0 0 0 1 R0
The first inequality is valid because the set of 4dr∗ 4xr 55xr ∈Xr
forms a feasible solution in Theorem 1. Thus, Z ∗ ¶ Zu∗ .
(ii) The bound Zu∗ is the optimal objective value to the
linear program with decision variables 4dr 1 t1 yrk 5r1 k :
Kr
R X
X
xr
i
= Zu∗ 0
Zu∗ = min t +
X
xr ∈Xr
h
r=1
∗0
and
‹ 4v5 = 11
∀ r = 11 0 0 0 1 R1
Zu∗ = min t +
Kr
R X
X
yrk
r=1 k=1
s0t0 t ¾ 01
t ¾ max d1 1 d2 1 d3 1 d4 1
t ¾ max d1 + d2 1 d1 + d3 1 d1 + d4 1
t ¾ max d2 + d3 1 d2 + d4 1 d3 + d4 1
yrk ¾ prk c0rk xr − prk d0r xr 1
∀ xr ∈ 801 192 1 k = 11 0 0 0 1 Kr 1 r = 11 20
Doan and Natarajan: Multivariate Marginal Bounds
146
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
Let P1 be a discrete uniform distribution with K1 = 3
scenarios: 461 85, 451 15, and 471 95, each occurring with
probability 1/3. Likewise, let P2 be a discrete uniform distribution with K2 = 3 scenarios: 411 95, 491 35, and 491 65,
each occurring with probability 1/3. By solving the linear programs, we verify that the tight bound Z ∗ = 50/3 is
strictly smaller than the bound Zu∗ = 17.
An example of combinatorial optimization problem that
satisfies the linear independence condition in Theorem 3(ii)
is the PERT problem with the partition formed by subsets
of incoming arcs into nodes. In the next section, we identify instances of the PERT problem where the multivariate
marginal bound is computable in polynomial time.
3.1. Application to PERT Networks
Let V = 811 0 0 0 1 M9 denote the set of nodes in a PERT
network where nodes 1 and M represent the start and end
of the project and E denotes the set of arcs. A total of N
arcs are present in the network. Each arc is associated with
a length or the time needed to complete the activity. Let
4i1 j5 denote an arc originating from node i and terminating at node j with arc length cij . The project duration is
determined by the length of the longest path from the start
node to end node in this network. It can be computed as the
optimal objective value to the combinatorial optimization
problem:
Z4c5 = max
X
cij xij
4i1 j5∈E
s.t.
X
xji −
i2 4j1 i5∈E
X
xij
i2 4i1 j5∈E

 11 if j = 11
= −11 if j = M1

01 if j = 21 0 0 0 1 M − 11
xij ∈ 801 191
(17)
∀ 4i1 j5 ∈ E0
For project networks with deterministic activity durations,
Z4c5 can be computed in polynomial time by solving the
linear programming relaxation of formulation (17). This is
due to the unimodular structure of the constraint matrix
for network flow problems. Whereas the deterministic optimization problem is easy, the computation of the expected
optimal value is still a challenging problem. Hagstrom
(1988) formally established the complexity of the stochastic version of PERT networks under the assumption of
independent activity durations. Specifically, she addressed
the complexity of the following two problems:
(1) MEAN: Given a PERT network with discrete, independent activity durations, compute the expected project
duration.
(2) CDF: Given a PERT network with discrete, independent activity durations and a due date T , compute the
probability that the project will be completed in time less
than or equal to T .
Her main results are summarized in the next theorem.
Theorem 4 (Hagstrom 1988). (i) CDF and the two-state
version of MEAN are #P-complete.
(ii) Furthermore, MEAN and CDF cannot be computed
in time polynomial in the number of points in the range of
the project duration unless P = NP.
The classes #P and #P-complete are the counting versions of the NP and NP-complete recognition problems.
This computational complexity class was first introduced
by Valiant (1979). Any counting problem is at least as
hard as the corresponding recognition problem. However, there exist recognition problems that are solvable
in polynomial time for which the counting versions are
#P-complete. Valiant (1979) and Provan and Ball (1983)
provide instances of such problems. The PERT problem falls under this category. Because the existence of
a polynomial-time algorithm for solving a #P-complete
problem would imply P = NP, it seems highly unlikely
that these problems can be solved in polynomial time.
Although the complexity of MEAN with activity durations taking more than two values is still open, part (ii) of
Theorem 4 indicates that problems with longer encoding
might be difficult to solve.
A special class of graphs for which efficient algorithms have been derived are series-parallel graphs. These
graphs can be constructed by a sequence of series and
parallel compositions starting from single-arc graphs. The
inverse operations, series and parallel reductions, recursively reduce series-parallel graphs to single arcs. Given
independent activity durations, the computation for these
graphs can therefore be reduced to a sequence of convolutions (series reductions) and products (parallel reductions) of distribution functions. However, even in this case,
there is an inherent complexity to the PERT problem that
depends on the way distributions are generated along the
reduction sequence. Potentially, the number of points in
the range of the project duration could be exponential in
the size of the network. The complexity results for these
graphs are summarized in the next theorem.
Theorem 5 (Ball et al. 1995, Möhring 2001). (i) The
two-state version of MEAN and CDF for a series-parallel
graph is NP-hard in the weak sense.
(ii) For a series-parallel graph with arbitrary discrete
distributions, MEAN and CDF can be computed in time
polynomial in the size of the network and the maximum
number of distinct values that the project duration takes
along a series-parallel reduction sequence.
(iii) For a series-parallel graph with activity durations
restricted to the set 801 11 21 0 0 0 1 q9, MEAN and CDF can
be computed in time polynomial in the size of the network and q.
Several authors, including Kleindorfer (1971), Spelde
(1976), Dodin (1985) and Möhring (2001), have proposed
bounds for general graphs by transforming them to seriesparallel graphs and using bounds for series-parallel graphs.
Doan and Natarajan: Multivariate Marginal Bounds
147
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
In the PERT context, a natural partition is formed by the
set of incoming arcs into each node. Given a joint (possibly dependent) discrete distribution for the set of activities
coming into each node and assuming independence among
activity durations at different nodes, Fulkerson (1962) and
Shogan (1983) developed bounds on the expected project
completion time. Meilijson and Nadas (1979) developed a
polynomial-time computable upper bound on the expected
project completion time, assuming marginal distribution
information allowing for any possible dependence among
activity durations. Our next theorem identifies instances of
the PERT problem when the multivariate marginal bound
is computable in polynomial time.
Theorem 6. (i) Given a scenario representation for the
set of activity durations entering each node, the tight multivariate marginal upper bound Z ∗ on the expected project
duration is computable in time polynomial in the maximum number of scenarios in any subset and the size of the
network.
(ii) Given a discrete distribution for each activity duration with independence among the activity durations that
enter a node, the tight multivariate marginal upper bound
Z ∗ on the expected project duration is computable in time
polynomial in the maximum number of supporting points
of the activity duration distribution and the size of the
network.
By using the dual formulation for the linear programming
representation of Z4d5, Z ∗ is computed as
Z ∗ = min wM − w1 +
M−1
i+
X h
Z ∗ = min Z4d5 +
Ɛ max 4c̃ij − dij 5
d
j=2
+Ɛ
h
i2 4i1 j5∈E
i
max 4c̃iM − diM 5 0
i2 4i1 M5∈E
(18)
k=1 yjk
j=2
∀ 4i1 j5 ∈ E1
s.t. wj − wi ¾ dij 1
yjk ¾ pjk cijk − pjk dij 1
∀ i2 4i1 j5 ∈ E1
(19)
k = 11 0 0 0 1 Kj 1 j = 21 0 0 0 1 M1
yjk ¾ 01
∀ k = 11 0 0 0 1 Kj 1 j = 21 0 0 0 1 M − 10
The linear program in (19) is polynomial sized in the size
of the network and the maximum number of scenarios in
any subset, proving the desired result.
(ii) To prove the polynomial complexity of computing
the bound, we reformulate (18) as follows:
Z ∗ = min t
s0t0 t ¾ Z4d5 +
M−1
X
Ɛ
h
j=2
+Ɛ
h
max 4c̃ij − dij 5
i+
i2 4i1 j5∈E
i
max 4c̃iM − diM 5 0
i2 4i1 M5∈E
(20)
The separation version of this problem is as follows:
Given t and d, check if
t ¾ Z4d5 +
M−1
X
Ɛ
j=2
Proof. (i) Consider the subvector of random activity durations c̃j = 4c̃ij 5i2 4i1 j5∈E for arcs entering a node j. Let
cjk denote a realization of this subvector occurring with
probability Pj 4c̃j = cjk 5 = pjk for k = 11 0 0 0 1 Kj where
P
k pjk = 1. Without loss of generality, we assume that
for each node j = 21 0 0 0 1 M − 1, there exists at least
one directed path from the start to the end node that
passes through j and one directed path that does not pass
through j. Else, we can decompose the problem into two
independent problems of finding the longest path from node
start 1 to node j and the longest path from node j to end
node M. The projection of the feasible region of (17) onto
the space of incoming arcs into node j is then the extreme
points of the standard simplex. This follows by observing
that any directed path in the acyclic PERT network from
the start to the end node consists of at most one arc entering
node j. Hence, Theorem 3(ii) is applicable to this problem.
The tight multivariate marginal upper bound is
PM PK j
+Ɛ
+
max 4c̃ij − dij 5
i2 4i1 j5∈E
max 4c̃iM − diM 5 0
i2 4i1 M5∈E
If not, find a violated inequality.
For a fixed d, Z4d5 can be evaluated in polynomial time
in the size of the network. We focus on the computation
of the expected value of the maximum of independent random variables; namely, a parallel graph. Assuming that the
maximum number of supporting points among the activity
distributions is Kmax = maxi Ki , the random variable c̃max =
maxi=110001n c̃i has at most nKmax supporting points. From
Theorem 5(ii), MEAN and CDF problem for this parallel
graph is solvable in time polynomial in n and Kmax . The
maximum number of incoming arcs into a node is M − 1.
Thus, testing for feasibility in Problem (20) is possible in
polynomial time in the maximum number of supporting
points Kmax and the size of the network.
To find a violated inequality, consider the case when
the solution is infeasible. We compute subgradients of
Z4d5 and functions of the form Ɛ6maxi=110001n 4c̃i − di 57 with
respect to d. For Z4d5, the optimal solution to Problem (17)
provides the subgradient. The piecewise-linear function
4n5
f 4d3 c5 = maxi=110001n 4c̃i − di 5 has g4d3 c5 = −eid 4c5 , where
min
d
imin
4c5 = min8j2 cj − dj = f 4d3 c59, as one of its subgradients. Therefore, Ɛ6g4d3 c̃57 is a subgradient of Ɛ6f 4d3 c̃57.
In order to calculate Ɛ6g4d3 c̃57, we need to calculate
Doan and Natarajan: Multivariate Marginal Bounds
148
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
d
Pi 4d5 = P 4c̃i −di = f 4d3 c̃51 i = imin
4c̃55 for all i = 11 0 0 0 1 n.
We have
Pi 4d5 = P c̃i − di > max 4c̃j − dj 51 c̃i − di
j=110001i−1
¾ max 4c̃j − dj 5 1 i = 11 0 0 0 1 n0
j=i+110001n
i
The random variable c̃max
4d5 = maxj=11 0001 i−1 4c̃j − dj 5 has at
most 4i −15Kmax supporting points and its distribution again
can be specified in polynomial time (instances of CDF
problem). We have similar results for the random variable
−i
c̃max
4d5 = maxj=i+11 0001 n 4c̃j − dj 5. The random variables c̃i ,
i
−i
c̃max
4d5, and c̃max
4d5 are independent. The total supporting points of the joint distribution is bounded from above
3
by 4i − 154n − i − 15Kmax
. Thus, the probability Pi 4d5 can
be calculated in polynomial time, which implies that the
subgradient Ɛ6g4d3 c̃57 is computable in polynomial time.
The violated inequality is then constructed in polynomial
time in the maximum number of supporting points Kmax
and the size of the network. Combining these two results,
we obtain the polynomial-time solvability of the separation
problem. The equivalence of separation and optimization
(Grötschel et al. 1988) proves the desired result. ƒ
As an application of the worst-case bound on the
expected project completion time, we consider a two-stage
stochastic program for project crashing. Formally, the duration of each activity 4i1 j5 ∈ E is described as c̃ij −tij , where
c̃ij is the bounded discrete random variable and the decision
variable tij is bounded between 0 and uij . The crashing cost
is assumed to be linear and the cost per unit change in tij is
fij ¾ 0 for all 4i1 j5 ∈ E. The goal is to determine the crashing duration for each activity that minimizes the sum of the
crashing costs and the expected project completion time.
The standard two-stage stochastic optimization formulation
for this problem is provided by Wollmer (1985):
X
fij tij + Ɛ6Z4c̃ − t57
min 
4i1 j5∈E
s0t0 0 ¶ tij ¶ uij 1
∀ 4i1 j5 ∈ E1
(21)
where  ¾ 0 is a parameter that provides a trade-off
between crashing costs and expected project completion
time. One can conceive more complicated variants of
the model to account for budget constraints of shared
resources. For simplicity, we focus on the basic model and
develop a distributional robust optimization counterpart of
this problem. Assume as in part (i) of Theorem 6 that a
scenario representation is provided for the set of activity
durations entering a node. However, the entire joint distribution for all activity durations in the project is unknown.
The stochastic program in (21) is replaced by a distributional robust optimization problem:
X
min 
fij tij +
sup
ƐP 6Z4c̃ − t57
4i1 j5∈E
s0t0
0 ¶ tij ¶ uij 1
P ∈P4P1 10001PR 5
∀ 4i1 j5 ∈ E1
(22)
where P is the set of all probability measures P with
the given marginals. Because the worst-case value is used
in computing the second-stage expected project duration,
the first-stage crashing decisions are robust to dependence.
A straightforward application of the dual formulation in
(19) to the inner supremum in (22) provides a linear optimization formulation for the distributional robust problem:
min 
X
fij tij +wM −w1 +
Kj
M X
X
yjk
j=2 k=1
4i1j5∈E
s0t0 0 ¶ tij ¶ uij 1
∀4i1j5 ∈ E1
∀ 4i1j5 ∈ E1
wj −wi ¾ dij 1
yjk ¾ pjk cijk −pjk tij −pjk dij 1
(23)
∀i2 4i1j5 ∈ E1
k = 110001Kj 1 j = 210001M1
yjk ¾ 01
∀ k = 110001Kj 1 j = 210001M −10
Because this linear program is polynomial sized, the distributional robust project-crashing problem under multivariate
marginals can be solved in polynomial time.
4. Conclusion
In this paper, the Meilijson and Nadas (1979) bound
for probabilistic combinatorial optimization problems is
extended from univariate marginals to nonoverlapping multivariate marginals. The bound is robust against dependence and valid across all joint distributions with the given
marginals. Furthermore, this bound is tight, in that there
exists a multivariate distribution that attains the bound. Our
result thus provides a way to improve on the univariate
marginal bound when additional distributional information
is available. Importantly, we identify new instances in the
subset selection and PERT network problem, where the
bound on the expected value is computable in polynomial
time. One interesting question that remains is whether these
bounds can be tightened when information on overlapping
multivariate marginals are available. Furthermore, is it possible to develop polynomial-time computable tight bounds
in this case?
Acknowledgments
The authors thank the associate editor and two anonymous reviewers for their valuable comments and suggestions on improving
the manuscript. The research was done when the first author was
at the Operations Research Center at Massachusetts Institute of
Technology and the Department of Combinatorics and Optimization at the University of Waterloo.
References
Agarwal, A., V. Zolotov, D. T. Blaauw. 2003. Statistical timing analysis
using bounds and selective enumeration. IEEE Trans. Comput.-Aided
Design Integrated Circuits Systems 22(9) 1243–1260.
Ball, M. O., C. J. Colbourn, J. S. Provan. 1995. Network reliability. M. O.
Ball, T. L. Magnanti, C. L. Monma, G. L. Nemhauser, eds. Network
Models. Handbooks in Operations Research and Management Science, Vol. 7. Elsevier Science, Amsterdam, 673–762.
Doan and Natarajan: Multivariate Marginal Bounds
Operations Research 60(1), pp. 138–149, © 2012 INFORMS
Bean, J. C. 1984. A Lagrangian algorithm for the multiple choice integer
program. Oper. Res. 32(5) 1185–1193.
Bertsimas, D., K. Natarajan, C. P. Teo. 2004. Probabilistic combinatorial
optimization: Moments, semidefinite programming and asymptotic
bounds. SIAM J. Optim. 15(1) 185–209.
Bertsimas, D., K. Natarajan, C. P. Teo. 2006. Persistence in discete
optimization under data uncertainty. Math. Programming 108(2–3)
251–274.
Birge, J. R., M. J. Maddox. 1995. Bounds on expected project tardiness.
Oper. Res. 43(5) 838–850.
Blaauw, D. T., K. Chopra, A. Srivastava, L. Scheffer. 2008. Statistical
timing analysis: From basic principles to state of the art. IEEE Trans.
Comput.-Aided Design Integrated Circuits Systems 27(4) 589–607.
Dodin, B. 1985. Bounding the project completion time distribution in
PERT networks. Oper. Res. 33(4) 862–881.
Dudzinski, K., S. Walukiewicz. 1987. Exact methods for the knapsack
problem and its generalizations. Eur. J. Oper. Res. 28(1) 3–21.
Embrechts, P., G. Pucetti. 2006. Bounds for functions of multivariate risks.
J. Multivariate Anal. 97(2) 526–547.
Fulkerson, D. R. 1962. Expected critical path length in PERT networks.
Oper. Res. 10(6) 808–817.
Genest, C., J. J. Quesada Molina, J. A. Rodríguez Lallena. 1995. De
l’impossibilité de construire des lois à marges multidimensionnelles
données à partir de copules. Comptes rendus de l’Académie des sciences de Paris 320(6) 723–726.
Grötschel, M., L. Lovász, A. Schrijver. 1988. Geometric Algorithms and
Combinatorial Optimization, Vol. 2. Springer-Verlag, Berlin.
Hagstrom, J. N. 1988. Computational complexity of PERT problems.
Networks 18(2) 139–147.
Kirkpatrick, T., N. Clark. 1966. PERT as an aid to logic design. IBM J.
Res. Development 10(2) 135–141.
Kleindorfer, G. B. 1971. Bounding distributions for a stochastic acyclic
network. Oper. Res. 19(7) 1586–1601.
Klein Hanevel, W. K. 1986. Robustness against dependence in PERT:
An application of duality and distributions with known marginals.
Math. Programming Stud. 27 153–182.
Lai, T. L., H. Robbins. 1976. Maximally dependent random variables.
Proc. Natl. Acad. Sci. USA 73(2) 286–288.
Li, H., M. Scarsini, M. Shaked. 1996a. Bounds for the distribution of a
multivariate sum. L. Rüschendorf, B. Schweizer, M. D. Taylor, eds.
Distributions with Fixed Marginals and Related Topics. IMS Lecture
Notes, Vol. 28. Institute of Mathematical Statistics, Hayward, CA,
198–212.
Li, H., M. Scarsini, M. Shaked. 1996b. Linkages: A tool for the construction of multivariate distributions with given nonoverlapping multivariate marginals. J. Multivariate Anal. 56(1) 20–41.
McNeil, A. J., R. Frey, P. Embrechts. 2005. Quantitative Risk Management: Concepts, Techniques and Tools. Princeton Series in Finance.
Princeton University Press, Princeton, NJ.
Meilijson, I. 1991. Sharp bounds on the largest of some linear combinations of random variables with given marginal distributions. Probab.
Engrg. Informational Sci. 5(1) 1–14.
149
Meilijson, I., A. Nadas. 1979. Convex majorization with an application to
the length of critical path. J. Appl. Probab. 16(3) 671–677.
Möhring, R. H. 2001. Scheduling under uncertainty: Bounding the
makespan distribution. H. Alt, ed. Computational Discrete Mathematics: Advanced Lectures, Lecture Notes in Computer Science,
Vol. 2122. Springer, Berlin, 79–97.
Möhring, R. H., F. J. Radermacher. 1989. The order-theoretic approach
to scheduling: The stochastic case. R. Slowiński, J. Wegarlz,
eds. Advances in Project Scheduling. IMS Lecture Notes. Elsevier,
Amsterdam, 479–531.
Nadas, A. 1979. Probabilistic PERT. IBM J. Res. Development 23(3)
339–347.
Natarajan, K., M. Song, C.-P. Teo. 2009. Persistency model and its applications in choice modeling. Management Sci. 55(3) 453–469.
Provan, J. S., M. O. Ball. 1983. The complexity of counting cuts and of
computing the probability that a graph is connected. SIAM J. Comput.
12(4) 777–788.
Ringer, L. J. 1971. A statistical theory for PERT in which completion times of activities are inter-dependent. Management Sci. 17(11)
717–723.
Rüschendorf, L. 1983. Solution of statistical optimization problem by
rearrangement methods. Metrika 30 55–61.
Rüschendorf, L. 2004. Comparison of multivariate risks and positive
dependence. J. Appl. Probab. 41(2) 391–406.
Sapatnekar, S. 2004. Timing. Kluwer Academic Publishers, Boston.
Scarsini, M. 1989. Copulae of probability measures on product spaces.
J. Multivariate Anal. 31(2) 201–219.
Shogan, A. W. 1983. Bounding distributions for a stochastic PERT network. Networks 12(4) 777–788.
Sklar, A. 1959. Fonctions de répartition à n dimensions et leurs marges.
Publications de l’Institut de Statistique de l’Université de Paris 8
229–231.
Spelde, H. G. 1976. Stochastische Netzpläne und ihre Anwendungim Baubetrieb. Ph.D. thesis, Rheinisch-Westfälische Technische
Hochschule, Aachen, Germany.
Valiant, L. G. 1979. The complexity of enumeration and reliability problems. SIAM J. Comput. 8(3) 410–421.
van Dorp, J. R., M. R. Duffey. 1999. Statistical dependence in risk analysis for project networks using Monte Carlo methods. Internat. J.
Production Econom. 58(1) 17–29.
Visweswariah, C., K. Ravindran, K. Kalafala, S. G. Walker, S. Narayan,
D. K. Beece, J. Piaget, N. Venkateswaran, J. G. Hemmett. 2006.
First-order incremental block-based statistical timing analysis. IEEE
Trans. Comput.-Aided Design Integrated Circuits Systems 25(10)
2170–2179.
Weiss, G. 1986. Stochastic bounds on distributions of optimal value functions with applications to PERT, network flows and reliability. Oper.
Res. 34(4) 595–605.
Wollmer, R. D. 1985. Critical path planning under uncertainty. Math. Programming Stud. 25 164–171.