Download New formulation of minimum-bias central composite

Research Paper
Struct Multidisc Optim 28, 231–242 (2004)
DOI 10.1007/s00158-004-0433-9
New formulation of minimum-bias central composite
experimental design and Gauss quadrature
X. Qu, G. Venter and R.T. Haftka
Abstract Response surface methods provide a powerful
tool for constructing approximations to complex response
functions. Statistical design of experiments is usually
used to select optimal points that minimize the error in
the resulting response surface approximation. Traditionally, data points are selected using minimum-variance
designs, for example the D-optimal design, which may
result in large bias errors for low-order approximation.
Minimum-bias criteria have been developed for selecting data points to minimize the bias error of a response
surface approximation. The present work developed
a minimum-bias counterpart to the popular minimumvariance central composite designs. In addition, a new
formulation of the minimum-bias design that assigns
unequal weights to the design points, based on Gauss
quadrature, is explored. Example problems are evaluated
and the results obtained from D-optimal, the traditional
minimum-bias, and the new Gauss-quadrature-based
minimum-bias designs are compared. It is shown that the
Gauss-quadrature-based minimum-bias design criterion
results in the most accurate approximations and provides
analytical solutions to a wider range of approximation
domains than the traditional minimum-bias design. Response surface approximations based on minimum-bias
central composite designs are more accurate than those
constructed from traditional central composite design.
Moreover, it is shown that using weights in regression
has little influence on the accuracy of the response surReceived: 28 July 2003
Revised manuscript received: 16 April 2004
Published online: 27 July 2004
 Springer-Verlag 2004
X. Qu1, u , G. Venter2 and R.T. Haftka1
1
Dept. of Mechanical and Aerospace Engineering, University
of Florida, Gainesville, FL 32611-6250, USA
e-mail: [email protected], [email protected]
2
Vanderplaats Research and Development, Inc, 1767 S 8th
Street, Suite 200, Colorado Springs, CO 80906, USA
e-mail: [email protected]
Preliminary version of the paper presented at the Fifth
World Congress on Structural and Multidisciplinary Optimization, Venice, Italy, May 19–23, 2003
face approximation in Gauss-quadrature minimum-bias
designs.
Key words central composite design, Gauss quadrature,
minimum-bias design of experiment, response surface approximation
1
Introduction
Response surface methods are used to construct simple approximations to the response of complex systems.
These response surface approximations are used in a large
number of problem areas, including engineering optimization and reliability evaluations (Kaufman et al. 1996;
Venter et al. 1997; Qu et al. 2003). Response surface approximations are generally smooth, low-order polynomials and have the desirable property of eliminating numerical noise, which is inherent in most engineering computer
simulations (e.g. Giunta et al. 1994; Venter et al. 1997).
The designer is also presented with a global view of the
response over the design space (e.g. Mistree et al. 1994),
and the process of integrating several design codes, as is
typically required in the multidisciplinary optimization
process, is greatly simplified (Kaufman et al. 1996; Mason
et al. 1998).
Statistical design of experiments leads to a more efficient construction of the approximations by identifying
a set of points at which to evaluate the response function, selected to minimize the effect of variance (noise)
on the resulting approximation. For regularly shaped approximation domains, there are a number of standard
designs (i.e. sets of data points) available, for example
the central composite (Box and Wilson 1951) and the
Box–Behnken (Box and Behnken 1960) designs. For an
irregularly shaped approximation domain, a computergenerated design must be used, such as the most widely
used D-optimal design (e.g. Myers and Montgomery 1995,
pp. 364–366). The D-optimal design minimizes a measure of the variance associated with the estimates of the
coefficients of the response surface model. Variance-based
232
criteria assume that all errors associated with a response
model are random errors and do not address bias or modelling error due to an insufficient response model.
However, the exact functional form of the response
function to be approximated is rarely known, and loworder polynomials are generally used as response models,
often leading to large bias errors. For regularly shaped approximation domains (cuboidal or spherical regions), fullfactorial minimum-bias designs (Myers and Montgomery
1995, pp. 406–421; Box and Draper 1987) are available.
Balabanov et al. (1996) successfully applied these designs to find a minimum-bias design for a 25-dimensional
sphere. For an irregularly shaped approximation domain,
Venter and Haftka (1997) used a genetic algorithm to select a specified number of design points from a candidate
set of points (similar to the D-optimal design criterion).
The genetic algorithm selects the data points that best
satisfy the minimum-bias conditions. These minimumbias design of experiments are based on minimizing averaged bias errors. Papila et al. (2004) developed a design of experiment to minimize the maximum bias error
using both point-independent and point-dependent error
bounds. Polynomial examples of two variables were provided to demonstrate the methodology.
The first objective of the present work is to explore
a new minimum-bias formulation based on Gauss quadrature (e.g. Nakamura 1991; Burden and Faires 1989) where
unequal weights are assigned to the design points in order
to improve the accuracy of the resulting response model.
Gauss quadrature optimizes both the location and the
weights associated with the integration points to maximize the order of the polynomial that can be integrated
exactly. The Gauss-quadrature integration points and associated weights are a natural choice for minimum-bias
designs, since the conditions that minimize the bias error
may be written in a form that is equivalent to finding
a numerical integration scheme capable of exactly integrating a polynomial of predefined order.
Central composite design (CCD, e.g. Myers and
Montgomery 1995, p. 55) is a popular statistical design
of experiment intended to minimize variance errors. CCD
usually requires fewer points than the Gauss-quadrature
minimum-bias design. Motivated by the use of weights
in Gauss-quadrature minimum-bias design, the second
objective of this work is to develop the minimum-bias
equivalent of CCD, with and without the use of weights
associated with design points. The effects of using weights
in constructing designs of experiments and response surface approximations on the accuracy of the resulting
response surface approximation are also investigated.
Section 2 presents minimum-bias design of experiments with weights. Gauss-quadrature-based minimumbias formulation is proposed in Sect. 3, and applied to
a tubular column example problem in Sect. 4. Section 5
studies the effects of weights in Gauss-quadrature-based
minimum-bias design. Section 6 develops minimum-bias
central composite design and investigated the effects of
weights in constructing design of experiments and regres-
sion. The advantages of minimum-bias central composite
design are demonstrated with a beam optimization example in Sect. 7.
2
Minimum-bias design with weights
2.1
Regression with weights
Response surface methods are used to construct an approximate relationship between a dependent variable f
(the response) and a vector x of n independent variables
(the predictor variables). The response is generally evaluated experimentally (these experiments may be numerical in nature), in which case f denotes the mean or expected response value. It is assumed that the true model
of the response may be written as a linear combination of
β . The
given functions z̃ with some unknown coefficients β̃
experimentally obtained response values y differ from the
expected value f due to random experimental error δ as
follows:
β +δ
y(x) = f (x) + δ = z̃(x)T β̃
(1)
Since the exact dependence of f on x is generally unknown, a response model is used to approximate f (x) as
follows:
y(x) = z(x)T b + ε
(2)
In (2), z(x) contains the assumed functions in the response model and the coefficients of the response model
are b, estimated from experimentally obtained response
values, a process known as regression. Furthermore, ε denotes the error, which is the difference between the predicted and measured response values and includes both
random (variance) and modelling (bias) error. For N design points, (2) may be written in matrix form as:
y = ZB +εε
(3)
The resulting response model
ŷ(x) = z(x) T b
(4)
where ŷ denotes the predicted response values obtained
from the response model.
The unknown coefficients may be estimated by
a weighted least-squares procedure that minimizes the
L2 -norm of the model error, defined as (e.g. Strang 1986)
ε2 = (y − Zb)T WT W(y − Zb)
(5)
where N denotes the number of experimental data points
used in constructing the response model, wi denotes
the weight associated with each design point, and W is
233
a diagonal matrix. The approximate values of the unknown coefficients that minimize ε2 may then be solved
by setting the derivative of ε2 to zero, to obtain
−1
T
T
b = (WZ) WZ
(WZ) Wy
(6)
2.2
Bias and minimum-bias design of experiments
For many engineering applications of response surface approximations, the data are from computer simulations.
The random error in the data is small, but there can be
substantial modelling errors. Model bias occurs when an
insufficient response model is used to approximate the response of a system. Assuming that the true (or exact)
model is a polynomial of higher order than the response
model, the true model may be written as
f (x) = E(y(x)) = z1 (x)T β 1 + z2 (x)T β 2
(7)
where E denotes the expected or mean value, z1 contains the monomials of the true model that is contained
in the response model, while z2 contains the higher-order
monomials of the true model that are not included in the
response model. Furthermore, β 1 and β 2 are the exact,
but unknown, coefficients of the true model associated
with z1 and z2 , respectively. Using the same notation, the
corresponding response model may then be written as
ŷ(x) = z1 (x)T b1
The MSE depends on the design points considered over
the approximation domain, R, and the average mean
squared error (AMSE)
AMSE =
NK
σ2
2
E [ŷ(x) − E ŷ(x)] dx +
R
APV
2
[E ŷ(x) − f (x)] dx , where K = R
1
dx
(11)
R
ASB
and N/σ2 (number of observations divided by the variance of the model) is a mere scale factor. As shown
in (11), the AMSE has two components, namely the average prediction variance (APV) and the average square
bias (ASB). Only the ASB term is considered in the
derivation of the minimum-bias conditions. Using the
above defined quantities, the ASB of (11) becomes
NK
ASB = 2
σ
T T
β 2 −2β
β T2 z2 zT1 Aβ
β2 +
β 2 A z1 zT1 Aβ
R
β T2 z2 zT2 β 2 dx
(12)
At this point, it is convenient to introduce some additional notation as follows:
(8)
Moment matrices, M11 and M12 :
where b1 is a biased estimator of β 1 ( a biased estimator
is one that has a mean value not equal to the estimated
quantity), with its expected or mean value given by
β 2 , where A =
E(b1 ) = β 1 + Aβ
−1
(WZ1 )T WZ2
(WZ1 )T WZ1
M11 =
(WZ1 )T WZ1
N
wi
i=1
2
MSE = E [ŷ(xi ) − f (xi )] =
M11 =
ZT1 Z1
N
(13)
i=1
2
ZT1 Z2
N
(14)
z1 zT1 dx
µ 11 = K
R
z2 zT2 dx
µ 22 = K
E [ŷ(xi ) − E ŷ(xi )] + [E ŷ(xi ) − f (xi )]
M12 =
Region moment matrices µ 11 , µ 22 and µ 12 :
2
SquaredBias
(WZ1 )T WZ2
N
wi
For the traditional minimum-bias conditions where all
design points have an equal weight of one, the moment
matrices of (13) reduce to:
E {[ŷ(xi ) − E ŷ(xi )] + [E ŷ(xi ) − f (xi )]} =
2
M12 =
(9)
and Z1 and Z2 are the matrices corresponding to z1
and z2 .
Both Myers and Montgomery (1995, pp. 406–421) and
Box and Draper (1987) show the traditional derivation
of the conditions that minimize the average squared bias
(ASB) of a response model when all design points have
an equal weight of one (without weights). Following the
derivation of Myers and Montgomery, the more general
derivation where unequal weights are assumed is presented here. The derivation is based on the fact that the
variance and bias error of a response model at a specific
data point, say xi , may be combined to obtain the mean
squared error (MSE) as follows
Variance
NK
σ2
R
(10)
z1 zT2 dx
µ 12 = K
R
(15)
234
The vector α 2 :
α2 =
√
β 2 /σ
Nβ
(16)
Making use of this new notation and some algebraic manipulation, (12) may be written as
µT12µ −1
ASB = α T2 µ 22 −µ
11 µ 12 +
−1
T
µ−1
µ−1
M11 M12 −µ
µ 11 M−1
α2
11 µ 12
11 M12 −µ
11 µ 12
(17)
where µ 11 is positive definite. A sufficient condition for
a minimum-bias design is then to have:
M11 = µ 11
M12 = µ 12
(18)
Note that in order to evaluate (18), the true model must
be known; however, this is seldom the case, and we assume that the true model is a polynomial of higher order
than the response model. Denoting the order of the true
model by d2 , we say that the response model is protected
against a true model of order d2 . It is usually assumed
that the true model is a polynomial one order higher that
the response model.
3
Gauss-quadrature-based minimum-bias formulation
Equation (18) essentially means that minimum bias is
achieved when the integrated moments are equal to the
corresponding sums. Gauss quadrature appears to offer a way to meet these conditions more easily by using
weights. That is, for the minimum-bias design, we can
vary not only the location of the data points but also
the weights associated with them. The Gauss-quadrature
formulation of the minimum-bias design is based on writing the minimum-bias conditions of (18) in the following
equivalent form:
N
Θ dx
wiΘ i
R
=
dx
R
i=1
N
(19)
wi
i=1
where Θ denotes all monomials up to degree (d1 + d2 )
with d1 denoting the order of the response surface model
and d2 denoting the order of the true model. For example,
if the response surface is linear and the true model is
quadratic, then Θ denotes all monomials up to cubic.
In order to satisfy the minimum-bias conditions of
(18), (19) must hold for all monomials Θ . This is equivalent to finding a numerical integration scheme with N integration points that is capable of exactly integrating any
polynomial up to the degree (d1 + d2 ). In the traditional
minimum-bias approach, all weights are assumed equal
and only the location of the integration points may be optimized. However, when positions and weights associated
with the integration points are optimized, we can obtain
an integration scheme capable of integrating higher-order
polynomials exactly. In particular, Gauss quadrature can
integrate the highest possible order polynomial using n
data points.
Gauss quadrature is discussed in detail in a number
of numerical analysis textbooks, for example Nakamura
(1991) and Burden and Faires (1989). One-dimensional
Gauss quadrature with N integration points may be written as
+1
N
f (x) dx ≈
wi f (xi )
−1
(20)
i=1
where xi denotes the ith integration point and wi denotes the weight associated with that point. The Gauss
quadrature of (20) can integrate exactly polynomials up
to degree (2N − 1). For one dimensional problem, it may
be shown that Gauss quadrature of N points as a design
of experiment protects response surface approximation
against true models of the order of 2N − 1 − d1 . For higher
dimension, the number of Gauss-quadrature points required depends on the dimension of the problem and the
number of points along each dimension. Gauss quadrature is very popular in numerical analysis procedures
and is, for example, an integral part of the finite element method (e.g. Cook et al. 1989, pp. 170–173 and pp.
183–185). Due to the popularity of Gauss quadrature,
integration points and associated weights are tabulated
in a number of books (e.g. Stroud and Secrest 1966).
Gauss quadrature also has the advantage that any two or
more integration domains for which a Gauss quadrature
exists may be combined to obtain higher-order and/or
more complicated integration domains. This means that
closed-form solutions are available to a much wider range
of approximation domains as compared to the traditional
minimum-bias design.
4
Tubular column example problem
The first example problem approximates the buckling
load safety margin of a simply supported tubular column, and is an extension of the problem considered by
Venter and Haftka (1997). A quadratic response model
was constructed using a D-optimal minimum-variance design (D-Opt), the genetic-algorithm-based equal-weight
minimum-bias design of Venter and Haftka (1997) (EQMinBias), a Gauss-quadrature minimum-bias design with
weights (GQ-MinBias), and the same Gauss quadrature
minimum-bias design without weights (GQ-NoWeight).
The example problem is shown schematically in Fig. 1.
The corresponding buckling load safety margin, which is
defined as the difference between the critical Euler buck-
235
Fig. 1 Schematic representation of tubular column and optimal design points for the tubular column example
ling load and the applied load, σc , expressed as a stress, is
presented in (21).
σC (D, T ) =
π2 E 2
P
(D + T 2 ) −
8l2
πDT
(21)
In (21), E is Young’s modulus of the material, P is the applied load, and l, D, and T are the length, diameter, and
wall thickness of the column. In the present work, the approximation domain is defined in (22), which represents
a triangular region, as shown in Fig. 1.
D ≤ 5.0 [in] T ≥ 0.05 [in]
D
≥ 10
T
(22)
The buckling load safety margin was approximated
using a quadratic (six-parameter) response model with
twelve design points for estimating the unknown coefficients of the model. As in Venter and Haftka (1997),
a set of 231 evenly spaced data points were considered as candidate points for the D-optimal- and the
genetic-algorithm-based minimum-bias designs. For the
two Gauss-quadrature-based minimum-bias designs, the
same set of Gauss-quadrature integration points defined
for a triangular region (e.g. Cook et al. 1989, p. 184)
were used as design points. For a triangular integration
domain, the twelve-point Gauss quadrature integrates
a sixth-order polynomial exactly. The resulting Gaussquadrature-based minimum-bias design with weights
thus protects the quadratic response model against
a sixth-order true model. The corresponding genetic algorithm minimum-bias design protects the quadratic response model against a third-order true model.
The optimal data points obtained from the three design criteria are shown graphically in Fig. 1. The coordinates and associated weights of the Gauss-quadrature-
Table 1 Optimal data points (Gauss-quadrature minimum-bias design, equal-weight minimum-bias design, and D-optimal design)
D
1.0678
4.7161
4.7161
2.7436
3.8782
3.8782
2.1357
2.1357
3.6034
4.7608
3.6034
4.7608
Gauss quadrature
T
Weight
0.0784
0.0784
0.4432
0.1622
0.1622
0.2756
0.0739
0.1897
0.0739
0.1897
0.3364
0.3364
0.050845
0.050845
0.050845
0.116786
0.116786
0.116786
0.082851
0.082851
0.082851
0.082851
0.082851
0.082851
EW-MinBias
D
T
1.400
1.850
2.300
3.200
3.200
3.650
4.100
4.100
4.325
4.325
4.775
4.775
0.095
0.073
0.163
0.118
0.253
0.298
0.185
0.365
0.073
0.208
0.163
0.410
D-Opt
D
T
0.500
0.500
2.750
2.750
2.750
2.750
5.000
5.000
5.000
5.000
5.000
5.000
0.050
0.050
0.050
0.050
0.275
0.275
0.050
0.050
0.275
0.275
0.500
0.500
236
Table 2 Response models: the design for the EW-MinBias was obtained by genetic optimization of the bias error (Venter and
Haftka 1997) while the GQ-NoWeight design used the Gauss quadrature points in Table 1
Parameter
Intercept
D
T
D2
T2
DT
D-Opt
−85 501.72
38 618.08
80 287.87
−3396.26
−45 066.00
−8418.12
EW-MinBias
GQ-MinBias
GQ-NoWeight
−24 925.67
6709.62
65 240.90
584.85
−73 805.03
−4069.21
−31 491.98
10 317.55
72 853.69
146.79
−56 968.98
−7635.65
−32 781.23
11 088.47
71 224.10
48.03
−51 157.99
−7700.455
based minimum-bias design points obtained from Cook
et al., p. 184 (1989), and the equal-weight minimum-bias
design points obtained from the genetic algorithm and
D-optimal design points given by Venter and Haftka
(1997) are presented in Table 1. The response models
obtained from the data points shown in Table 1 and
Fig. 1 are summarized in Table 2. A response model based
on Gauss-quadrature minimum-bias design but without
using weights in regression is also constructed in order to
investigate the effects of weights on the accuracy of the
constructed approximation.
The predictive capabilities of the respective models
were evaluated based on the independent list of 231
evenly spaced candidate points. The average, RMS (root
mean square), and maximum values of the errors, as well
as the G-efficiency (see Appendix) were calculated and
the results are summarized in Table 3.
The models obtained from the minimum-bias designs
are more accurate (except for the maximum error) than
the corresponding models obtained from the D-optimal
criterion. Both Gauss-quadrature-based minimum-bias
designs outperformed the more traditional, geneticalgorithm-based minimum-bias design except that the
later is slightly better in terms of the averaged error.
Apart from being more accurate, the Gauss-quadraturebased designs were obtained from an analytical solution,
while the traditional designs were obtained by making
use of a computationally intensive genetic algorithm. The
G-efficiency is a measure of the maximum prediction variance of a response model and as such is a variance-type
criterion of the predictive capabilities of a model. As expected, the Geff of the D-optimal design is better than
that of the corresponding minimum-bias designs. Note
that the Gauss-quadrature-based minimum-bias designs
resulted in significantly higher Geff values that the corresTable 3 Predictive capabilities of the quadratic response
models
Design
D-Opt
EW-MinBias
GQ-MinBias
GQ-NoWeight
%AveErr
34.14
8.67
9.23
9.47
%RMSE %MaxErr
49.79
25.82
22.88
22.37
183.28
284.67
255.90
249.58
Geff
100.0%
11.5%
22.2%
23.65%
ponding traditional minimum-bias design. This indicates
that the Gauss-quadrature design is more efficient in reducing the variance error of a model, because the Gaussquadrature-based design selects design points closer to
the perimeter of the design space. An unexpected result
is that, even without using weights, the Gauss points
still perform comparably to that of the Gauss points
with weights. There is almost no difference in the performance of the Gauss-quadrature-based minimum-bias
design with weights and that without weights.
5
Effects of weights in Gauss-quadrature-based
minimum-bias design
The minimal effects of weight on the accuracy of the
approximations prompted us to study a simple quartic function y = x4 . Employing a four-point GQ design,
two response surfaces were built to approximate y = x4
using Gauss-quadrature minimum-bias design with and
without weights. The two response surfaces were exactly
the same, thus of the same accuracy. For a five-point
GQ design, (xi , wi ) = (0, 0.5689), (±0.5385, 0.4786) and
(±0.9062, 0.2369), the two response surfaces are no longer
the same, as shown in Table 4, but the difference in accuracy is minimal. It is worth noting that the RMSE predictor for the case with weights is based on the weight matrix
being the matrix of covariance of the error. Since we are
using weights from minimum-bias design, the RMSE predictor is no longer meaningful for the case with weights.
The true root mean square error (TRMSE) of the response surfaces is calculated exactly via analytical integration with MATLAB.
Weights associated with points in the design of experiments have two contributions in response surface approximations. One is to be used in regression analysis to solve
for the unknown coefficients in the response surface approximation. The results of the tubular and polynomial
examples indicate that using the weights of the Gaussquadrature minimum-bias design contributes little to improving the accuracy of the response surface approximation. The second contribution of weights is that weights
change the location of points in the resulting design of experiments, which is investigated in the next section.
237
Table 4 Comparison of statistics of quadratic response surfaces built from five-point GQ minimum-bias design with and without
using weights for test function x4
GQ Min-bias
GQ Min-bias without weights
Test function
−0.1016 + 0.9111x2
0.8956
0.3034
0.0779
0.4
y = x4
N/A
0.2
0
0.4
RS
−0.08572 + 0.85714x2
2
R Adj.
0.8367
Mean of response
0.3034
TRMSE (exact)
0.0762
Integrated x4 over [−1, 1]
0.4
N
6
Minimum-bias CCD
i=1
While the finite-element literature provides useful Gaussquadrature data for use as experimental designs, there is
no minimum-bias equivalent for the central composite design (CCD, e.g. Myers and Montgomery 1995, p. 55). The
CCD is a popular minimum-variance design that is useful for low dimensions. The number of points in CCD is
determined by (2n + 2n + 1), where n is the dimensionality of the problem (n = 3 to 6 is studied here). To protect quadratic polynomials against quartic polynomials,
Gauss-quadrature minimum-bias design needs 3n points,
which is more than that required by CCD. We decided
to develop a minimum-bias CCD to provide an attractive alternative to the GQ-based minimum-bias design
presented here. Minimum-bias CCD with and without
weights are developed for three to six dimensions, and the
contribution of the weights in constructing the design of
experiments is also investigated.
For minimum-bias CCD with weights, there are only
five design variables: the weight of the center point, the
weight and position of the vertices, and the weight and
position of the axial points. In n dimensional space, if
a quadratic response surface is employed and the true
function is a quartic, according to the minimum-bias design condition shown by (18), the following seven equations need to be satisfied by the coordinates xi and the
weights wi of the N points in the CCD
N
wi = 1 , N = 2n + 2n + 1 ;
i=1
N
i=1
N
i=1
1
wi ;
5 i=1
N
wi x4i =
N
1
wi ;
3 i=1
N
wi x2i =
1
wi ;
7 i=1
N
wi x6i =
i=1
N
1
wi ;
9 i=1
N
wi x2i u2i =
1 wi ;
27 i=1
i=1
−1 ≤ xi , ui , vi ≤ 1, 0 ≤ wi
(23)
where xi , ui , and vi are any three coordinates of the
points. The weights and coordinates of the points in the
above equations are substituted according to the position
of the points (center, vertices and axial points). Because
there are more equations than unknowns, the least-square
solver in MATLAB was used to solve the seven equations of (23). The solutions for three to six dimensions are
shown in Table 5. For comparison, CCDs without weight
(equal weights) were also obtained and shown in Table 6.
It is seen that while the L2 norm of the residual of (23) is
much higher without weights, the difference in positions
is small. It can be seen that almost all the designs have
points on the faces of the box, but the vertex points are
much closer to the center than in the standard CCD.
Using the minimum-bias CCD in 6-D, a quadratic
response surface is constructed to approximate a full
quartic polynomial with all coefficients equal to one.
To investigate the efficiency of minimum-bias CCD, two
other quadratic response surfaces were fitted to the test
function using a regular CCD and a face center central composite design (FCCCD, e.g. Myers and Montgomery, 1995, p. 313). Since the RMSE predictor is no
longer meaningful for the case with weights, the RMSE of
the response surfaces are compared by using test points.
The true root mean square error (TRMSE) is calculated
exactly via analytical integration with MATLAB, and
also approximated using 46 656 structured test points
that were chosen as a uniform 66 grid, and 46 656 ran-
n
XAxial
XVertex
WeightAxial
WeightCenter
WeightVertex
L2 Norm∗
3
4
5
6
1
1
1
1
0.6781
0.6781
0.6781
0.6934
0.04512
0.04512
0.04512
0.04119
0.2016
0.1113
0.02106
0.0
0.06591
0.03296
0.01648
0.007893
0.02109
0.02109
0.02109
0.02429
Residuals of (23)
i=1
1 wi ;
15 i=1
N
wi x2i u4i =
N
wi x2i u2i vi2 =
Table 5 Numerical solutions to minimum-bias CCD of 3, 4, 5 and 6 dimensions
∗
N
238
Table 6 Numerical solutions to minimum-bias CCD without
weights of 3, 4, 5 and 6 dimensions
n
XAxial
XVertex
L2 Norm∗
3
4
5
6
0.9313
1.0
1.0
1.0
0.6634
0.6488
0.6462
0.6390
0.4726
0.7939
2.6194
6.5138
∗
Residuals of (23)
domly generated points uniformly distributed in the design space.
The accuracy of the four response surfaces is compared in Table 7 for a 6-D quartic polynomial. It is seen
from the TRMSE that the accuracy of the response surface approximation based on minimum-bias CCD is the
best, and the difference between the designs with and
without weights is minimal. FCCCD is much better than
the regular CCD. The difference between the performance of regular CCD and FCCCD is due to the location
of the axial points. For a regular CCD, the axial points
are set at 2.8284 on the axis for a rotatable orthogonal
CCD, which will induce large errors with the presence of
bias errors. It is seen that 46 656 random test points provide close estimation to the TRMSE, while the estimation
from structured 46 656 points as 66 grid is less close to the
TRMSE. This reflects the fact that, of the 46 656 points,
42 560 (66 − 46 ) points are on the boundary of the domain. If we were using these points to estimate the RMSE
by integration, at least trapezoidal integration should be
used. It is seen in Table 7 that the use of the trapezoidal
rule improves the RMS error estimate substantially by
drastically reducing the weight of the boundary points
compared to the interior points. This demonstrates that,
when bias errors dominate random errors, designs of experiments and error estimation that arrange points inside
the domain generally provide better results than those
putting points on the boundary of the domain.
Numerical response is usually noisy, thus it is necessary to investigate the performance of minimum-bias
CCD for response with noise. We compare the four designs for the quartic polynomial with all coefficients one
and normally distributed noise that has a mean value of
0 and standard deviation of 10 percent of the average of
the test function over the unit cube, Normal (0, 0.5867).
The accuracy of the four response surface approximations
is compared in Table 8. It is seen that the accuracy of the
response surface approximations based on minimum-bias
CCD with or without weights is much better than that of
the response surface approximations based on CCD and
FCCCD. FCCCD is still better than the regular CCD.
It is seen from Table 8 that a smaller L2 Norm indicates better RS accuracy. But, if the L2 Norm is relatively small, such as those of minimum-bias CCD with
and without the use of weights, the difference in the accuracy of the RS is very small, which means that reducing
the L2 Norm beyond a certain level does not improve RS
accuracy significantly. For the cases where the L2 Norm is
large, such as in FCCCD and CCD, the errors of RS based
on the two designs are very large.
Table 7 Comparison of statistics of quadratic response surfaces for a six-variable full quartic polynomial with all coefficients
equal to one
Error Statistics
Minimum-bias CCD
with weights
Minimum-bias CCD
without weights
Regular CCD
FCCCD
0.9263
7.7445
1.8238
1.8557
3.6842
2.2248
0.9482
6.2581
1.8329
1.8497
4.3216
2.4543
0.8667
34.6623
32.5793
32.5255
25.9872
31.4441
0.9475
23.7533
7.4293
7.4718
7.5856
7.3776
Rsquare Adj.
Mean of response∗
TRMSE (exact)
RMSE (46 656 random points)
RMSE (46 656 structured points)
RMSE (46 656 trapezoidal integration)
∗
The exact mean of a full quartic polynomial with all coefficients equal to one is 88/15 = 5.8667.
Table 8 Comparisons of statistics of quadratic response surfaces based on full quartic test function with all coefficients one and
10 percent noise
Error Statistics
Rsquare Adj.
Mean of response
TRMSE (exact)
L2 Norm of residual of (23)
Minimum-bias CCD
with weights
Minimum-bias CCD
without weights
Regular CCD
FCCCD
0.9341
5.7420
1.8716
0.02109
0.9492
6.2554
1.9317
6.514
0.8665
34.5295
32.5732
132.3
0.9470
23.7137
7.4208
1097
239
7
Beam optimization example
Response surface approximations based on engineering
simulation are commonly used for optimization in engineering applications. In order to demonstrate the effect of
approximation accuracy based on various CCD designs,
an example of a six-variable beam optimization problem
is solved here.
7.1
Problem description
The beam is modelled by three segments and is subjected
to a tip load as shown in Fig. 2. The optimization problem is to minimize the weight of the beam, subject to
constraints on maximum bending stress, maximum tip
displacement and geometry. The weight is minimized by
changing the height and width of each segment, resulting in six design variables. The material properties and
problem parameters are shown in Fig. 2.
VisualDOC (Vanderplaats, 2003) was used to create
the approximations and to perform the optimization. The
geometric constraints are fairly simple and are not approximated. Instead, they are represented with linked relationships in VisualDOC as hi − 20bi ≤ 0. The volume
of the beam is used as the objective function. The tip
displacement and maximum bending stress in each component is constrained as y − 2.5 cm ≤ 0 and σi − σ̄ ≤ 0,
respectively. Approximations for the objective function
(volume), tip displacement and maximum bending stress
component in each segment were created, for the minimum bias CCD (with and without weights) and the FCCCD designs. All designs used 77 data points to fit the 28
polynomial coefficients.
Fig. 2 The cantilevered beam
Table 9 Optimization results
b1 (cm)
b2 (cm)
b3 (cm)
h1 (cm)
h2 (cm)
h3 (cm)
Objective (cm3 )
Tip disp (cm)
Sigma 1 (N/cm2 )
Sigma 2 (N/cm2 )
Sigma 3 (N/cm2 )
∗
∗∗
Exact
FCCCD
MB_CCD
with weights∗
MB_CCD
without weights∗∗
2.9920
2.6138
2.0746
59.8408
52.2758
41.4913
66 960.18
2.2069
14 000.00
13 999.99
14 000.00
2.8613
2.5859
2.1222
57.2260
51.7184
42.4434
64 592.49
2.1933
13 997.83
14 020.06
14 022.19
3.0001
2.5971
2.0855
59.9400
51.9415
41.7101
66 966.93
2.1901
14 005.48
14 000.36
13 959.24
3.0103
2.5961
2.0819
59.9400
51.9222
41.6381
66 986.30
2.1968
14 025.74
14 017.91
13 997.92
Minimum-bias CCD with weights from Table 5
Minimum-bias CCD without weights from Table 6
240
Table 10 Exact values of the objective, the displacement and the stresses at optima in Table 9
Objective (cm3 )
Tip disp (cm)
Sigma 1 (N/cm2 )
Sigma 2 (N/cm2 )
Sigma 3 (N/cm2 )
Exact
FCCCD
MB_CCD
with weights
MB_CCD
without weights
66 960.18
2.2069
14 000.00
13 999.99
14 000.00
64 592.50
2.2016
16 008.16
14 457.53
13 078.47
66 966.93
2.2277
13 916.34
14 272.07
13 780.89
66 986.3
2.2352
13 869.23
14 287.97
13 852.51
7.2
Results
The results without and with approximation are shown in
Table 9. In order to obtain baseline results for comparing
the results obtained when using the approximations, the
problem was first directly optimized without using any
approximations.
Table 10 provides exact values of the objective, the
displacement and the stresses at the optima found by
each response surface approximation as a comparison of
the accuracy of the response surface approximations. Recall that Table 9 provides the optimum function values
obtained from the approximations. As a comparison of
the accuracy of the response surface approximations,
Table 10 provides the exact function evaluations at the
optimum points reported in Table 9. Comparing Tables 9
and 10, it is clear that the volume is exactly approximated
in each case. This is to be expected since the volume is
a quadratic function of the design variables. However, the
stress constraints are not exactly approximated, with the
FCCCD approach resulting in the least accurate stress
approximations. For example, the FCCCD approximations led to an optimum that had a stress violation of
12.6% for Sigma 1, resulting in a lower volume value as
compared to the other approaches. In contrast the two
minimum-bias designs resulted in much more accurate
stress approximations, with a maximum stress violation
of about 2% at the optimum. It is seen that the effect of
using weights is insignificant.
8
Concluding remarks
Minimum-bias-based design of experiments can effectively reduce the bias errors in response surface approximations. Minimum-bias design is traditionally performed with equal weights assigned to each design point.
A new formulation of minimum-bias design, based on
Gauss quadrature that utilizes weights as in numerical
quadrature, is presented. Based on the tabulated values
of the Gauss quadrature integration points and associated weights, the new formulation appears to provide the
designer with a more accurate, closed-form minimumbias design for a wider range of approximation domains
with much lower computational intensity than traditional
minimum-bias design. In all cases considered where the
bias or modelling error dominates, the response surface
approximations obtained from the minimum-bias-based
designs are more accurate than similar response models
based on the D-optimal criterion.
Central composite design is a popular minimumvariance design of experiment and requires smaller numbers of design points than the Gaussian-quadrature
minimum-bias design, thus is more affordable for computationally expensive problems. Minimum-bias central
composite design with and without the use of weights
in low dimensions (e.g. three to six) were developed to
provide the minimum-bias equivalent of the central composite design. Minimum-bias central composite designs
with and without the use of weights outperformed the
traditional central composite design in terms of the accuracy of response surface approximation. It is demonstrated that, when bias errors dominate random errors,
designs of experiments and error estimation that arrange
points inside the domain generally provide better results than those putting points on the boundary of the
domain.
Weights associated with points in the design of experiments have two contributions in response surface approximations. One is to be used in regression analysis to solve
for the unknown coefficients in the response surface approximation. The second contribution of weights is that
the weights change the location of the design of experiment points. The results of the tubular buckling and polynomial examples indicate that using weights of Gaussquadrature minimum-bias design in the regression analysis contributes little toward improving the accuracy of the
constructed response surface approximation. The beam
design and polynomial examples demonstrated that the
use of weights in the construction of minimum-bias central composite design has minimal influence on the accuracy of the resulted response surface approximation.
For response surface approximation in high dimensions (> 10), the number of points required by Gauss
quadrature or central composite design can be very high.
Minimum-bias design based on the principle outlined in
this paper can be developed for currently available design
of experiments for high dimensions, such as fractional factorial design, orthogonal arrays or Latin hypercube sampling (Qu et al. 2003).
241
Acknowledgements This work was supported in part by Grant
NAG-1-2177 from NASA Langley Research Center and NSF
Grant DM5-9979711.
References
Balabanov, V.; Kaufman, M.; Knill, D.L.; Haim, D.; Golovidov, O.; Giunta, A.A.; Grossman, B.; Mason, W.H.; Watson, L.T.; Haftka, R.T. 1996: Dependence of Optimal Structural Weight on Aerodynamic Shape for a High Speed Civil
Transport. Proc. of 6 th Symp. on Multidisc. Anal. Optim.
(held in Bellevue, WA, USA), 599–612, AIAA-1996-4046
Box, G.E.P.; Wilson, K.B. 1951: On the Experimental Attainment of Optimum Conditions. J Roy Stat Soc B, 1–45
Box, G.E.P.; Behnken, D.W. 1960: Some New Three-Level
Designs for the Study of Quantitative Variables. Technometrics 30, 1–40
Box, G.E.P.; Draper, N.R. 1987: Empirical Model-Building
and Response Surfaces. 437–442, New York: Wiley
Burden, R.L.; Faires, J.D. 1989: Numerical Analysis. 198–203,
4th edn. Boston: PWS-Kent
Cook, R.D.; Malkus, D.S.; Plesha, M.E. 1989: Concepts and
Applications of Finite Element Analysis. New York: Wiley
Giunta, A.A.; Dudley, J.M.; Narducci, R.; Grossman, B.;
Haftka, R.T.; Mason, W.H.; Watson, L.T. 1994: Noisy Aerodynamic Response and Smooth Approximations in HSCT
Design. Proc. of 5 th Symp. on Multidisc. Anal. Optim. (held
in Panama City, Florida, USA), 1117–1128, AIAA-1994-4376
Kaufman, M.; Balabanov, V.; Grossman, B.; Mason, W.H.;
Watson, L.T.; Haftka, R.T. 1996: Multidisciplinary Optimization via Response Surface Techniques. Proc. of 36 th Israel
Conf. on Aero. Sci. (held in Israel), A-57–A-67
Mason, B.H.; Haftka, R.T.; Johnson, E.R.; Farley, G.L. 1998:
Variable Complexity Design of Composite Fuselage Frames by
Response Surface Techniques. Thin Wall Struct 32, 235–261
Papila, M.; Haftka, R.T.; Watson, L. 2004: Bias Error Bounds
for Response Surface Approximations and Min-Max Bias Design. in preparation for AIAA J
Mistree, F.; Patel, B.; Vadde, S. 1994: On Modeling Objectives and Multilevel Decisions in Concurrent Design. Proc. of
20 th ASME Design Automation Conf. (held in Minneapolis,
Minnesota, USA), 151–161.
Strang, G. 1986: Introduction to Applied Mathematics.
137–154, Massachusetts: Wellesley-Cambridge
Stroud, A.D.; Secrest, D. 1966: Gaussian Quadrature Formulas. New Jersey: Prentice-Hall
Vanderplaats, G. 2003: VisualDOC 3.0 User Manual. Colorado: Vanderplaats Research & Development
Venter, G.; Haftka, R.T. 1997: Minimum-Bias Based Experimental Design for Constructing Response Surfaces in Structural Optimization. Proc. of 38th AIAA/ASME/ASCE/AHS/
ASC Struct., Struct. Dyn., Mater. Conf. (Held in Kissimmee,
Florida, USA), 1225–1238, AIAA-1997-1053
Venter, G.; Haftka, R.T.; Chirehdast, M. 1997: Response Surface Approximations for Fatigue Life Prediction. Proc. of
38th AIAA/ASME/ASCE/AHS/ASC Struct., Struct. Dyn.,
Mater. Conf. (Held in Kissimmee, Florida, USA), 1383–1396,
AIAA-1997-1331
Appendix:
Predictive capabilities
The percent average error (%AveErr), the percent root
mean square error (%RMSE) and the percent maximum
error (%MaxErr) calculated using m test points in the
present paper are defined as:
100 %AveErr =
|yi − ŷi|
ȳm i=1
m
m
100 1
%RMSE =
(yi − ŷi )2
ȳ
m i=1
%MaxErr =
100
Max [|yi − ŷi|]
i
ȳ
(A.1)
(A.2)
(A.3)
where:
1 |yi |
m i=1
m
ȳ =
(A.4)
The G-Efficiency (Geff ) is defined as (Myers and Montgomery 1995, p. 367)
Geff =
p
Max [v(xi )]
(A.5)
i
Myers, R.H.; Montgomery, D.C. 1995: Response Surface
Methodology-Process and Product Optimization Using Designed Experiments. New York: Wiley
where p denotes the number of parameters (monomials)
in the response model and v(xi ) denotes the scaled prediction variance. The scaled prediction variance, defined
as
Nakamura, S. 1991: Applied Numerical Methods with Software. 123–130, New Jersey: Prentice-Hall
v(xi ) =
Qu, X.; Venkataraman, S.; Haftka, R.T.; Johnson, T.F. 2003:
Deterministic and Reliability-based Optimization of Composite Laminates for Cryogenic Environments. AIAA J 41,
2029–2036
N Var(ŷi )
σ2
(A.6)
satisfies the property
Max [v(xi )] ≥ p
i
(A.7)
242
Also, σ2 denotes the variance of the error and Var(ŷi ) denotes the prediction variance of data point i. For the case
of unequal weights, the prediction variance may be derived from the formulation of the response model
−1 T
ŷi = z(xi ) T b b = ZT W̄Z
Z W̄y
(A.8)
as follows
coefficients b is the diagonal elements of the covariance
matrix, which is defined as:
C = Var(b) = (ZT W̄Z)−1 ZT W̄Var(y)W̄Z(ZT W̄Z)−1 =
−1
(ZT W̄Z)−1 ZT W̄Iσ2 W̄Z ZT W̄Z
=
T
−1 T
T
−1
2
σ Z W̄Z
Z W̄W̄Z Z W̄Z
(A.9)
where the variance σ2 of the model is constant and is approximated by (Myers and Montgomery, 1995, p. 673):
Var(ŷi ) = xTi V ar(b)xi
where:
W̄ = WW
The matrix Var (b) is known as the covariance matrix and
is denoted by C in the present work. The variance of the
m
σ̂2 =
(yi − ŷi)2
i=1
m−p
(A.10)
The prediction variance of data point i is then defined as:
−1 T
−1
Z W̄W̄Z ZT W̄Z
xi
Var(ŷi ) = σ2 xTi ZT W̄Z
(A.11)