Download Barizi.; (1973)An assessment and some aplications of the empirical Bayes approach to random regression models." Thesis.

AN ASSESSMENT AND SOME APPLICATIONS
O~M EM.PIRICAL BAYES APPROACH
.,., .IANroM REGRESSION M>rELS
by
BARIZI
Institute of Statistics
Mimeograph Series No. 85P,
January 1973
vi
TABLE OF CONTENTS
Page
LIST OF TABLES •
. •• vii
1.
INTRODUCTION..
1
2.
REVIEW OF LITERATURE
4
3.
BAYES AND EMPIRICAL BAYES ESTIMATION IN RANDOM REGRESSION
MODELS • • • • • • • • . • . • • • . . • • •
3.1 Bayes Estimation In General • • . • . . . . .
3.2 Least Squares Estimation . . . . • • . . . • •
3.3 Bayes Estimation In Random Regression Models
10
12
3.5 Remarks on Empirical
43
13
23
3.4 Empirical Bayes Estimation • • . . • • • • •
4.
~aye~
Estimation . . . . .
COMPARISON OF EMPJ;RICAL BAYES TO OlIDINARY LEAST SQUARES
ESTIMATORS • • . • • • • • • • •
49
4.1 Objeotives Of The Study. • • . . . . . . . •
4.2 Generation of R8Jldom Variables c, ex, 13 and ~
4.3 Comp~tations Of The OLS And EB Estimates
4.4 Results . . . . , . . . . . . . . . .
0
5.
6.
10
•
50
52
53
58
•
APPLICATIONS TO SOME PROBLEMS IN ECONOMETRICS
68
5.1 lnvestment Analysis • • • • • • • . .
5.2 Elasticities of Substitution •••
68
SUMMARY, CONCLUSIONS AND RECOMMENDAT;I:ONS • • .
74
6. 1
SUIllIIlary. . • . • • • • . • . . .
6.2 Conclusions and Recommendations
7.
REFERENCES
8.
APPENDIX...
......
..
...
72
..
74
75
77
80
vii
LIST OF TABLES
Page
3·1
Relation of C.V. and b/a • . . •
4.1
Several
4.2
4.3
~
. . . . . . .
~
and
~
2
Estimates for the marginal bias of
3
.
52
••••
58
. . . . . ...
Ratio of the average squared error of EB to 016
estimators for cas\=s 1, 2 and 3 • • • • • ••
arld
4.4
of prior densities of c, a,
form~
studied. .
48
~2
~
for cases 1, 2
. . . . . . . . . . . . .
60
The +atio R for cases 1, 2 and 3 after applying several
~
correction
fa~tors
(C.F.) for
~2
~
• • . • • . •
62
~2
Estimates for the marginal bias of .80 ~ for cases 1,
2 8J1d 3
. . . . . . . . . . . . . . . . .
4.6
The average of EB estimates for ~2 (uncorrected) and
estimates of its marginal bias for case 1, where
p = 2 and N = 10 . • • • • • •
. . . .
64
The ratio of the average squared error of EB to OLS
estimators for cases 4, 5, 6, 7, 8 and 9 . . .
OLS and EB estimates for regression coefficients and
error variances for the ten corporations • • •
70
5·2
Zellner's two-stage L8 estimates for regression coefficients for the ten corporations • • • • • • • •
71
5·3
OLS and EB estimates for elasticities of substitution
in import demand between flue-cured tobacco from the
United states and tobacco from other countries . • •
73
Generated values of c, a, ~, ~2 and their OLS and EB
estimates, taken from one replication, where N = 10,
T = 15, for cases 1 through 9 . . . . . . . . . . .
81
5·1
8.1
INTRODUCTION
1.
Linear regression models have been widely used in econometric
analysis using time series and/or cross section data.
In many cases
reported, the regression coefficient vector as well as the variance
of disturbances is assumed to be fixed in successive observations.
However, in some applications, the constancy of the coefficient
vector may be questioned.
For example, suppose a particular co-
efficient represents the response of a plant to nitrqgen fertilizer.
It is well-known that this response is strongly influenced by
temperature,
~ainfall
and
~oil
characteristics.
If these can be held
constant, we might expect that the coefficient is constant.
If they
vary but can be observed, it is desirable to incorporate their
influences into the model, but if they are unobserved, we might be
better to assume the coefficient to be random across locations and
fixed within each location.
Similar considerations may also apply in econometric studies.
observed by Klein
As
(1953), it is unlikely that interindividual
differences observed in a cross section sample can be explained by a
simple regression with a few independent variables.
In deriving the
production function, the supply function for factors, and the demand
function for a prOduct, Nerlove
(1965) found it appropriate to treat
the elasticities of output with respect to inputs and of factor
supplies and product demand as random variables differing from firm to
firm.
Kuh
(1959) and Nerlove (1965) treated the intercept as random
and the slopes as fixed parameters in estimating a relationship from a
time series of cross sections.
Zellner
(1966) pointed out that there
2
would be no aggregation bias in the least squares estimates of coefficients in a macro-equation, if we assumed the coefficients of the
micro-equations to be random.
By assuming the coefficient vector to
be random across units, Swamy (1970) presented a consistent and
asymptotically efficient estimator for the mean and an unbiased
estimator for the variance-covariance matrix of the coefficient vector.
If we can assume the coefficient vector to be random across units,
but fixed within units as proposed by Swamy (1970) there is no reason
why we cannot assume that the
X matrix and/or the variance of
residuals are also random across units but fixed within units, since
we may consider these units as a random sample from a larger
population.
These are the regression models,
!.
~.,
by assuming that
the coefficient vector or the variance of residuals or the
X matrix
or some combinations of them are random across units but fixed within
units, that will be studied in this thesis and we may call them random
regression models.
The regression models described by Swamy (1970)
above are called random coefficient regression models.
As pointed out by Swamy (1970), there is a close connection
between random coefficient regression models and Bayesian inference,
since the latter is also based on the assumption that a parameter
in this case the coefficient vector -- is considered as a random
variable with a fixed known
~
priori distribution.
Any restriction or
prior knowledge about the possible values of a parameter can be incorporated quite readily via an
ing the analysis of the data.
~
priori distribution, before perform-
However, a common objection for
Bayesian inference is that the requirement of the form of the
~
priori
3
distribution to be assumed known, which many statisticians consider it
to be unrealistic, although Jeffreys
(1961) suggested to use a diffuse
prior in case there is no knowledge at all about its form.
Bayesian
analysis has been applied to several econometric problems, for
examples, Tiao and Zellner
(1965) on mUltiple regression models with
autocorrelated errors, Zellner and Park
(1965) on a class of
distributed lag models, Zellner and Geisel
(1970) on distributed lag
models with applications to quarterly consumption function estimation,
Chetty
(1971) on Solow's distributed lag models and Chetty (1968) on
pooling time series and cross section data.
In this thesis we shall present another approach, called
empirical Bayes (Robbins,
1955), in estimation of the coefficient
vector and the variance of disturbances for random regression models.
Four multiple linear regression models are considered and empirical
Bayes estimators for each model are presented.
The difference between
empirical Bayes and Bayesian inference is that the first does not
assume at all about the knowledge of the form of the prior distribution
of a parameter.
A comparison of empirical Bayes to ordinary least squares
estimators is also given for a special case of a simple linear
regression model.
4
2.
REVIEW OF LITERATURE
The empirical Bayes approach was introduced by Robbins (1955).
It is applicable when the same decision problem presents itself
repeatedly and independently with a fixed but unknown
distribution of the parameter.
~
priori
The statistical decision problem that
we consiqer comprises:
e
(1)
A parameter space
(2)
An observable random variable
space
1
fined.
on which a
f(xle)
~
function of
X has a density
~.
~
=e
and
&.
In
& is a
x.
A loss function
L(&,e)~ 0 , representing the loss
we incur in taking
0
the true parameter is
(5)
e,
is de-
~
with generic element
estimation problem we take
(4)
measure
~-finite
with respect to
A decision space
e.
X belonging to a
When the parameter is
function
(3)
with generic element
as an estimator of
e, when
e.
An ~ priori distribution of
G of
e
defined on
® , which is unknown.
The problem is to choose a function
observe
x
we shall take
incur the loss
L[&(x),eJ
o(x)
0, such that when we
as an estimate of
The expected loss when
& and thereby
e
is the true
parameter is given by
R(o,e)
SlL[&(x),e]f(x\e)$(x) .
(2.1)
5
Hence, the overall expected loss when
e
has an
~
priori distribution
G is
(2.2)
which is called the Bayes risk relative to
G.
Rewrite (2.2) by using (2.1),
R(5,G) :;
Ie Sx. L[5 (x), eJf (x le)d,u (x)dG(e)
(2.3 )
:; .fx. ~G[5 (x),x]d,u (x)
where
~G[5(x),xJ :;
The objective is to find
IeL[5(x),eJf(x\e)dG(e)
5
G
(2.4)
•
such that
~G[5G(x),xJ :; min ~G[5(X),XJ
5E$)
which means that for any
(2.5)
OEl), we have
(2.6)
Since no assumption is made about the form of
(2.6) cannot be derived and, therefore,
However, an asymptotically optimal
0G
5
G, (2.2) through
cannot also be obtained.
relative to
5 , was proposed by Robbins (1964). We may consider
N
estimate of 5 . The procedures are as follows.
G
by
G, denoted
5
as an
N
6
Let
(xu-' eN)
(xl,e l ), (X ,S2)' .•• ,
2
be a random sample of size
N from a papulation of
(x,e)'s whose joint distribution function is
F(x\S)G(e).
Xl' X2 ' ••• , X are observable random
N
Note that
variables, while
e , e , .•. , eN
l
2
are unobservable.
Now, construct the following function
so that we shall take
thereby incur the loss
6N(~+1) E ~
as an estimate of
L[oN(~+1),eN+1J.
the expected loss in estimating
e + , and
N l
For a given sequence
e N+l , given
Xl' x 2 ' ... ,
~
(ON}
, by
(2.3), is
and hence the overall expected loss is given by
where
E
denotes the expectation with respect to the
N independent
Xl' X ' ... , xu- ' which have a common (marginal)
2
density with respect to g on 1, given by
random variables
(2.8)
From
(2.6)
and
(2.7),
it follows that
,
7
If a sequencr
(ON}
can be found such that
lim R( oN' G)
==
R(G) ,
N-ioo
we say that
[ON}
is asymptotically optimal relative to
G.
The problem now is whether such a sequenre exists and, further,
whether a sequence
every G
€
q
[ON}
which is asymptoticallY optimal relative to
q
can be found;
distribution on
may be the class of all possible
®.
One way to obtain [ON}
is to find a sequence
as a sequence of distribution functions of
as
N
~ 00,
i.
e
such that
d
GN(e) ~ G(e)
~.,
lim GN(e)
==
G(e)
N~oo
at every continuity point of
G.
(1964) presented a general
Robbins
method for constructin~ a particular sequenc~ GN(e)
for an unknown
G(e)
and then proved a theorem that under appropriate
conditions on the family of
hold for any
as estimators
G whatever.
f(xle), GN(e) ~ G(e)
Denoting
FG{x)
==
as
N->
00
will
LOOooF(xla)dG(e) ,
these conditions are:
(1)
For every x, F(xle)
is a continuous function of
e ;
(2)
Both
lim
e~
F(x I 00
- 00
F(xle)
==
F(xl-
00)
exist for every x
and
lim
e~oo
f(xle)
==
8
(3)
Neither
F(xl-c:o)
nor
F(xlc:o)
isa
distribution function;
(4)
If
Gl , G
2
of
e
(1964)
Robbins
are any distribution functions
such that
, then
also gave a method of obtaining directly a sequence
(&N} , which bypasses the estimation of
of
f(xle)
G, if the parametric family
and the loss function are given.
Several other works on empirical Bayes techniques on estimation
or testing hypotheses were reported by Samuel
(1968),
Clemmer and Krutchkoff
and Krutchkoff
(1969)
Martz and Krutchkoff
(1969).
and Maritz
(1969)
of Martz and Krutchkoff
(1963),
(1967),
Krutchkoff
(1969),
Rutherford
Among these works, only that
will be cited below, since it is closely
related to the work presented in this thesis.
They considered a sequence of experiments using a multiple linear
regression model
y. = 2$. +
where
X..J.
E.,
-~
-~
is a
k X 1
for
-~
i
(2.10)
= 1, 2, .•• , N
observable random vector,
X
is a
k X P
matrix of nonstochastic independent variables which is common to all
experiments,
~.
-~
is a
p X 1
realization of a random vector
prior distribution
G(~)
and
coefficient vector which is the ith
~
E.
-~
with an unknown and unspecified
is a
k X 1
random vector
distributed as normal with mean 0 and common known variance
~2I
By
making use of observations from previous experiments, they derived an
empirical Bayes estimator for the coefficient vector in the Nth
9
experiment.
They also pointed out, by a Monte Carla study using a
simple linear regression model, that the larger
N the smaller the
ratio of the average squared error for the empirical Bayes estimator
of
~N
to the average squared error of the maximum likelihood
estimator.
With the number of experiences as small as five, a
substantial improvement for the empirical Bayes estimator was
obtained in many cases.
The degree of improvement is also affected by the form of the
prior distribution of
an exceptionally diffuse prior will give a
~,
ratio of average squared errors close to one for every value of
If
~
2
N.
is unknown, as is the case in practice, they suggested to
use
the least squares estimator for
experiment) data, where
By using
s2
instead of
£N
~
2
by using the present (the Nth
is the least squares estimator of
2
~
~N
, they found only a slight effect on the
ratio of the averag! squared errors.
10
3.
BAYES AND EMPIRICAL BAYES ESTIMATION
IN RANDOM REGRESSION MODELS
Before we consider the mUltiple linear regression model, let us
briefly review Bayes estimation in general.
3.1
Bayes Estimation In General
Let
be an observable random variable in a Euclidean space
X
with a conditional density
f(xle), where
e
is the parameter.
1
e
is assumed to be a realization of an unobservable random variable
whose (prior) density is
over a parameter space ® •
g(e)
The joint density of
X and
h(x,e)
the marginal density of
e
e
is
f(xle)g(e)
(3.1)
X is
(3.2)
and the posterior density of
e
p(alx) =
given
X =x
is
~«X»)
(3.3)
g
By using a quadratic loss function
an estimator of
e,
,..
,..
L(a,e) = (a-e)
2
,..
,where
e
the overall (Bayes) risk is given by
(3.4)
The Bayes estimator for
,..
respect to
a, namely,
a
is obtained by minimizing (3.4) with
is
11
(3.5)
Note that
....
e
is a function of
B
x
only, therefore, its expectation
is given by
(3.6)
= SxSeeh(x,e)dedX = Jeeg(e)de = E(e) •
....
e
unbiased.
Further, it will be shown that Bayes estimator based on a
sufficient statistic for
Let
is called marginally
B
e
is the same as that of (3.5).
T be a sufficient statistic for
factorization theorem (Kendall and stuart,
e.
By the Neyman
1961),
f(xle)
can be
1
factorized into
(3·7)
f(xle) = f(tle)q(x)
where
f (t Ie)
is a function of
of
e
given
T over 3'
is the conditional density of
x
X=x
that does not involve
as in
(3.3),
and
q (x)
The posterior density
e
by substituting
(3.7),
can be written
as
p(elx) =
1
~he symbol
1~~:I:~~t:~de = ...,....--;.:~~~.f--
= f(e\t) •
(3.8)
f in f(xle), f(tle) and f(elt) do not represent the
same functional form, its use 1S merely to indicate the density
function, whatever form it might take.
12
Substituting
(3.8) into (3.5) gives us
(3.9)
which is what we wanted to show.
In the next presentation about mUltiple linear regression model,
Bayes estimators will be based on sufficient statistics rather than
the original random variables.
3.2
Least Squares Estimation
Consider the following mUltiple linear regression model
(3.10)
y.. is
where
~
T Xl,
X is a
is distributed with mean
unknown fixed parameters.
~
estimators for
and
~
0
TXP
nonstochastic with full rank,
and variance
~
2
2
(~,~)
I ,and
are
It is well-known that the least squares
2
are respectively given by
(3.11)
and
By additional assumption of normality of
are also maximum likelihood estimators for
after correction for bias.
for
~
and
s
2
b
~
(3.11) and (3.12)
and
2
~
, respectively,
is the best linear unbiased estimator
is the best unbiased estimator for
~
2
.
13
The density function of
S = (T-p)s
2
•
(Kendall and Stuart,
2
~
and
S
can be expressed as
1
exp[- ~[S + (b - t3)'X'X(b - t3)J}
(a\[2II)T
2ci
--
=
where
~
Therefore, by the Neyman factorization theorem
1961),
b
and
S
3.3
are jointly sufficient for
Further, it has been shown (Graybill,
1961), that
N[~,~2(X,x)-lJ and
are independently distributed as
(T-p)
(3.13)
£
~
and
X2~2 with
degrees freedom, respectively.
Bayes Estimation In Random Regression Models
In this section, four variants of multiple linear regression model
and their respective Bayes estimation will be presented.
The four
variants of the model will be denoted by model A, B, C and D as
described below.
and Krutchkoff
Bayes estimation for model A follows that of Martz
(1969), while those for models B, C and D were developed
analogous to their results.
Model A:
~
'IX 1
=
X
~
'IXp
IP< 1
+
€
'IX 1
where
(a)
€
(b)
X and
is distributed as
r:l
2
NlQ,~ I) ;
are fixed and known, where
is a matrix of full rank;
X
14
(c)
~
is an unknown realization of a random
variaqle
~
with a known prior density
Note that once the random variable
assumed to be constant over
estimator for
T
~
g(~)
takes a value
successive observations.
will be based on a sufficient statistic
~
~,
The
b
it is
Bayes
for
~
as in (3.9).
The conditional density of
b
is given by
2
IX/Xr~
1
f(bIX,R~ ) = ~ exp(- ---2(Xb - ~)/(Xb
-
2~
(o\[11)P
1:1.
-
-
-
- ~)} •
-
(3.14 )
Aside,
1
- --- X/X(b - ~)
2
-~
or
~ = £ + ~2(XIX)-1
1
2 ~b f(£lx,~,~2)
f(£ IX,~, ~) -
The posterior density of
~
given
2
(X,£,~)
is
p(~IX,b,~2) = h(Q,~, Ix,~2) = ~X,~,~'g(~)
where
-
h (blx,~2)
g-
(3.15 )
h (blx,~2)
g-
(3.16 )
h(£,~IX,~2) is the conditional joint density of b and ~,
while
Therefore, from (3.9), the Bayes estimator for
~
is given by
15
By substituting (3.15) into (3.17), we obtain
1
A
~=----
h (b
\x,<i)
S[£
2
-1
+ ~ (XIX)
g-
1
I
a
2
2
ab f(£ X,~,~ )Jf(£IX,~,~ )g(~)~
f(£IX,~,~) 2
(3.18 )
where
h (bIX,~2)
g-
the first partial derivative of
From
with respect to
b.
(3.6), we have that ~ is marginal~ unbiased, ~.~., E(~) = E(~) .
Note that
hg(£lx,~2) depends on g(~) which we denote by putting a
subscript
g.
Model B:
~
.l=x
'IX 1
+€
'IXp pXl
']Xl
where
is distributed as
N(Q,~2I) ;
(a)
€
(b)
X is fixed, known and with full rank;
(c)
(~,~2) is an unknown realization of a random
variable
density
(~A,_, ,.,.2)
v
2
g(~, ~
Wl"
) •
th a kn own JOln
." t prlor
"
16
From (3.13), we know that
b
and
=
(T-p)s
2
are jointly sufficient
(~,~2) and are independently distributed as N[~,(X/X)-1~2J and
for
X2~2
with
(T-p)
~
estimators for
degrees of freedom, respectively.
and
2
~
density of
b
conditional density of
freedom.
and
Denote
f(£IX,~,~2)
N[~, (X/X)-1~2J
which is
S
The Bayes
, therefore, will be based on these
2
sufficient statistics.
X
S
which is
to be the conditional
and
X2~2
fT_P(sl~2)
with
Note that the conditional density of
(T-p)
S
as the
degrees of
does not depend on
~.
For a given
(X,~,~2), it can be shown that £
and
independent, therefore, the conditional joint density of
denoted by
b
are
and
S,
2
fT_p(£,SIX,~,~), can be expressed as the product of the
conditional density of
Since
S
S
b
and that of
is distributed as
S,
~.~.,
X2~2 with
(T-p)
degrees of
freedom, we can write
(3.20 )
or,
2
The symbol f does not have to represent the same functional form,
it merely indicates the density function and the index (T-p) or
(T-p-2) of f indicates the degrees of freedom.
17
s
t(~)
S ~(T-p-2)-1 S(-2)
e
cr
2
= -cr--....-01.,.(T--2.....,)~T--2--T-,
-2- = -T_-S-_-2 f T_p _2 (S Icr )
~
-p~
(-p-)
P
)
(
2 2
2 r
2
(3.21)
f _ _ (slcr2 ) is the conditional density of
TP 2
type with (T-p-2) degrees of freedom.
S which is a X2 _
where
The posterior joint density of
2
f(~,cr IX,E.,S)
and
~
cr2 , given
(x,.!?, S) , is
2
2
fT_P(E.,slx,~,cr )g(~,cr )
f
(b, S IX)
T-p -
(3.22)
where the numerator of (3.22) is the conditional joint density of E.,
s,
~ and cr2 , while
~
From (3.9), the Bayes estimator for
A
~
~
2
= E(~lX,.!?,S) = SJ~f(~,cr
is given by
IX,E.,S)~dcr
2
.
substituting (3.15) and (3022) into (3.23) gives us
A
f3
-
=
SS[b
-
+
cr2( XI X)-1 0h
f (I
E. X,~,cr 2)
0....
f(bIX,~,cr2)
2
fT_P(E.,S IX,~,cr2) g(~,cr)
]
-
Further, by making use of (3.19), we have,
fT_p(E.,slx)
~dcr2
18
==
l? + f
Next,
(X/X)-l
(b SfX)
T-p -'
app~ng
-b+
- -
SS[ab
0
-
I
2
2
I2
2
2
f(l? X,~,O" )J[O" fT_P(S 0" )Jg(~,O" )~dO" •
(3.21), we obtain
S(X/X)-l
T-p-2
f T(1)
__ 2(l?,S IX)
~
f -p -b,S X)
T
(3.24 )
where
Similarly, from (3.9), the Bayes estimator for
,,2
0"
==
E(O"2\ X,l?,S)
==
0"
2
is given by
SS 2f(~,O" 21 X,l?,S)~dO" 2 •
0"
Substituting (3,22) and (3.19) successively, gives us
2
2
2
2
,,2 _
0" f T-p (l?, S IX,~, 0" )g (~, 0" )~dO"
0"
fT_p(l?,slx)
SS
_
- SS
f(l?IX,~,O"
2 )[0"2fT_;p(S 10"2 )Jg~,O" 2 )~dO" 2
f
Now, by using (3.21), we have
(b,S\X)
T-p -
•
19
Note that both
"
~
,," 2
and
and
0-
g (~,
depend on
f(l) (b SIX)
T-p-2 -'
2
0-
)
f _ (£,
T p
since
,
g(~,
depend on
2
0- )
•
Model C:
X
Y..
'IX 1
f3_
'IXp :px 1
€
+
-
'IX 1
where
(a)
E
(b)
(X,o-)
is distributed as
2
variable
2
N(Q,o- I)
is a known realization of a random
2
(X, 0- ) , while
~ is an unknown
realization of a random variable
three random variables
X,
~
The
~.
and
2
0-
have a known joint prior density
(c)
X is a matrix of full rank.
Observe that Model C is similar to Model A in the sense that
0-
2
X and
are known, therefore the only parameter to be estimated is
The posterior density of
~
given
2
(X,~,o-)
~.
is given by
(3.26 )
20
where
and the numerator of
x,
~
and
2
CJ
(3.26)
is the joint conditional density of
b,
•
~,
Therefore, the Bayes estimator for
By substituting
(3.15)
the Bayes estimator for
into
(3.27),
using
(3.9)
analogous to
and
(3.18),
is
we obtain
as follows:
~
(1)
2
1 h
CJ (X/X)g
2
(b,X,CJ)
2
h (b,X,CJ )
g-
where
Model D:
Y..
'lXl
~
X
=
pXl
'JXp
+ E
'lXl
where
2
N(0,CJ I) ;
(a)
€
(b)
X is a known rea.lization of a random variable
is distributed as
X , while
(3.26)
?
(~,CJ-)
is an unknown realization of
(3.28 )
21
a random variable (~, (l).
variables
X,
prior density
(c)
2
and
cr
g(X,~,cr
2
~
The three random
have a ,known joint
)
X is a matrix of full rank.
Note that Model D is similar to Model B in the sense that
2
(~,cr)
and
X is known
is the parameter to be estimated.
The derivation of Bayes estimators for
to that of Model B, namely the Bay'es estimator for
...
~==b+
S(X/X)-l
T-p-2
cr2
and
~
~
is analogous
is given by
f~:~_2(X,:£,S)
(3.29)
f T-p (X,b,S)
-
where
(1)
0
f _ _2 (X,:£,S) == 0:£ f _ _2 (X,:£,S) ,
TP
TP
and the Bayes estimator for
... 2
(J'
In this case
...
~
2
cr
S
f T_ _2 (X,£,S)
P
== T-p-2 f
(X,b,S)
T-p -
and ~2
depend on
and
(X,:£, S) ,
Remarks:
is given by
g(X,~,S)
f (1)
_ _ (X,:£,S)
TP 2
(3.30)
since all of
depend on
f
.
T-p,
g(X,~,cr2).
If we are dealing with a single mUltiple linear
regression model, Model A and Model B,
~.~.,
the models with fixed
X,
are the ones that are commonly used in Bayes estimation.
However, when we are dealing with several regression equations,
where the
X matrix as well as
~ and cr2 may differ from one
regression equation to another, we might be willing to assume Model C
or D.
22
For Mode.l A and C, we assume ttat
cases is not the case.
s
'::J
O"~
If
0"
2
is known, whi chin mos t
is unknown, one might wonder of using
2
2 2
, the least squares estimate for O' , instead of the true 0"
to obtain the Bayes estimate for
Bayes estimate for
~,
say
"
.~,
~.
IE this case, the obtained
wil.l not minimize the overall risk
(Bayes risk) for all possib.le values of
the Bayes risk for a special case when
On the other hand, when
2
0"
0"
0"
2
rather it only minimizes
,
2
s
2
is unkrlOwn as assumed in Models B
'::J
(~,O"~),
and D, the Bayes estimate for
,
denoted by
minimize the Bayes risk over all. possible values of
(~, 0-2 ) , wi 11
2
(~, 0" )
•
Note that for al.l models A, B, C and D, the knowledge of the
exact form of the prior density must be assumed to obtain the Bayes
estimators.
If there is no knowledge at all about the prior density,
Jeffreys (1961) suggested to use a uniform prior density as follows.
When the range of a parameter
e
is from
-c
to
c, where
and may take value
+00, the prior density is taken to be
For the case where
eE(O,C)
density of
log
lei
or
6E(-C,O), where
c > 0
g(e)o:;l.
C > 0 , the prior
is taken to be uniform, ~.~.,
f (log
Ie \ )0:;1
or
gee)
0:;
\e\-l.
Applying Jeffrey's suggestion of uniform prior density for Mode1S
A, B, C and D, we have:
23
Model A:
g (~)cx:l , since
_
~
E
E
:P
(p dimension Euclidean srace),
Model B:
··1
g (~, cr )cx:cr·
, since
and
cr>O.
Models C and D:
2
-1
g (X"~' cr )cx:cr· , since
3.4
X c E'D<p , ~ (:.E
p
and
cr > 0 •
Empirical Bayes Estimation
Recal.l that the Bayes estimators for Models A, B, C and D cannot
g(~)
be obtained, unless the prior densitips
for Model B,
2
g (X,~, cr)
for Mode.l A,
for Model C and Mode 1 D are known.
2
g(~,cr )
In
practice, this is usually not t.he case and it is unrealistic to assume
a specific form of a prior density.
In this section, this assumption about the knowledge of the prior
density will be rel.axed and a two-stage Bayes estimator, which is
called empirical Bayes
(~stimator,
will. be derived.
the four models discussed in Section
denoted by Model. A*,
B*, C*
~Dd
D*,
3.3,
the following four models,
will be considered.
Model A*:
Y.,,=X
~.
-],-1.
']Xl 'D<:p pXl
where
+
c.
-].
']Xl '
In accordance with
i
1, 2, •.• , N
24
~l' ~2' ... , ~N
(a)
are independent with a
2
common distribution
2
(X,o-)
(b)
all
is
N(.Q,a I)
fixed and known, common for
i = 1, 2, ... , N.
The matrix
X is
of full rank;
~l' ~2'
(c)
~N
0."
are unknown independent
realizations of a random variable
an
unknow~
with
~
g(~).
prior density
This is the model analyzed by Martz and Krutchkoff
they derived an empirical Bayes estimator for
We shall review
~N'
their derivation below and obtain an empirical Bayes
13. (i = 1, 2, .. u, N)
N) , by
g(~)
been known, the Bayes estimate for
= -b.l +
2
0-
-1 h
13.
-l
(i
1, 2, ... ,
(X'X)·
(1) (£. IX, (j2 )
g
l
h (b.
g
Now, since
2
(X,o-)
-l
IX,
is common for all
(£l'~l)' (£2'~2)' •.. ,
of a random variable
equation
for
(3.18), would be given by
p.
-l
consider
~stimate
0
-l
Had
(1969), where
(£,~)
(£N'~N)
(3.31)
2
(J
)
i = 1, 2, •.• , N,
we can
as independent realizations
whose joint density is denoted (recall
(3.16) for Model A of Section 3.3) by
E. 2 ,
Therefore, we can also consider that
£.1.'
realizations of a random variable
whose (marginal) density is
b
••• , £N
are independent
25
hg(~lx,~2) in (3.16).
denoted by
common for all
i
1, 2, .•. , N , we can. just denote this marginal
h (b) .
density by
g-
Noting that
variable
(X,~2) is
For simplicity, since
b
hg-l
(b.)
is a multivariate densi ty of a random
evaluated at
b = E.i ' and
~l' ~2'
•.. ,
~N
are
observable and independent, several methods for estimating
are available such as the method described by Cacoullus
Let
f(~)
~
= x.
-l
!
3
A quadratic mean consistent
asymptotically unbiased estimate for
independent observations
g
(1966) below.
be a multivariate density of a random vector
(]Xl) , evaluated at
h (b.)
!l' !2' .•• ,
f(~),
~
P
N II D..
based on the sample of
~,
of
... ,
1
and
x -x
.p
D.
is given by
kp)
(3.32)
P
j=i J
where
IKCyJ I <00,
(a)
sup
.lEEp
(b)
SI KCyJ IdX < 00,
(c)
~.~.
,
K is bounded;
K is integrable;
\y\PK(y) = 0 , where
lim
Ixl-'
~.~.,
oo
length of vector
Iyl
denotes the
X
3
Consistency here is in the sense of mean squared error consistent,
~.~.,
lim
N-.oo
E[fN(~)-f(~)J2 = 0 .. at every continuity point of f .
-
N
26
(d)
SK(I)dl = 1
;
(e)
b.. = b.. (N)
satisfies
J
lim b.. (N) = 0
N-+ CO J
J
lim N[b. . (N)J P
N-+ co
J
= co, for j =
By using £1' £2' ..• , £N
as
and
1, 2, .•• , P •
N independent observations and
setting
7~
l
>
x -x.
*#
••
,
P
l
1
P
(2II)p j=l
2
.
k J]
25 j
Sln
p) = --:::- II
b.p
b .. -b
[lJ
•
(3.33 )
[bij-bkjJ
25.
J
and defining
. (bij-bkj )
25.
Sln
_ .........._,....;JII--_ = 1 ,
for
b .. -b .
(lJ k J)
k
=i
,
20 j
a consistent and asymptotically unbiased estimate for
hg (b.)
,
-l
therefore, is given by
2
l
1+
N
L:
P
II
b .. -b
k=l j=l
(lJ
kfi
25.
J
where
(1)
5. = 0 (N)cr
J
\/t (X/Xf 1 } JJ..'
;
.
kJ
]
(3.34 )
27
(2)
,
(X/X)-l}jj
is the
(j,j)th
element of
(X'X) -1 ;
(3)
..
0 (N)
= 0 and
lim 0 (N)
satisfies
N... ex:>
lim N[6 (N)]P
N... ex:>
1
6 (N) = N)
q
= ex:>, for example, take
q > p •
wi th
Note that all of the conditions (a) to (e) given in
met by the function
K of
(3.33).
(3.32) are
We might use functions other than
(3.33) as long as it satisfies conditions (a) to (e) given in (3.32);
the choice of
(3.33) is made so that this work will be comparable to
that of Martz and Krutchkoff (1969).
At this point, a study to find a
function that will give better empirical Bayes estimators, if possible
the best in some sense, is still
~en.
of (3.32) is a
Furthermore, observe that
p X 1
vector
and can be wri tten as
h(l\b.)
g
-~
where
b.
-~
=
[~bO
() il
h (b.), •.. , ~bO
h (b. )JI
g -~
() ip g-~
= [b. l , b. 2, ..• , b. J/.
~
~
By the definition of the first
~p
derivative of a function, (3.35) .can be expressed as
1
h ( ) (b.)
g
-~
= [hg 1 (b. ), ... , hgp (b.)
J'
-J.
(3.36 )
-~
where
h . (b.)
gJ
-~
h ([b. l' ••• , b. . +
g
~
= lim --!ii!._=
€... O
~J
~
€
€,
••• ,
b.
JI)
- h (b.)
__
~p
--:;;~
g
-~
-.:.L~
28
for
,
= 1, 2, •.. , p
j
By choosing
E
such that
lim
E
=0
for example we can choose
N... co
E =
5.J
of
(3.34) -- and using (3.32),
h .(b.)
gJ
of
-J.
(3.36) can be
estimated consistently by
which in turn gives a consistent estimator for
h-~l) (b.)
-J.
-1'4
An
as follows:
[h-_l(b.),
... , -l.\J"p
h-_ (b.)J
I
-1'4 -J.
-J.
=
empirical Bayes estimator for
(3.38)
•
is obtained by substituting
~.
-J.
(3.34) and (3.38) into (3.31), namely
,
i = 1, 2, .•• , N •
By using an argument as given by Rutherford and Krutchkoff
(1969),
~.
it can be shown that the overall risk of
"~.
the Bayes risk of
of
-J.
asymptotically optimal.
(3.31), as
~.
-J.
of
(3.39) converges to
N ... co, ~.~.,
Rutherford and Krutchkoff
~
~i
of
(3.39) is
(1969) gave a set
of sufficient conditions for an empirical Bayes estimator to be
asymptotically optimal, when the squared error loss function is used.
Denoting
e,
a
and
~
e
as the Bayes and empirical Bayes estimators for
respectively, those sufficient conditions are given at the top of
the next page.
29
"
plim "e
(a)
= "e .
N-+co
,
(b)
)' > 0
For some
we have t hat
""
~i
Indeed,
of
and some real number
E( \ e \2+)') ~ M < CO
M < co ,
•
(3.39) can be shown to satisfy condition (a) as follows.
~(£i)
By the consistency of Cacoullus' estimators
and
~l)(£i)'
we have that
h (b.)
g
-~
and
Therefore, for
~
plim~.
N-+co -~
= b.
-~
~
~.
-~
of
(3.39), we have
= -~
b.
+
+
Further, by imposing a mild condition on the prior density
such that their third absolute moments are bounded, we have that
condition (b) is also satisfied.
Model B*:
Y...~ =
':IX 1
where
x
~.+
-~
'D<p pX 1
§.~,
.....
'D<l
i
1, 2, •.• , N
g(~)
30
(a)
(b)
E.
is distributed as
.§.2'
... ,
-~
2
N(O,ao.I)
-
and .§.l'
~
are independent;
.§.N
X is fixed, known and common for all
i
=
1, 2, ..• , N ;
independent realizations of a random variable
(~, 0"2)
g(~, 0"
2
wi th an unknown pri or densi ty
) •
Note that once the random variable (~,0"2) takes a value, say
2
(~.,O".),
-~
~
it is fixed for
regression equation.
f3.
(i =
-~
Had
successive observations in each ith
T
g(~,O"
2
)
been known, the Bayes estimate for
1, 2, .•• , N) , from (3.25), is given by
A
f3.
= b.
-~
-~
(3.40)
and the Bayes estimate for
o-~
2
=
~
_S_i_ f T- E- 2( O?·i' Si1xr'X))
T-p-2 f -p -2
b.,S.~ X
T
Observe that since
can consider
2
0". , from (3.26), is given by
X is common for all
(3.41)
i
=
1, 2, ••• , N , we
2 2 2
(£l'Sl'~l'O"l)' (!?2,S2'~2'0"2)' ... , (!2-N,SN'~N'O"N)
independent realizations of a random variable
2
(£,S,~,O")
as
whose joint
density is denoted (recall equation (3.22) for Model B of Section 3.3)
Thus,
can also be considered as independent realizations of a random variable
(£,S) whooo (marginal) density is fT_P(£,sIX)
from (3.22).
31
Thus,
(T-p)
:.
f -p (b.,S.
IX)
T -1 1
degrees of freedom of a random variable
(b,S)
= (b.,S.).
-1
1
Similarly,
multivariate density with
variable evaluated at
i
t::
is the value of a multivariate density with
1, 2, ..• , N,
f -p- 2(b.,S.
Ix)
-1
1
T
(T-p-2)
(b.,S.).
-1
1
evaluated at
is the value of a
degrees of freedom of a random
Since
f -p (b.,S.\X)
T -1 1
(£,S)
and
X is common for all
f -p- 2(b.,s.lx)
-1
1
T
can simply be
denoted by
f -p (b.,S.)
and f -p - 2(b.,S.),
respectively.
-1
1
T -1 1
T
Again, in this case, Cacoullus' method can be applied to obtain
consistent and asymptotically unbiased estimates for
f
T-p
(b.,S.),
-1
1
(1)
f T-p- 2(b.
,S.)
and f -p- 2(b.,
S.)
, which will be denoted by
-1
1
-1
1
T
(1)
(1)
f T (b"Si)' f T 2(b.,S.) and f T 2(b.,S.), respectively.
N, -p ~
N, -p- -1 1
N, -p- ~ 1
Therefore, SUbstituting these estimates into (3.40) we obtain an
empirical Bayes estimate for
~
~
...
A
t:.i '
(3.42)
= -i
b +
Similarly, by substituting
into
denoted by
f N, T-p (b.,S.)
-1
1
and
f N,-pT 2(b.,S.)
-1
1
(3.41), we obtain an empirical Bayes estimate for
2
~.
1
, denoted
by
The derivation of
f (1)
T 2(b.,S.)
N, -p- /~1 1
f N, T-p (b.,S.),
-1
1
will be presented below.
S.
1
=
T
2
1::
e ..
j=l
1J
f N, T-p - 2(b.,S.)
-1
1
Since
and
32
has
(T-p)
T (b.,S.)
N, -p - l l
as follows:
degrees of freedom,
(3.34),
away, analogous to
can be obtained right
f
1
,Sol )
, -p (b.
-l
fN T
(2TT )
p+L_
P
n 6. .
-n'Y
. 1 lJ
J==
(
t
2
S'-Sk
2". ] np
'Y
sine
N
1 + E
k==l
kfi
S.-S
[l
k
~J
.
J==l
. cbij-bkjj
Sln
2J:
U ••
lJ
kJ ]
b. -b .
[lj
(3.44 )
26 ..
lJ
where
(1)
'Y ==
(2)
§ == -
(3 )
6(N)
~-
1
E
N l=
. 1
(S.-S)
2
l
1 N
E S. ;
N l:;::
. 1
J:
u
(N)
ij == U
l:
(4 )
(5)
N
l
_
~
S.
1
..2:-r
(X
'X ) - } .
T-pt
jJ
is the
6 (N)
satisfies
lim N[ 6 (N)
N-+ co
1
t+1
(j,j)th
lim (N) == 0
N-+co
=co,
the random variable
and
for example, take
6(N) == N q, where q > p
The derivation of
element of
f , T-p- 2(b.,S.)
-l
l
N
+ 1 •
is rather complicated, since
33
=
S
has
(T-p)
variable
T
.E
e.
j=l
J
degrees of freedom.
S*
2
However, we can construct a random
a random variabJ,.e with
derived from
S,
i
j'
= 1,
S*
possibili ty of which pair of
free ej's.
,
2, .•. , T .
T > p + 2,
But, for
of
(3.45) gives more than one
e.' s to be thrown away out of
J
For uniformity, it is suggested to take
T.
j
S*
(T-p)
as the
(S-e~-e:
)'s and all possible (T-p) free
J JI
average of all possible
e 's out of
(T-p)
~.~.,
S* = S _ e 2 - e 2 1
j
j
j
degrees of freedom --
S, for example by throwing away two out of
degrees freedom of
where
(T-p-2)
Suppose the first
(T-p)
of e.'s are free, then we
J
have
s* =
1
T-p
T-p
1:
.E
(T-p) j=l
2
1
= (T-P)
2
j
j/=l
J
J
< j'
T-p
.E
2
2
(S-e. - e. 1 )
T-:p
T-P
.E [.E
j '=1 k=l
j=l
j < j
1
kfj
k:/=j
1
T-p
T
T
T-E-2
2
2
T-E-2
2
2
'" T.E e k +
.E
ek =
S + T-P
.E
ek •
p k=l
k=T-p+l
T-p
k=T-p+l
(3.46 )
34
Since the free
overall average of
ej's can be any
S*
in
(T-p)
out of
T of ej's, the
(3.46) is given by
s* = T-p-2 S + 1
~ ~ (T-l)e 2 = [T-E-2 + ~]s = T-2S
T-p
( T ) T-p k=l p-l k
T-p
T(T-p)
T
T-p
By using the random variable
degrees of freedom, instead of
again analogous to
S*
(3.47) which has
of
S, we can derive
(T-p-2)
f , T-p- 2(b.,S.),
-~
~
N
(3.34), as follows:
2
fN T
,
-p-
(
1
2(b.,S.) = --..:;;:;..----~
~
p+ L
P
(2II) ~'Y II 5 ..
j=l ~J
(Sin[ k
S.-S*
+ N.
L:
k=l
k/=i
S.-S*
[~ K]
2y
J
k
P
.
II
J=l
in[~J)
S.-S'¥"
[~
2y
~]
b..-bkj ) 2]
sine ~J]
25 ..
~J
(3.48 )
b .. -b .
[~t .kJ ]
~J
where
(a)
Sf
T-2
= ~ Si ' for
(b)
'Y
and
5ij
i = 1, 2, ... , N ;
are as defined in
(3.44)
f (1)T 2(b.,S.) is obtained, analogous to
N, -p- -~ ~
making use of (3.48), as follows.
Finally,
(3.38,
) by
fN(lT)
= [fN,· l , T-p- 2(b.,S.),
••• , f N, p, T-p- 2(b.,S.
)],
, -p- 2(b.,S.)
-~
~
-:l.
~
-~
~
(3.49)
35
where
f N,J,
. T-p- 2(b.,S.)
-2
2
1
= --~--(fN
v
, T-p- 2([b·2 l ,
ij
.
• •• , b.2p J', S.)
- f
2
N,
. , S.2 )}
T-p- 2 (b
-2
for j "" 1, 2, ..• , p .
Before we discuss Model C* and D*,
x = c..~,X*J
, where
X*
= [!2'
obse~ve
~, ... , ~J
that, in general,
.
Model C*:
Yi
=
'D<l
~i
Xi
= 1,
i
'D<p pxl
2, ••• , N
or
~i
'D<l
=:
[1,
Xf
J ~i + §.i '
'lXl 'IX (p-l) pXl
'D<l
= 1, 2, ..• , N
i
where
(a)
€.
-2
(i
= 1,
and §.l' §.2' ..• , §.N
(b)
(X!' o-~),
is distributed as
2, ... , N)
(X~, o-~),
are independent;
2
••• , (Xj,o-N)
are known independent
realizations of a random variable
~l' ~2'
••. ,
of a random
X* ,
~
g(X*,~,o-
and
2
~
(X*,0-2), while
are unknown independent realizations
variable~.
0-
2
N(O,o-.I)
2
2
The three random variables
have an unknown prior joint density
) ;
(c) X is a matrix of full rank.
36
Note that the assumptions for Model c* is the same as those for
g(X*,~,~2) is unknown.
Model C, except that the prior joint density
Tne parameters to be estimated in Model C* are
g(x*,~,~2) been known, the Bayes estimate for
~l' ~2' ••• , ~N'
~.
-~
Had
(i = 1, 2, •.• , N) ,
from (3.29) is given by
2 '
-1
'"
~ = b. + ~. ( X.X.)
... -~
~
*
h (1) (X. , b . , ~.2 )
g
~-~~
(3·50)
2
h (Xt,b.,~.)
~ ~
g
~
-~
~
where, as before,
(1)
2
0
2
hg (X1E",b.,~.)
="'-::-b
hg (X1E",b.,~.)
•
~ -~
~
0 h •
~ -~
~
-~
Again, by a similar argument as for Models A* and B*, we can
(Xl'£l,~i), (X~'£2'~~)'
consider
realizations of a random variable
2
hg (X*,b,~
).
to obtain a
.•• ,
~
-~
2
(X*,£,~)
as independent
whose density is
Therefore, Cacoullus ' method is again applicable here
consi~tent
and asymptotically unbiased estimate for
h (X~,b.,~~) , a multivariate density
g
(XN'£N'~~)
~
(X*,b,~2)
= (Xt,b.~~).
~ -~ ~
h (X*,b,~2) evaluated at
g
-
Denote this estimate by
h (Xt,b.,~~) •
Note that we can write
Xf. .=
[~;2' .•• , x .. , ••• , x. J,
....
-~J
-~p
for
i = 1, 2, ••• , N where
x .. = [x.~J'1' ••. , x ~Jm
.. , ••• , x.~J'T]',
.
-~J
for j
2, 3, •.• , p , which allows us to write
N
~ -J.,
~
37
hg (X-tE-,b.,cl)
== hg (x.
, ... , -~p
x. ,b.,cr~).
~ -~
~
-~ 2
~
~
Applying Cacoullus' method, analogous to
(3.44), we can obtain
N
I:
k==l
k/i.
(3.51)
where
(1)
Y == 5 (N)
(2)
cr
-2
1 N
2
== - I: cr; ;
N . 1 ...
~==
lljm
(4)
= o(N)
_
1
x.
• Jm
O. .
~J
(6 )
-2)2
( cr2 -cr
i
I-~-----­
\~ i~l
N
I: x
N i==l ijm
=-
=
0 (N)cr. \ Ir-(X-!-X.-)--l-}..~
~ ~
JJ
\It
f(X!X. )-l} ..
t
l
(xijm-X.jm)2
~
JJ
is the (j,j)th element of
-
(X!X. )-1
~
l
38
(7)
0 (N) satisfies
lim
0 (N)
= 0 and
N.... oo
.
( ) (p-l)T+p+l
lim
N[ 0 N J
N.... 00
o{N)
While
(1)
lL~
-~
2
(X~,b.,O".),
J. -J.
e;
for example take
= N q ,where q > {P-l)T+p+l .
J.
h (1) (X.* , b . , 0".2 )
estimate for
=00,
1
l
-l
J.
a consistent and asymptotically unbiased
is obtained, analogous to (3.49), as
follows:
[h--l(X~,~,O"~),
... , h-_
(Xt,b.,O"~)J'
N
J. ~.l.
J.
Np
J. -J.
J.
(3.52)
where
... ,
lL_. (X~,
b., O"~)
= ~[lL_{X~,
[b.J. l'
J. -J.
J.
u. . -~
J.
-~J
b .. + 0.. , •.. , b. JI ,O"~)
J.J
J.J
J.J
J.p
J.
for j = 1, 2, ..• , p .
Finally, by substituting (3.51) and (3.52) into (3.50), an
empirical Bayes estimate for
,.,
'"
~. = b
-J.
-i +
2
0".
J.
(i = 1, 2, ... , N), is given by
~.
-J.
(X!X. )
1
-
*
h (1)
\
(X. , b.
-~
J. -J.
2
,0". )
J.
2
h-_(X~,b.,O".)
J. J.
. N
J. -J.
J.
Model D*:
VJ.'
:J..
'D<l
=::
X.J . - 13.
J.
'D<p IP< 1
+
E.
-J.
'lXl
or, as in Model C*, we can write
,
i
= 1, 2, ... , N
(3.53 )
39
li
']Xl
=
[1: , xf
]
']Xl 'IX (p-l)
+
,
E.
-~
'lXl
i
= 1,
2, ... , N
where
(a)
E.
-~
is distributed as
2
N(Q, (Ji I)
... ,
.§.l' .§.2'
i = 1, 2,
N,
and
,
for
... ,
.§.N
are independent;
(b)
... ,
x* are observable (known) inN
dependent reali~ations of a random variable
Xi, X*,
2
X* , while
ar~
unobservable (unknown) independent
realizations of a random variable
(~,
The three random variables
and
X*,
~
2
(J ) .
(J2
have an unknown prior joint density
g(X*,~,(J
(c)
2
) ;
X is a matrix of full rank.
In this model, the
(~2,(J2)' ••• ,
2
2
(~N,(JN)'
par~eters
to be estimated are
Observe that the assumptions for this model
is the same as those for Model D, except that the prior joint
density
When
... ,
g(X*,~,(J2) is unknown.
2
g(X*,~,(J)
N) , from
is known, the Bayes es timate for
13. (i
-~
= 1, 2,
(3.30), will be given by
(3·54 )
40
2(.1
and the Bayes estimate for
(J'.
~
:::;
1, 2, ..• , N) , from (3.31), will
be given by
S. f T-p- 2(X1E-,b
J. - i ,S.J. )
~i - T-p-2 f
(X~,b.,S.)
T-p J. -J. J.
"
2
~_
(3.55 )
By the same arguments as before, we can consider
variable
(X*,£,S)
whose (marginal) density is
f
T-p
fore, Cacoullus' method is applicable for estimating
f -p- 2(X~,b.,S.)
J. -J. J.
T
and
fN,T_P(Xf'£i,Si) ,
fT(l)
-p-
2(X~,b.,S.)
J. -J. J.
f N, T-p- 2(X~,b.,S.)
J. -J. J.
as before.
and
fN(lT)
, -p-
(X*,b,S).
f
T-p
There-
(X~,b.,S.),
J. -J.
J.
Denoting
2(X*.,b.,S.)
J. --J.
as
J.
their respective (Cacoullus) estimates, empirical Bayes estimators
for
~.
-J.
2
and
~.
J.
will be given by substituting these estimates as
before into (3.54)and (3.55) as follows:
-1
A
~
.
-J.
S. (X!X. )
J.
J. J.
= -J.
b. + --=-T--=::;"":;2::""--p-
and
"2 _ _ _
S.J._ f N, T-:8- 2(X~,b.,S.)
J. -J.
J.
fri - T-p-2 f T (X~,b.,S.)
N, -p J. -J. J.
(3.57 )
Needless to show algebraic derivations, since they are analogous
to those described in Model B* and C*, we have the following results:
41
f
T
N, -p
1
(X~, b. ,8. )
1. -1. 1.
N
~
k=l
kfi
. [Xi jm-XkjmJ)
s1.n
i
p
T
n
n
26 jm
( j=2 m:;::l
b .. -b
. [1.J2~ kj J
s1.n
p
Vij
n
b .. -b ,
[1.J k J]
j=l
26 ..
1.J
wbere
(1) Y = 6(N)
_
(2 )
8
(3 )
6
(4 )
1
=-
\j
1
-N
N
_ 2
~ (S.-8)
. 1 1.
1.=
N
~
8.
N i=l 1.
is as defined in (3.51) ;
8.
1
6., = 6(N)
T1. ((X!X.)- } .. ;
1.J
-p
1. 1.
JJ
jm
(5 )
((X!X.
)-1},.
1. 1.
JJ
(6 )
6(N)
is tbe (j,j)th element of
sa.tisfies
-
lim
6(N)
=0
(X!X.
)-1
1. 1.
and
N... oo
lim N[6{N)] (p-1)T+p+1 '" 00, for example we
N... cn
42
1
can take
q
=N
q, where
> (p-1)T+p+1 .
T 2(X~,bi'S,)
,-p~.,.. ~
fN
5(N)
1
= ---------------=~-------------
(2II)(p-1)T+P+~y( ~
~
6. ) ~ 5..
j=2 111=1 JIll j=l 1J
S'-S~02
Sin[~J
(
S. -S*k
. [Xi jIll_X kjIll ] )
12
] ~p
T s~n
2·
6.
---.....:::;.y"--II
II
JIll
S. -S'lE"
. 2
1
x .. -x ,
[~]
J= m=
[lJm k JIll]
2y
26
jm
. [
s~n
N
+ I:
k=l
k/=i
P
S.-S1E"
[-2:.-2.J
2y
b .. -b .
sine 1J k J]
25
II
j=l
2
.
ij
where
and
*
(1)
T-2 S
Si = T
i'
(2)
y, 5 , 6 jm
ij
for
and
l'
_-
5(N)
1, 2, .•• , N ;
are as defined in (3.58 )
fN(ll
2(X~,b"S,)
,-I--P~ -;L
~
~
[fN 1 T
2(X~,b.,S,),
~"
,
-p~ -~
~
••• , f N,p, T-p- 2 (X1E-,
b, , S,~ ) l'
~ -~
(3.60)
where
1
f N,J,
' T-p- 2(X~,b"S.)
--(fN,-I--pm
2(X1E-,Cb'l'
1 -~ ~ ~ --~
U,.
~
~
:LJ
... , b, ]
~p
for
j
~
I ,
S.) - f
~
T
2 (X~ , b , , S, )}
~ -~
~
N, -p-
1, 2, ••• , P •
3.5 Remarks On Empirical Bayes Estimation
As indicated by the assumptions, Model A* is applicable when we
'
have a se t a f regress:Lon
equat'~ons W:L'th common
known.
o2
(X,cr )
and
cr2
4S
~
Martz and Krutchkoff (1969) pointed out, by means of a
simulation study of a simple linear regression model, that the
empirical Bayes estimator for
~
'"
(denoted by ~N ) is better than
the ordinary least squares estimate (denoted by £N ), in the sense
....
that the average squared error for
'"
~N
is smaller than that of
The more improvement will be gained, the larger
N
£N.
(the number of
experiments ).
Note that Model B* is a generalization of Model A*, since if the
random variable
2
cr
in Model B* is degenerate of a known value
Model B* will become Model A*.
Similarly Model c* is also a
generalization of Model A*, in this case if the random variable
2
(X*,cr)
of Model c* is degenerate.
generalization of Model B*.
Model D*, however, is a
cr
2
,
44
By assuming the
and
~
2
X matrix to be a random variable besides
, as described in Model C* and D*, empirical Bayes estimation
for these models will be more cumbersome as
the
~
X matrix) becomes larger.
TX P
(the dimension of
By putting some restrictions on this
X matrix, we can obtain special cases of Model C* or D*.
For example, the
fol~owing
model, denoted by Model
TIt'
is a
special case of Model D*.
,Model
Dt:
,x~J
::;: [_1
4
-'l.
'D<l 'JXl
'D<l
~.
-~
+
E.
,
-:L
i
'D<l
~l
1, 2, ... , N
where the assumptions are the same as those for Model D* with
additional restrictions below:
(a)
T is an odd number;
(b)
The elements of
x
(c)
x
i2
::;:
iT
x1E"
x il ::;: x
i3
-
2.5x
,
for
il
are equally spaced,
-~
x
...
::;:
i2
i
::;:
::;: x
1, 2,
. ..,
iT
.!.. ~. ,
xi,T-l
N
.
The chaise of restriction (c) will be discussed later at the end
of this section.
Since
is odd, we have a middle element of
T
By restrictions (b) and (c),
be expressed in terms of
c.
~
7(T-l)
~m
can
as follows:
i
4T+6m-10
xim ::;:
x.
ci '
:;:: 1, 2, .•• , N
(3.61)
m ::;: 1, 2, •.. , T
From the above restrictions, we see that random variables
and
c
assuming
density
have a
one~to-one
2
(~*,~,O" )
correspondence.
x*
Therefore, instead of
as a random variable with an unknown prior
2
2
g(~*,~,O" ) , we can as well assume that
4
random variable with an unknown prior density
(c,~,O" )
g( c,~, 0"
2
)
is a
•
Cacoullus' estimates of the required multivariate densities as
given in (3.58), (3.59) and (3.60) will become
f
1
T (c.,~,S.):=: - - - - : : ; . - - - N, -p ~ -~ ~
2
P
n
(2Il )P+ N)'t.
. 1
J:=:
&..
~J
2
s. -Sk
N
I:
k=l
k}i
Sin[?r
O
[Si-Sk
2'Y J
c.-c k
] Sin[-k-J
[ci-c k ]
p
.
Sln
. )
b .. -b k J]
[~J
n
2& ..
~J
j:;;l
2t.
where
(1)
A
= 6(NlVl:.N.
~
~=
(2)
c
(3)
)'
1
- 2
(c. -c)
~
N
1
I: c
N ~=
. 1 i
and
&••
~J
are as defined in (3.58);
4 2 2
The symbol g in g(~*,~,O") and g(c,~,O") do not necessarily
represent the same functional form, it-merely indicates the prior
density function.
46
(4)
6 (N)
satisfies
lim
6 (N)
=0
and
N.... ex:>
N[ &(N) ]P+2
lim
N.... ex:>
= ex:>,
for example take
1
- -
6 (N) = N
q > p + 2 .
q, where
2
f
T
2(c.,b.,S.)
N, -p~ -~
~
1
= -----+2
P
(2rI )p N)' 6 n 6..
. 1
J=
N
+
~J
S'-S~0
sin[-¥J
Q
S.-S~
[.2:...-..2:.
J
2)'
p
.
s~n
b .. -b k . ) 2
[~J
21:V • • JJ
n
t
k=l
. 1
J=
k/=i
~J
b .. -b k ·
[~J
J]
20 ..
~J
(3.63 )
where
)',
f
6
ij
,
6
and
(1)
T
2(c.,b.,S.)
~ -~
~
N, -p-
6(N)
=
are as defined in
(3.62).
[fN , 1 , T -P- 2(c.,b.,S.),
... ,
~ -~
~
f
N,p,
T
)J'
-p- 2(c.,b.,S.
~ -~
~
where
..• , b.~p J/,S.)
- f , T -p- 2(c.,b.,S.)}
~
~ -~
~
N
for
j =
1, 2, ••• , P .
(3.64 )
EB estimators for
2
and
~.
-~
rr.~ , from (3.56) and (3.57), are
therefore given by
~
S. (X!X. )
~.
-~
~
-1
~
(1)
f N, T-p- 2(c.,b.,S.)
~ -~
~
:;:: b. + --=T=---2-=---~
-pf N, T-p- 2(c.,b.,S.)
~""""l
~
~2
S..
rr
",,--=~,::"
f
b . , S. )
N T -p- 2 ( c ~. , -~
~
(3.66)
'
T-p-2 f , T-p (ci,b.,S.)
-~
~
N
i
where
~
X. :;:: [l,x~J .
--:I.
~
The essence of restricting the
X matrix in Model D*, is actually
to make it possible to have a one-to-one correspondence between
matrix and a scalar variable
simplified.
X
c, so that the estimation is much more
Other kinds of restrictions on
X might as well be
imposed to have a one-to-one correspondence between
X matrix and a
scalar variable.
The matrix
T X (p-l).
X*, which is one-to-one to X, has a dimension of
X*
Any restriction on
one correspondence between
dimension of
X*
Y is less than
such that we can have a one-to-
and a multivariate
T X (p-l)
Y, where the
will, therefore, simplify
the estimation, in the sense that we have variables with less
dimension in the joint prior density.
Now, some arguments for choosing restriction (c), ~.~.,
x
iT
:;:: 2.5xil (i :;:: 1, 2, ••. , N) , will be given below:
Consider a uniform distribution
1
f(x) :;::
[ b-a '
o ,
for
elsewhere
48
The mean and variance of (3.67), denoted by
given by
g
x
= ~(b+a)
212
~x = 12(b-a) .
and
g
x
and
2
~x
' are
Therefore, the coefficient
of variation will be given by
~
c. v. = - x =
fJ.
x
b-a
·577350 b+a
(3.68 )
Relation (3.68) can be rewritten as
b
a
1 + l.732052 C.V.
1 - l.732052 C.V.
and is tabulated below for several values of C. V.
Table 3.l
C. V.
-ba
.05
1.2
Relation of C.V. and b/a
.20
.10
1.4
2.1
If we design the
.25
2·5
b = 2.5a.
3.2
.40
5·5
·50
l3· 9
·55
41. 2
.60
·577350
-52.0
ro
X matrix such that the elements of
uniformly distributed with
that
·30
x~
-~
are
C. V. = .25 , then from Table 3.1 we have
In practice, many independent variables used in
regression analysis in economic studies have C.V. between .20 to ,30.
When we are dealing with an independent variable whose C.V. is not
close to .25, the restriction (c) can be revised accordingly by using
the relation given in Table 3.l.
Further designing of the
X matrix
is by imposing restriction (b) to have an equally spaced elements of
x~.
-~
Recall that a part of designing an experiment is designing the
construction of the
X matrix.
4.
COMPARISON OF EMPIRICAL BAYES TO
ORDINARY LEAST SQUARES ESTIMATORS
In this chapter, a simulation study will be presented to compare
empirical Bayes (EB) to the ordinary least squares (OLS) estimators.
The OLS estimators are chosen for comparison, since by the assumptions
3.4, particularly the
for Models A*, B*, C* and D* in Section
~l' ~2'
independence of
estimators (Theil, 1971).
pt,
work with Model
..• ,
~N
' they are the best linear unbiased
For simplicity of the study, we shall only
as described in Section 3.5.
Let us rewrite Model
Dt as
follows:
= 1,
i
where
,
1:i
of lIs and
and
E.
-l
and
x.
-1.
Q'i
,
~i
E.
-1.
are
15
X
are scalars.
are as given for Model
1
(4.1)
2, ... , N
vectors, 1
is a
Assumptions about
Dl in Section 3· 5.
Q'i
15
,
1
X
~i
Since here
vector
,
-l
T
= 15
x.
from (3.61), we can write
x.
lm
(the middle element of
and
i
1, 2, ••• , N
m
1, 2, " ., 15
(4.2)
for
x. ) .
-l
In addition, for ease of programming, the random variables
and
~
2
c,
are chosen to be independent, although according to
the assumptions for model D*, it does not necessarily require so.
generation of random variables
c, Q' , ~
and
The
~2 will be described
,
50
in later sections of this chapter and several cases concerning the
form of the prior density
g(c,a,~,~
In each case, the whole set of
(experiments) will be replicated
r
2
)
will be considered.
N regression equations
= 100
times and the values of
are varied to be 2, 3, 4, 5, 10, 15, 20, 30 and 40.
replication we generate at random
N values of
N
Note that in each
(c,a,~,~
2
) ,
so that
in general, it will be different from replication to replication.
4.1
Objectives Of The Study
Bayes estimators for a mUltiple linear regression model as seen
in Section 3.3 are dependent on the prior density.
Since EB
estimators as given in Section 3.4 are actually two-stage Bayes
estimators, therefore, they will indirectly depend on the true form of
the prior density, although the knowledge of the exact form of this
prior density is by-passed in their derivation.
Therefore, we may
suspect that the magnitude of improvement for EB over OLS estimators,
if there is any, will somewhat depend on the true form of the prior
density.
This study will attempt to show that no matter what form the true
prior density was, an improvement for EB over OLS estimators will
always be gained,
~.~.,
EB estimators are at least as good as the OLS
and in some cases they are much better in the mean squared error sense.
The ratio of the average squared error, denoted by
R
ABE
ASE
EB estimator
OLS estimator
(4.3)
51
will be used as a measure of the improvement for EB over OLB
•
estimators; the smaller
of
~
2
~
~2
(a.,b.,s.), (~.,~.,~.) as the 018 and EB estimators for
Denote
2
(~i'~i'~i)
the larger improvement gained.
R
l
l
l
l
l
l
of the model given in (4.1), respectively.
~.,~.,~~
(i = 1, 2, ... , N)
l
l
l
2
The true values
are known by generation of random
2
~
~
~2
variables
~,~,~
computed.
The average squared errors for EB and 018 estimators re-
, while
(a.,b.,s.)
l
l
(~.,~.,~.)
and
l
l
l
l
can be
quired for obtaining (4.3) are computed as follows:
A8E(a)
(4.4)
r
~
N
l
= 1 Z [-N Z (~. - ~l' )2 Jk
r k=l
and similar expressions for
average squared error for
and
~
~
i=l
~
2
Hence, the ratio of the
is given by
A
A
ASE~~
R~ = A8E a
wi th similar expressions for
l
R~
and
j
(4.5)
R
~
It is also an objective of this study to investigate the relation
between
number of
N and
R.
In practice we are usually dealing with small
N, say around 10, because if
N is very large, it is im-
practical to report individual estimates for regression parameters.
In this case (if
N is large), estimates for the means and variances
of regression parameters, for example as given by 8wamy (1970), would
be desirable.
We can expect that the larger
since the larger
N, the smaller
N the better the estimation of the required
multivariate densities to obtain EB estimators.
R,
52
4.2
•
Generation of Random Variables
c, a, ~
~2
and
c, a ,
For ease of programming, the random variables
~
and
~
2
are taken to be independent, although the assumption for the model
does not necessarily require so.
density of
c, a ,
and
~
individual prior densities,
~
2
This means that the prior joint
can be expressed as the products of the
~.~.,
Several forms of prior densities such as normal, U-shaped and L-shaped
with given mean and variance will be used for the generation of
values of
Table 4.1
c, a, ~
and
~2, as described in Table 4.1.
Several forms of prior densities of
a
studied
c, a, ~
and
~2
2 b
g4(~ )
Cases
gl(C)
g2(a)
g3 (~)
1
2
3
4
5
6
7
8
9
N(10,9)
L(10,9)
US(10,9)
N(10,144)
N(10,9)
N(0,9)
N(-10,9)
N(lO, 9)
N(10,9)
N(10,9)
L(10,9)
US (10, 9)
N(10,9)
N(10,144)
N(10,9)
N(10,9)
N(0,9)
D(lO,O)
N(5, 2. 25)
L(5,2.25)
US(5,2.~5)
N(5,2.25)
N(5,36)
N(5,2.25)
N(5,2.25)
N(0,2.25)
D(5,0)
N(50,225)
L(50,225)
US (50, 225)
N(50,225)
N(50, 225)
N(50,225)
N(50,225)
N(50,225)
N(50,225)
~=Normal, L=L-shaped, US=U-shaped, D=degenerate, the numbers within
the bracket indicate the mean and variance, respectively.
bIn the case of g4(~2) is normal, since ~2 > 0 , the prior density is
taken to be
.000008 + N(50,225) ,
=[
o
for
o<
otherwise
~
2
< 100
53
The L-shaped and U-shaped distributed random variables are
derived by linear transformation of random variables
W and
Z,
whose distributions are as follows:
1
(W_33 )2
few) := (
o
,
for
,
otherwise
(4.6)
and
1
z
2
,
1
-(1.5)3 < z < (1.5)3
for
f(z) := (
o ,
Note that
respectively.
otherwise
W and
Z
are L-shaped and U-shaped random variables,
By the following linear transformations of
we can obtain an L-shaped random variable
standard deviation
and a U-shaped random variable
cry
/.LV ' standard deviation
y :=
with mean
y
W and
Z,
/.L y ,
V with mean
cr '
V
4(3
- t.
(4.8)
\(5)cryW + /.L y - crY,fr
1
5
~(3 - b\[5 )cryZ
V
4.3
+ /.LV •
Computations Of The 018 And EB Estimates
By the generation of random variables
case described in Section
.•. , N).
From
we can obtain
c. , the elements
~
can be derived by using
error component
4.2,
€.
-1
,
(4.2),
since
€.
-1
x. (m:=
1m
and also from
c,
Of,
f3
and
cr
2
2
for each
ci ,Of i ,f3 ,cri (i := 1,
i
2,
1, 2, .'0' 15) of x.
~
cr~,
we can generate the
1
is distributed
N(o,cr~I).
-1
01
a.
1
[~i] , denoted by
The 018 estimate for
[b~J, by using (3.11)
is computed as follows:
a.
[b.~J
(4.10)
~
Further, (4.10) can be rewritten as
1
15
2
15
a i = OI i + n-( E xi
E
i m=l
m m=l
=~. +
b.
1
1
1
--D (15
15
E
15
E x.
i
m=l
15
(4. 11)
E x.
E x. E. )
im - m=l
1m m=l ~m ~m
E.
1m 1m
-
15
E x.
m=l
15
E
1m m=l
(4.12)
E. )
1m
where
15
D. = 15
2
15
E x. - (E x. )
m=l 1m
m=l 1m
1
2
~~1 from (3.12) is given by
The 018 estimate for
2 1 1
s. = -T (y. - X.b. )/(V. - X.b.) = -
-p
~
where
1
Since we have
2
1
1-1
~1
(X~X.
flx!~
1 ~
M. = I - X.
~
1
T-p
1-~
and I
is a
15
2
15
E x.
m=l
TXT
identity matrix.
] , (4.13) can be expressed as
= 15, Xi = [l,x.
- -1
T
1
15
2
15
15
E x.
E.
s. = 13 C E E. - D--[ E x. (E E. )
1
m=l lm
1. m= 1 ~m m= 1 1m
- 2
(4.13 )
E!M.E.
-~ ~-~
15
E
1m m=l
E.
1m m=l
1m 1m
2
15
2
+ 15( E x. E.
) }]
m=l
where, as before,
D
i
=
15
15 2
E x.
m=l
~m
15
2
(E x. )
m=l ~m
~m
1m
.
(4.14 )
55
EB estimates for
~.,
1
2
and
~.
1
are computed, by applying
~.
1
(3.65) and (3.66) to model in (4.1), as follows:
15
- f N 2 ll(c.,a.,b.,s.) Z x. J
"
1 1 1 1 m=l 1m
~
~1' = b. + lID f
(4.15)
~
~
b S )[- f N,1 ,11(c.,a.,b.,S.)
Z x.
. N, 13 ( c.,
1 1 1 1 m=l
1m
1
1 a.,
1 1., 1.
1
+ 15 f N 2 11(c.,a.,b.,S.)J
"
1 111
(4.16)
?2
S.f 11(c.,a.,b.,S.)
~. = 1 N,
1 1 1 1)
1
Ilf , 13(c.,a.,b.,s.
1 l 1 1
N
(4.17)
where
S.
1
= Us.12 ,
15 2
15
2
D. = 15 Z x. - (Z x. )
m=l 1m
m=l 1m
1
and
f N, 13(c.,a.,b.,S.),
1 1 1 1
f N, 11(c.,a.,b.,S.)
1 111 ,
f N" 1 11(c.,a.,b.,S.),
1
1
1 1
f N,2,11(c.,a.,b.,S.)
1 1 1 1
are as defined below, by applying (3.62), (3.63) and (3.64) on the
model in (4.1).
Let
1
V
N
2
. 1
1=
1
1
N
-N[ Z c. - -N( Z c.)
. 1
1=
1
2
l
J
56
1
6
i1
=N
10
2
si
-
15
r:
D.~ m=1
x
2
im
S* - 13
i - 15 Si '
Then,
f N, 1) (c i , ai' bi' Si)
= ii
t
+
c. -c
sine
C
~6
a. -a
k]sin[
b -b
~1
S._S~)2
f
1
11(c.,a.,b.,S.)
= -K.
~
~
~
~
N,
~
S
-s
0
~6.k]sin[ ~6 k]sin[~]
sine ~ ~l
O
S.-Sf
[~
'4
~J
i2
'4
2
(4.18 )
57
(4.19)
2
f
N"
8.) =
1 lie c . , a. , b . ,
l
l
J. J.
1
K l:
i vi 1
N
+
r:
k=1
~i
a.-a k +5· 1
b.-b
k
Sin[~Jsin[ l25. l JSin[~J
~
c.-ck
II
c.-ck
.
l
[26 J
J.2
a.-a
l k +5·l 1
[ 2o
J
i1
b.-b
l k
[--25----]
i2
2
8._8*0
sin[F]
8.-8*
[l k J
'4
1
- --l:--[f
N'
V
il
11(c.,a.,b.,8.)J
l
J.
J.
J.
(4.20)
f
N"
2 11(c.,a.,b.,8.)
J. J. J. J.
=K
1
l:
i V i2
8'-8~02
sin[.5]sin[ ~ l]
(
8.
[.5J
-S~
[~ l]
58
(4.21)
a.,
In programming the simulation,
~
2?
~
s., ex., t3;.....
b.,
~
~
are
~
computed by (4.11), (4.12), (4.14), (4.15), (4.16) and (4.17),
respectively, so that we do not have to generate the values of
y ..
-~
4.4 Results
The ratios of the average squared errors of EB to OLS estimators,
denoted by
R,
ex
RR
.....
and R0- , as described in Section 4.1 and 4.2 were
first computed for cases 1, 2, and 3 (see Table 4.1). The results are
given be low in Tab le 4.2.
Table 4.2
Ratio of the average squared error of EB to 013
estimators for cases 1, 2 and 3 a
Case 1
N
2
3
4
5
10
15
20
30
40
R
R
·943
·905
.867
.832
·719
.674
.628
.587
·553
·913
.895
.862
.803
.664
.634
.600
.540
·513
ex
t3
Case 2
R
0-
1.'488
1.428
1.359
1.318
1. 222
1.147
1.107
1. 043
1.013
R
ex
·939
.884
.846
.817
·728
.681
.642
.584
·557
Case 3
R
R
t3
·925
.872
.826
.808
·705
.660
.621
·569
.544
0-
1.430
1. 313
1. 277
1. 265
1. 220
1.183
10139
10075
1. 032
aCases 1, 2, and 3 correspond to Table 4.1.
R
ex
·953
.867
.861
.826
·729
.678
.625
·577
·546
R
t3
·944
.881
.860
.819
·721
.674
.623
·575
.547
R
0-
1.636
1.355
1.318
1.306
1.122
1. 097
1. 069
1.026
·976
59
From Table 4.2, we see that the ratios
smaller than 1 and are always decreasing as
means that EB estimators for
a
and
~
and
R
a
R~
are all
N becomes larger.
This
for cases 1, 2, and 3 are
better than the OL6 in the average squared error sense.
More
improvements are gained for EB over OL8 estimators, the larger the
number of experiments
a
and
~
(N).
are notable, for
The improvements for EB estimators for
N
= 10 or larger the gain
is more than
Recall that cases 1, 2, and 3 (see Table 4.1) have different shape
of prior densities, but their respective means and variances are the
same.
It seems that the shape of prior densities of
does not affect much the ratios of
R ,
and
a
~
respective means and variances are the same.
Note from Table 4.2, however, the ratio
than 1, except in case 3 for
that EB estimator for
the ratio
R(J'
(J'2
N
R(J'
R(J'
c,
,
a,
and
~
given their
is always greater
= 40 where R(J' = .976 , which means
is worse than the OL8 for
is also decreasing as
N ~ 40 , but,
N becomes larger.
What
so that the
follows is an attempt to improve the EB estimator for
improvement will be gained for small values of
N, say for
N around
10 or even less.
The high ratio of
R(J'
in Table 4.2 (for
caused by the large variance of
~2
(J'
, or both.
~2
(J'
N ~ 40 ) might be
, or, the large marginal bias of
An estimate for the marginal bias for
~2
(J'
, if
is known, is computed as
(4.22)
60
Using (4.22) with
r = 100 , the marginal biases of
~2
cr
for
cases 1, 2 and 3 were computed and the results are given in Table 4.3
below.
Table 4.3
Estimates for the marginal bias of
and 3
~2
cr
for cases 1, 2
N
Case 1
Case 2
Case 3
2
3
4
5
10
15
20
30
40
6.10
7.4 4
8·50
7·60
9·11
9·12
B.50
8·52
7·92
5·44
6.87
6.24
7·58
7·67
8.25
8.15
7·99
7·29
5·43
6.66
6·50
6.29
4·94
6·77
7· 03
6.31
5·72
The results in Table 4.3 reveal to us that the direction of the
marginal bias for
N
~
40 is positive.
Therefore, correcting
~2
cr.
l
of
(3.66) by a constant positive mUltiple factor of less than one will
simultaneously reduce the variance as well as the marginal bias of
~2
cr
, for
N
~
40 .
Several positive correction factors less than one were examined
in the study, namely, .95, .90, .85, .80 and ·75·
selected since estimates of the marginal bias of
around 10% to 15% of
1
r
N
~2
i=l
l
-- E [E cr. J
Nr k=l
k
These numbers were
~2
cr
in Table 4.3 are
61
~2
By applying the above correction factors for
cr
, the Ratio
R
cr
was recomputed for cases 1, 2 and 3 and the results are given in Table
4.4.
Observing the values of
R
cr
of .80 gives a good result where
~2
From now on, .80 cri
p
= 15
R is less than one for
cr
~2
(4.1), instead of cr.
~
that this correction factor of
T
4.4, a correction factor
will be used as an EB estimator for
simulation of model
where
in Table
~2
cr
as given in
N
2
~
3 .
cr
i
for the
(4.17).
Note
is recommended only for model
(4.1),
(the number of observations in the ith experiment) and
= 2 (the number of independent variables including the dummy
variable, which is the same as the number of regression coefficients
including the intercept).
Remarks:
~2
Note that the correction factor of
cr
of .80 (Table
4.4) was chosen since it gave the smallest ratio Rcr (among all
correction factors considered) for cases 1, 2 and 3. However, this
2
~2
corrected EB estimator for cr , i'~"
.80 cr , seems to give a downward marginal bias as a trade off for small variance.
.80 ~2
cr
for cases
marginal biases of
Reca11
.
ag~n,
th a t th e Bayes es t·~mat or f or
2
cr
2(c.,b.,S.)
~ -~
~
T-pf T (ci,b.,S.)
~
~
N, -p
2 , denoted by
Acr2,
cr
~2
cr
(EB
, which is a two-stage Bayes estimator) must be due
to the estimation of the multivariate densities
f
3 are given in Table
Therefore, the marginal bias for
is marginally unbiased.
estimator for
1, 2 and
Estimates of
of
and
f
T-p
(c.,b.,S.)
~
-~
~
and
(3.62) and (3.63) whose estimates are denoted by
f
T 2(c.,b.,S.).
~ -~
~
N, -p-
These multivariate
densities and their estimates, as indicated by the subscript of
depend on the degrees of freedom of
S, i.~.,
f,
(T-p), therefore, we
62
Table 4.4
N
The ratio R for cases 1, 2 and 3 after applying
~
~2
several correction factors (C. F.) for ~
C. F. =1. 00
C.F~=·95
C.F·=·90
C.F.=.85
c.F.=.80
C.F·=·75
1.091
·987
.871
.848
·761
·717
.692
.655
.642
1. 045
·943
.814
.801
' ·709
.668
.653
.627
.620
1. 041
·951
.809
.810
·707
.668
.664
.649
.649
1. 035
.898
.874
.850
.800
·751
·710
.677
.662
·979
.839
• 8~7
·791
·748
.697
.661
.639
.631
.961
.821
.822
·774
·739
.688
.660
.647
.647
1.286
·937
.899
.885
.807
·741
·701
.675
.657
1. 250
.899
.862
.842
·798
·718
.675
.656
.646
1. 254
0911
.876
.849
.837
.742
.698
.686
.685
Case 1
2
3
4
5
10
15
20
30
40
1. 488
1.428
1.359
1.318
1.222
1.147
1.107
1. 043
1. 013
1·313
1. 229
1.144 .
1.106
1.018
·956
·919
.863
.839
1.181
1.082
·981
·950
.864
.812
·781
.734
·715
Case 2
2
3
4
5
10
15
20
30
40
1. 430
1.312
1.277
1.265
1. 220
1.183
1.139
1.075
1. 032
1. 261
1.134
1.093
1. 093
1.036
·994
·949
.895
.862
1.129
·996
.962
·951
.896
.850
.806
·763
·739
Case 3
2
3
4
5
10
15
20
30
40
1.639
1.355
1.318
1.306
1.122
1. 097
1.069
1. 026
·976
1.479
1.165
1.127
1.117
·969
·931
.898
.860
.821
1.362
1. 025
.988
.976
.864
.813
·775
.743
·715
63
Estimates for the marginal bias of
2 and 3
Table 4.5
N
2
3
4
5
10
15
20
30
40
~2
(j
for cases 1,
Case 1
Case 2
Case 3
-5·12
-4.05
-3.20
-3·92
-2·71
-2·71
-3·20
-3.18
-3.66
-5.64
-4.51
-5·01
-3.94
-3.86
-3.40
-3.48
-3.60
-4.17
-9·66
-4.()8
-4.80
-4.97
-6.05
-4.58
-4.38
-4·95
-5·42
simulation for case 1 by varying
~2
(T-p).
A
(T-p) -- in this case by varying
T
can expect that the marginal bias of
only and fixed
.80
(j
also depends on
p = 2 -- was conducted with N = 10
Computing the average of
~2
(j
(j
1
r = 200 •
as
A
:2
and
N
r
= -- t
[t
~2
(j]
Nr k=l i=l i k
and the estimate of the marginal bias of
~2
(j
as given in (4.22), we
obtain the results as presented in Table 4.6.
From colums (1) and (3) of Table 4.6, we see that the marginal
bias of
~2
(j
is inversely related to the number of degrees of freedom
(T-p) , which suggests us to regress column:(3) on the inverse of
column (1).
Regressing the marginal bias of
A
....
1 -2
(j
T-p
gives the following relation:
~2
(j
on
64
Table 4.6
The average of EB estimates for ~2 (uncorrected) and
estimates of its marginal bias for case 1, where p = 2
and N = 10
(1)
(2)
(3 )
(4)
T-p
-2
~
Bias (~2)
1.77 -2
--~
T-p
7
9
11
65· 85
62.01
62.59
58.10
55.46
54.01
54.31
55·83
55.44
15· 85
12.01
12·59
8.10
5.46
4.01
4.31
5.83
5.44
16.66
12.19
10.07
7·91
6.55
5.63
...
A-
13
15
17
19
21
23
A
A-
5.06
4·71
4.27
AA-
1·77 ~2
T-p
The values of
For removing the
m~rginal bias
~2
EB estimator for
~
~2
Note that for
of
~2,
by relation (4.23), the
T-E-l.77 , i.~., the corrected
T-p
is given by T-~-1.77 &2 .
correction factor for
comes
are given in column (4) of Table 4.6.
should be
-p
T
=
15
and p
=
2 , this correction factor be-
.86. The discrepancy of the new correction factor (.86) to the
old one (.80) can be explained as follows.
The correction factor of
.86 will almost remove the marginal bias but give a higher variance of
EB estimator for
~2 compared to that with correction factor of .80.
In general, the correction factor
~2
~
such that the
margin~l
an EB estimator for
T-;p-l·77
T-p
can be used for correcting
bias is mostly removed, however to obtain
~2 with a lower ratio R , the correction factor
must be slightly smaller.
~
End of remarks.
Applying the correction factQr of .80 for
~2
~
, the ratio of the
average squared errQr of EB to 018 estimators, denoted by R ,
a
R~
and
R~,
were computed for cases
~,
5, 6, 7, 8 and 9 (See Table 4.1) as
outlined in Section 4.1 and 4.2 and the results are given in Table
4·7·
Table 4.7
The ratio of the average squared error of EB to OLS
estimators for cases 4, 5, 6, 7, 8 and 9a
Case 4
01
~
·952
·920
.888
.863
·730
.679
.621
.577
.537
·707
.607
·703
.508
.453
.435
.435
.316
.314
N
R
2
3
4
5
10
15
20
30
40
R
~
1.040
·950
.820
.807
·721
.680
.664
.637
.630
·958
·921
.851
.807
·743
.691
.647
.604
·570
10
15
20
30
40
01
~
.978
·979
·971
·975
·943
·927
.918
.896
.885
·958
·964
.964
·957
·953
·928
·930
·903
.894
R
R
~
1.075
.986
.864
.861
·784
.749
·736
·708
.699
Case 8
Case 7
2
3
4
5
Case 6
Case 5
1. 043
·920
.806
.766
·750
·709
.637
.580
.543
•!t26
·946
.804
·793
·705
.671
.658
.633
.626
·943
·905
.867
.832
·719
.674
.628
.587
.553
01
~
.898
.826
·764
·715
·557
.499
.455
.403
.375
.683
·590
.544
.473
.311
.272
.094
.038
.033
R
R
~
1. 042
·926
.800
·785
.696
.656
.643
.616
.614
Case 9
·913
.895
.862
.803
.664
.634
.600
.540
.513
1. 045
·943
.814
.801
·709
.668
.653
.627
.620
.871
·775
·705
.649
·527
.471
.428
.384
.359
.872
·768
·703
.633
.489
.447
.412
.352
.340
1.024
·922
·785
·773
.680
.644
.632
.607
.604
aSee Table 4.1 for the description of cases 4, 5, 6, 7, 8 and 9·
Comparing the values of R
OI
,
R~
from Table 4.2 wi th
R~
taken from Table 4.4 (C.F. = .80) for cases 1, 2 and 3 to those values
of
R
01,
FL
~
and
R
~
in Table 4.7 for cases 4, 5, 6, 7, 8 and 9,
several results on the next page can be observed.
66
(a)
The shape of the prior densities of
and
~
~
2
R ,
a
does not affect much the ratios
and
R~,
~
c, a,
given their respective means and
variances are the same (comparisons of cases 1,
2 and 3.
That is why the shapes of prior
densities for other cases were, then, taken to
be normal).
(b)
The higher the variability of c -- which represents the
X matrix
better EB estimator for
affect much the ratios
mean of
a,
c
2
~,~
among experiments, the
~
R
a
but it does not
and
R , given the
~
and the respective prior densities of
are the same (comparison of case 1 to
case 4).
(c)
The smaller the absolute value of the mean of
c--
which represents the absolute magnitude of the
X
matrix -- , the better EB estimators for
expecially for
ratio
R~,
~,but
a
and
it does not affect much the
given the variance of
c
and the
respective prior densities for
same (comparisons of cases 1, 6 and 7).
are the
The
reason for this is that when the magnitude of the
elements of the
X matrix is small, the magnitude
of the elements of
(X/x)-l
will be large, which in
turn makes the variance of an L8 estimator large.
That is why the ratio of the average squared
error of EB to that of an OLS (which reflects
its variance, since OLS estimators are unbiased) estimators becomes smaller.
(d)
The ratios
and
R
CT
seem to be un-
affected by the change of the prior means of
and
,
given the variances of
01
the respective prior densities for
c
01
~
,
~
and
and
CT
2
are the same (comparison of case 1 to case 8 ).
{~)
The smaller the variances of
better EB estimators for
01
01
and
and
~
respective prior densities of
the same.
c
~.~.,
,
and
,
the
given the
CT
2
are
The best EB estimators will be
obtained when the variances of
zero,
~
when
Of
and
~
01
and
are
~
are actually fixed
(comparisons of cases 1, 5 and 9).
Results of the simulation for the generated values of
(represented the
2
(OI,~,CT)
X
mat~ix),
the generated parameter values of
with their respective 018 and EB estimates, taken from one
replication of cases 1 through 9 for the special case of
T
= 15
c
, are presented in the Appendix.
N
= 10
,
Since the true parameter
values are known, we can see from the Appendix the direction as well
as the magnitude of improvement of each of EB over its corresponding
OLS estimate.
68
5.
APPLICATIONS TO SOME PROBLEMS
IN ECONOMETRICS
In economic studies, the nature of experiments described by Models
A*, B*, C* and D* of Section
3.4 might be relationships of micro-unit
variables such as firms, farms, households, etc., observed over a
period of time, or it might be relationships of regions with
observations over a time series or over several sub-units in each
region.
The estimate of the U.S. consumption function (a distributed lag
model), for example, as obtained by Griliches et al.
Zellner and Geisel
(1962) or by
(1970) might still be improved if we incorporate
the data from several other countries.
Similarly, if the constancy
of regression coefficients over a long time period is doubtful, we
might break down this series into several sub-series and assume that
the regression coefficients are constant within a sub-series but they
are random across sub-series.
By assumptions as described in Section
3.4, empirical Bayes estimates for regression coefficients in each
sub-series will be better than ordinary least squares.
Two examples will be given on the next pages to illustrate the
applications of empirical Bayes approach in economic studies.
5.1 Investment Analysis
Grunfeld
(1958) constructed a multiple linear regression model to
study the determinants of corporate investments.
variable I, the independent variables
model are described as follows:
F_
l
and
The dependent
C_
l
used in the
I
~
gross investment
additions to plant and
~
equipment plus maintenance and repairs in
millions of dollars, deflated by
F_
l
~
value of the firm
= price
l
;
of common and
preferred shares at December
price of December
P
31 (or average
31 and January 31 of the
following year) times number of common and
preferred shares outstanding plus total book
value of debt at December
dollars deflated by
C_
l
31 in million
P , lagged one year;
2
= the stock of plant and equipment =
accumulated sum of net additions to plant and
equipment deflated by
P
allowance deflated by
P
l
3
minus depreciation
, lagged one year;
where
P
l
= implicit
price deflator of producers durable
equipment (base
P
2
1947);
= implicit price deflator of G.N.P. (base 1947);
P3. = depreciation expense deflator = ten years
moving average of wholesale price index of
metals and metal products (base
1947).
The description of the above variables and the time series data
(1935-54) were also given in an article by Boot and de Wit (1960).
70
By assuming Model D* of Section 3.4 for these investment data,
where we have
= 10,
N
20
T
and p
= 3 , EB estimates for
regression coefficients and error variance are computed by using
(3.56) and (3.57).
The results, as well as the OLS estimates, are
presented in Table 5.1 below.
Table 5.1
Corporation
1
2
3
4
5
6
7
8
9
10
OLS and EB estimates for regression coefficients and
error variances for the ten corporations a
OLS estimates
Inter- Coeff. Coeff.
cept.
of F_ of C_
l
l
-149.78
-- 49·20
9·96
- 6.19
-
22·71
8.69
4.50
0·51
7·72
0.16
.1193
.1749
.0266
.0779
.1624
.1315
.0875
.0529
.0753
.0046
.3714
.3896
.1517
.3157
.0031
.0854
.1238
.0924
.0821
.4347
s
2
8423.87
9299.60
777·45
176.32
82.17
65.33
88.67
104.31
82.83
1.18
Intercept
EBestimates
Coeff. Coeff.
of F_ l of C_ l
.1205
.1638
.0310
.0786
22.26 .1646
- 7·25 .1172
- 3·01 .0854
- 3.17 .0579
- 7.94 .0725
0.03 .0071
-157·28
- 16.81
- 15.60
- 6.40
.3670
.3572
.1453
.3128
.0028
.1291
.1198
.0829
.0859
.4292
.. 2
cr
8311.60
9134·70
777·32
176.32
82.17
65.33
88.69
104.33
82.84
1.18
~he ten corporations are: (1) General Motors, (2) U.S. Steel,
(3) General Electric, (4) Chrysler, (5) Atlantic Refineries,
(6) I.B.M., (7) Union Oil, (8) Westinghouse, (9) Goodyear, and
(10) Diamond Match.
On the other hand, there is another different approach due to
Zellner (1962) for analyzing these data of ten corporations, based on
the following model:
y.
-].
where
= X.".
].-].
+
E.
-].
,
i
= 1,
2, ••. , N
(5·1)
71
(a)
~i (i = 1, 2,
(b)
x.1 (i = 1, 2,
... , N)
... , N)
is fixed;
is non-stochastic
with full rank;
(c)
E (e. )
-J.
= -0 for i = 1, 2,
... ,
N
,
i, j
= 1, 2,
E(e.en
=
l-J
eJ •• 1
lJ
for all
and
... , N .
Applying Zellner's two-stage least squares on the above data from
ten corporations, we obtain estimates for regression coefficients as
given in Table 5.2 below.
Table 5.2
Zellner's two-stage LS estimates for regression coefficients for the ten corporations
Corporations
Intercept
1.
2.
3.
4.
5·
6.
7·
8.
9·
10.
-133·00
- 18.60
- 11.20
2.45
26.50
5.56
9.67
4.11
2.58
2.20
G.M.
U.S. Steel
G.E.
Chrysler
Atl. Ref.
I. B.M.
U. Oil
West.
G. Year
D. Match.
Coefficient
of F_
l
.1130
.1700
.0332
.0672
·1310
.1310
.1120
.0525
.0760
-.0181
Coefficient
of C_
l
.3860
.3200
.1240
.3060
.0102
.0571
.1280
.0412
.0641
.3650
Zellner (1963) also claimed that only if the absolute value of
the true correlation coefficient of error components from two
different experiments,
~.~.,
72
is in the neighborhood of zero and/or
(T-2p)
is small, then the OLS
estimators are slightly more efficient than the two-stage least squares;
more efficiency of the two-stage LS will be gained the higher
and/or
(T-2p).
XiX.
2 J
most when
Ipl
The efficiency over the OLS estimates will be gained
= ~0
for all
i
Fj
.
EB estimators, therefore, are not comparable to Zellner's twostage 18 since they are based on different assumptions about error
components; EB estimators are superior over the 018 in the case where
p =0
and
N (the number of experiments) is large enough
while Zellner's two-stage LS does not depend on
efficient than the 018 when
(~5),
N and it is more
p is not close to zero.
At this point,
however, a study can be conducted to investigate the robustness of EB
estimators when
p
F0
and compare them to Zellner's two-stage least
squares estimators.
5.2
Elasticities of Substitution
018 estimates for elasticities of substitution in the demands of
the selected importers between flue-cured from the United States and
tobacco from other sources were given by Capel (1966) using annual
data for 1955-1964.
He constructed the following linearized model.
where
A,B
= quantities
of United States flue-cured and other
tobacco imported by a country in a particular
year, respectively;
73
Pa'Pb
= their
prices;
A
= elasticity
k
= intercept;
fJ.
= random
of substitution between A and B;
error.
By assuming Model D* of Section 3.4, where we have
T = 10
and
N=9 ,
p = 2 , EB estimates for the elasticities of substitution
for the nine countries are presented in Table 5.3, along with their
OLS estimates.
Table 5.3
018 and EB estimates for elasticities of substitution in
demand between flue-cured tobacco from the United
States and tobacco from other countries a
i~ort
Importing
Countries
018 estimates
Uni ted Kingdom
West Germany
Netherlands
Belgium
Japan
Egypt
Ireland
Denmark
Sweden
~he data were taken from Capel
-2.47
-3.57
-1.47
-5·15
-0.81
-2.19
-0·99
-1.04
-0.26
EB estimates
-2.463
-3.581
-l. 588
-4.257
-0.825
-1.889
-l. 002
-1.805
-0• .l38
(1966), which sources are: Egypt:
Uni ted Arab Republic tobacco report, unpublished report, Foreign
Agricultural Service, Cairo, Egypt, 1965; other countries: United
Nations (1956-1965).
6.
SUMMARY, CONCLUSIONS AND RECOMMENDATIONS
6. 1 Summary
Consider the following mUltiple linear regression model:
E.
f3.
+ -J.
-J.
'D<p p<l
'lXl
X.
J.
,
i
= 1,
2, ••• , N
where
(a) .§.l' .§.2'
..., .§.N
are independent and
2
is distributed as N(O, cr. I)
-
(b)
2
f3 i ,cr.
-
J.
E.
-J.
J.
are fixed parameters and
X.J.
is a
non-stochastic full rank matrix.
The ordinary least squares estimator for
f3. , denoted by
-J.
is the best linear unbiased and
2
1
s.J. = T-p (y.J. - X.J.-J.
b. ) I (;Z. - X. b. )
J.
J.-J.
is the minimum variance unbiased estimator for
By assuming that the coefficient vector
residuals
2
cr
or the
2
cr. •
J.
~,
or the variance of
X matrix or some combinations of them to be
random across units, but fixed within units, four variants of mUltiple
linear regression models were constructed.
When the
~
priori
distribution is assumed known, the Bayes estimation for each model was
75
presented.
~
However, when no assumption is made about the form of the
priori distribution, a two-stage Bayes estimator
which is called
the empirical Bayes estimator -- was developed for each model.
A comparison of empirical Bayes to the ordinary least squares was
performed by means of a simulation study using a mUltiple linear
regression model; and,- finally, some applications of the method in
econometrics were pointed out.
It was found that for all cases stUdied, the empirical Bayes
estimators are always better than the ordinary least squares, in the
sense that their corresponding average squared errors are smaller, no
matter what was the true form of the unknown joint prior density of
x,
~
6.2
and
0-
2
Conclusions and Recommendations
Relationships, in this case multiple linear regression, of micro-
unit variables such as firms, farms, households, regions, etc., might
be considered as a random sample from a larger popUlation.
the case, the independent variables and
~arameters
If this is
can be treated as
random variables across micro-units, but they are fixed as a
realization of a random variable over
T
successive observations with-
in a micro-unit.
Empirical Bayes estimators for
~
and
2
0-
,
for the simple
linear regression, are always better than the 018 in the sense that
their average squared errors are smaller.
The ratio of the average
squared error for EB to OLS estimators is practically not affected by
different forms of the joint prior density of
X, a,
~,
and
(J'
2
given their mean and variance are the same.
More improvement for EB estimators will be gained over the OLS,
the smaller the prior variance of a
and/or
~,~.~., the closer the
random regression model to a fixed (parameter) regression model.
Similarly, the higher the variability of the
X matrix across micro-
uni ts, the more improvement gained for EB over the OLS estimators.
This is the case often found in practice, when we are dealing with
different sizes of micro-units such as firms, farms, regions,
countries, etc., where most likely the variability of independent
variables across micro-units is high.
Here, the EB estimation is
mostly recommended.
EB estimators are better than the 018, when the error components
across, as well as within, the micro-units are independent.
The
robustness of EB estimator when the error components are correlated
need to be investigated further, in this case the appropriate
comparison is to the Zellner's two-stage least squares instead of the
ordinary least squares estimators.
Different types of functions other than the one used in
(3.33) or
even different multivariate density estimators other than that of
Cacoullus might be investigated to obtain better EB estimators than
those outlined in this thesis.
Throughout this thesis, only the point estimation is discussed.
The EB interval estimation as well as testing of
hypotheses for
random regression models are recommended for future research.
77
7.
1.
REFERENCES
Boot, J.C.S. and S.M. de Wit (1960). Investment demand; an
empirical contribution. International Economic Review,
1:3-30.
,
.
2.
Cacoullus, T. (1966). Estimation of a multivariate density.
Inst. Statist. Math. (Tokyo, Japan), 18:174-183.
3.
Capel, R. E. (1966). An analysis of the export demand for
United States flue-cured tobacco. Ph.D. Thesis, Department
of Economics,
N.C. State University, Raleigh, N.C.
Univ. Microfilms, Ann Arbor, Mich.
4.
Chetty, V. K. (1968). Pooling of time series and cross section
data. Econometrica, 36:279-290.
5·
Chetty, V. K. (1971). Estimation of Solow's distributed lag
models. Econometrica, 39:99-117.
6.
Clemmer, B.A. and R. S. Krutchkoff (1968). The.,use of empirical
Bayes estimators in a linear regression model. Biometrika,
Ann.
55: 525-534.
7.
Graybill, F.A. (1961). An Introduction to Statistical Linear
Models. Vol. 1. McGraw-Hill Book Co., Inc., New York, N.Y.
8.
Griliches, z., G. S. Maddala, R. Lucas and N. Wallace (1962).
Notes on estimated aggregate quarterly consumption functions.
Econometrica, 30:491-500.
9.
Grunfeld, Y. (1958). The determinants of corporate investment.
Unpublished Ph.D. Thesis, Department of Economics, Univ. of
Chicago, Chicago, Ill. Univ. Microfilms, Ann Arbor, Mich.
10.
Jeffreys, H. (1961). Theory of Probability.
Clarendon Press, Oxford, England.
11.
Kendall, M. G. and A. Stuart (1961). The Advanced Theory of
Statistics. Vol. 2. Hafner Publishing Co., New York, N.Y.
12.
Klein, L. R. (1953). A Textbook of Econometrics.
Peterson and Co., Evanston, IL~nois.
13.
Krutchkoff, R. G. (1967). A supplementary sample non-parametric
empirical Bayes approach to some statistical decision
problems. Biometrika, 54:451-458.
Third Edition.
Row,
14.
Kuh, E. (1959). The validity of cross-sectionally estimated
behavior equations in time series applications. Econometrica,
27:197-214.
15.
Maritz, J. S. (1969). Empirical Bayes estimation for the Poisson
distribution. Biometrika, 56:349-349.
16.
Martz, H. F. and R. S. Krutchkoff (1969). Empirical Bayes
estimators in a multiple linear regression model. Biometrika, 56:367-374.
17.
Nerlove, M. (1965). Estimation and Identification of CobbDouglas Production Functions. Rand McNally and Company,
Chicago, Ill.
18.
Robbins, H. (1955). An empirical Bayes approach to statistics.
Proc. Third Berkeley Symposium on Math. Stat. and Prob.,
1:157, Berkeley, Calif.
19.
Robbins, H. (1964). The empirical Bayes approach to statistical
decision problems. Annals of Math. Stat., 35:1-20.
20.
Rutherford, J.R. and R. S. Krutchkoff (1969).
optimality of empirical Bayes estimators.
€ -asymptoti c
Biometrika,
56 :221-223.
21.
Samuel, E. (1963). An empirical Bayes approach to testing
certain parametric hypotheses. Annals of Math. Stat.
34 :1.370-1.385·
22.
Swamy, P.A.V.B. (1970). Efficient inference in a random coefficient regression model. Econometrica, 38:311-323.
23.
Theil, H. (1971). Principles of Econometrics.
Sons, Inc., New York, N. Y.
24.
Tiao, G.C. and A. Zellner (1965). Bayes' theorem and the use of
prior knowledge in regression analysis. Biometrika,
John Wiley and
51:219-230.
25.
Zellner, A. (1962). An efficient method of estimating seemingly
unrelated regressions and tests for aggregation bias. J.
Amer. Stat. Assoc., 57:348-368.
26.
Zellner, A. (1963). Estimators for seemingly unrelated
regression equations; some exact finite sample results.
Amer. Stat. Assoc., 58:977-992.
J.
79
*
27.
Zellner, A. (1966). On the aggregation problem: A new approach
to a troublesome problem. Report
6628, Center for
Mathematical Studies in Business and Economics, University
of Chicago, Chicago, Ill.
28.
Zellner, A. and M. S. Geisel (1970). Analysis of distributed
lag models with applications to consumption function
estimation. Econometrica, 38:865-888.
29.
Zellner, A. and C. J. Park (1965). Bayesian analysis of a class
distributed lag models. Econometric Annual of the Indian
Journal, 13:432-444.
80
e,
8.
APPENDIX
e
e
e
Table 8.1
Generated values of c,~,~, ~2 and their OLS and EB estimates, taken from one
replication, where N = 10, T = 15, for cases 1 through 9
OLS estimates
Experiments
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
c
~
~
2
a
~
b
EB estimates
s
2
~
ex
""
~
,,2
"~ a
2
Case 1:
c ~ N(10,9), ~ ~ N(10,9), ~ ~ N(5,2.25), ~ ~ N(50,225)
11.60
5.84
9.45
13.31
7·29
7·55
6.85
-9.67
8.40
11.37
10.41
10.83
Case 2:
c ~ L(10,9), ~ ~ L(10,9) ~ ~
10·52
9·12
6.98
10·93
10.36
10·73
8.38
8.42
14.56
18.65
9.36
12·57
7.61
8.25
12·51
9.84
7·17
6.61
13.27
7·52
8.~2
1l·50
5.54
9·41
12·93
10·52
9·36
5.89
3.39
3·90
7·47
7· 54
5·77
4.67
2.59
.3.03
6.80
2.04
4.88
7.41
4.79
7·71
6.68
5.45
4.18
3·70
9·02
7·13
68.22
69.09
46.58
33.89
36.82
48.06
58.58
36.25
51.66
26.06
58.14
69· 78
55·58
36·78
33·90
59.46
43.59
60.19
61.12
62.14
13.84
2.02
11·55
15·55
-1·57
9·22
20.03
0.44
18.61
10.88
3.05
5.54
7·15
7.34
6.86
4.42
2.04
4.10
5·55
1.70
2
L(5,2.25), ~ ~
12.02
25.67
9·98
5·03
23.18
12.22
3.54
0.22
11. 23
-3·97
4.70
6.04
4.46
7·91
5·76
5.30
4.43
4.22
9·11
7· 79
97.56
60.50
41. 95
44.84
45.28
72.41
63.35
47·54
53·90
29·77
3.94
12.36
15.36
2.35
8.53
14.63
1. 01
16.09
10·95
3.19
5.22
7.06
7·35
6.33
4.50
2.80
4.05
5.83
1.10
77·13
55·40
40.22
41.81
43.68
63.35
60.11
48·57
51.32
30.26
11. 29
24.20
9·43
4.13
22·75
12.13
4.04
1.68
12.08
-3.41
4.76
6.17
4.53
7·99
5·79
5.30
4.38
4.06
9.04
7.76
25·25
56.01
25·10
33.40
32·57
79·60
66.16
20.36
46.62
22·79
12.12
L(50,225)
2Y.·94
59.66
24.88
35.22
32.34
94.24
72.40
20.17
50.08
24.05
():)
~
e
e
e
Table
8.1. (Continued)
OLB estimares
Experiments
c
Case
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
3:
c ~
US(10,9),
11.85
6.54
13·52
13.76
13.47
11.65
9·27
6.50
7·19
12·,79
6·73
6.47
13.66
8.15
7· 53
-6.79
13· 76
13.32
6·96
6.82
Case
t3
Of
4:
8.33
-0·55
14.24
4.07
13·79
-8·99
31.10
-22.04
24.52
9·06
C N
~
Of
6.63
3.09
6.84
3.82
6.85
3.26
3.86
3.24
3·33
6.85
N(10,144),
9·90
7.66
13.67
8·07
11.10
9·33
9·20
7.86
7·37
12.16
0"
2
US(10,9),
65.25
67.75
61.68
64.12
63.67
66.52
65.96
38.18
35·72
36.25
Of
5.04
2·92
6.01
4.13
5·25
7·14
2.84
4.97
6.42
1. 83··
~
b
a
t3
~
US(5,2.25),
EB estimates
s
2
"-
"Of
50.08
71.16
45·96
36.61
64·76
65·77
50.20
52.34
76.11
37· 97
4·96
14.48
12·94
-2·70
10·55
3.32
-5·15
6.28
-0.69
18;21
5.63
6.07
6.81
5·29
6.88
3.17
4·93
6.93
1.31
~2
0"
2 ~ US
(50, 225 )
6.06
2.64
6·73
4·99
6.03
3.19
4.25
3.66
4.02
8.13
16~65
"-
t3
0"
75.36
15·23
61. 29
8.49
- 83.70
17·68
72.13
7.49
33.84
17.68
68.26
7·51
60.36
3·51
2.09
19·76
0.10
20.38
-1·75
2.48
4.24
26.69
N(10,9), ~ ~ N(5,2.25), 0"2 ~ N(50,225)
14·93
12·91
16.99
6.51
18·96
10.31
3.28
2.94
"-
39.69
60.28
34.60
36.66
94.68
72.25
29.84
24.62
149,70
30.23
4.35
9.54
11.68
0·52
6.63
0.04
-5·13
6.56
1.31
17·91
5·99
3·30
6.67
4.87
6.18
3·59
4.24
3·72
3·77
7·88
67,61
57·53
74.44
65.37
35.47
62.48
56.08
19·74
20.64
26.09
5·70
7·99
6.15
6.04
5.56
6.52
3.16
4·94
6.84
1.33
39·10
57·38
34.05
35·83
87"27
68.54
28.20
23.82
129.68
28.64
OJ
f\)
e
e
Table 8.1
e
(Continued)
OL8 estimates
Experiments
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
c
Case 5:
c ,..,. N(lO,9),
9·74
17·83
10·55
9·92
9·64
9·99
11. 50
12.69
11.67
8·74
l7·05
11.68
-3.66
27· 93
-6.21
23.83
8.00
l8·93
7·88
Case 6:
c ,..,. N(0,9),
-4.03
0.85
1. 84
-3.08
1.35
-1. 53
-3· 53
-3· 92
-1. 80
6.46
9·53
13.32
8.74
15.06
~8·98
6.34
13.87
12.49
13.30
13· 25
9·15
2
f'
a
a
(J"
Of "'"
-0.14
1. 97
10.12
18.68
13.09
3.38
loll
-0·37
5·75
2·79
N(10, 144),
74.82
70.89
59·95
29·09
60.39
22.64
25.25
44.38
41.25
62.61
ex ,.., N(1O,9),
l.19
4.64
7·29
6.05
3·00
6·30
6.35
5·45
3.14
5·01
36.24
23.20
29· 72
26.33
43.83
45.05
33·76
70.80
47·35
40.46
EB estimates
s
b
f' "'" N(5,36 ),
24.16
ll.14
13.29
1. 40
32.43
-5· 75
25·03
12.27
21+.34
4.c23
'-0.80
2.17
10.24
18.21
12·78
3.17
0.96
-0·75
5.36
3.45
t5 ,.., N(5, 2.25),
12.05
25·10
4.26
11.34
14.09
17· 98
19·36
11. 74
8.83
10.11
1. 74
-8.61
9·94
7·13
-2.64
7·77
7·45
4·94
1.65
5·ll
(J"
2
,..,
2
""Of
""-
f'
~2
(J"
N(50,225)
44.89
21·90
68.81
10.80
12.80
68.98
22.'30
1.38
56.51
32.48
-4.48
36.30
20.49
24·93
13.28
39· 75
24.28
19·48
84.88
4.78
2 ,..,. N(50,225)
(J"
-0·58
2.18
10.29
18.20
12·77
3· 05
0·96
-0.82
5.36
3.38
41. 70
63.23
63.45
21. 05
52·75
37.16
19.40
37·51
18.41
74.14
12.12
20.17
8.16
9·89
11. 73
14.45
15.68
ll.41
9·68
9·56
1. 77
-3.00
7·87
6.66
-0·95
5·54
6.45
4.86
2.11
5·19
32.03
30·98
31. 54
35.86
46.46
46.81
42.16
45.00
45·55
39.66
29·32
29·05
20·51
34.28
56.57
59·20
44.25
52.31
53.13
41. 97
OJ
\jJ
e
e
e
Table 8.1
(Continued)
018 estimates
Experiments
c
Case 7:
I
i
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
-9·05
-13· 83
~14.76
-10.85
-8.02
-14.29
-13.03
-7·25
-11. 96
-9·65
~
ex
C "'"
0-
N(-10, 9),
11.36
9·35
11.33
5.89
12.63
3.86
4.68
11.32
6.81
13·96
c "'" N(10,9),
10·95
6.17
5·24
9·16
11. 98
5·71
6·96
12·75
8.04
10.35
1.36
-0.65
1.33
-4.10
2.63
-6.14
-5.32
1.32
-3.19
3·96
66.06
48.53
32.03
47·58
43.10
55·06
62.45
32.87
40.87
38.37
ex ,.,. N(0,9),
-2.42
0.65
-1.89
-3· 09
-0·27
1.34
-2.18
-3.37
0·54
1·39
66.06
48.53
32.03
47·58
43.10
55.06
62.45
32.87
40.87
38.37
b
a
~
ex "'" N(10, 9),
2·58
5.65
3.10
1. 90
4·73
6.34
2.82
1..63
5·54
6.39
Case 8:
2
"'" N(5,2. 25),
26
5·58
9·32
18.22
13.80
0·98
5·32
21. 94
8.14
8.77
-1.
~
1. 76
5.43
3·02
3·30
5.04
6.01
2.81
2·76
5· 71
6.10
,.,. N(0,2.25),
-11. 26
-4.42
-0.68
8.22
3.80
-9·02
-4.68
11. 94
-1. 86
-1. 22
-1. 74
1.16
-1. 65
-4·74
-0.48
2.16
-2.18
-4.01
0.29
1.67
EB estimates
s
0-
""
ex
2
2
,..,
""
~
~2
0-
N(50, 225)
16.00
-1. 21
47·16
5·52
17·07
9·04
52.80
16.52
12.16
27·98
54.48
-0.07
81. 42
6.39
21. 01
35·20
6·72
26·77
43·97
5·95
2
0- ,... N(50,225)
16.00
47·16
17· 07
52.81
27· 98
54.48
81. 42
32.50
26·77
43·97
-10.25
-3.69
-2.45
6.21
2.45
-6.06
-6.05
12.16
-2·58
-1. 04
1. 77
5·43
3·00
3.15
4.85
5· 93
2.89
2.65
5.60
5·82
15· 55
43.59
17· 95
49.42
27·33
49.18
69.15
34.19
26.64
40.86
-1.83
1. 04
-1.32
-4·53
-0.38
1.65
-1. 98
-4.04
0.37
1.64
15.84
44.00
17·28
48.56
26.38
49·49
68.00
33·90
26.15
41.31
(»
+'"
-
•
Table 8.1
(Continued )
QLS estimates
Experiments
1
2
3
4
5
6
7
8
9
10
e
•
c
ex
2
t3
Case 9:
c --- N(10,9),
16.25
12.14
17.47
9·58
10.02
0.81
12·77
13.61
13.15
13.35
10.0
10.0
10.0
10.0
10.0
10.0
10.0
10.0
10.0
10.0
5·0
5·0
5·0
5·0
5·0
5·0
5·0
5·0
5·0
5·0
ex
=
10, t3
68.49
36.95
20·74
45·03
56.02
55· 27
36.01
51. 85
42.52
46.88
~his is a corrected EB estimate using C.F.
b
a
(J"
=
5,
cr2 ___
17· 72
3·52
7·29
12.35
12.30
10.65
8·76
8·53
9·23
3.19
EB estimates
2
""
ex
62.84
46.13
23.10
38.83
26.40
84.10
36.47
27·82
64·77
54.01
13.56
5.86
8.28
11·74
10.39
10.43
8.12
8.20
9·24
6.08
s
'"
'"t3
~2
cr
N(50, 225)
4.48
5.32
5.26
5·12
4.65
5·28
4,84
4.92
5·06
5.48
4·73
5.1lJ.
5·20
5.18
4.83
5·46
4.88
4·94
5·05
5.26
55·42
43·61
23·52
37·95
26.81
69·60
35·66
27·97
56.60
49·41
= .80, which is slightly biased downward.
co
\.Jl