Download The Maximum Likelihood Method is used to estimate the normal

Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
Maximum Likelihood Estimation of Truncated Normal
Regression Model
Received: 12/10/2005
Accepted: 21/1/2007
Faris M. Alathari*
&
Zaki J. Al-sarraf**
‫ملخص‬
‫وزيل ي اح ي لي ي لةي غيييا اليل ي ي‬
‫ال وزي ي الطبيل ي‬
‫ف ي الحي ي ا الليةيييل ا لا ي ي ي ي ي‬
‫ ولكن ف أكثاهي يكيون الي غييا اليل يي اييا طةييأ ليأ يذ أييل يييل يين يي‬,‫لنيوذج االنح اا‬
‫يمي ي ه ا ييي الي غي ييا اليل يي ي في ي الف ي ياا‬
‫ ( في ييثا ي ي ال ي ي ي طي الا حي ي‬,) ‫الف ي ياا‬
‫( و إنيي ي ي ي ي ي ي ي ي طي ف ي ي ي ي يمي ي ي ي ه ا ال ي ي ييي اليوج ي ي ييو ا في ي ي ي‬b,) ‫ ( أو الف ي ي ي ياا‬, a )
‫اي ال وزيي الطبيلي‬
‫ ( وفي هيذ الح ليل في ن ال يي اليمي ه ا لةي غييا اليل يي ي‬a , b ) ‫الف ياا‬
.)truncated normal distribution( ‫اليب وا‬
Maximum likelihood ( ‫ا طاي يل اإليكي ن األعظي‬
‫إليا ا ي‬
‫يهي‬
‫إن هذا الاح‬
‫) نيوذج االنح اا ال ط الطبيلي اليب يوا‬parameters( ‫) ف إيج ي اات ليل ل‬method
‫) ي ي اي ازي ي طاي ييل اإليك ي ن األعظ ي‬truncated normal linear regression model(
. ‫ق و الكف ءا ال ايبيل وال طبي ال ايب‬
‫عيين أ ييةوا ال ي يا اليل ي ي عةييا البي ن ي ت‬
‫ وال ييذ‬Verbeek )2000
‫ي يين بي ي‬
‫ي ةي‬
‫اليلاوفل و الي يثةل ب واص اال‬
‫إن أ ييةوا ال ي يا ف ي هييذا الاح ي‬
‫) أو األ ييةوا الي ي ي‬Censored data( ‫الي اا ا ييل‬
. ‫عينل ايا عموائيل ين اليج ي‬
‫ي‬
Abstract
The Maximum Likelihood Method is used to estimate the normal linear
regression model when the truncated normal data is the only available data.
This procedure, unlike the procedure of estimation from censored data,
utilizes the relationship between the normal distribution and the truncated
normal distribution to estimate the model. This model utilizes a random
sample from the truncated normal distribution. It is unlike the so- called
truncated regression model mentioned by Verbeek (2000), which utilizes a
non- random sample. Under the regularity conditions (Zacks, 1971, p.194),
*
**
professor, Mathematics Dep, The Hashemite University.
professor, Arabic Academy for Financial and Exchange Seiences.
Al-Manarah, Vol. 14, No. 3, 2008 .
23
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
this procedure provides consistent, asymptotic normal, and asymptotic
efficient estimators (Montagomery, Peck, and Vining, 2001).
Keywords: Maximum Likelihood Estimation, Truncated Normal istribution,
Domain of study, Censored Regression Model.
1. Introduction:
In practice, the normal distribution is often used as the distribution for the
dependent variable, Yt , of the regression model. But in most real- life problems,
the dependent variable is usually not free to take any value in the interval
(   ,  ). For instant, the dependent variable may be observed only when
a< Yt <b where a and b are extended real numbers such that a<b. To these cases,
the observed values of Yt may follow a truncated normal distribution especially
when the values of the observations in one or both of the normal distribution tails
are unknown. Such data often arise when an item is measured using an instrument
with a limited scale. Observations that are beyond the scale of measurement are
unknown. The truncated distributions are very important and widely used in
statistics. Joshi(1979), Balakrishnan and Joshi (1982), Khan and Ali (1987),
Ahmad (2001), Ahmad and Fawzy (2003) among others, treated the doubly
truncated distributions in defferent disciplines.To estimate such models, most
researchers assume that the population of the restricted data represents a
subpopulation or domain of study for which the censored regression model is the
most suitable one to consider. For example, see Verbeek (2000), and Johnston and
Dinardo (1997).
In this paper, a regression situation is considered to find the maximum
likelihood estimates of the linear relationship between the normal dependent
variable and some independent variables as well as the estimated residual
variance by assuming that the population of the restricted data a< Yt <b is the
only available sampled population from which we can get a random sample. The
estimated relationship is then used to predict the values of the missing
observations of the normal populations. Such populations are available in most
real- life problems. For example, the wages or salaries of some companies may be
limited to the values a< Yt <b and follow a truncated normal distribution.
Al-Manarah, Vol. 14, No. 3, 2008 .
24
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
Similarly, the population of the test scores for an exam may also be limited to
a< Yt <b and follow a truncated normal distribution. This procedure provides us
with consistent, asymptotic normal, and asymptotic efficient estimators
(Montagomery, Peck, and Vining,2001).
2.Truncated Distribution:
Let X be a random variable with probability density function f (.; ) and
cumulative distribution F (.;  ) , then the probability density function of Y the
truncated version of the random variable X truncated on the left at a and on the
right at b is given by
g( y; ) 
f ( y; ) I (a, b ) ( y )
…………. (1)
F(b; )  F(a; )
3.The truncated Normal Distribution:
Let  ( z ) and  ( z ) be respectively the p.d.f. and the distribution function of a
standard normal distribution. Let Wt be a random variable with normal p.d.f. of
mean  t and a constant variance  2 . Then the p.d.f. of Yt the truncated version
of Wt truncated on the left at a and on the right at b is given by
y  t
( t
) I (a, b ) ( y t )
2

g( y t ;  t ,  ) 
.
b  t
a  t
 [ (
)(
)]


……….. (2)
To find E( Yt ) , we have
E[( Yt   t )] 
 [ (
yt  t
) dy t
 ( y t   t ) (

a
b
1
b  t
a  t
)(
)]




[ (
b  t
a  t
)(
)]


Al-Manarah, Vol. 14, No. 3, 2008 .
25
[ (
b  t
a  t
) (
)]


Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
And hence
E( Yt )   t 

[ (
b  t
a  t
)(
)]


[ (
b  t
a  t
) (
)] ….…(3)


4. The model:
Let Wt   t   t
and the location parameter  t is given in terms of p independent variables
X1 , X 2 , ..., Xp such that  t = X t  where Xt  1 x t1 x t 2 ...x tp ,




   0 1 ...p and  t ~NID(0,  2 ).
Then the p.d.f. of Yt is given by
y  xt 
( t
) I (a, b ) ( y t )

g( y t ;  t ,  2 ) 
b  xt 
a  xt 
 [ (
)(
)]


. ………… (4)
For simplicity, let
Pt ( ,  2 )   (
b  xt 
a  xt 
)(
).


And hence,
y  xt 
( t
) I (a, b ) ( y t )

g( y t ; ,  2 ) 
, … (5)
 Pt (,  2 )
and
E( Yt x t )  xt  
Suppose
that
a  xt  
 b  xt 
(
) (
) … (6)




Pt ( ,  2 ) 

we
have
n
random
sample
data
say
{ ( y t , x t1 , x t 2 , ..., x tp) : t  1, 2, ...,n} taken from the truncated normal distribution
given in (5) to be used in estimating the value of the parameter  . This leads to
Al-Manarah, Vol. 14, No. 3, 2008 .
26
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
the so called truncated regression model. This model is different from the model
mentioned by Verbeek (2000, pg 201). His model based on different sample
selection rule. This rule based on drawing Wt randomly from the non- truncated
normal distribution. If Wt falls in the interval (a, b), then we observe Wt ,
otherwise we do not observe Wt . In this case we no longer have a random
sample and we have to take this into account when making inference.
For the purposes of interpreting the coefficients of the truncated regression
model, consider the following two derivatives with respect to a particular x tk for
observation t,
 E( Wt x t )
 k
 x tk
and
k (
E( Yt x t )
 k 
 x tk
b  xt 
b  xt 
a  xt 
a  xt 
) (
)  k (
) (
)





Pt ( ,  2 )
 b  x 
a  xt  
t
 k  (
) (
)




[Pt ( ,  2 )]2
2
2

a  xt   
 b  xt 




b

x

b

x

a

x

a

x

t
t
t
t
 (
) (
) 
) (
)(
) (
)  (

  






  k 1 


2
P
(

,

)
[Pt ( ,  2 )]2
t






This tells us that the marginal effect of a change in x tk upon the expected
outcome Wt is  k whereas that the marginal effect in x tk upon the expected
outcome Yt will be different from  k . It will also depend upon the values of x t .
Al-Manarah, Vol. 14, No. 3, 2008 .
27
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
5. Maximum likelihood estimation:
The likelihood function for this model is thus given by
n
L( ,  2 )   g( y t ; ,  2 )
t 1

n
y  xt 
 ( t
) I (a, b ) ( y t )

t 1

n
n
…..… (7)
2
 Pt (,  )
t 1
Let L*  ln L(,  2 ).
Then,
L*  
n ( y  x ) 2
n
n
n
t
ln( 2 )   t
  ln P t (,  2 )  ln(  2 ) ……. (8)
2
2
t 1
t 1
2 2
and hence
n ( y  x  ) x
n
 Pt (,  2 )
1
 L*
t
t
 
  t
2


t 1
2
t  1 Pt (,  )
where,
 Pt (,  2 ) 
b  xt 
a  xt 

[ (
)  (
)]





b  xt 
1 a  xt 
[(
)  (
)] x t



…….. (9)
and hence,
n ( y  x  ) x
n
a  xt 
b  xt 
1
 L*
t
t
t 1
 
 
[(
)  (
)] x t …..(10)
 t  1 Pt (,  2 )



t 1
2
Similarly,
 L*
n ( y  x ) 2
n
 Pt (,  2 )
1
n
t
t
 

2
t  1 Pt (,  )
2 4
 2
2 2
 
  2 t 1
Al-Manarah, Vol. 14, No. 3, 2008 .
28
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
and
 Pt (,  2 )
 2



 2
1
23
[ (
b  xt 
a  xt 
)  (
)]


[(a  xt )(
a  x t 
b  xt 
)  (b  xt )(
)]


...(11)
and hence,
 L*
 2

n ( y  x ) 2
a  xt 
b  xt 
1 n
1
n

[(a  xt )(
)  (b  xt )(
)] 
 t 4t

3
2


t 1
2
2 t 1 Pt (,  )
2 2
….… (12)
*
*
L
L
in equation (10) and
in equation (12) to zero we see

 2
~
~ 2 satisfy the equations :
that the maximum likelihood estimates  and 
Equating
n

~
~
( y t  xt  ) xt  
t 1
~
~
a  x t 
b  x t 
1
)  (
)] x t =0
~ ~ 2 [( 
~
~

t  1 Pt ( ,  )
n

...(13)
and
n
~
~
1
~ a  x t 
~ b  x t 
~2 = 0


[(
a

x

)

(
)

(
b

x

)

(
t
t
~ ~2
~
~ )]  n


t 1 Pt ( ,  )
~ 2 ~n
 ( y t  x t  )  

t 1
…… (14)
~
Multiplying equation (13) by  and adding the result to equation (14) yields,
n
~
 ( y t  x t  ) y t
t 1
~2 

~
~
b  x t 
a  x t 
[
b

(
)

a

(
)]
n
~
~


n 
~ ~2
~ P (
t 1

t , )
Al-Manarah, Vol. 14, No. 3, 2008 .
29
. ..…(15)
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
Equations (13) and (15) in matrix form will be:
~
  ( X X) 1 X( Y  V )
….…(16)
and
~2 

~
Y  Y    X Y
n
~ ~2
n   h 2t ( , 
)
t 1
….…(17)
Where
~
~
b  xt 
a  xt 
[( ~ )  ( ~ )]
~ ~2


h 1t (  ,  ) 
,
~ ~2
Pt ( ,  )
~ ~2
h 2t (  , 
)
1

1
X =
.
1

x11
x 21
.
x n1
[b(
~
~
b  xt 
a  xt 
)

a

(
~
~ )]


~ ~2
~ P (

t , )
x12 ... x1p 

x 22 ... x 2p 

. . . 
x n 2 ... x np 
~ h
Y  y1 y 2 ... y n  and V  
11 h12 ... h1n 
As special case, when a    and b   , then the maximum likelihood
~
~ 2 of equations (16) and (17) lead to the OLS estimate
estimators  and 

the
maximum
likelihood
  ( X X) 1 X Y and

Y  Y    X Y

2 
under non-truncation, respectively.
n
estimator
~
~ 2 are found by solving the equations (16) and (17) by the
The estimators  and 
trial and error method. A standard procedure is to calculate the estimate
~ 
0 =   ( X  X) 1 X  Y from the OLS method and use it as an initial guess to
Al-Manarah, Vol. 14, No. 3, 2008 .
30
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf
~
 in equation (16) with which to begin finding a solution
~ 2 in equation (17) to find another value
substitute the value 
0
~ 2 for

0
~
1 for
~ 2 . Then

~
 which in
~ 2 . We continue this
~ 2 for 
turn substituted in equation (16) to find a new value 
1
process until we get
~ ~
~2  
~ 2  .00001 and 

i   i  1  .00001 and the
i
i 1
solution is found.
References:
Ahmad,A. A., Moments of order Statistics from doubly truncated continuous
distributions. Statistics 35(4),479-494, 2001.
Ahmad,A. A. and Fawzy, M., Recurrence relations for single moments of
generalized order statistics from doubly truncated distributions, Journal of
Statistical Planning and Inference 117, 241-249, 2003.
Balakrishnan, N. and Joshi, P. C., Moments of order Statistics from doubly
truncated pareto distribution, Journal of Communications in StatisticsTheory and Methods 23(10), 2841- 2852, 1982.
Johnston, J. and Dinardo, J., Econometric Methods, McGraw- Hill, New York,
1997.
Joshi, P. C., A note on the moments of order Statistics from doubly truncated
exponential distribution, Annals of the institute of statistical mathematics 31,
321- 324, 1979.
Kendall M. and Stuard A., The advanced theory of Statistics, London, 1970
Khan, A. H. and Ali, M. M., Characterization of probability distributions through
higher order gap, Journal of Communications in Statistics- Theory and
Methods 16(5), 1281- 1287, 1987
Montgomery, D. C., Peck, E. A. and Vining, G. Geoffrey, Introduction to Linear
Regression Analysis, 3rd ed. Wiley, New York, 2001
Tryfos, Peter, Sampling Methods for Applied research, Wiley, New York, 1996
Verbeek, Marno, A guide To Modern Econometrics, Wiley, New York, 2000
Zacks, S., The Theory of statistical Inference, John Wiley, New York, 1971.
Al-Manarah, Vol. 14, No. 3, 2008 .
31