Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf Maximum Likelihood Estimation of Truncated Normal Regression Model Received: 12/10/2005 Accepted: 21/1/2007 Faris M. Alathari* & Zaki J. Al-sarraf** ملخص وزيل ي اح ي لي ي لةي غيييا اليل ي ي ال وزي ي الطبيل ي ف ي الحي ي ا الليةيييل ا لا ي ي ي ي ي ولكن ف أكثاهي يكيون الي غييا اليل يي اييا طةييأ ليأ يذ أييل يييل يين يي,لنيوذج االنح اا يمي ي ه ا ييي الي غي ييا اليل يي ي في ي الف ي ياا ( في ييثا ي ي ال ي ي ي طي الا حي ي,) الف ي ياا ( و إنيي ي ي ي ي ي ي ي ي طي ف ي ي ي ي يمي ي ي ي ه ا ال ي ي ييي اليوج ي ي ييو ا في ي ي يb,) ( أو الف ي ي ي ياا, a ) اي ال وزيي الطبيلي ( وفي هيذ الح ليل في ن ال يي اليمي ه ا لةي غييا اليل يي يa , b ) الف ياا .)truncated normal distribution( اليب وا Maximum likelihood ( ا طاي يل اإليكي ن األعظي إليا ا ي يهي إن هذا الاح ) نيوذج االنح اا ال ط الطبيلي اليب يواparameters( ) ف إيج ي اات ليل لmethod ) ي ي اي ازي ي طاي ييل اإليك ي ن األعظ يtruncated normal linear regression model( . ق و الكف ءا ال ايبيل وال طبي ال ايب عيين أ ييةوا ال ي يا اليل ي ي عةييا البي ن ي ت وال ييذVerbeek )2000 ي يين بي ي ي ةي اليلاوفل و الي يثةل ب واص اال إن أ ييةوا ال ي يا ف ي هييذا الاح ي ) أو األ ييةوا الي ي يCensored data( الي اا ا ييل . عينل ايا عموائيل ين اليج ي ي Abstract The Maximum Likelihood Method is used to estimate the normal linear regression model when the truncated normal data is the only available data. This procedure, unlike the procedure of estimation from censored data, utilizes the relationship between the normal distribution and the truncated normal distribution to estimate the model. This model utilizes a random sample from the truncated normal distribution. It is unlike the so- called truncated regression model mentioned by Verbeek (2000), which utilizes a non- random sample. Under the regularity conditions (Zacks, 1971, p.194), * ** professor, Mathematics Dep, The Hashemite University. professor, Arabic Academy for Financial and Exchange Seiences. Al-Manarah, Vol. 14, No. 3, 2008 . 23 Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf this procedure provides consistent, asymptotic normal, and asymptotic efficient estimators (Montagomery, Peck, and Vining, 2001). Keywords: Maximum Likelihood Estimation, Truncated Normal istribution, Domain of study, Censored Regression Model. 1. Introduction: In practice, the normal distribution is often used as the distribution for the dependent variable, Yt , of the regression model. But in most real- life problems, the dependent variable is usually not free to take any value in the interval ( , ). For instant, the dependent variable may be observed only when a< Yt <b where a and b are extended real numbers such that a<b. To these cases, the observed values of Yt may follow a truncated normal distribution especially when the values of the observations in one or both of the normal distribution tails are unknown. Such data often arise when an item is measured using an instrument with a limited scale. Observations that are beyond the scale of measurement are unknown. The truncated distributions are very important and widely used in statistics. Joshi(1979), Balakrishnan and Joshi (1982), Khan and Ali (1987), Ahmad (2001), Ahmad and Fawzy (2003) among others, treated the doubly truncated distributions in defferent disciplines.To estimate such models, most researchers assume that the population of the restricted data represents a subpopulation or domain of study for which the censored regression model is the most suitable one to consider. For example, see Verbeek (2000), and Johnston and Dinardo (1997). In this paper, a regression situation is considered to find the maximum likelihood estimates of the linear relationship between the normal dependent variable and some independent variables as well as the estimated residual variance by assuming that the population of the restricted data a< Yt <b is the only available sampled population from which we can get a random sample. The estimated relationship is then used to predict the values of the missing observations of the normal populations. Such populations are available in most real- life problems. For example, the wages or salaries of some companies may be limited to the values a< Yt <b and follow a truncated normal distribution. Al-Manarah, Vol. 14, No. 3, 2008 . 24 Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf Similarly, the population of the test scores for an exam may also be limited to a< Yt <b and follow a truncated normal distribution. This procedure provides us with consistent, asymptotic normal, and asymptotic efficient estimators (Montagomery, Peck, and Vining,2001). 2.Truncated Distribution: Let X be a random variable with probability density function f (.; ) and cumulative distribution F (.; ) , then the probability density function of Y the truncated version of the random variable X truncated on the left at a and on the right at b is given by g( y; ) f ( y; ) I (a, b ) ( y ) …………. (1) F(b; ) F(a; ) 3.The truncated Normal Distribution: Let ( z ) and ( z ) be respectively the p.d.f. and the distribution function of a standard normal distribution. Let Wt be a random variable with normal p.d.f. of mean t and a constant variance 2 . Then the p.d.f. of Yt the truncated version of Wt truncated on the left at a and on the right at b is given by y t ( t ) I (a, b ) ( y t ) 2 g( y t ; t , ) . b t a t [ ( )( )] ……….. (2) To find E( Yt ) , we have E[( Yt t )] [ ( yt t ) dy t ( y t t ) ( a b 1 b t a t )( )] [ ( b t a t )( )] Al-Manarah, Vol. 14, No. 3, 2008 . 25 [ ( b t a t ) ( )] Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf And hence E( Yt ) t [ ( b t a t )( )] [ ( b t a t ) ( )] ….…(3) 4. The model: Let Wt t t and the location parameter t is given in terms of p independent variables X1 , X 2 , ..., Xp such that t = X t where Xt 1 x t1 x t 2 ...x tp , 0 1 ...p and t ~NID(0, 2 ). Then the p.d.f. of Yt is given by y xt ( t ) I (a, b ) ( y t ) g( y t ; t , 2 ) b xt a xt [ ( )( )] . ………… (4) For simplicity, let Pt ( , 2 ) ( b xt a xt )( ). And hence, y xt ( t ) I (a, b ) ( y t ) g( y t ; , 2 ) , … (5) Pt (, 2 ) and E( Yt x t ) xt Suppose that a xt b xt ( ) ( ) … (6) Pt ( , 2 ) we have n random sample data say { ( y t , x t1 , x t 2 , ..., x tp) : t 1, 2, ...,n} taken from the truncated normal distribution given in (5) to be used in estimating the value of the parameter . This leads to Al-Manarah, Vol. 14, No. 3, 2008 . 26 Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf the so called truncated regression model. This model is different from the model mentioned by Verbeek (2000, pg 201). His model based on different sample selection rule. This rule based on drawing Wt randomly from the non- truncated normal distribution. If Wt falls in the interval (a, b), then we observe Wt , otherwise we do not observe Wt . In this case we no longer have a random sample and we have to take this into account when making inference. For the purposes of interpreting the coefficients of the truncated regression model, consider the following two derivatives with respect to a particular x tk for observation t, E( Wt x t ) k x tk and k ( E( Yt x t ) k x tk b xt b xt a xt a xt ) ( ) k ( ) ( ) Pt ( , 2 ) b x a xt t k ( ) ( ) [Pt ( , 2 )]2 2 2 a xt b xt b x b x a x a x t t t t ( ) ( ) ) ( )( ) ( ) ( k 1 2 P ( , ) [Pt ( , 2 )]2 t This tells us that the marginal effect of a change in x tk upon the expected outcome Wt is k whereas that the marginal effect in x tk upon the expected outcome Yt will be different from k . It will also depend upon the values of x t . Al-Manarah, Vol. 14, No. 3, 2008 . 27 Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf 5. Maximum likelihood estimation: The likelihood function for this model is thus given by n L( , 2 ) g( y t ; , 2 ) t 1 n y xt ( t ) I (a, b ) ( y t ) t 1 n n …..… (7) 2 Pt (, ) t 1 Let L* ln L(, 2 ). Then, L* n ( y x ) 2 n n n t ln( 2 ) t ln P t (, 2 ) ln( 2 ) ……. (8) 2 2 t 1 t 1 2 2 and hence n ( y x ) x n Pt (, 2 ) 1 L* t t t 2 t 1 2 t 1 Pt (, ) where, Pt (, 2 ) b xt a xt [ ( ) ( )] b xt 1 a xt [( ) ( )] x t …….. (9) and hence, n ( y x ) x n a xt b xt 1 L* t t t 1 [( ) ( )] x t …..(10) t 1 Pt (, 2 ) t 1 2 Similarly, L* n ( y x ) 2 n Pt (, 2 ) 1 n t t 2 t 1 Pt (, ) 2 4 2 2 2 2 t 1 Al-Manarah, Vol. 14, No. 3, 2008 . 28 Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf and Pt (, 2 ) 2 2 1 23 [ ( b xt a xt ) ( )] [(a xt )( a x t b xt ) (b xt )( )] ...(11) and hence, L* 2 n ( y x ) 2 a xt b xt 1 n 1 n [(a xt )( ) (b xt )( )] t 4t 3 2 t 1 2 2 t 1 Pt (, ) 2 2 ….… (12) * * L L in equation (10) and in equation (12) to zero we see 2 ~ ~ 2 satisfy the equations : that the maximum likelihood estimates and Equating n ~ ~ ( y t xt ) xt t 1 ~ ~ a x t b x t 1 ) ( )] x t =0 ~ ~ 2 [( ~ ~ t 1 Pt ( , ) n ...(13) and n ~ ~ 1 ~ a x t ~ b x t ~2 = 0 [( a x ) ( ) ( b x ) ( t t ~ ~2 ~ ~ )] n t 1 Pt ( , ) ~ 2 ~n ( y t x t ) t 1 …… (14) ~ Multiplying equation (13) by and adding the result to equation (14) yields, n ~ ( y t x t ) y t t 1 ~2 ~ ~ b x t a x t [ b ( ) a ( )] n ~ ~ n ~ ~2 ~ P ( t 1 t , ) Al-Manarah, Vol. 14, No. 3, 2008 . 29 . ..…(15) Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf Equations (13) and (15) in matrix form will be: ~ ( X X) 1 X( Y V ) ….…(16) and ~2 ~ Y Y X Y n ~ ~2 n h 2t ( , ) t 1 ….…(17) Where ~ ~ b xt a xt [( ~ ) ( ~ )] ~ ~2 h 1t ( , ) , ~ ~2 Pt ( , ) ~ ~2 h 2t ( , ) 1 1 X = . 1 x11 x 21 . x n1 [b( ~ ~ b xt a xt ) a ( ~ ~ )] ~ ~2 ~ P ( t , ) x12 ... x1p x 22 ... x 2p . . . x n 2 ... x np ~ h Y y1 y 2 ... y n and V 11 h12 ... h1n As special case, when a and b , then the maximum likelihood ~ ~ 2 of equations (16) and (17) lead to the OLS estimate estimators and the maximum likelihood ( X X) 1 X Y and Y Y X Y 2 under non-truncation, respectively. n estimator ~ ~ 2 are found by solving the equations (16) and (17) by the The estimators and trial and error method. A standard procedure is to calculate the estimate ~ 0 = ( X X) 1 X Y from the OLS method and use it as an initial guess to Al-Manarah, Vol. 14, No. 3, 2008 . 30 Maximum Likelihood ……………...........….………..Faris Alathari & Zaki Al-sarraf ~ in equation (16) with which to begin finding a solution ~ 2 in equation (17) to find another value substitute the value 0 ~ 2 for 0 ~ 1 for ~ 2 . Then ~ which in ~ 2 . We continue this ~ 2 for turn substituted in equation (16) to find a new value 1 process until we get ~ ~ ~2 ~ 2 .00001 and i i 1 .00001 and the i i 1 solution is found. References: Ahmad,A. A., Moments of order Statistics from doubly truncated continuous distributions. Statistics 35(4),479-494, 2001. Ahmad,A. A. and Fawzy, M., Recurrence relations for single moments of generalized order statistics from doubly truncated distributions, Journal of Statistical Planning and Inference 117, 241-249, 2003. Balakrishnan, N. and Joshi, P. C., Moments of order Statistics from doubly truncated pareto distribution, Journal of Communications in StatisticsTheory and Methods 23(10), 2841- 2852, 1982. Johnston, J. and Dinardo, J., Econometric Methods, McGraw- Hill, New York, 1997. Joshi, P. C., A note on the moments of order Statistics from doubly truncated exponential distribution, Annals of the institute of statistical mathematics 31, 321- 324, 1979. Kendall M. and Stuard A., The advanced theory of Statistics, London, 1970 Khan, A. H. and Ali, M. M., Characterization of probability distributions through higher order gap, Journal of Communications in Statistics- Theory and Methods 16(5), 1281- 1287, 1987 Montgomery, D. C., Peck, E. A. and Vining, G. Geoffrey, Introduction to Linear Regression Analysis, 3rd ed. Wiley, New York, 2001 Tryfos, Peter, Sampling Methods for Applied research, Wiley, New York, 1996 Verbeek, Marno, A guide To Modern Econometrics, Wiley, New York, 2000 Zacks, S., The Theory of statistical Inference, John Wiley, New York, 1971. Al-Manarah, Vol. 14, No. 3, 2008 . 31