Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
CSCK MLE BIAS CALCULATION Paul H. Johnson, Jr.H1L Department of Mathematics, University of Illinois at Urbana-Champaign, Champaign, IL [email protected] Yongxue Qi Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, IL [email protected] Yvonne C. Chueh Department of Actuarial Science, Statistics, and Mathematics, Central Washington University, Ellensburg, WA [email protected] (1) Corresponding Author ABSTRACT Actuarial Model Optimal Outcome Fit (AMOOF) is an open source, high performance computation program that is currently under development at Central Washington University. AMOOF is an add-in to both Microsoft Excel and Mathematica that is designed to assist actuaries and other financial analysts in efficient stochastic modeling of financial outcomes arising from areas including capital determination, reserve setting, risk analysis, and product pricing. AMOOF utilizes the method of maximum likelihood to estimate the unknown parameters of probability distributions fit to stochastic financial outcome model data. Maximum likelihood estimators (MLEs) possess highly desirable properties such as asymptotic unbiasedness, consistency, and asymptotic normality. Unfortunately, many of these properties, specifically unbiasedness, may not be valid for small sample sizes (for example, data of less than 100 observations). In this paper, we consider the Cox and Snell / Cordeiro and Klein (CSCK) methodology for determining the analytic bias associated with MLEs that were calculated using small sample data. Researchers from the University of Illinois at UrbanaChampaign have developed a Mathematica program, entiled “CSCK MLE Bias Calculation,” that enables the user to calculate the analytic CSCK MLE bias vectors for various probability distributions with up to four unknown parameters. The process for arriving at the CSCK bias associated with each MLE is outlined step-by-step in a fashion that makes the entire process transparent to the user. Once the CSCK MLE bias has been determined for a given MLE, the bias corrected MLE (BMLE) can be obtained by subtracting the CSCK MLE bias (evaluated at the MLEs) from the MLE resulting in a parameter estimate that is unbiased even with small sample sizes. Ultimately, this CSCK methodology will be implemented in AMOOF to improve the quality of MLEs generated via that program. the analytic CSCK MLE bias vectors for various probability distributions with up to four unknown parameters. The process for 2 arriving at the CSCK bias associated with each MLE is outlined step-by-step in a fashion that makes the entire process CSCK_MLE_Bias_Calculation_063012.nb transparent to the user. Once the CSCK MLE bias has been determined for a given MLE, the bias corrected MLE (BMLE) can be obtained by subtracting the CSCK MLE bias (evaluated at the MLEs) from the MLE resulting in a parameter estimate that is unbiased even with small sample sizes. Ultimately, this CSCK methodology will be implemented in AMOOF to improve the quality of MLEs generated via that program. INTRODUCTION Actuarial Model Optimal Outcome Fit HAMOOFL@1D is the current pilot project of the departments of Actuarial Science, Statistics, and Mathematics and Computer Science at Central Washington University. AMOOF is an add-in to both Microsoft Excel@2D and Mathematica@3D that is being designed to assist actuaries and other financial analysts in efficiently analyzing financial outcomes arising from areas including capital determination, reserve setting, risk analysis, and product pricing. For a stochastic financial outcome data set, AMOOF considers candidate probability density functions that can be fit to the data and, using various statistical criteria, finds the density function that best fits the data. AMOOF not only considers standard probability distributions, but also transformed, inverse, and mixed probability distributions in order to arrive at the optimal probability density function for the given stochastic model data. AMOOF also enables its user to fit a probability density function to the tails of a distribution and to consider extreme values, which are often of interest to various risk management and regulatory authorities. AMOOF is both open source and high performance computation program that will be made available to insurance companies and other financial institutions. Financial outcomes that were once too time prohibitive to consider can be analyzed using the easy-to-implement AMOOF program. More information regarding the Actuarial Model Optimal Outcome Fit project can be found at http://www.cwu.edu/~chueh/AMOOF.html. AMOOF uses the method of maximum likelihood to estimate the parameters of probability distributions fit to stochastic model data. The method of maximum likelihood is widely used to estimate the unknown parameters of probability distributions that are fit to data. Maximum likelihood estimators (MLEs) have many desirable properties. For example, MLEs are asymptotically unbiased, consistent, and asymptotically normal. However, many of these properties rely on having a large sample size. This means that MLE properties, such as unbiasedness, may not be valid for small sample sizes. While “small” is not a strictly defined term, we will use it to refer to data sets comprised of fewer than 100 observations. Researchers from various fields have highlighted the asymptotic requirement in order to obtain unbiased parameter estimates via the method of maximum likelihood. Cox and Snell@4D first considered analytic expressions for the bias of MLEs calculated with small samples as part of their study of a general definition of residuals. Cordeiro and Klein@5D later extended the Cox and Snell analysis and generalized their result. We shall refer to the method for obtaining analytic expressions for small sample MLE bias as the Cox and Snell / Cordeiro and Klein (CSCK) method. The CSCK methodology provides a “corrective approach’’ for mitigating small sample MLE bias, where a bias-corrected MLE (BMLE) is obtained by subtracting the CSCK MLE bias (evaluated at the MLEs) from the MLE. While the CSCK method has been around for decades, it is only recently that computational software has allowed for efficient numerical calculation of the CSCK MLE bias for various distributions. Giles and co-authors have considered the CSCK MLE bias of many common 2-parameter distributions, such as the gamma distribution@6D . In this paper, we introduce a Mathematica program developed by researchers from the University of Illinois at Urbana- Cordeiro and Klein (CSCK) method. The CSCK methodology provides a “corrective approach’’ for mitigating small sample MLE bias, where a bias-corrected MLE (BMLE) is obtained by subtracting the CSCK CSCK_MLE_Bias_Calculation_063012.nb MLE bias (evaluated at the MLEs) from 3 the MLE. While the CSCK method has been around for decades, it is only recently that computational software has allowed for efficient numerical calculation of the CSCK MLE bias for various distributions. Giles and co-authors have considered the CSCK MLE bias of many common 2-parameter distributions, such as the gamma distribution@6D . In this paper, we introduce a Mathematica program developed by researchers from the University of Illinois at UrbanaChampaign, entiled “CSCK MLE Bias Calculation,” that implements the CSCK methodology. The user will be able to determine the CSCK MLE bias for several probability distributions that are commonly utilized in the analysis of stochastic financial outcomes. The program is designed so that for a probability distribution with at most four unknown parameters, the CSCK MLE bias vector is calculated in a fashion that makes the entire process transparent to the user. The entire CSCK method is outlined step-by-step throughout the program with the the desired CSCK MLE bias vector outputted at the end. Once the CSCK MLE bias has been determined for a given MLE, the BMLE can be obtained as an unbiased parameter estimate even with small sample sizes. The ultimate goal of this project is to improve the quality of MLEs used in fitting probability distributions to stochastic financial outcome model data in AMOOF. OVERVIEW OF THE CSCK METHODOLOGY Consider a probability distribution with p unknown parameters: θ = Iθ1 , θ2 , ..., θp M . Let l(θ) denote the total ' loglikelihood function, based on a sample of n observations. It is assumed throughout this discussion that the observations are not necessarily identically distributed. Assume l(θ) is regular with respect to all derivatives of the elements of θ up to and including the third order. Define the following joint cumulants of l(θ) for i, j, l = 1, 2, ..., p: 2 3 kij =E[ ¶¶θi θl j ], kijl =E[ ¶θ¶i θlj θl ], and kij,l 2 =E[ ¶¶θi θl j ¶l ]. ¶θl Also, define the cumulant derivative: k HlL ij = ¶kij ¶θl . The total Fisher information Matrix of order p for θ is denoted as K is denoted as K -1 = { -kij }. The inverse of total Fisher information Matrix = { -k ij }. ` θs denote the MLE of the s-th element (parameter) of θ. Cox and Snell proved that for independent observations, the ` bias of θs , bs , for s = 1, 2, ..., p, can be expressed as: Let bs ` p = E[θs - θs ] = Úi, j,l=1 k si k jl [0.5kijl + kij,l ] + O(n-2 ). The total Fisher information Matrix of order p for θ is denoted as K 4 = { -kij }. The inverse of total Fisher information Matrix CSCK_MLE_Bias_Calculation_063012.nb -1 ij is denoted as K = { -k }. ` Let θs denote the MLE of the s-th element (parameter) of θ. Cox and Snell proved that for independent observations, the ` bias of θs , bs , for s = 1, 2, ..., p, can be expressed as: bs ` p = E[θs - θs ] = Úi, j,l=1 k si k jl [0.5kijl + kij,l ] + O(n-2 ). Cordeiro and Klein verified that the assumption of independence in the preceding equation can be relaxed as long as all k’s are assumed to be O(n), resulting in the following expression: bs ` p p = E[θs - θs ] = Úi=1 k si Új,l=1 [k HlL ij - 0.5kijl ]k jl + O(n-2 ). Giles and co-authors point out that this second equation for bs is more computationally efficient than the first equation for bs as there are no kij,l terms to evaluate; we will consider this second equation for bs exclusively from this point on. The second equation for bs can be expressed in matrix notation to provide a convenient expression for the MLE bias vector, b. Let A ` = {AH1L | AH2L |--- AHpL } HlL where A If θ denotes the MLE vector: b = k HlL ij - 0.5kijl for l = 1, 2, ..., p. ` = E[θ - θ] = K -1 Avec(K -1 ) + O(n-2 ). ` The bias-corrected MLE (BMLE) vector, θ , under the CSCK methodology is the difference between the MLE vector (θ) and ` the MLE bias vector in evaluated at the MLEs (b): ` ` θ = θ - b. Note, as the true values of the distributional parameters θ are unknown, the CSCK MLE bias is estimated by plugging in the corresponding MLEs. Componentwise, for the s-th parameter, the BMLE of θs , ` ` θs , is equal to θs - bs . Johnson and co - authors@7D determined the CSCK MLE biases for the lognormal, two-parameter gamma, and two-parameter Weibull distributions. They also provided two simulation analyses. The first simulation demonstrated that BMLEs have preferable empirical properties when compared to MLEs for the lognormal, two-parameter gamma, and two-parameter Weibull distributions. The second simulation showed that BMLEs were preferable to MLEs for the loss reserving of an illustrative 20period equity-linked insurance contract for both the lognormal and two-parameter Weibull distributions, but not for the twoparameter gamma distribution. Johnson and co - authors@7D determined the CSCK MLE biases for the lognormal, two-parameter gamma, and two-parameter Weibull distributions. They also provided two simulation analyses. The first simulationCSCK_MLE_Bias_Calculation_063012.nb demonstrated that BMLEs have 5 preferable empirical properties when compared to MLEs for the lognormal, two-parameter gamma, and two-parameter Weibull distributions. The second simulation showed that BMLEs were preferable to MLEs for the loss reserving of an illustrative 20period equity-linked insurance contract for both the lognormal and two-parameter Weibull distributions, but not for the twoparameter gamma distribution. PROGRAMMING The CSCK MLE Bias Calculation program is a Mathematica application that implements the CSCK methodology. The program is set up in the following step-by-step format: 1. Mathematica will load the SMLE package, which was created and described in detail in Rose and Smith’s excellent tutorial article@8D . SMLE stands for “Symbolic Maximum Likelihood Estimation.” Using the SMLE package, the “Log’’ function in Mathematica is redefined via the “SuperLog’’ function to allow for a full symbolic representation of the total loglikelihood function. The SMLE package effectively gives the Log function the information it needs to covert the loglikelihood function into a sum of functions involving natural logarithms. For example, Mathematica is explicitly informed via the SMLE package that the sample size is a positive integer, variables and parameters are real-valued, and the arguments of natural logarithms are positive. With the SMLE package, the user does not have to manually determine the total loglikelihood function; this is automatically done in CSCK MLE Bias Calculation. The SMLE package is provided in Appendix A. 2. The user inputs both the probability density function and the number of distributional parameters. Appendix B provides an inventory of continuous probability distributions that we have considered so far using CSCK MLE Bias Calculation, obtained from a popular actuarial science textbook@9D . In some cases, the user will also have to input the assumptions for the distributional parameters in the function that defines the expected value function, Expect[x_], used in the cumulant calculation. The default assumes that all parameters are positive, real-valued numbers. Finally, it can be verified that all of the probability distributions in Appendix B are O(n) as is required to use the CSCK methodology. 3. The loglikelihood function is automatically determined using the SMLE package. 4. Various partial derivatives of the loglikelihood function are computed with respect to the distributional parameters (first order partial derivatives, second order partial derivatives, third order partial derivatives each with respect to a specific parameter; and mixed partial derivatives with respect to two and three parameters. These partial derivatives are calculated in order to determine the cumulants in the next step. 5. Various cumulants required for the CSCK method are calculated. 6. Various cumulant derivatives required for the CSCK method are calculated. 7. The total Fisher Information Matrix K and its inverse K -1 are constructed. 8. The A matrix is constructed. -2 ` 5. Various cumulants required for the CSCK method are calculated. 6 CSCK_MLE_Bias_Calculation_063012.nb 6. Various cumulant derivatives required for the CSCK method are calculated. 7. The total Fisher Information Matrix K and its inverse K -1 are constructed. 8. The A matrix is constructed. 9. The analytic CSCK MLE bias vector is outputted, ignoring the O(n-2 ) term which is assumed to be negligible: b ` = E[θ - θ] = K -1 Avec(K -1 ). For a distribution with p parameters, the CSCK MLE bias vector would be of the form: ` ` ` 9b@θ̀1 D, b@θ2 D, ..., b@θp D=, where b@θsD denotes the CSCK bias of the MLE of θs . In practice, the BMLE vector would be determined by taking the MLE and subtracting from it b evaluated at the MLEs. CSCK MLE BIAS CALCULATION PROGRAM The CSCK MLE Bias Calculation program will be illustrated using the 2-parameter gamma distribution. The CSCK MLE bias for the 2-parameter gamma distribution has been previously calculated in full detail by Giles and co - authors@6D ; it can be verified that our program provides the same results. 1. Mathematica loads the SMLE package and turns on the SuperLog function: << SMLE.m SuperLog@OnD SuperLog is now On. 2. USER INPUTS: The user inputs both the probability density function (f@yi D) and the number of distributional parameters (p) in the light blue sections. It is not needed for the 2-parameter gamma distribution, but if necessary, the user should change the assumptions for the parameters in the Expect[x_] function in the light blue section. Note: If the distribution of interest has less than four parameters, the additional parameters are treated as null throughout the remainder of the CSCK MLE bias calculation: f@yi D = Hyi Θ2L ^ Θ1 * Exp@- yi Θ2D * H1 Hyi * Gamma@Θ1DLL; p = 2; Expect@x_D := Integrate@x * f@yi D, 8yi , 0, ¥<, Assumptions ® 8Θ1 Î Reals, Θ2 Î Reals, Θ3 Î Reals, Θ4 Î Reals, Θ4 > 0, Θ3 > 0, Θ2 > 0, Θ1 > 0<D CSCK_MLE_Bias_Calculation_063012.nb 3. The loglikelihood function (l[Θ1,Θ2,Θ3,Θ4]) is automatically determined using the SMLE package (“Log” is redefined via “SuperLog” in Step 1): L = ä f@yi D n i=1 l@Θ1, Θ2, Θ3, Θ4D = Log@LD ã- Θ2 I yi ä n i=1 yi Θ2 M Θ1 Gamma@Θ1D yi 1 Θ2 n Θ2 HΘ1 Log@Θ2D + Log@Gamma@Θ1DDL + HΘ2 - Θ1 Θ2L â Log@yi D + â yi n n i=1 i=1 4. Various partial derivatives of the loglikelihood function are computed with respect to distributional parameters: H* Compute ¶l@Θ1,Θ2,Θ3,Θ4D *L ¶Θ1 firstlΘ1 = Simplify@¶8Θ1,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l@Θ1,Θ2,Θ3,Θ4D *L ¶Θ2 firstlΘ2 = Simplify@¶8Θ2,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l@Θ1,Θ2,Θ3,Θ4D *L ¶Θ3 firstlΘ3 = Simplify@¶8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l@Θ1,Θ2,Θ3,Θ4D *L ¶Θ4 firstlΘ4 = Simplify@¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ12 secondlΘ1 = Simplify@¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ22 secondlΘ2 = Simplify@¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ32 secondlΘ3 = Simplify@¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ42 secondlΘ4 = Simplify@¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ13 thirdlΘ1 = Simplify@¶8Θ1,3< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ23 thirdlΘ2 = Simplify@¶8Θ2,3< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ33 thirdlΘ3 = Simplify@¶8Θ3,3< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ43 thirdlΘ4 = Simplify@¶8Θ4,3< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ1¶Θ2 secondlΘ1Θ2 = Simplify@¶8Θ1,1<,8Θ2,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ1¶Θ3 secondlΘ1Θ3 = Simplify@¶8Θ1,1<,8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD 7 8 CSCK_MLE_Bias_Calculation_063012.nb secondlΘ1Θ3 = Simplify@¶8Θ1,1<,8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ1¶Θ4 secondlΘ1Θ4 = Simplify@¶8Θ1,1<,8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ2¶Θ3 secondlΘ2Θ3 = Simplify@¶8Θ2,1<,8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ2¶Θ4 secondlΘ2Θ4 = Simplify@¶8Θ2,1<,8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l2 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ3¶Θ4 secondlΘ3Θ4 = Simplify@¶8Θ3,1<,8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ1¶Θ2¶Θ3 thirdlΘ1Θ2Θ3 = Simplify@¶8Θ1,1< ¶8Θ2,1< ¶8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ1¶Θ2¶Θ4 thirdlΘ1Θ2Θ4 = Simplify@¶8Θ1,1< ¶8Θ2,1< ¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ1¶Θ3¶Θ4 thirdlΘ1Θ3Θ4 = Simplify@¶8Θ1,1< ¶8Θ3,1< ¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ2¶Θ3¶Θ4 thirdlΘ2Θ3Θ4 = Simplify@¶8Θ2,1< ¶8Θ3,1< ¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶l3 @Θ1,Θ2,Θ3,Θ4D *L ¶Θ1¶Θ22 thirdlΘ1secondΘ2 = Simplify@¶8Θ1,1< ¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ1¶Θ32 thirdlΘ1secondΘ3 = Simplify@¶8Θ1,1< ¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ1¶Θ42 thirdlΘ1secondΘ4 = Simplify@¶8Θ1,1< ¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ2¶Θ12 thirdlΘ2secondΘ1 = Simplify@¶8Θ2,1< ¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ2¶Θ32 thirdlΘ2secondΘ3 = Simplify@¶8Θ2,1< ¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ2¶Θ42 thirdlΘ2secondΘ4 = Simplify@¶8Θ2,1< ¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ3¶Θ12 thirdlΘ3secondΘ1 = Simplify@¶8Θ3,1< ¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ3¶Θ22 thirdlΘ3secondΘ2 = Simplify@¶8Θ3,1< ¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ3¶Θ42 thirdlΘ3secondΘ4 = Simplify@¶8Θ3,1< ¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ4¶Θ12 thirdlΘ4secondΘ1 = Simplify@¶8Θ4,1< ¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ4¶Θ22 thirdlΘ4secondΘ2 = Simplify@¶8Θ4,1< ¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD H* Compute ¶Θ4¶Θ32 thirdlΘ4secondΘ3 = Simplify@¶8Θ4,1< ¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD CSCK_MLE_Bias_Calculation_063012.nb - n HLog@Θ2D + PolyGamma@0, Θ1DL + â Log@yi D n - n Θ1 Θ2 + Úni=1 yi Θ22 0 0 - n PolyGamma@1, Θ1D n Θ1 Θ2 - 2 Úni=1 yi Θ23 0 0 - n PolyGamma@2, Θ1D - 2 n Θ1 Θ2 + 6 Úni=1 yi Θ24 0 0 n Θ2 0 0 0 0 0 0 0 0 0 n Θ22 0 0 0 0 0 0 0 0 i=1 9 10 CSCK_MLE_Bias_Calculation_063012.nb 0 0 0 5. Various cumulants required for the CSCK method are calculated: H* Compute k11 =EA ¶2 l E *L ¶Θ1 2 k11 = Factor@Expect@secondlΘ1DD H* Compute k22 =EA ¶2 l E *L ¶Θ2 2 k22 = Factor@Expect@secondlΘ2DD H* Compute k33 =EA ¶2 l E *L ¶Θ3 2 k33 = Factor@Expect@secondlΘ3DD H* Compute k44 =EA ¶2 l E *L ¶Θ4 2 k44 = Factor@Expect@secondlΘ4DD H* Compute k12 =k21 =EA ¶2 l E *L ¶Θ1 ¶Θ2 k12 = Factor@Expect@secondlΘ1Θ2DD H* Compute k13 =k31 =EA ¶2 l E *L ¶Θ1 ¶Θ3 k13 = Factor@Expect@secondlΘ1Θ3DD H* Compute k14 =k41 =EA ¶2 l E *L ¶Θ1 ¶Θ4 k14 = Factor@Expect@secondlΘ1Θ4DD H* Compute k23 =k32 =EA ¶2 l E *L ¶Θ2 ¶Θ3 k23 = Factor@Expect@secondlΘ2Θ3DD H* Compute k24 =k42 =EA ¶2 l E *L ¶Θ2 ¶Θ4 k24 = Factor@Expect@secondlΘ2Θ4DD H* Compute k34 =k43 =EA ¶2 l E *L ¶Θ3 ¶Θ4 k34 = Factor@Expect@secondlΘ3Θ4DD H* Compute k111 =EA ¶3 l E *L ¶Θ1 3 k111 = Factor@Expect@thirdlΘ1DD H* Compute k222 =EA ¶2 l E *L ¶Θ2 3 k222 = Factor@Expect@thirdlΘ2DD H* Compute k333 =EA ¶3 l E *L ¶Θ3 3 k333 = Factor@Expect@thirdlΘ3DD H* Compute k444 =EA ¶3 l E *L ¶Θ4 3 k444 = Factor@Expect@thirdlΘ4DD CSCK_MLE_Bias_Calculation_063012.nb k444 = Factor@Expect@thirdlΘ4DD H* Compute k112 =k121 =k211 =EA ¶3 l E *L ¶Θ1 2 ¶Θ2 k112 = Factor@Expect@thirdlΘ2secondΘ1DD H* Compute k113 =k131 =k311 =EA ¶3 l E *L ¶Θ1 2 ¶Θ3 k113 = Factor@Expect@thirdlΘ3secondΘ1DD H* Compute k114 =EA ¶3 l E *L ¶Θ1 2 ¶Θ4 k114 = Factor@Expect@thirdlΘ4secondΘ1DD H* Compute k122 =k212 =k221 =EA ¶3 l E *L ¶Θ2 2 ¶Θ1 k122 = Factor@Expect@thirdlΘ1secondΘ2DD H* Compute k133 =k313 =k331 =EA ¶3 l E *L ¶Θ3 2 ¶Θ1 k133 = Factor@Expect@thirdlΘ1secondΘ3DD H* Compute k144 =k414 =k441 =EA ¶3 l E *L ¶Θ4 2 ¶Θ1 k144 = Factor@Expect@thirdlΘ1secondΘ4DD H* Compute k223 =k232 =k322 =EA ¶3 l E *L ¶Θ2 2 ¶Θ3 k223 = Factor@Expect@thirdlΘ3secondΘ2DD H* Compute k224 =k242 =k422 =EA ¶3 l E *L ¶Θ2 2 ¶Θ4 k224 = Factor@Expect@thirdlΘ4secondΘ2DD H* Compute k334 =k343 =k433 =EA ¶3 l E *L ¶Θ3 2 ¶Θ4 k334 = Factor@Expect@thirdlΘ4secondΘ3DD H* Compute k123 =k321 =k231 =k213 =k132 =k312 =EA k123 = Factor@Expect@thirdlΘ1Θ2Θ3DD H* Compute k124 =k421 =k241 =k214 =k142 =k412 =EA k124 = Factor@Expect@thirdlΘ1Θ2Θ4DD H* Compute k134 =k431 =k341 =k314 =k143 =k413 =EA k134 = Factor@Expect@thirdlΘ1Θ3Θ4DD H* Compute k234 =k432 =k342 =k324 =k243 =k423 =EA k234 = Factor@Expect@thirdlΘ2Θ3Θ4DD H*Compute k233 =k332 =k323 =EA ¶3 l E *L ¶Θ3 2 ¶Θ2 k233 = Factor@Expect@thirdlΘ2secondΘ3DD H* Compute k244 =k442 =k424 =EA ¶3 l E *L ¶Θ4 2 ¶Θ2 k244 = Factor@Expect@thirdlΘ2secondΘ4DD H* Compute k344 =k443 =k434 =EA ¶3 l E *L ¶Θ4 2 ¶Θ3 k344 = Factor@Expect@thirdlΘ3secondΘ4DD - n PolyGamma@1, Θ1D ¶3 l E *L ¶Θ1 ¶Θ2 ¶Θ3 ¶3 l E *L ¶Θ1 ¶Θ2 ¶Θ4 ¶3 l E *L ¶Θ1 ¶Θ2 ¶Θ3 ¶3 l E *L ¶Θ1 ¶Θ2 ¶Θ3 11 12 CSCK_MLE_Bias_Calculation_063012.nb n Θ1 Θ22 0 0 n Θ2 0 0 0 0 0 - n PolyGamma@2, Θ1D 4 n Θ1 Θ23 0 0 0 0 0 n Θ22 0 0 0 0 0 0 0 0 0 0 0 0 6. Various cumulant derivatives required for the CSCK method are calculated: H* Compute k11 H1L = ¶k11 ¶Θ1 *L ka111 = Factor@¶8Θ1,1< k11D CSCK_MLE_Bias_Calculation_063012.nb ka111 = Factor@¶8Θ1,1< k11D H* Compute k11 H2L = ¶k11 ¶Θ2 *L ka112 = Factor@¶8Θ2,1< k11D H* Compute k11 H3L = ¶Θ 11 *L ¶k 3 ka113 = Factor@¶8Θ3,1< k11D H* Compute k11 H4L = ¶k11 ¶Θ4 *L ¶k12 ¶Θ1 *L ¶k12 ¶Θ2 *L ¶k12 ¶Θ3 *L ¶k12 ¶Θ4 *L ¶k13 ¶Θ1 *L ¶k13 ¶Θ2 *L ¶k13 ¶Θ3 *L ¶k13 ¶Θ4 *L ¶k14 ¶Θ1 *L ¶k14 ¶Θ2 *L ¶k14 ¶Θ3 *L ¶k14 ¶Θ4 *L ¶k22 ¶Θ1 *L ¶k22 ¶Θ2 *L ¶k22 ¶Θ3 *L ¶k22 ¶Θ4 *L ¶k23 ¶Θ1 *L ¶k23 ¶Θ2 *L ka114 = Factor@¶8Θ4,1< k11D H* Compute k12 H1L = ka121 = Factor@¶8Θ1,1< k12D H* Compute k12 H2L = ka122 = Simplify@¶8Θ2,1< k12D H* Compute k12 H3L = ka123 = Factor@¶8Θ3,1< k12D H* Compute k12 H4L = ka124 = Factor@¶8Θ4,1< k12D H* Compute k13 H1L = ka131 = Factor@¶8Θ1,1< k13D H* Compute k13 H2L = ka132 = Factor@¶8Θ2,1< k13D H* Compute k13 H3L = ka133 = Factor@¶8Θ3,1< k13D H* Compute k13 H4L = ka134 = Factor@¶8Θ4,1< k13D H* Compute k14 H1L = ka141 = Factor@¶8Θ1,1< k14D H* Compute k14 H2L = ka142 = Factor@¶8Θ2,1< k14D H* Compute k14 H3L = ka143 = Factor@¶8Θ3,1< k14D H* Compute k14 H4L = ka144 = Factor@¶8Θ4,1< k14D H* Compute k22 H1L = ka221 = Factor@¶8Θ1,1< k22D H* Compute k22 H2L = ka222 = Factor@¶8Θ2,1< k22D H* Compute k22 H3L = ka223 = Factor@¶8Θ3,1< k22D H* Compute k22 H4L = ka224 = Factor@¶8Θ4,1< k22D H* Compute k23 H1L = ka231 = Factor@¶8Θ1,1< k23D H* Compute k23 H2L = ka232 = Factor@¶8Θ2,1< k23D 13 14 CSCK_MLE_Bias_Calculation_063012.nb ka232 = Factor@¶8Θ2,1< k23D H* Compute k23 H3L = ¶k23 ¶Θ3 *L ¶k23 ¶Θ4 *L ¶k24 ¶Θ1 *L ¶k24 ¶Θ2 *L ¶k24 ¶Θ3 *L ¶k24 ¶Θ4 *L ¶k33 ¶Θ1 *L ¶k33 ¶Θ2 *L ¶k33 ¶Θ3 *L ¶k33 ¶Θ4 *L ¶k34 ¶Θ1 *L ¶k34 ¶Θ2 *L ¶k34 ¶Θ3 *L ¶k34 ¶Θ4 *L ¶k44 ¶Θ1 *L ¶k44 ¶Θ2 *L ¶k44 ¶Θ3 *L ¶k44 ¶Θ4 *L ka233 = Factor@¶8Θ3,1< k23D H* Compute k23 H4L = ka234 = Factor@¶8Θ4,1< k23D H* Compute k24 H1L = ka241 = Factor@¶8Θ1,1< k24D H* Compute k24 H2L = ka242 = Factor@¶8Θ2,1< k24D H* Compute k24 H3L = ka243 = Factor@¶8Θ3,1< k24D H* Compute k24 H4L = ka244 = Factor@¶8Θ4,1< k24D H* Compute k33 H1L = ka331 = Factor@¶8Θ1,1< k33D H* Compute k33 H2L = ka332 = Factor@¶8Θ2,1< k33D H* Compute k33 H3L = ka333 = Factor@¶8Θ3,1< k33D H* Compute k33 H4L = ka334 = Factor@¶8Θ4,1< k33D H* Compute k34 H1L = ka341 = Factor@¶8Θ1,1< k34D H* Compute k34 H2L = ka342 = Factor@¶8Θ2,1< k34D H* Compute k34 H3L = ka343 = Factor@¶8Θ3,1< k34D H* Compute k34 H4L = ka344 = Factor@¶8Θ4,1< k34D H* Compute k44 H1L = ka441 = Factor@¶8Θ1,1< k44D H* Compute k44 H2L = ka442 = Factor@¶8Θ2,1< k44D H* Compute k44 H3L = ka443 = Factor@¶8Θ3,1< k44D H* Compute k44 H4L = ka444 = Factor@¶8Θ4,1< k44D - n PolyGamma@2, Θ1D 0 0 0 CSCK_MLE_Bias_Calculation_063012.nb 0 n Θ22 0 0 0 0 0 0 0 0 0 0 n Θ22 2 n Θ1 Θ23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 16 CSCK_MLE_Bias_Calculation_063012.nb 0 0 7. The total Fisher Information Matrix K and its inverse K -1 are constructed: H* INFORMATION MATRIX K *L L = 88- k11, - k12, - k13, - k14<, 8- k12, - k22, - k23, - k24<, 8- k13, - k23, - k33, - k34<, 8- k14, - k24, - k34, - k44<< Kinformation = L@@1 ;; p, 1 ;; pDD MatrixForm@KinformationD ::n PolyGamma@1, Θ1D, ::n PolyGamma@1, Θ1D, n PolyGamma@1, Θ1D n Θ2 n Θ2 n Θ2 n Θ2 n Θ1 , 0, 0>, : >, : n n n Θ1 , Θ2 n Θ1 , Θ22 Θ2 Θ22 >> , 0, 0>, 80, 0, 0, 0<, 80, 0, 0, 0<> Θ22 H* INVERSE OF INFORMATION MATRIX K *L Kinv = Inverse@KinformationD; MatrixForm@KinvD n Θ1 n2 Θ22 - + n2 Θ1 PolyGamma@1,Θ1D Θ22 n Θ2 - Θ22 n2 Θ2 - + + n2 Θ1 PolyGamma@1,Θ1D Θ22 Θ22 n PolyGamma@1,Θ1D n - n2 n2 Θ1 PolyGamma@1,Θ1D Θ22 Θ22 - n2 + n2 Θ1 PolyGamma@1,Θ1D Θ22 Θ22 H* CONSTRUCTION: VEC@K INVERSED *L vecKinv = Flatten@Transpose@KinvDD; MatrixForm@vecKinvD n Θ1 n2 Θ22 - + n2 Θ1 PolyGamma@1,Θ1D Θ22 Θ22 n Θ2 - n2 + n2 Θ1 PolyGamma@1,Θ1D Θ22 Θ22 n Θ2 - n2 + n2 Θ1 PolyGamma@1,Θ1D Θ22 Θ22 n PolyGamma@1,Θ1D - n2 Θ22 + n2 Θ1 PolyGamma@1,Θ1D Θ22 8. The A matrix is constructed: H* Compute a11 H1L =k11 H1L -Hk111 2L; Do similar computation for the rest aij HlL =kij HlL -Ikijl 2M, where i=1,2,3, 4; j=1,2,3,4; l=1,2,3,4 *L a111 = ka111 - k111 2 CSCK_MLE_Bias_Calculation_063012.nb a111 a121 a131 a141 a211 a221 a231 a241 a311 a321 a331 a341 a411 a421 a431 a441 a112 a122 a132 a142 a212 a222 a232 a242 a312 a322 a332 a342 a412 a422 a432 a442 a113 a123 a133 a143 a213 a223 a233 a243 a313 a323 a333 a343 a413 a423 a433 a443 a114 a124 a134 a144 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = ka111 - k111 2 ka121 - k112 2 ka131 - k113 2 ka141 - k114 2 ka121 - k112 2 ka221 - k122 2 ka231 - k123 2 ka241 - k124 2 ka131 - k113 2 ka231 - k123 2 ka331 - k133 2 ka341 - k134 2 ka141 - k114 2 ka241 - k124 2 ka341 - k134 2 ka441 - k144 2 ka112 - k112 2 ka122 - k122 2 ka132 - k123 2 ka142 - k124 2 ka122 - k122 2 ka222 - k222 2 ka232 - k223 2 ka242 - k224 2 ka132 - k123 2 ka232 - k223 2 ka332 - k233 2 ka342 - k234 2 ka142 - k124 2 ka242 - k224 2 ka342 - k234 2 ka442 - k244 2 ka113 - k113 2 ka123 - k123 2 ka133 - k133 2 ka143 - k134 2 ka123 - k123 2 ka223 - k223 2 ka233 - k233 2 ka243 - k234 2 ka133 - k133 2 ka233 - k233 2 ka333 - k333 2 ka343 - k334 2 ka143 - k134 2 ka243 - k234 2 ka343 - k334 2 ka443 - k344 2 ka114 - k114 2 ka124 - k124 2 ka134 - k134 2 ka144 - k144 2 17 18 CSCK_MLE_Bias_Calculation_063012.nb a144 a214 a224 a234 a244 a314 a324 a334 a344 a414 a424 a434 a444 = = = = = = = = = = = = = ka144 - k144 2 ka124 - k124 2 ka224 - k224 2 ka234 - k234 2 ka244 - k244 2 ka134 - k134 2 ka234 - k234 2 ka334 - k334 2 ka344 - k344 2 ka144 - k144 2 ka244 - k244 2 ka344 - k344 2 ka444 - k444 2 H* A MATRIX *L H* Construction of AH1L Matrix *L La1 = 88a111, a121, a131, a141<, 8a211, a221, a231, a241<, 8a311, a321, 8a411, a421, a431, a441<<; A1 = La1@@1 ;; p, 1 ;; pDD; MatrixForm@A1D H* Construction of AH2L Matrix *L La2 = 88a112, a122, a132, a142<, 8a212, a222, a232, a242<, 8a312, a322, 8a412, a422, a432, a442<<; A2 = La2@@1 ;; p, 1 ;; pDD; MatrixForm@A2D H* Construction of AH3L Matrix *L La3 = 88a113, a123, a133, a143<, 8a213, a223, a233, a243<, 8a313, a323, 8a413, a423, a433, a443<<; A3 = La3@@1 ;; p, 1 ;; pDD; MatrixForm@A3D H* Construction of AH4L Matrix *L La4 = 88a114, a124, a134, a144<, 8a214, a224, a234, a244<, 8a314, a324, 8a414, a424, a434, a444<<; A4 = La4@@1 ;; p, 1 ;; pDD; MatrixForm@A4D LA = Join@A1, A2, A3, A4, 2D; A = LA@@1 ;; p, 1 ;; p * pDD; MatrixForm@AD 1 n PolyGamma@2, Θ1D 2 0 0 0 0 3n 2 Θ22 0 0 0 0 0 0 0 a331, a341<, a332, a342<, a333, a343<, a334, a344<, CSCK_MLE_Bias_Calculation_063012.nb 0 0 0 0 n 2 Θ22 0 0 n 2 Θ22 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 20 CSCK_MLE_Bias_Calculation_063012.nb 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - 1 2 n PolyGamma@2, Θ1D 0 3n 2 Θ22 2 Θ22 n 2 Θ22 K - n 0 K 0 0 0 0 O 0 0 0 0 O 0 0 - 1 2 n PolyGamma@2, Θ1D 0 0 - 0 3n n 2 Θ22 2 Θ22 n 2 Θ22 0 9. The analytic CSCK MLE bias vector is outputted, ignoring the O(n-2 ) term which is assumed to be negligible. We also formally turn off the SuperLog function employed by the SMLE package: CSCKMLEBIAS = [email protected] SuperLog@OffD : - 2 + Θ1 PolyGamma@1, Θ1D - Θ12 PolyGamma@2, Θ1D 2 n H- 1 + Θ1 PolyGamma@1, Θ1DL SuperLog is now Off. 2 , Θ2 HPolyGamma@1, Θ1D + Θ1 PolyGamma@2, Θ1DL 2 n H- 1 + Θ1 PolyGamma@1, Θ1DL2 > The CSCK MLE bias vector for the 2-parameter gamma distribution contains the polygamma function, which may be unfamilar to some users. More information about this function can be found at http://reference.wolfram.com/mathematica/ref/PolyGamma.html. Also, as the sample size n approaches infinity, the CSCK MLE bias of both parameters approaches zero - MLEs are asymptotically unbiased. CSCK_MLE_Bias_Calculation_063012.nb 21 The CSCK MLE bias vector for the 2-parameter gamma distribution contains the polygamma function, which may be unfamilar to some users. More information about this function can be found at http://reference.wolfram.com/mathematica/ref/PolyGamma.html. Also, as the sample size n approaches infinity, the CSCK MLE bias of both parameters approaches zero - MLEs are asymptotically unbiased. The above code was setup in such a way to make the details of the CSCK MLE bias calculation transparent to the user. The user can follow the determination of the CSCK MLE bias step-by-step. If the user only cares about the outputted CSCK MLE bias, Appendix C contains a Mathematica function that simply outputs the bias. To determine the CSCK MLE bias for a given distribution, use the function b[f_, p_]. The argument f_ is replaced with the probability density function of the distribution of interest. The argument p_ is replaced by the set of parameters, expressed as 9θ1 , θ2 , ..., θp =. In principle, the range of possible values of 9θ1 , function b[f_, p_] can accomodate a probability distribution with any number of θ2 , ..., θp = are reflected in the Expect[x_] function; the user may need to adjust both parameters, unlike the above code which can consider a distribution with at most four parameters. For each distribution, the the number of parameters and the parameter assumptions in Expect[x_] to suit the distribution of interest. RESEARCH PROGRESS Thus far, we have analyzed the CSCK MLE bias for a total of 24 continuous distributions; these distributions are provided in Appendix B. For each of the following distributions, the probability density function (f[yi]), MLEs, concentrated loglikelihood function, and the analytic CSCK MLE bias vector (MLEBias) are provided, if calculable: • 3 one-parameter distributions: exponential, inverse exponential, single-parameter Pareto • 14 two-parameter distributions: gamma, inverse gamma, Weibull, inverse Weibull, Pareto (Type II), inverse Pareto, generalized Pareto, Gumbel, lognormal, inverse Gaussian, beta, loglogistic, paralogistic, inverse paralogistic • 6 three-parameter distributions: transformed gamma, inverse transformed gamma, generalized Pareto 3, generalized beta, Burr, inverse Burr • 1 four-parameter distribution: transformed beta. Of the 24 continuous distributions, we obtained analytic CSCK MLE bias vectors for 17. We could NOT obtain bias vectors for the following 7 distributions: loglogistic, paralogistic, inverse paralogistic, generalized beta, Burr, inverse Burr, and the transformed beta. The first five distributions are part of the Burr distribution family, and the last two distributions are beta distributions. The remainder of this section details the ongoing and future research direction of the CSCK MLE Bias Calculation project: ISSUES REGARDING CERTAIN CUMULANTS The primary reason that analytic CSCK MLE bias vectors could not be obtained for the 7 afore mentioned distributions is because certain cumulants cannot be calculated using Mathematica. For example, consider the loglogistic distribution: 1. The probability density function for the loglogistic distribution (f[yi]) is: The remainder of this section details the ongoing and future research direction of the CSCK MLE Bias Calculation project: 22 CSCK_MLE_Bias_Calculation_063012.nb ISSUES REGARDING CERTAIN CUMULANTS The primary reason that analytic CSCK MLE bias vectors could not be obtained for the 7 afore mentioned distributions is because certain cumulants cannot be calculated using Mathematica. For example, consider the loglogistic distribution: 1. The probability density function for the loglogistic distribution (f[yi]) is: 2. The loglikelihood function for the loglogistic distribution (l[ ]) is: Note: Θ3 is treated as null. 3 = The third partial derivative of the loglikelihood function for the loglogistic distribution with respect to Θ1: 2n Θ 13 - 2 Úni=1 K 2 I-Θ 2-Θ 1 L o g@Θ 2D yiΘ 1 +Θ 2-Θ 1 L o g@yi D yiΘ 1 M 3 I1+Θ 2 -Θ 1 3 yiΘ 1 M - 1 I1+Θ 2-Θ 1 yiΘ 1 M 2 3 I-Θ 2-Θ 1 L o g@Θ 2D yiΘ 1 + Θ 2-Θ 1 L o g@yi D yiΘ 1 M IΘ 2-Θ 1 L o g@Θ 2D2 yiΘ 1 - 2 Θ 2-Θ 1 L o g@Θ 2D L o g@yi D yiΘ 1 + Θ 2-Θ 1 L o g@yi D2 yiΘ 1 M + 1 1+Θ 2-Θ 1 yiΘ 1 I-Θ 2-Θ 1 L o g@Θ 2D3 yiΘ 1 + 3 Θ 2-Θ 1 L o g@Θ 2D2 L o g@yi D yiΘ 1 - 3 Θ 2-Θ 1 L o g@Θ 2D L o g@yi D2 yiΘ 1 + Θ 2-Θ 1 L o g@yi D3 yiΘ 1 MO Note how complicated this derivative is even after applying the Mathematica function “Simplify.” 4. The corresponding cumulant based on the third partial derivative of the loglikelihood function for the loglogistic distribution with respect to Θ1 Hk111L i s : = 4. The corresponding cumulant based on the third partial derivative of the loglikelihood function for the loglogistic distribution with respect to Θ1 Hk111L i s : CSCK_MLE_Bias_Calculation_063012.nb 23 = It can be verified that the k111 integral is incalculable for any range of values for the variable yi or the parameters Θ1 and Θ2 using Mathematica. We attempted to further simplify the integrand, using Mathematica functions such as “Simplify” and “FullSimplify,” but this did not result in the integral computing. We have also tried looking at the sum of integrals instead of the integral of the sum but the integral still did not compute. As k111 cannot be calculated using Mathematica, the analytic CSCK MLE bias vector for the loglogistic distribution cannot be calculated via Mathematica. Similarly, for other six distributions, certain cumulants cannot be calculated. Therefore, it is currently not possible to calculate their analytic CSCK MLE bias vectors via Mathematica. We currently attempting to remedy this problem by considering various numerical analysis techniques involving series expansions, as well as looking at other symbolic calculation packages such as Maple. COMPLEXITY OF CSCK MLE BIAS VECTORS For many of the continuous distributions, their probability density functions involve complex mathematical functions such as the gamma function. Consequently, the analytic CSCK MLE bias vectors for those distributions are incredibly complicated and involve complex mathematical functions. This is especially true for the 3-parameter distributions listed in Appendix B. We have attempted to apply operations such as “Simplify” and “FullSimplify” throughout the CSCK MLE bias vector calculations to decrease the complexity, but it has no effect. When you take third partial derivatives of complex 3-parameter loglikelihood functions, you are going to obtain complex output functions for the CSCK MLE bias. Nonetheless, we are exploring all of the simplification functions of Mathematica in an attempt to further simplify the outputted CSCK MLE bias vectors. This is necessary in order to add these complicated CSCK MLE bias formulas into the Microsoft Excel version of AMOOF. OTHER MLE BIAS CORRECTION APPROACHES Currently, researchers are focused on the CSCK approach to mitigate MLE bias in small samples. However, there are other methods in the literature for determining and accounting for small sample MLE bias. For example, some authors have have attempted to apply operations such as “Simplify” and “FullSimplify” throughout the CSCK MLE bias vector calculations to decrease the complexity, but it has no effect. When you take third partial derivatives of complex 3-parameter loglikelihood CSCK_MLE_Bias_Calculation_063012.nb 24 functions, you are going to obtain complex output functions for the CSCK MLE bias. Nonetheless, we are exploring all of the simplification functions of Mathematica in an attempt to further simplify the outputted CSCK MLE bias vectors. This is necessary in order to add these complicated CSCK MLE bias formulas into the Microsoft Excel version of AMOOF. OTHER MLE BIAS CORRECTION APPROACHES Currently, researchers are focused on the CSCK approach to mitigate MLE bias in small samples. However, there are other methods in the literature for determining and accounting for small sample MLE bias. For example, some authors have considered a preventive approach where the score vector itself is adjusted to mitigate bias. We have not considered whether such a “preventive” approach is preferable to the corrective approach in this paper, where MLEs are adjusted for potential bias ex post. APPLICATION OF CSCK MLE BIAS RESULTS TO AMOOF The ultimate goal of this project is to improve the quality of MLEs used in the process of fitting probability distributions to stochastic financial outcome model data in AMOOF. CSCK MLE Bias Calculation has obtained analytic CSCK MLE bias vectors for most of the probability distributions that are utilized in AMOOF in Appendix B. These CSCK MLE vectors will now be programmed into AMOOF and BMLEs can replace the MLEs that are generated. AMOOF will be subsequently be able to optimally fit probability distributions such that the estimated parameters of those distributions will be unbiased even in the presence of small sample stochastic model data. ACKNOWLEDGMENTS The authors would like to acknowledge the financial support of The Actuarial Foundation, who provided the grant which funds the project “Developing and Testing AMOOF2.” The authors also appreciate the hard work of students at both Central Washington University and the University of Illiois at Urbana-Champaign, who provided much need analytical and technical support. REFERENCES [1] Y.C. Chueh and W.D. Curtis. 2005. “Optimal Probability Density Function Models for Stochastic Model Outcomes: Parametric Model Fitting on Tail Distributions.” New Ideas in Symbolic Computation:Proceedings of the 6th International Mathematica Symposium: 17 pages. [2] Microsoft. 2007. Microsoft Excel [computer software]. Redmond, Washington: Microsoft. [3] Wolfram Research, Inc. 2010. Mathematica Edition: Version 8.0 [computer software]. Champaign, IL: Wolfram Research, Inc. [4] D.R. Cox and E.J. Snell. 1968. “A General Definition of Residuals.” Journal of the Royal Statistical Society B(30): 248-275. [5] G.M. Cordeiro, G.M. and R. Klein. 1994. “Bias Correction in ARMA Models.” Statistics and Probability Letters 19: 169-176. [6] D.E. Giles and H. Feng. 2009. “Bias of the Maximum Likelihood Estimators of theTwo-Parameter Gamma Distribution Revisited.” Econometrics Working Paper EWP0908:1-9. [7] P.H. Johnson, Jr., Y. Qi, and Y.C. Chueh. 2011. “Bias-Corrected Maximum Likelihood Estimation in Actuarial Science.” Available at: http://www.math.uiuc.edu/~pjohnson/Johnson_etal_BMLE_Paper.pdf. [8] C. Rose and M.D. Smith. 2000. “Symbolic Maximum Likelihood Estimation with Mathematica.” The Statistician 49(2): 229-240. [9] S.A. Klugman, S.A., H.H. Panjer, and G.E. Willmot. 2008. “Loss Models: From Data to Decisions, Third Edition.” Hoboken, [5] G.M. Cordeiro, G.M. and R. Klein. 1994. “Bias Correction in ARMA Models.” Statistics and Probability Letters 19: 169-176. [6] D.E. Giles and H. Feng. 2009. “Bias of the Maximum Likelihood Estimators of theTwo-Parameter Gamma Distribution CSCK_MLE_Bias_Calculation_063012.nb 25 Revisited.” Econometrics Working Paper EWP0908:1-9. [7] P.H. Johnson, Jr., Y. Qi, and Y.C. Chueh. 2011. “Bias-Corrected Maximum Likelihood Estimation in Actuarial Science.” Available at: http://www.math.uiuc.edu/~pjohnson/Johnson_etal_BMLE_Paper.pdf. [8] C. Rose and M.D. Smith. 2000. “Symbolic Maximum Likelihood Estimation with Mathematica.” The Statistician 49(2): 229-240. [9] S.A. Klugman, S.A., H.H. Panjer, and G.E. Willmot. 2008. “Loss Models: From Data to Decisions, Third Edition.” Hoboken, NJ: Wiley. APPENDIX A: SMLE MATHEMATICA PACKAGE The following Mathematica package is from Appendix B of Rose and Smith@8D and is used in the CSCK MLE Bias Calculation program to automatically generate a symbolic representation of the loglikelihood function. ________________________________________________________________________________________________ (* :Name: SMLE *) (* :Authors: Colin Rose and Murray D. Smith *) (* :Version: Mathematica v3 or higher *) (* :Legal: Copyright 1999 *) (* :Summary: Symbolic Maximum Likelihood Estimation *) BeginPackage[“SMLE`”] SuperLog::usage = “SuperLog[On] activates the enhanced Log operator, so that Log[Product[__]] objects get converted into sums of logs. SuperLog[Off] switches the enhancement off.” Begin[“`Private`”] SuperLog[Q_] := Module[{erk, iii, nnn}, Product[iii, {iii, nnn}]; (* Pre-Load Product *) Which[ Q === On, Unprotect[Log]; Clear[Log]; Log[Product[x_, {k_, a_, b_}]] := Log[Product[Times[erk, x], {k, a, b}]] /. erk -> 1; Log[Product[HoldPattern[Times[x__]], {k_, a_, b_}]] := Simplify[Map[Sum[#, {k, a, b}]&, Plus@@Map[Expand[PowerExpand[Log[#]]]&, List[x]]] //. Sum[u_. w_, {kk_, aa_, bb_}] :> u Sum[w, {kk, aa, bb}] /; FreeQ[u, kk] == True]; Protect[Log]; Print[“SuperLog is now On.”], Q === Off, Unprotect[Log]; Clear[Log]; Protect[Log]; Print[“SuperLog is now Off.”], True, Print[“Carumbah! Please use SuperLog[On] or SuperLog[Off].”] ]] End[ ] Protect[SuperLog]; EndPackage[ ] APPENDIX B: INVENTORY OF CONTINUOUS DISTRIBUTIONS H********************************************************************* 26 CSCK_MLE_Bias_Calculation_063012.nb APPENDIX B: INVENTORY OF CONTINUOUS DISTRIBUTIONS H********************************************************************* INVENTORY OF CONTINUOUS DISTRIBUTIONS ******************************************************************** H* For each distribution, there is: H1L The probability density function, f@yi D, as provided in Klugman, Panjer, and Willmot; "Loss Models: From Data to Decisions, Third Edition;" Appendix A. H2L The maximum likelihood estimators, MLEs. MLEs are denoted with "hats" over the parameter. In some cases, it is not possible to obtain a closed form for the MLEs. H3L When applicable, the concentrated loglikelihood function, ConLog. Sometimes, it is possible to solve for the MLE of one distributional parameter as a function of the remaining parameters; this MLE can be substituted into the original loglikelihood function, resulting in a function with fewer parameters. H4L The Cox and Snell Cordeiro and Klein HCSCKL analytic MLE bias vectors HK^8-1<AvecHK^8-1<LL, MLEBias. For some distributions, Mathematica was unable to calculate the CSCK MLE bias, because certain cumulants for those distributions apparently be calculated via Mathematica. *L H****************************** ONE PARAMETER DISTRIBUTIONS ******************************L H* EXPONENTIAL *L f@yi D = 1 Θ1 * Exp@- yi Θ1D ` MLEs = :Θ1 ® Úni=1 yi ConLog = N A MLEBias = 80< n > H4L The Cox and Snell Cordeiro and Klein HCSCKL analytic MLE bias vectors HK^8-1<AvecHK^8-1<LL, MLEBias. For some distributions, Mathematica was unable to calculate the CSCK MLE bias, becauseCSCK_MLE_Bias_Calculation_063012.nb certain cumulants for those distributions apparently be calculated via Mathematica. *L H****************************** ONE PARAMETER DISTRIBUTIONS ******************************L H* EXPONENTIAL *L f@yi D = 1 Θ1 * Exp@- yi Θ1D ` MLEs = :Θ1 ® Úni=1 yi n > ConLog = N A MLEBias = 80< H* INVERSE EXPONENTIAL *L f@yi D = HΘ1 yi ^ 2L * Exp@- Θ1 yi D ` MLEs = :Θ1 ® ConLog = N A MLEBias = 9 n Úni=1 Θ1 = n 1 yi > 27 28 CSCK_MLE_Bias_Calculation_063012.nb H* INVERSE EXPONENTIAL *L f@yi D = HΘ1 yi ^ 2L * Exp@- Θ1 yi D ` MLEs = :Θ1 ® n Úni=1 1 yi > ConLog = N A MLEBias = 9 Θ1 = n H* SINGLE PARAMETER PARETO - yi takes values greater than Θ2. Note: Θ2 must be determined prior to any analysis and is therefore fixed; the only true unknown parameter is Θ1. f@yi D = Θ1 * HΘ2L ^ HΘ1L * yi ^ H- Θ1 - 1L ` MLEs = :Θ1 ® - ConLog = N A n > n Log@Θ2D-Úni=1 Log@yi D *L CSCK_MLE_Bias_Calculation_063012.nb H* WEIBULL *L f@yi D = - ã yi Θ1 Θ2 Θ1 J Θ2 yi Θ2 N -1+Θ1 1 ` ` Θ1 MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® n-1Θ1 IÚni=1 yΘ1 i M > Θ1 ConLog = n Log@Θ1D - Θ1 LogBn-1Θ1 IÚni=1 yΘ1 i M F + 1 -1Θ1 IÚn Θ1 Θ1 H- 1 + Θ1L Úni=1 Log@yi D - IÚni=1 yΘ1 i M n i=1 yi M MLEBias = : - 18 Θ1 IΠ2 -2 Zeta@3DM 1 2 n Π4 Θ12 n Π4 1 -Θ1 , Θ2 IΠ4 H- 1 + 2 Θ1L - 6 Π2 I1 + EulerGamma2 + 5 Θ1 - 2 EulerGamma H1 + 2 Θ1LM 72 H- 1 + EulerGammaL Θ1 Zeta@3DM> H* INVERSE WEIBULL *L f@yi D = Θ1 * HΘ2 yi L ^ Θ1 * Exp@- HΘ2 yi L ^ Θ1D * H1 yi L ` ` MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® J N Θ1 > 1 n Úni=1 y-Θ1 i 29 H****************************** THREE 30 CSCK_MLE_Bias_Calculation_063012.nb PARAMETER DISTRIBUTIONS ******************************L H* TRANSFORMED GAMMA *L f@yi D = Θ3 * Hyi Θ2L ^ HΘ1 * Θ3L * Exp@- Hyi Θ2L ^ Θ3D * H1 Hyi * Gamma@Θ1DLL ` ` ` Ún yΘ3 MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® J i=1 i N Θ3 , Θ3 ® No Closed Form Solution> 1 n Θ1 ConLog = n Log@Θ3D - Log@Gamma@Θ1DD - Θ1 Θ3 LogB n Θ1 Úni=1 1 yi F + H- 1 + Θ1 Θ3L Úni=1 Log@yi D - MLEBias = 9- I4 Θ12 PolyGamma@0, Θ1D3 - 4 Θ12 PolyGamma@0, 1 + Θ1D3 + -Θ3 n Θ1 Úni=1 1 yi Úni=1 yΘ3 i H1 + Θ1 PolyGamma@1, 1 + Θ1DL I2 - Θ1 PolyGamma@1, Θ1D H1 + 2 Θ1 PolyGamma@1, 1 + Θ1DL + Θ12 PolyGamma@2, Θ1D + Θ1 PolyGamma@1, 1 + Θ1D I3 + Θ12 PolyGamma@2, Θ1DMM + Θ1 PolyGamma@0, 1 + Θ1D2 I1 + Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@2, 1 + Θ1DM + Θ1 PolyGamma@0, Θ1D2 I1 - 12 Θ1 PolyGamma@0, 1 + Θ1D + Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@2, 1 + Θ1DM - Θ1 PolyGamma@0, 1 + Θ1D I2 + 8 Θ1 PolyGamma@1, 1 + Θ1D + 4 Θ12 PolyGamma@1, 1 + Θ1D2 + Θ1 PolyGamma@2, 1 + Θ1D Θ1 PolyGamma@1, Θ1D H6 + 8 Θ1 PolyGamma@1, 1 + Θ1D + Θ1 PolyGamma@2, 1 + Θ1DLM - Θ1 PolyGamma@0, Θ1D I- 2 - 12 Θ1 PolyGamma@0, 1 + Θ1D2 - 8 Θ1 PolyGamma@1, 1 + Θ1D - 4 Θ12 PolyGamma@1, 1 + Θ1D2 - Θ1 PolyGamma@2, 1 + Θ1D + Θ1 PolyGamma@1, Θ1D H6 + 8 Θ1 PolyGamma@1, 1 + Θ1D + Θ1 PolyGamma@2, 1 + Θ1DL + 2 PolyGamma@0, 1 + Θ1D I1 + Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@2, 1 + Θ1DMMM I2 n IΘ1 PolyGamma@0, Θ1D2 - 2 Θ1 PolyGamma@0, Θ1D PolyGamma@0, 1 + Θ1D + Θ1 PolyGamma@0, 1 + Θ1D2 - H- 1 + Θ1 PolyGamma@1, Θ1DL H1 + Θ1 PolyGamma@1, 1 + Θ1DLM M, 2 - IΘ2 IΘ1 H- 1 + Θ3L PolyGamma@0, Θ1D + Θ1 PolyGamma@0, Θ1D HH2 - 3 Θ3L PolyGamma@0, 1 + Θ1D + 2 Θ3 H- 2 + Θ1 PolyGamma@1, 1 + Θ1DLL - Θ1 Θ3 H1 + Θ1 PolyGamma@1, 1 + Θ1DL I1 + Θ1 PolyGamma@0, 1 + Θ1D2 + Θ1 PolyGamma@1, 1 + Θ1DM PolyGamma@2, Θ1D + 4 3 PolyGamma@0, Θ1D2 I- 1 + 2 Θ3 + Θ1 PolyGamma@0, 1 + Θ1D2 H- 1 + 3 Θ3 + Θ1 PolyGamma@1, Θ1DL Θ1 PolyGamma@1, 1 + Θ1D + 3 Θ1 Θ3 PolyGamma@1, 1 + Θ1D + 2 Θ1 Θ3 PolyGamma@0, 1 + Θ1D H4 + Θ1 PolyGamma@1, Θ1D - 2 Θ1 PolyGamma@1, 1 + Θ1DL + Θ1 PolyGamma@1, Θ1D H2 - 2 Θ3 + Θ1 H2 - 3 Θ3L PolyGamma@1, 1 + Θ1DL - 2 Θ12 Θ3 PolyGamma@2, 1 + Θ1DM Θ1 PolyGamma@1, Θ1D2 IΘ1 PolyGamma@0, 1 + Θ1D2 H1 + Θ1 PolyGamma@1, 1 + Θ1DL H1 + Θ1 PolyGamma@1, 1 + Θ1DL H- 1 + Θ3 + Θ1 H- 1 + 2 Θ3L PolyGamma@1, 1 + Θ1DL + Θ1 Θ3 PolyGamma@0, 1 + Θ1D H4 + 6 Θ1 PolyGamma@1, 1 + Θ1D + Θ1 PolyGamma@2, 1 + Θ1DLM + PolyGamma@1, Θ1D I2 Θ12 Θ3 PolyGamma@0, 1 + Θ1D3 + Θ12 PolyGamma@0, 1 + Θ1D4 - H1 + Θ1 PolyGamma@1, 1 + Θ1DL H- 1 + 2 Θ3 + Θ1 H- 1 + 3 Θ3L PolyGamma@1, 1 + Θ1DL + Θ12 Θ3 PolyGamma@0, 1 + Θ1D I6 PolyGamma@1, 1 + Θ1D + 4 Θ1 PolyGamma@1, 1 + Θ1D2 + PolyGamma@2, 1 + Θ1DM - Θ1 PolyGamma@0, 1 + Θ1D2 I- 2 + Θ3 + Θ1 H- 2 + Θ3L PolyGamma@1, 1 + Θ1D + 2 Θ12 Θ3 PolyGamma@2, 1 + Θ1DMM - PolyGamma@0, Θ1D IΘ1 PolyGamma@0, 1 + Θ1D3 HΘ3 + 2 Θ1 PolyGamma@1, Θ1DL + 2 Θ1 Θ3 PolyGamma@0, 1 + Θ1D2 H2 + 2 Θ1 PolyGamma@1, Θ1D - Θ1 PolyGamma@1, 1 + Θ1DL + Θ3 I2 + 6 Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@1, 1 + Θ1D2 + Θ1 PolyGamma@2, 1 + Θ1D + Θ1 PolyGamma@1, Θ1D I- 6 - 6 Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@1, 1 + Θ1D2 - Θ1 PolyGamma@2, 1 + Θ1DMM - PolyGamma@0, 1 + Θ1D IΘ3 I- 2 + Θ12 PolyGamma@2, Θ1D + Θ1 PolyGamma@1, 1 + Θ1D I- 3 + Θ12 PolyGamma@2, Θ1DM + 2 Θ12 PolyGamma@2, 1 + Θ1DM + Θ1 PolyGamma@1, Θ1D I- 2 + 3 Θ3 + 2 Θ1 H- 1 + 2 Θ3L PolyGamma@1, 1 + Θ1D + 2 Θ12 Θ3 PolyGamma@2, 1 + Θ1DMMMMM I2 n Θ3 IΘ1 PolyGamma@0, Θ1D2 - 2 Θ1 PolyGamma@0, Θ1D PolyGamma@0, 1 + Θ1D + 2 Θ1 PolyGamma@0, 1 + Θ1D2 - H- 1 + Θ1 PolyGamma@1, Θ1DL H1 + Θ1 PolyGamma@1, 1 + Θ1DLM M, 2 IΘ3 I- Θ1 PolyGamma@0, Θ1D3 + Θ1 PolyGamma@0, 1 + Θ1D3 + Θ1 PolyGamma@0, Θ1D2 H6 + 3 PolyGamma@0, 1 + Θ1D - 4 Θ1 PolyGamma@1, Θ1D - 2 Θ1 PolyGamma@1, 1 + Θ1DL 2 Θ1 PolyGamma@0, 1 + Θ1D2 H- 3 + 2 Θ1 PolyGamma@1, Θ1D + Θ1 PolyGamma@1, 1 + Θ1DL + H- 1 + Θ1 PolyGamma@1, Θ1DL I- 4 - 8 Θ1 PolyGamma@1, 1 + Θ1D - 2 Θ12 PolyGamma@1, 1 + Θ1D2 - Θ1 PolyGamma@2, 1 + Θ1D + MLEBias = Mathematica unable to calculate CSCK_MLE_Bias_Calculation_063012.nb H****************************** FOUR PARAMETER DISTRIBUTIONS ******************************L H* TRANSFORMED BETA *L f@yi D = HGamma@Θ1 + Θ4D HGamma@Θ1D * Gamma@Θ4DLL * HΘ3 * Hyi Θ2L ^ HΘ3 * Θ4LL Hyi * H1 + Hyi Θ2L ^ Θ3L ^ HΘ1 + Θ4LL ` ` MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® No Closed Form Solution, ` ` Θ3 ® No Closed Form Solution, Θ4 ® No Closed Form Solution> ConLog = N A MLEBias = Mathematica unable to calculate APPENDIX C: CSCK MLE BIAS CALCULATION FUNCTION: The following function will output only the CSCK MLE bias for an inputted probability density function (f_) and set of parameters (p_). It needs to be used in conjunction with the SMLE Mathematica package: b@f_, p_D := Module@8l, Gradient, Hessian, ThirdPartialDer, ExpectHessian, ExpectThirdPartialDer, DerivativeExpectHessian, aijk, Amatrix, Kinv, vecKinv, BIAS, Expect<, Expect@x_D := Integrate@x * f, 8yi , 0, ¥<, Assumptions ® 8Θ1 Î Reals, Θ2 Î Reals, Θ3 Î Reals, Θ3 > 0, Θ2 > 0, Θ1 > 0<D; SuperLog@OnD; l = Log@Ûni=1 fD; Gradient = D@l, 8p<D; Hessian = D@l, 8p, 2<D; ThirdPartialDer = D@l, 8p, 3<D; ExpectHessian = Map@Expect@ðD &, HessianD; ExpectThirdPartialDer = Map@Expect@ðD &, ThirdPartialDerD; DerivativeExpectHessian = D@ExpectHessian, 8p<D; aijk = DerivativeExpectHessian - ExpectThirdPartialDer 2; Amatrix = Apply@Join, aijk~ Join~ 82<D; Kinv = Inverse@- ExpectHessianD; vecKinv = Flatten@Transpose@KinvDD; BIAS = [email protected]; SuperLog@OffD; BIASD 31