Download csck mle bias calculation

CSCK MLE BIAS CALCULATION
Paul H. Johnson, Jr.H1L
Department of Mathematics, University of Illinois at Urbana-Champaign, Champaign, IL
[email protected]
Yongxue Qi
Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, IL
[email protected]
Yvonne C. Chueh
Department of Actuarial Science, Statistics, and Mathematics, Central Washington University, Ellensburg, WA
[email protected]
(1) Corresponding Author
ABSTRACT
Actuarial Model Optimal Outcome Fit (AMOOF) is an open source, high performance computation program that is currently
under development at Central Washington University. AMOOF is an add-in to both Microsoft Excel and Mathematica that is
designed to assist actuaries and other financial analysts in efficient stochastic modeling of financial outcomes arising from
areas including capital determination, reserve setting, risk analysis, and product pricing. AMOOF utilizes the method of
maximum likelihood to estimate the unknown parameters of probability distributions fit to stochastic financial outcome model
data. Maximum likelihood estimators (MLEs) possess highly desirable properties such as asymptotic unbiasedness,
consistency, and asymptotic normality. Unfortunately, many of these properties, specifically unbiasedness, may not be valid
for small sample sizes (for example, data of less than 100 observations).
In this paper, we consider the Cox and Snell / Cordeiro and Klein (CSCK) methodology for determining the analytic bias
associated with MLEs that were calculated using small sample data. Researchers from the University of Illinois at UrbanaChampaign have developed a Mathematica program, entiled “CSCK MLE Bias Calculation,” that enables the user to calculate
the analytic CSCK MLE bias vectors for various probability distributions with up to four unknown parameters. The process for
arriving at the CSCK bias associated with each MLE is outlined step-by-step in a fashion that makes the entire process
transparent to the user. Once the CSCK MLE bias has been determined for a given MLE, the bias corrected MLE (BMLE) can
be obtained by subtracting the CSCK MLE bias (evaluated at the MLEs) from the MLE resulting in a parameter estimate that is
unbiased even with small sample sizes. Ultimately, this CSCK methodology will be implemented in AMOOF to improve the
quality of MLEs generated via that program.
the analytic CSCK MLE bias vectors for various probability distributions with up to four unknown parameters. The process for
2
arriving
at the CSCK bias associated with each MLE is outlined step-by-step in a fashion that makes the entire process
CSCK_MLE_Bias_Calculation_063012.nb
transparent to the user. Once the CSCK MLE bias has been determined for a given MLE, the bias corrected MLE (BMLE) can
be obtained by subtracting the CSCK MLE bias (evaluated at the MLEs) from the MLE resulting in a parameter estimate that is
unbiased even with small sample sizes. Ultimately, this CSCK methodology will be implemented in AMOOF to improve the
quality of MLEs generated via that program.
INTRODUCTION
Actuarial Model Optimal Outcome Fit HAMOOFL@1D is the current pilot project of the departments of Actuarial Science, Statistics,
and Mathematics and Computer Science at Central Washington University. AMOOF is an add-in to both Microsoft Excel@2D and
Mathematica@3D that is being designed to assist actuaries and other financial analysts in efficiently analyzing financial
outcomes arising from areas including capital determination, reserve setting, risk analysis, and product pricing. For a
stochastic financial outcome data set, AMOOF considers candidate probability density functions that can be fit to the data and,
using various statistical criteria, finds the density function that best fits the data. AMOOF not only considers standard
probability distributions, but also transformed, inverse, and mixed probability distributions in order to arrive at the optimal
probability density function for the given stochastic model data. AMOOF also enables its user to fit a probability density
function to the tails of a distribution and to consider extreme values, which are often of interest to various risk management
and regulatory authorities. AMOOF is both open source and high performance computation program that will be made
available to insurance companies and other financial institutions. Financial outcomes that were once too time prohibitive to
consider can be analyzed using the easy-to-implement AMOOF program. More information regarding the Actuarial Model
Optimal Outcome Fit project can be found at http://www.cwu.edu/~chueh/AMOOF.html.
AMOOF uses the method of maximum likelihood to estimate the parameters of probability distributions fit to stochastic model
data. The method of maximum likelihood is widely used to estimate the unknown parameters of probability distributions that
are fit to data. Maximum likelihood estimators (MLEs) have many desirable properties. For example, MLEs are asymptotically
unbiased, consistent, and asymptotically normal. However, many of these properties rely on having a large sample size. This
means that MLE properties, such as unbiasedness, may not be valid for small sample sizes. While “small” is not a strictly
defined term, we will use it to refer to data sets comprised of fewer than 100 observations. Researchers from various fields
have highlighted the asymptotic requirement in order to obtain unbiased parameter estimates via the method of maximum
likelihood.
Cox and Snell@4D first considered analytic expressions for the bias of MLEs calculated with small samples as part of their study
of a general definition of residuals. Cordeiro and Klein@5D later extended the Cox and Snell analysis and generalized their
result. We shall refer to the method for obtaining analytic expressions for small sample MLE bias as the Cox and Snell /
Cordeiro and Klein (CSCK) method. The CSCK methodology provides a “corrective approach’’ for mitigating small sample
MLE bias, where a bias-corrected MLE (BMLE) is obtained by subtracting the CSCK MLE bias (evaluated at the MLEs) from
the MLE. While the CSCK method has been around for decades, it is only recently that computational software has allowed
for efficient numerical calculation of the CSCK MLE bias for various distributions. Giles and co-authors have considered the
CSCK MLE bias of many common 2-parameter distributions, such as the gamma distribution@6D .
In this paper, we introduce a Mathematica program developed by researchers from the University of Illinois at Urbana-
Cordeiro and Klein (CSCK) method. The CSCK methodology provides a “corrective approach’’ for mitigating small sample
MLE bias, where a bias-corrected MLE (BMLE) is obtained by subtracting the CSCK CSCK_MLE_Bias_Calculation_063012.nb
MLE bias (evaluated at the MLEs) from 3
the MLE. While the CSCK method has been around for decades, it is only recently that computational software has allowed
for efficient numerical calculation of the CSCK MLE bias for various distributions. Giles and co-authors have considered the
CSCK MLE bias of many common 2-parameter distributions, such as the gamma distribution@6D .
In this paper, we introduce a Mathematica program developed by researchers from the University of Illinois at UrbanaChampaign, entiled “CSCK MLE Bias Calculation,” that implements the CSCK methodology. The user will be able to
determine the CSCK MLE bias for several probability distributions that are commonly utilized in the analysis of stochastic
financial outcomes. The program is designed so that for a probability distribution with at most four unknown parameters, the
CSCK MLE bias vector is calculated in a fashion that makes the entire process transparent to the user. The entire CSCK
method is outlined step-by-step throughout the program with the the desired CSCK MLE bias vector outputted at the end.
Once the CSCK MLE bias has been determined for a given MLE, the BMLE can be obtained as an unbiased parameter
estimate even with small sample sizes.
The ultimate goal of this project is to improve the quality of MLEs used in fitting probability distributions to stochastic financial
outcome model data in AMOOF.
OVERVIEW OF THE CSCK METHODOLOGY
Consider a probability distribution with p unknown parameters: θ
= Iθ1 , θ2 , ..., θp M . Let l(θ) denote the total
'
loglikelihood function, based on a sample of n observations. It is assumed throughout this discussion that the observations are
not necessarily identically distributed. Assume l(θ) is regular with respect to all derivatives of the elements of θ up to and
including the third order.
Define the following joint cumulants of l(θ) for i, j, l = 1, 2, ..., p:
2
3
kij =E[ ¶¶θi θl j ], kijl =E[ ¶θ¶i θlj θl ], and kij,l
2
=E[ ¶¶θi θl j
¶l
].
¶θl
Also, define the cumulant derivative: k
HlL
ij
=
¶kij
¶θl
.
The total Fisher information Matrix of order p for θ is denoted as K
is denoted as K
-1
= { -kij }. The inverse of total Fisher information Matrix
= { -k ij }.
`
θs denote the MLE of the s-th element (parameter) of θ. Cox and Snell proved that for independent observations, the
`
bias of θs , bs , for s = 1, 2, ..., p, can be expressed as:
Let
bs
`
p
= E[θs - θs ] = Úi, j,l=1 k si k jl [0.5kijl + kij,l ] + O(n-2 ).
The total Fisher information Matrix of order p for θ is denoted as K
4
= { -kij }. The inverse of total Fisher information Matrix
CSCK_MLE_Bias_Calculation_063012.nb
-1
ij
is denoted as K
= { -k }.
`
Let θs denote the MLE of the s-th element (parameter) of θ. Cox and Snell proved that for independent observations, the
`
bias of θs , bs , for s = 1, 2, ..., p, can be expressed as:
bs
`
p
= E[θs - θs ] = Úi, j,l=1 k si k jl [0.5kijl + kij,l ] + O(n-2 ).
Cordeiro and Klein verified that the assumption of independence in the preceding equation can be relaxed as long as all k’s
are assumed to be O(n), resulting in the following expression:
bs
`
p
p
= E[θs - θs ] = Úi=1 k si Új,l=1 [k HlL ij - 0.5kijl ]k jl + O(n-2 ).
Giles and co-authors point out that this second equation for
bs is more computationally efficient than the first equation
for
bs as there are no kij,l terms to evaluate; we will consider this second equation for bs exclusively from this point on. The
second equation for
bs
can be expressed in matrix notation to provide a convenient expression for the MLE bias vector, b.
Let A
`
= {AH1L | AH2L |--- AHpL }
HlL
where A
If θ denotes the MLE vector:
b
= k HlL ij - 0.5kijl for l = 1, 2, ..., p.
`
= E[θ - θ] = K -1 Avec(K -1 ) + O(n-2 ).
Ž
`
The bias-corrected MLE (BMLE) vector, θ , under the CSCK methodology is the difference between the MLE vector (θ) and
`
the MLE bias vector in evaluated at the MLEs (b):
Ž ` `
θ = θ - b. Note, as the true values of the distributional parameters θ are
unknown, the CSCK MLE bias is estimated by plugging in the corresponding MLEs. Componentwise, for the s-th parameter,
the BMLE of θs ,
` `
Ž
θs , is equal to θs - bs .
Johnson and co - authors@7D determined the CSCK MLE biases for the lognormal, two-parameter gamma, and two-parameter
Weibull distributions. They also provided two simulation analyses. The first simulation demonstrated that BMLEs have
preferable empirical properties when compared to MLEs for the lognormal, two-parameter gamma, and two-parameter Weibull
distributions. The second simulation showed that BMLEs were preferable to MLEs for the loss reserving of an illustrative 20period equity-linked insurance contract for both the lognormal and two-parameter Weibull distributions, but not for the twoparameter gamma distribution.
Johnson and co - authors@7D determined the CSCK MLE biases for the lognormal, two-parameter gamma, and two-parameter
Weibull distributions. They also provided two simulation analyses. The first simulationCSCK_MLE_Bias_Calculation_063012.nb
demonstrated that BMLEs have
5
preferable empirical properties when compared to MLEs for the lognormal, two-parameter gamma, and two-parameter Weibull
distributions. The second simulation showed that BMLEs were preferable to MLEs for the loss reserving of an illustrative 20period equity-linked insurance contract for both the lognormal and two-parameter Weibull distributions, but not for the twoparameter gamma distribution.
PROGRAMMING
The CSCK MLE Bias Calculation program is a Mathematica application that implements the CSCK methodology. The program
is set up in the following step-by-step format:
1. Mathematica will load the SMLE package, which was created and described in detail in Rose and Smith’s excellent tutorial
article@8D . SMLE stands for “Symbolic Maximum Likelihood Estimation.” Using the SMLE package, the “Log’’ function in
Mathematica is redefined via the “SuperLog’’ function to allow for a full symbolic representation of the total loglikelihood
function. The SMLE package effectively gives the Log function the information it needs to covert the loglikelihood function into
a sum of functions involving natural logarithms. For example, Mathematica is explicitly informed via the SMLE package that
the sample size is a positive integer, variables and parameters are real-valued, and the arguments of natural logarithms are
positive. With the SMLE package, the user does not have to manually determine the total loglikelihood function; this is
automatically done in CSCK MLE Bias Calculation. The SMLE package is provided in Appendix A.
2. The user inputs both the probability density function and the number of distributional parameters. Appendix B provides an
inventory of continuous probability distributions that we have considered so far using CSCK MLE Bias Calculation, obtained
from a popular actuarial science textbook@9D . In some cases, the user will also have to input the assumptions for the
distributional parameters in the function that defines the expected value function, Expect[x_], used in the cumulant calculation.
The default assumes that all parameters are positive, real-valued numbers. Finally, it can be verified that all of the probability
distributions in Appendix B are O(n) as is required to use the CSCK methodology.
3. The loglikelihood function is automatically determined using the SMLE package.
4. Various partial derivatives of the loglikelihood function are computed with respect to the distributional parameters (first order
partial derivatives, second order partial derivatives, third order partial derivatives each with respect to a specific parameter;
and mixed partial derivatives with respect to two and three parameters. These partial derivatives are calculated in order to
determine the cumulants in the next step.
5. Various cumulants required for the CSCK method are calculated.
6. Various cumulant derivatives required for the CSCK method are calculated.
7. The total Fisher Information Matrix K and its inverse K
-1
are constructed.
8. The A matrix is constructed.
-2
`
5. Various cumulants required for the CSCK method are calculated.
6
CSCK_MLE_Bias_Calculation_063012.nb
6. Various cumulant derivatives required for the CSCK method are calculated.
7. The total Fisher Information Matrix K and its inverse K
-1
are constructed.
8. The A matrix is constructed.
9. The analytic CSCK MLE bias vector is outputted, ignoring the O(n-2 ) term which is assumed to be negligible: b
`
= E[θ -
θ] = K -1 Avec(K -1 ). For a distribution with p parameters, the CSCK MLE bias vector would be of the form:
`
`
`
9b@θ̀1 D, b@θ2 D, ..., b@θp D=, where b@θsD denotes the CSCK bias of the MLE of θs . In practice, the BMLE vector
would be determined by taking the MLE and subtracting from it b evaluated at the MLEs.
CSCK MLE BIAS CALCULATION PROGRAM
The CSCK MLE Bias Calculation program will be illustrated using the 2-parameter gamma distribution. The CSCK MLE bias
for the 2-parameter gamma distribution has been previously calculated in full detail by Giles and co - authors@6D ; it can be
verified that our program provides the same results.
1. Mathematica loads the SMLE package and turns on the SuperLog function:
<< SMLE.m
SuperLog@OnD
SuperLog is now On.
2. USER INPUTS: The user inputs both the probability density function (f@yi D) and the number of distributional parameters
(p) in the light blue sections. It is not needed for the 2-parameter gamma distribution, but if necessary, the user should change
the assumptions for the parameters in the Expect[x_] function in the light blue section. Note: If the distribution of interest has
less than four parameters, the additional parameters are treated as null throughout the remainder of the CSCK MLE bias
calculation:
f@yi D = Hyi  Θ2L ^ Θ1 * Exp@- yi  Θ2D * H1  Hyi * Gamma@Θ1DLL;
p = 2;
Expect@x_D := Integrate@x * f@yi D, 8yi , 0, ¥<, Assumptions ®
8Θ1 Î Reals, Θ2 Î Reals, Θ3 Î Reals, Θ4 Î Reals, Θ4 > 0, Θ3 > 0, Θ2 > 0, Θ1 > 0<D
CSCK_MLE_Bias_Calculation_063012.nb
3. The loglikelihood function (l[Θ1,Θ2,Θ3,Θ4]) is automatically determined using the SMLE package (“Log” is redefined via
“SuperLog” in Step 1):
L = ä f@yi D
n
i=1
l@Θ1, Θ2, Θ3, Θ4D = Log@LD
ã- Θ2 I
yi
ä
n
i=1
yi
Θ2
M
Θ1
Gamma@Θ1D yi
1
Θ2
n Θ2 HΘ1 Log@Θ2D + Log@Gamma@Θ1DDL + HΘ2 - Θ1 Θ2L â Log@yi D + â yi
n
n
i=1
i=1
4. Various partial derivatives of the loglikelihood function are computed with respect to distributional parameters:
H* Compute
¶l@Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1
firstlΘ1 = Simplify@¶8Θ1,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l@Θ1,Θ2,Θ3,Θ4D
*L
¶Θ2
firstlΘ2 = Simplify@¶8Θ2,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l@Θ1,Θ2,Θ3,Θ4D
*L
¶Θ3
firstlΘ3 = Simplify@¶8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l@Θ1,Θ2,Θ3,Θ4D
*L
¶Θ4
firstlΘ4 = Simplify@¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ12
secondlΘ1 = Simplify@¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ22
secondlΘ2 = Simplify@¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ32
secondlΘ3 = Simplify@¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ42
secondlΘ4 = Simplify@¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ13
thirdlΘ1 = Simplify@¶8Θ1,3< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ23
thirdlΘ2 = Simplify@¶8Θ2,3< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ33
thirdlΘ3 = Simplify@¶8Θ3,3< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ43
thirdlΘ4 = Simplify@¶8Θ4,3< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1¶Θ2
secondlΘ1Θ2 = Simplify@¶8Θ1,1<,8Θ2,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1¶Θ3
secondlΘ1Θ3 = Simplify@¶8Θ1,1<,8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD
7
8
CSCK_MLE_Bias_Calculation_063012.nb
secondlΘ1Θ3 = Simplify@¶8Θ1,1<,8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1¶Θ4
secondlΘ1Θ4 = Simplify@¶8Θ1,1<,8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ2¶Θ3
secondlΘ2Θ3 = Simplify@¶8Θ2,1<,8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ2¶Θ4
secondlΘ2Θ4 = Simplify@¶8Θ2,1<,8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l2 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ3¶Θ4
secondlΘ3Θ4 = Simplify@¶8Θ3,1<,8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1¶Θ2¶Θ3
thirdlΘ1Θ2Θ3 = Simplify@¶8Θ1,1< ¶8Θ2,1< ¶8Θ3,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1¶Θ2¶Θ4
thirdlΘ1Θ2Θ4 = Simplify@¶8Θ1,1< ¶8Θ2,1< ¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1¶Θ3¶Θ4
thirdlΘ1Θ3Θ4 = Simplify@¶8Θ1,1< ¶8Θ3,1< ¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ2¶Θ3¶Θ4
thirdlΘ2Θ3Θ4 = Simplify@¶8Θ2,1< ¶8Θ3,1< ¶8Θ4,1< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶l3 @Θ1,Θ2,Θ3,Θ4D
*L
¶Θ1¶Θ22
thirdlΘ1secondΘ2 = Simplify@¶8Θ1,1< ¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ1¶Θ32
thirdlΘ1secondΘ3 = Simplify@¶8Θ1,1< ¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ1¶Θ42
thirdlΘ1secondΘ4 = Simplify@¶8Θ1,1< ¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ2¶Θ12
thirdlΘ2secondΘ1 = Simplify@¶8Θ2,1< ¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ2¶Θ32
thirdlΘ2secondΘ3 = Simplify@¶8Θ2,1< ¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ2¶Θ42
thirdlΘ2secondΘ4 = Simplify@¶8Θ2,1< ¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ3¶Θ12
thirdlΘ3secondΘ1 = Simplify@¶8Θ3,1< ¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ3¶Θ22
thirdlΘ3secondΘ2 = Simplify@¶8Θ3,1< ¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ3¶Θ42
thirdlΘ3secondΘ4 = Simplify@¶8Θ3,1< ¶8Θ4,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ4¶Θ12
thirdlΘ4secondΘ1 = Simplify@¶8Θ4,1< ¶8Θ1,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ4¶Θ22
thirdlΘ4secondΘ2 = Simplify@¶8Θ4,1< ¶8Θ2,2< l@Θ1, Θ2, Θ3, Θ4DD
H* Compute
¶Θ4¶Θ32
thirdlΘ4secondΘ3 = Simplify@¶8Θ4,1< ¶8Θ3,2< l@Θ1, Θ2, Θ3, Θ4DD
CSCK_MLE_Bias_Calculation_063012.nb
- n HLog@Θ2D + PolyGamma@0, Θ1DL + â Log@yi D
n
- n Θ1 Θ2 + Úni=1 yi
Θ22
0
0
- n PolyGamma@1, Θ1D
n Θ1 Θ2 - 2 Úni=1 yi
Θ23
0
0
- n PolyGamma@2, Θ1D
- 2 n Θ1 Θ2 + 6 Úni=1 yi
Θ24
0
0
n
Θ2
0
0
0
0
0
0
0
0
0
n
Θ22
0
0
0
0
0
0
0
0
i=1
9
10
CSCK_MLE_Bias_Calculation_063012.nb
0
0
0
5. Various cumulants required for the CSCK method are calculated:
H* Compute k11 =EA
¶2 l E *L
¶Θ1 2
k11 = Factor@Expect@secondlΘ1DD
H* Compute k22 =EA
¶2 l E *L
¶Θ2 2
k22 = Factor@Expect@secondlΘ2DD
H* Compute k33 =EA
¶2 l E *L
¶Θ3 2
k33 = Factor@Expect@secondlΘ3DD
H* Compute k44 =EA
¶2 l E *L
¶Θ4 2
k44 = Factor@Expect@secondlΘ4DD
H* Compute k12 =k21 =EA
¶2 l E *L
¶Θ1 ¶Θ2
k12 = Factor@Expect@secondlΘ1Θ2DD
H* Compute k13 =k31 =EA
¶2 l E *L
¶Θ1 ¶Θ3
k13 = Factor@Expect@secondlΘ1Θ3DD
H* Compute k14 =k41 =EA
¶2 l E *L
¶Θ1 ¶Θ4
k14 = Factor@Expect@secondlΘ1Θ4DD
H* Compute k23 =k32 =EA
¶2 l E *L
¶Θ2 ¶Θ3
k23 = Factor@Expect@secondlΘ2Θ3DD
H* Compute k24 =k42 =EA
¶2 l E *L
¶Θ2 ¶Θ4
k24 = Factor@Expect@secondlΘ2Θ4DD
H* Compute k34 =k43 =EA
¶2 l E *L
¶Θ3 ¶Θ4
k34 = Factor@Expect@secondlΘ3Θ4DD
H* Compute k111 =EA
¶3 l E *L
¶Θ1 3
k111 = Factor@Expect@thirdlΘ1DD
H* Compute k222 =EA
¶2 l E *L
¶Θ2 3
k222 = Factor@Expect@thirdlΘ2DD
H* Compute k333 =EA
¶3 l E *L
¶Θ3 3
k333 = Factor@Expect@thirdlΘ3DD
H* Compute k444 =EA
¶3 l E *L
¶Θ4 3
k444 = Factor@Expect@thirdlΘ4DD
CSCK_MLE_Bias_Calculation_063012.nb
k444 = Factor@Expect@thirdlΘ4DD
H* Compute k112 =k121 =k211 =EA
¶3 l E *L
¶Θ1 2 ¶Θ2
k112 = Factor@Expect@thirdlΘ2secondΘ1DD
H* Compute k113 =k131 =k311 =EA
¶3 l E *L
¶Θ1 2 ¶Θ3
k113 = Factor@Expect@thirdlΘ3secondΘ1DD
H* Compute k114 =EA
¶3 l E *L
¶Θ1 2 ¶Θ4
k114 = Factor@Expect@thirdlΘ4secondΘ1DD
H* Compute k122 =k212 =k221 =EA
¶3 l E *L
¶Θ2 2 ¶Θ1
k122 = Factor@Expect@thirdlΘ1secondΘ2DD
H* Compute k133 =k313 =k331 =EA
¶3 l E *L
¶Θ3 2 ¶Θ1
k133 = Factor@Expect@thirdlΘ1secondΘ3DD
H* Compute k144 =k414 =k441 =EA
¶3 l E *L
¶Θ4 2 ¶Θ1
k144 = Factor@Expect@thirdlΘ1secondΘ4DD
H* Compute k223 =k232 =k322 =EA
¶3 l E *L
¶Θ2 2 ¶Θ3
k223 = Factor@Expect@thirdlΘ3secondΘ2DD
H* Compute k224 =k242 =k422 =EA
¶3 l E *L
¶Θ2 2 ¶Θ4
k224 = Factor@Expect@thirdlΘ4secondΘ2DD
H* Compute k334 =k343 =k433 =EA
¶3 l E *L
¶Θ3 2 ¶Θ4
k334 = Factor@Expect@thirdlΘ4secondΘ3DD
H* Compute k123 =k321 =k231 =k213 =k132 =k312 =EA
k123 = Factor@Expect@thirdlΘ1Θ2Θ3DD
H* Compute k124 =k421 =k241 =k214 =k142 =k412 =EA
k124 = Factor@Expect@thirdlΘ1Θ2Θ4DD
H* Compute k134 =k431 =k341 =k314 =k143 =k413 =EA
k134 = Factor@Expect@thirdlΘ1Θ3Θ4DD
H* Compute k234 =k432 =k342 =k324 =k243 =k423 =EA
k234 = Factor@Expect@thirdlΘ2Θ3Θ4DD
H*Compute k233 =k332 =k323 =EA
¶3 l E *L
¶Θ3 2 ¶Θ2
k233 = Factor@Expect@thirdlΘ2secondΘ3DD
H* Compute k244 =k442 =k424 =EA
¶3 l E *L
¶Θ4 2 ¶Θ2
k244 = Factor@Expect@thirdlΘ2secondΘ4DD
H* Compute k344 =k443 =k434 =EA
¶3 l E *L
¶Θ4 2 ¶Θ3
k344 = Factor@Expect@thirdlΘ3secondΘ4DD
- n PolyGamma@1, Θ1D
¶3 l E *L
¶Θ1 ¶Θ2 ¶Θ3
¶3 l E *L
¶Θ1 ¶Θ2 ¶Θ4
¶3 l E *L
¶Θ1 ¶Θ2 ¶Θ3
¶3 l E *L
¶Θ1 ¶Θ2 ¶Θ3
11
12
CSCK_MLE_Bias_Calculation_063012.nb
n Θ1
Θ22
0
0
n
Θ2
0
0
0
0
0
- n PolyGamma@2, Θ1D
4 n Θ1
Θ23
0
0
0
0
0
n
Θ22
0
0
0
0
0
0
0
0
0
0
0
0
6. Various cumulant derivatives required for the CSCK method are calculated:
H* Compute k11 H1L =
¶k11
¶Θ1
*L
ka111 = Factor@¶8Θ1,1< k11D
CSCK_MLE_Bias_Calculation_063012.nb
ka111 = Factor@¶8Θ1,1< k11D
H* Compute k11 H2L =
¶k11
¶Θ2
*L
ka112 = Factor@¶8Θ2,1< k11D
H* Compute k11 H3L = ¶Θ 11 *L
¶k
3
ka113 = Factor@¶8Θ3,1< k11D
H* Compute k11 H4L =
¶k11
¶Θ4
*L
¶k12
¶Θ1
*L
¶k12
¶Θ2
*L
¶k12
¶Θ3
*L
¶k12
¶Θ4
*L
¶k13
¶Θ1
*L
¶k13
¶Θ2
*L
¶k13
¶Θ3
*L
¶k13
¶Θ4
*L
¶k14
¶Θ1
*L
¶k14
¶Θ2
*L
¶k14
¶Θ3
*L
¶k14
¶Θ4
*L
¶k22
¶Θ1
*L
¶k22
¶Θ2
*L
¶k22
¶Θ3
*L
¶k22
¶Θ4
*L
¶k23
¶Θ1
*L
¶k23
¶Θ2
*L
ka114 = Factor@¶8Θ4,1< k11D
H* Compute k12 H1L =
ka121 = Factor@¶8Θ1,1< k12D
H* Compute k12 H2L =
ka122 = Simplify@¶8Θ2,1< k12D
H* Compute k12 H3L =
ka123 = Factor@¶8Θ3,1< k12D
H* Compute k12 H4L =
ka124 = Factor@¶8Θ4,1< k12D
H* Compute k13 H1L =
ka131 = Factor@¶8Θ1,1< k13D
H* Compute k13 H2L =
ka132 = Factor@¶8Θ2,1< k13D
H* Compute k13 H3L =
ka133 = Factor@¶8Θ3,1< k13D
H* Compute k13 H4L =
ka134 = Factor@¶8Θ4,1< k13D
H* Compute k14 H1L =
ka141 = Factor@¶8Θ1,1< k14D
H* Compute k14 H2L =
ka142 = Factor@¶8Θ2,1< k14D
H* Compute k14 H3L =
ka143 = Factor@¶8Θ3,1< k14D
H* Compute k14 H4L =
ka144 = Factor@¶8Θ4,1< k14D
H* Compute k22 H1L =
ka221 = Factor@¶8Θ1,1< k22D
H* Compute k22 H2L =
ka222 = Factor@¶8Θ2,1< k22D
H* Compute k22 H3L =
ka223 = Factor@¶8Θ3,1< k22D
H* Compute k22 H4L =
ka224 = Factor@¶8Θ4,1< k22D
H* Compute k23 H1L =
ka231 = Factor@¶8Θ1,1< k23D
H* Compute k23 H2L =
ka232 = Factor@¶8Θ2,1< k23D
13
14
CSCK_MLE_Bias_Calculation_063012.nb
ka232 = Factor@¶8Θ2,1< k23D
H* Compute k23 H3L =
¶k23
¶Θ3
*L
¶k23
¶Θ4
*L
¶k24
¶Θ1
*L
¶k24
¶Θ2
*L
¶k24
¶Θ3
*L
¶k24
¶Θ4
*L
¶k33
¶Θ1
*L
¶k33
¶Θ2
*L
¶k33
¶Θ3
*L
¶k33
¶Θ4
*L
¶k34
¶Θ1
*L
¶k34
¶Θ2
*L
¶k34
¶Θ3
*L
¶k34
¶Θ4
*L
¶k44
¶Θ1
*L
¶k44
¶Θ2
*L
¶k44
¶Θ3
*L
¶k44
¶Θ4
*L
ka233 = Factor@¶8Θ3,1< k23D
H* Compute k23 H4L =
ka234 = Factor@¶8Θ4,1< k23D
H* Compute k24 H1L =
ka241 = Factor@¶8Θ1,1< k24D
H* Compute k24 H2L =
ka242 = Factor@¶8Θ2,1< k24D
H* Compute k24 H3L =
ka243 = Factor@¶8Θ3,1< k24D
H* Compute k24 H4L =
ka244 = Factor@¶8Θ4,1< k24D
H* Compute k33 H1L =
ka331 = Factor@¶8Θ1,1< k33D
H* Compute k33 H2L =
ka332 = Factor@¶8Θ2,1< k33D
H* Compute k33 H3L =
ka333 = Factor@¶8Θ3,1< k33D
H* Compute k33 H4L =
ka334 = Factor@¶8Θ4,1< k33D
H* Compute k34 H1L =
ka341 = Factor@¶8Θ1,1< k34D
H* Compute k34 H2L =
ka342 = Factor@¶8Θ2,1< k34D
H* Compute k34 H3L =
ka343 = Factor@¶8Θ3,1< k34D
H* Compute k34 H4L =
ka344 = Factor@¶8Θ4,1< k34D
H* Compute k44 H1L =
ka441 = Factor@¶8Θ1,1< k44D
H* Compute k44 H2L =
ka442 = Factor@¶8Θ2,1< k44D
H* Compute k44 H3L =
ka443 = Factor@¶8Θ3,1< k44D
H* Compute k44 H4L =
ka444 = Factor@¶8Θ4,1< k44D
- n PolyGamma@2, Θ1D
0
0
0
CSCK_MLE_Bias_Calculation_063012.nb
0
n
Θ22
0
0
0
0
0
0
0
0
0
0
n
Θ22
2 n Θ1
Θ23
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
15
16
CSCK_MLE_Bias_Calculation_063012.nb
0
0
7. The total Fisher Information Matrix K and its inverse
K -1 are constructed:
H* INFORMATION MATRIX K *L
L = 88- k11, - k12, - k13, - k14<, 8- k12, - k22, - k23, - k24<,
8- k13, - k23, - k33, - k34<, 8- k14, - k24, - k34, - k44<<
Kinformation = L@@1 ;; p, 1 ;; pDD
MatrixForm@KinformationD
::n PolyGamma@1, Θ1D,
::n PolyGamma@1, Θ1D,
n PolyGamma@1, Θ1D
n
Θ2
n
Θ2
n
Θ2
n
Θ2
n Θ1
, 0, 0>, :
>, :
n
n
n Θ1
,
Θ2
n Θ1
,
Θ22
Θ2
Θ22
>>
, 0, 0>, 80, 0, 0, 0<, 80, 0, 0, 0<>
Θ22
H* INVERSE OF INFORMATION MATRIX K *L
Kinv = Inverse@KinformationD; MatrixForm@KinvD
n Θ1
n2
Θ22 -
+
n2 Θ1 PolyGamma@1,Θ1D
Θ22
n
Θ2 -
Θ22
n2
Θ2 -
+
+
n2 Θ1 PolyGamma@1,Θ1D
Θ22
Θ22
n PolyGamma@1,Θ1D
n
-
n2
n2 Θ1 PolyGamma@1,Θ1D
Θ22
Θ22
-
n2
+
n2 Θ1 PolyGamma@1,Θ1D
Θ22
Θ22
H* CONSTRUCTION: VEC@K INVERSED *L
vecKinv = Flatten@Transpose@KinvDD; MatrixForm@vecKinvD
n Θ1
n2
Θ22 -
+
n2 Θ1 PolyGamma@1,Θ1D
Θ22
Θ22
n
Θ2 -
n2
+
n2 Θ1 PolyGamma@1,Θ1D
Θ22
Θ22
n
Θ2 -
n2
+
n2 Θ1 PolyGamma@1,Θ1D
Θ22
Θ22
n PolyGamma@1,Θ1D
-
n2
Θ22
+
n2 Θ1 PolyGamma@1,Θ1D
Θ22
8. The A matrix is constructed:
H* Compute a11 H1L =k11 H1L -Hk111 2L;
Do similar computation for the rest aij HlL =kij HlL -Ikijl ‘2M,
where i=1,2,3, 4; j=1,2,3,4; l=1,2,3,4 *L
a111 = ka111 - k111  2
CSCK_MLE_Bias_Calculation_063012.nb
a111
a121
a131
a141
a211
a221
a231
a241
a311
a321
a331
a341
a411
a421
a431
a441
a112
a122
a132
a142
a212
a222
a232
a242
a312
a322
a332
a342
a412
a422
a432
a442
a113
a123
a133
a143
a213
a223
a233
a243
a313
a323
a333
a343
a413
a423
a433
a443
a114
a124
a134
a144
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
ka111 - k111  2
ka121 - k112  2
ka131 - k113  2
ka141 - k114  2
ka121 - k112  2
ka221 - k122  2
ka231 - k123  2
ka241 - k124  2
ka131 - k113  2
ka231 - k123  2
ka331 - k133  2
ka341 - k134  2
ka141 - k114  2
ka241 - k124  2
ka341 - k134  2
ka441 - k144  2
ka112 - k112  2
ka122 - k122  2
ka132 - k123  2
ka142 - k124  2
ka122 - k122  2
ka222 - k222  2
ka232 - k223  2
ka242 - k224  2
ka132 - k123  2
ka232 - k223  2
ka332 - k233  2
ka342 - k234  2
ka142 - k124  2
ka242 - k224  2
ka342 - k234  2
ka442 - k244  2
ka113 - k113  2
ka123 - k123  2
ka133 - k133  2
ka143 - k134  2
ka123 - k123  2
ka223 - k223  2
ka233 - k233  2
ka243 - k234  2
ka133 - k133  2
ka233 - k233  2
ka333 - k333  2
ka343 - k334  2
ka143 - k134  2
ka243 - k234  2
ka343 - k334  2
ka443 - k344  2
ka114 - k114  2
ka124 - k124  2
ka134 - k134  2
ka144 - k144  2
17
18
CSCK_MLE_Bias_Calculation_063012.nb
a144
a214
a224
a234
a244
a314
a324
a334
a344
a414
a424
a434
a444
=
=
=
=
=
=
=
=
=
=
=
=
=
ka144 - k144  2
ka124 - k124  2
ka224 - k224  2
ka234 - k234  2
ka244 - k244  2
ka134 - k134  2
ka234 - k234  2
ka334 - k334  2
ka344 - k344  2
ka144 - k144  2
ka244 - k244  2
ka344 - k344  2
ka444 - k444  2
H* A MATRIX *L
H* Construction of AH1L Matrix *L
La1 = 88a111, a121, a131, a141<, 8a211, a221, a231, a241<, 8a311, a321,
8a411, a421, a431, a441<<; A1 = La1@@1 ;; p, 1 ;; pDD; MatrixForm@A1D
H* Construction of AH2L Matrix *L
La2 = 88a112, a122, a132, a142<, 8a212, a222, a232, a242<, 8a312, a322,
8a412, a422, a432, a442<<; A2 = La2@@1 ;; p, 1 ;; pDD; MatrixForm@A2D
H* Construction of AH3L Matrix *L
La3 = 88a113, a123, a133, a143<, 8a213, a223, a233, a243<, 8a313, a323,
8a413, a423, a433, a443<<; A3 = La3@@1 ;; p, 1 ;; pDD; MatrixForm@A3D
H* Construction of AH4L Matrix *L
La4 = 88a114, a124, a134, a144<, 8a214, a224, a234, a244<, 8a314, a324,
8a414, a424, a434, a444<<; A4 = La4@@1 ;; p, 1 ;; pDD; MatrixForm@A4D
LA = Join@A1, A2, A3, A4, 2D;
A = LA@@1 ;; p, 1 ;; p * pDD; MatrixForm@AD
1
n PolyGamma@2, Θ1D
2
0
0
0
0
3n
2 Θ22
0
0
0
0
0
0
0
a331, a341<,
a332, a342<,
a333, a343<,
a334, a344<,
CSCK_MLE_Bias_Calculation_063012.nb
0
0
0
0
n
2 Θ22
0
0
n
2 Θ22
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
19
20
CSCK_MLE_Bias_Calculation_063012.nb
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
-
1
2
n PolyGamma@2, Θ1D
0
3n
2 Θ22
2 Θ22
n
2 Θ22
K
-
n
0
K
0
0
0 0
O
0 0
0 0
O
0 0
-
1
2
n PolyGamma@2, Θ1D
0
0
-
0
3n
n
2 Θ22
2 Θ22
n
2 Θ22
0
9. The analytic CSCK MLE bias vector is outputted, ignoring the O(n-2 ) term which is assumed to be negligible. We also
formally turn off the SuperLog function employed by the SMLE package:
CSCKMLEBIAS = [email protected]
SuperLog@OffD
:
- 2 + Θ1 PolyGamma@1, Θ1D - Θ12 PolyGamma@2, Θ1D
2 n H- 1 + Θ1 PolyGamma@1, Θ1DL
SuperLog is now Off.
2
,
Θ2 HPolyGamma@1, Θ1D + Θ1 PolyGamma@2, Θ1DL
2 n H- 1 + Θ1 PolyGamma@1, Θ1DL2
>
The CSCK MLE bias vector for the 2-parameter gamma distribution contains the polygamma function, which may be unfamilar
to some users. More information about this function can be found at
http://reference.wolfram.com/mathematica/ref/PolyGamma.html. Also, as the sample size n approaches infinity, the CSCK
MLE bias of both parameters approaches zero - MLEs are asymptotically unbiased.
CSCK_MLE_Bias_Calculation_063012.nb
21
The CSCK MLE bias vector for the 2-parameter gamma distribution contains the polygamma function, which may be unfamilar
to some users. More information about this function can be found at
http://reference.wolfram.com/mathematica/ref/PolyGamma.html. Also, as the sample size n approaches infinity, the CSCK
MLE bias of both parameters approaches zero - MLEs are asymptotically unbiased.
The above code was setup in such a way to make the details of the CSCK MLE bias calculation transparent to the user. The
user can follow the determination of the CSCK MLE bias step-by-step.
If the user only cares about the outputted CSCK MLE bias, Appendix C contains a Mathematica function that simply outputs
the bias. To determine the CSCK MLE bias for a given distribution, use the function b[f_, p_]. The argument f_ is replaced with
the probability density function of the distribution of interest. The argument p_ is replaced by the set of parameters, expressed
as 9θ1 ,
θ2 , ..., θp =. In principle, the
range of possible values of 9θ1 ,
function b[f_, p_] can accomodate a probability distribution with any number of
θ2 , ..., θp = are reflected in the Expect[x_] function; the user may need to adjust both
parameters, unlike the above code which can consider a distribution with at most four parameters. For each distribution, the
the number of parameters and the parameter assumptions in Expect[x_] to suit the distribution of interest.
RESEARCH PROGRESS
Thus far, we have analyzed the CSCK MLE bias for a total of 24 continuous distributions; these distributions are provided in
Appendix B. For each of the following distributions, the probability density function (f[yi]), MLEs, concentrated loglikelihood
function, and the analytic CSCK MLE bias vector (MLEBias) are provided, if calculable:
•
3 one-parameter distributions: exponential, inverse exponential, single-parameter Pareto
•
14 two-parameter distributions: gamma, inverse gamma, Weibull, inverse Weibull, Pareto (Type II), inverse Pareto,
generalized Pareto, Gumbel, lognormal, inverse Gaussian, beta, loglogistic, paralogistic, inverse paralogistic
•
6 three-parameter distributions: transformed gamma, inverse transformed gamma, generalized Pareto 3, generalized
beta, Burr, inverse Burr
•
1 four-parameter distribution: transformed beta.
Of the 24 continuous distributions, we obtained analytic CSCK MLE bias vectors for 17. We could NOT obtain bias vectors
for the following 7 distributions: loglogistic, paralogistic, inverse paralogistic, generalized beta, Burr, inverse Burr, and the
transformed beta. The first five distributions are part of the Burr distribution family, and the last two distributions are beta
distributions.
The remainder of this section details the ongoing and future research direction of the CSCK MLE Bias Calculation project:
ISSUES REGARDING CERTAIN CUMULANTS
The primary reason that analytic CSCK MLE bias vectors could not be obtained for the 7 afore mentioned distributions is
because certain cumulants cannot be calculated using Mathematica. For example, consider the loglogistic distribution:
1.
The probability density function for the loglogistic distribution (f[yi]) is:
The remainder of this section details the ongoing and future research direction of the CSCK MLE Bias Calculation project:
22
CSCK_MLE_Bias_Calculation_063012.nb
ISSUES REGARDING CERTAIN CUMULANTS
The primary reason that analytic CSCK MLE bias vectors could not be obtained for the 7 afore mentioned distributions is
because certain cumulants cannot be calculated using Mathematica. For example, consider the loglogistic distribution:
1.
The probability density function for the loglogistic distribution (f[yi]) is:
2.
The loglikelihood function for the loglogistic distribution (l[ ]) is:
Note: Θ3 is treated as null.
3
=
The third partial derivative of the loglikelihood function for the loglogistic distribution with respect to Θ1:
2n
Θ 13
- 2 Úni=1 K
2 I-Θ 2-Θ 1 L o g@Θ 2D yiΘ 1 +Θ 2-Θ 1 L o g@yi D yiΘ 1 M
3
I1+Θ 2
-Θ 1
3
yiΘ 1 M
-
1
I1+Θ 2-Θ 1 yiΘ 1 M
2
3 I-Θ 2-Θ 1 L o g@Θ 2D yiΘ 1 + Θ 2-Θ 1 L o g@yi D yiΘ 1 M
IΘ 2-Θ 1 L o g@Θ 2D2 yiΘ 1 - 2 Θ 2-Θ 1 L o g@Θ 2D L o g@yi D yiΘ 1 + Θ 2-Θ 1 L o g@yi D2 yiΘ 1 M +
1
1+Θ 2-Θ 1 yiΘ 1
I-Θ 2-Θ 1 L o g@Θ 2D3 yiΘ 1 + 3 Θ 2-Θ 1 L o g@Θ 2D2 L o g@yi D yiΘ 1 - 3 Θ 2-Θ 1 L o g@Θ 2D L o g@yi D2 yiΘ 1 + Θ 2-Θ 1 L o g@yi D3 yiΘ 1 MO
Note how complicated this derivative is even after applying the Mathematica function “Simplify.”
4. The corresponding cumulant based on the third partial derivative of the loglikelihood function for the loglogistic distribution
with respect to Θ1 Hk111L i s :
=
4. The corresponding cumulant based on the third partial derivative of the loglikelihood function for the loglogistic distribution
with respect to Θ1 Hk111L i s :
CSCK_MLE_Bias_Calculation_063012.nb
23
=
It can be verified that the k111 integral is incalculable for any range of values for the variable yi or the parameters Θ1 and Θ2
using Mathematica. We attempted to further simplify the integrand, using Mathematica functions such as “Simplify” and
“FullSimplify,” but this did not result in the integral computing. We have also tried looking at the sum of integrals instead of the
integral of the sum but the integral still did not compute. As k111 cannot be calculated using Mathematica, the analytic CSCK
MLE bias vector for the loglogistic distribution cannot be calculated via Mathematica.
Similarly, for other six distributions, certain cumulants cannot be calculated. Therefore, it is currently not possible to calculate
their analytic CSCK MLE bias vectors via Mathematica. We currently attempting to remedy this problem by considering
various numerical analysis techniques involving series expansions, as well as looking at other symbolic calculation packages
such as Maple.
COMPLEXITY OF CSCK MLE BIAS VECTORS
For many of the continuous distributions, their probability density functions involve complex mathematical functions such as
the gamma function. Consequently, the analytic CSCK MLE bias vectors for those distributions are incredibly complicated and
involve complex mathematical functions. This is especially true for the 3-parameter distributions listed in Appendix B. We
have attempted to apply operations such as “Simplify” and “FullSimplify” throughout the CSCK MLE bias vector calculations to
decrease the complexity, but it has no effect. When you take third partial derivatives of complex 3-parameter loglikelihood
functions, you are going to obtain complex output functions for the CSCK MLE bias. Nonetheless, we are exploring all of the
simplification functions of Mathematica in an attempt to further simplify the outputted CSCK MLE bias vectors. This is
necessary in order to add these complicated CSCK MLE bias formulas into the Microsoft Excel version of AMOOF.
OTHER MLE BIAS CORRECTION APPROACHES
Currently, researchers are focused on the CSCK approach to mitigate MLE bias in small samples. However, there are other
methods in the literature for determining and accounting for small sample MLE bias. For example, some authors have
have attempted to apply operations such as “Simplify” and “FullSimplify” throughout the CSCK MLE bias vector calculations to
decrease
the complexity, but it has no effect. When you take third partial derivatives of complex 3-parameter loglikelihood
CSCK_MLE_Bias_Calculation_063012.nb
24
functions, you are going to obtain complex output functions for the CSCK MLE bias. Nonetheless, we are exploring all of the
simplification functions of Mathematica in an attempt to further simplify the outputted CSCK MLE bias vectors. This is
necessary in order to add these complicated CSCK MLE bias formulas into the Microsoft Excel version of AMOOF.
OTHER MLE BIAS CORRECTION APPROACHES
Currently, researchers are focused on the CSCK approach to mitigate MLE bias in small samples. However, there are other
methods in the literature for determining and accounting for small sample MLE bias. For example, some authors have
considered a preventive approach where the score vector itself is adjusted to mitigate bias. We have not considered whether
such a “preventive” approach is preferable to the corrective approach in this paper, where MLEs are adjusted for potential
bias ex post.
APPLICATION OF CSCK MLE BIAS RESULTS TO AMOOF
The ultimate goal of this project is to improve the quality of MLEs used in the process of fitting probability distributions to
stochastic financial outcome model data in AMOOF. CSCK MLE Bias Calculation has obtained analytic CSCK MLE bias
vectors for most of the probability distributions that are utilized in AMOOF in Appendix B. These CSCK MLE vectors will now
be programmed into AMOOF and BMLEs can replace the MLEs that are generated. AMOOF will be subsequently be able to
optimally fit probability distributions such that the estimated parameters of those distributions will be unbiased even in the
presence of small sample stochastic model data.
ACKNOWLEDGMENTS
The authors would like to acknowledge the financial support of The Actuarial Foundation, who provided the grant which funds
the project “Developing and Testing AMOOF2.” The authors also appreciate the hard work of students at both Central
Washington University and the University of Illiois at Urbana-Champaign, who provided much need analytical and technical
support.
REFERENCES
[1] Y.C. Chueh and W.D. Curtis. 2005. “Optimal Probability Density Function Models for Stochastic Model Outcomes:
Parametric Model Fitting on Tail Distributions.” New Ideas in Symbolic Computation:Proceedings of the 6th International
Mathematica Symposium: 17 pages.
[2] Microsoft. 2007. Microsoft Excel [computer software]. Redmond, Washington: Microsoft.
[3] Wolfram Research, Inc. 2010. Mathematica Edition: Version 8.0 [computer software]. Champaign, IL: Wolfram Research,
Inc.
[4] D.R. Cox and E.J. Snell. 1968. “A General Definition of Residuals.” Journal of the Royal Statistical Society B(30): 248-275.
[5] G.M. Cordeiro, G.M. and R. Klein. 1994. “Bias Correction in ARMA Models.” Statistics and Probability Letters 19: 169-176.
[6] D.E. Giles and H. Feng. 2009. “Bias of the Maximum Likelihood Estimators of theTwo-Parameter Gamma Distribution
Revisited.” Econometrics Working Paper EWP0908:1-9.
[7] P.H. Johnson, Jr., Y. Qi, and Y.C. Chueh. 2011. “Bias-Corrected Maximum Likelihood Estimation in Actuarial Science.”
Available at: http://www.math.uiuc.edu/~pjohnson/Johnson_etal_BMLE_Paper.pdf.
[8] C. Rose and M.D. Smith. 2000. “Symbolic Maximum Likelihood Estimation with Mathematica.” The Statistician 49(2):
229-240.
[9] S.A. Klugman, S.A., H.H. Panjer, and G.E. Willmot. 2008. “Loss Models: From Data to Decisions, Third Edition.” Hoboken,
[5] G.M. Cordeiro, G.M. and R. Klein. 1994. “Bias Correction in ARMA Models.” Statistics and Probability Letters 19: 169-176.
[6] D.E. Giles and H. Feng. 2009. “Bias of the Maximum Likelihood Estimators of theTwo-Parameter
Gamma Distribution
CSCK_MLE_Bias_Calculation_063012.nb
25
Revisited.” Econometrics Working Paper EWP0908:1-9.
[7] P.H. Johnson, Jr., Y. Qi, and Y.C. Chueh. 2011. “Bias-Corrected Maximum Likelihood Estimation in Actuarial Science.”
Available at: http://www.math.uiuc.edu/~pjohnson/Johnson_etal_BMLE_Paper.pdf.
[8] C. Rose and M.D. Smith. 2000. “Symbolic Maximum Likelihood Estimation with Mathematica.” The Statistician 49(2):
229-240.
[9] S.A. Klugman, S.A., H.H. Panjer, and G.E. Willmot. 2008. “Loss Models: From Data to Decisions, Third Edition.” Hoboken,
NJ: Wiley.
APPENDIX A: SMLE MATHEMATICA PACKAGE
The following Mathematica package is from Appendix B of Rose and Smith@8D and is used in the CSCK MLE Bias Calculation
program to automatically generate a symbolic representation of the loglikelihood function.
________________________________________________________________________________________________
(* :Name: SMLE *)
(* :Authors: Colin Rose and Murray D. Smith *)
(* :Version: Mathematica v3 or higher *)
(* :Legal: Copyright 1999 *)
(* :Summary: Symbolic Maximum Likelihood Estimation *)
BeginPackage[“SMLE`”]
SuperLog::usage = “SuperLog[On] activates the enhanced Log operator, so that Log[Product[__]] objects get converted into
sums of logs. SuperLog[Off] switches the enhancement off.”
Begin[“`Private`”]
SuperLog[Q_] := Module[{erk, iii, nnn},
Product[iii, {iii, nnn}]; (* Pre-Load Product *)
Which[
Q === On,
Unprotect[Log]; Clear[Log];
Log[Product[x_, {k_, a_, b_}]] := Log[Product[Times[erk, x], {k, a, b}]] /. erk -> 1;
Log[Product[HoldPattern[Times[x__]], {k_, a_, b_}]] := Simplify[Map[Sum[#, {k, a, b}]&,
Plus@@Map[Expand[PowerExpand[Log[#]]]&, List[x]]] //.
Sum[u_. w_, {kk_, aa_, bb_}] :> u Sum[w, {kk, aa, bb}] /; FreeQ[u, kk] == True];
Protect[Log]; Print[“SuperLog is now On.”],
Q === Off,
Unprotect[Log]; Clear[Log]; Protect[Log]; Print[“SuperLog is now Off.”],
True,
Print[“Carumbah! Please use SuperLog[On] or SuperLog[Off].”]
]]
End[ ]
Protect[SuperLog];
EndPackage[ ]
APPENDIX B: INVENTORY OF CONTINUOUS
DISTRIBUTIONS
H*********************************************************************
26
CSCK_MLE_Bias_Calculation_063012.nb
APPENDIX B: INVENTORY OF CONTINUOUS
DISTRIBUTIONS
H*********************************************************************
INVENTORY OF CONTINUOUS
DISTRIBUTIONS ********************************************************************
H* For each distribution, there is:
H1L The probability density function, f@yi D, as provided in Klugman, Panjer,
and Willmot; "Loss Models: From Data to Decisions, Third Edition;" Appendix A.
H2L The maximum likelihood estimators,
MLEs. MLEs are denoted with "hats" over the parameter. In some cases,
it is not possible to obtain a closed form for the MLEs.
H3L When applicable, the concentrated loglikelihood function,
ConLog. Sometimes, it is possible to solve for the MLE of one
distributional parameter as a function of the remaining parameters;
this MLE can be substituted into the original loglikelihood function,
resulting in a function with fewer parameters.
H4L The Cox and Snell  Cordeiro and Klein HCSCKL analytic MLE
bias vectors HK^8-1<AvecHK^8-1<LL, MLEBias. For some distributions,
Mathematica was unable to calculate the CSCK MLE bias, because certain
cumulants for those distributions apparently be calculated via Mathematica. *L
H****************************** ONE PARAMETER DISTRIBUTIONS ******************************L
H* EXPONENTIAL *L
f@yi D = 1  Θ1 * Exp@- yi  Θ1D
`
MLEs = :Θ1 ®
Úni=1 yi
ConLog = N  A
MLEBias = 80<
n
>
H4L The Cox and Snell  Cordeiro and Klein HCSCKL analytic MLE
bias vectors HK^8-1<AvecHK^8-1<LL, MLEBias. For some distributions,
Mathematica was unable to calculate the CSCK MLE bias, becauseCSCK_MLE_Bias_Calculation_063012.nb
certain
cumulants for those distributions apparently be calculated via Mathematica. *L
H****************************** ONE PARAMETER DISTRIBUTIONS ******************************L
H* EXPONENTIAL *L
f@yi D = 1  Θ1 * Exp@- yi  Θ1D
`
MLEs = :Θ1 ®
Úni=1 yi
n
>
ConLog = N  A
MLEBias = 80<
H* INVERSE EXPONENTIAL *L
f@yi D = HΘ1  yi ^ 2L * Exp@- Θ1  yi D
`
MLEs = :Θ1 ®
ConLog = N  A
MLEBias = 9
n
Úni=1
Θ1
=
n
1
yi
>
27
28
CSCK_MLE_Bias_Calculation_063012.nb
H* INVERSE EXPONENTIAL *L
f@yi D = HΘ1  yi ^ 2L * Exp@- Θ1  yi D
`
MLEs = :Θ1 ®
n
Úni=1
1
yi
>
ConLog = N  A
MLEBias = 9
Θ1
=
n
H* SINGLE PARAMETER PARETO - yi takes values greater than
Θ2. Note: Θ2 must be determined prior to any analysis and is therefore fixed;
the only true unknown parameter is Θ1.
f@yi D = Θ1 * HΘ2L ^ HΘ1L * yi ^ H- Θ1 - 1L
`
MLEs = :Θ1 ® -
ConLog = N  A
n
>
n Log@Θ2D-Úni=1 Log@yi D
*L
CSCK_MLE_Bias_Calculation_063012.nb
H* WEIBULL *L
f@yi D =
-
ã
yi
Θ1
Θ2
Θ1 J
Θ2
yi
Θ2
N
-1+Θ1
1
`
`
Θ1
MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® n-1Θ1 IÚni=1 yΘ1
i M >
Θ1
ConLog = n Log@Θ1D - Θ1 LogBn-1Θ1 IÚni=1 yΘ1
i M F +
1
-1Θ1 IÚn
Θ1 Θ1
H- 1 + Θ1L Úni=1 Log@yi D - IÚni=1 yΘ1
i M n
i=1 yi M
MLEBias = :
-
18 Θ1 IΠ2 -2 Zeta@3DM
1
2 n Π4 Θ12
n Π4
1
-Θ1
,
Θ2 IΠ4 H- 1 + 2 Θ1L - 6 Π2 I1 + EulerGamma2 + 5 Θ1 - 2 EulerGamma H1 + 2 Θ1LM 72 H- 1 + EulerGammaL Θ1 Zeta@3DM>
H* INVERSE WEIBULL *L
f@yi D = Θ1 * HΘ2  yi L ^ Θ1 * Exp@- HΘ2  yi L ^ Θ1D * H1  yi L
`
`
MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® J
N Θ1 >
1
n
Úni=1 y-Θ1
i
29
H****************************** THREE
30
CSCK_MLE_Bias_Calculation_063012.nb
PARAMETER DISTRIBUTIONS ******************************L
H* TRANSFORMED GAMMA *L
f@yi D = Θ3 * Hyi  Θ2L ^ HΘ1 * Θ3L * Exp@- Hyi  Θ2L ^ Θ3D * H1  Hyi * Gamma@Θ1DLL
`
`
`
Ún yΘ3
MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® J i=1 i N Θ3 , Θ3 ® No Closed Form Solution>
1
n Θ1
ConLog = n Log@Θ3D - Log@Gamma@Θ1DD - Θ1 Θ3 LogB
n Θ1
Úni=1
1
yi
F + H- 1 + Θ1 Θ3L Úni=1 Log@yi D -
MLEBias = 9- I4 Θ12 PolyGamma@0, Θ1D3 - 4 Θ12 PolyGamma@0, 1 + Θ1D3 +
-Θ3
n Θ1
Úni=1
1
yi
Úni=1 yΘ3
i
H1 + Θ1 PolyGamma@1, 1 + Θ1DL I2 - Θ1 PolyGamma@1, Θ1D H1 + 2 Θ1 PolyGamma@1, 1 + Θ1DL +
Θ12 PolyGamma@2, Θ1D + Θ1 PolyGamma@1, 1 + Θ1D I3 + Θ12 PolyGamma@2, Θ1DMM +
Θ1 PolyGamma@0, 1 + Θ1D2 I1 + Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@2, 1 + Θ1DM +
Θ1 PolyGamma@0, Θ1D2 I1 - 12 Θ1 PolyGamma@0, 1 + Θ1D +
Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@2, 1 + Θ1DM - Θ1 PolyGamma@0, 1 + Θ1D
I2 + 8 Θ1 PolyGamma@1, 1 + Θ1D + 4 Θ12 PolyGamma@1, 1 + Θ1D2 + Θ1 PolyGamma@2, 1 + Θ1D Θ1 PolyGamma@1, Θ1D H6 + 8 Θ1 PolyGamma@1, 1 + Θ1D + Θ1 PolyGamma@2, 1 + Θ1DLM -
Θ1 PolyGamma@0, Θ1D I- 2 - 12 Θ1 PolyGamma@0, 1 + Θ1D2 - 8 Θ1 PolyGamma@1, 1 + Θ1D -
4 Θ12 PolyGamma@1, 1 + Θ1D2 - Θ1 PolyGamma@2, 1 + Θ1D +
Θ1 PolyGamma@1, Θ1D H6 + 8 Θ1 PolyGamma@1, 1 + Θ1D + Θ1 PolyGamma@2, 1 + Θ1DL +
2 PolyGamma@0, 1 + Θ1D I1 + Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@2, 1 + Θ1DMMM ‘
I2 n IΘ1 PolyGamma@0, Θ1D2 - 2 Θ1 PolyGamma@0, Θ1D PolyGamma@0, 1 + Θ1D +
Θ1 PolyGamma@0, 1 + Θ1D2 - H- 1 + Θ1 PolyGamma@1, Θ1DL H1 + Θ1 PolyGamma@1, 1 + Θ1DLM M,
2
- IΘ2 IΘ1 H- 1 + Θ3L PolyGamma@0, Θ1D + Θ1 PolyGamma@0, Θ1D HH2 - 3 Θ3L PolyGamma@0, 1 + Θ1D +
2 Θ3 H- 2 + Θ1 PolyGamma@1, 1 + Θ1DLL - Θ1 Θ3 H1 + Θ1 PolyGamma@1, 1 + Θ1DL
I1 + Θ1 PolyGamma@0, 1 + Θ1D2 + Θ1 PolyGamma@1, 1 + Θ1DM PolyGamma@2, Θ1D +
4
3
PolyGamma@0, Θ1D2 I- 1 + 2 Θ3 + Θ1 PolyGamma@0, 1 + Θ1D2 H- 1 + 3 Θ3 + Θ1 PolyGamma@1, Θ1DL Θ1 PolyGamma@1, 1 + Θ1D + 3 Θ1 Θ3 PolyGamma@1, 1 + Θ1D + 2 Θ1 Θ3 PolyGamma@0, 1 + Θ1D
H4 + Θ1 PolyGamma@1, Θ1D - 2 Θ1 PolyGamma@1, 1 + Θ1DL + Θ1 PolyGamma@1, Θ1D
H2 - 2 Θ3 + Θ1 H2 - 3 Θ3L PolyGamma@1, 1 + Θ1DL - 2 Θ12 Θ3 PolyGamma@2, 1 + Θ1DM Θ1 PolyGamma@1, Θ1D2 IΘ1 PolyGamma@0, 1 + Θ1D2 H1 + Θ1 PolyGamma@1, 1 + Θ1DL H1 + Θ1 PolyGamma@1, 1 + Θ1DL H- 1 + Θ3 + Θ1 H- 1 + 2 Θ3L PolyGamma@1, 1 + Θ1DL +
Θ1 Θ3 PolyGamma@0, 1 + Θ1D H4 + 6 Θ1 PolyGamma@1, 1 + Θ1D + Θ1 PolyGamma@2, 1 + Θ1DLM +
PolyGamma@1, Θ1D I2 Θ12 Θ3 PolyGamma@0, 1 + Θ1D3 + Θ12 PolyGamma@0, 1 + Θ1D4 -
H1 + Θ1 PolyGamma@1, 1 + Θ1DL H- 1 + 2 Θ3 + Θ1 H- 1 + 3 Θ3L PolyGamma@1, 1 + Θ1DL +
Θ12 Θ3 PolyGamma@0, 1 + Θ1D I6 PolyGamma@1, 1 + Θ1D +
4 Θ1 PolyGamma@1, 1 + Θ1D2 + PolyGamma@2, 1 + Θ1DM - Θ1 PolyGamma@0, 1 + Θ1D2
I- 2 + Θ3 + Θ1 H- 2 + Θ3L PolyGamma@1, 1 + Θ1D + 2 Θ12 Θ3 PolyGamma@2, 1 + Θ1DMM -
PolyGamma@0, Θ1D IΘ1 PolyGamma@0, 1 + Θ1D3 HΘ3 + 2 Θ1 PolyGamma@1, Θ1DL +
2 Θ1 Θ3 PolyGamma@0, 1 + Θ1D2 H2 + 2 Θ1 PolyGamma@1, Θ1D - Θ1 PolyGamma@1, 1 + Θ1DL +
Θ3 I2 + 6 Θ1 PolyGamma@1, 1 + Θ1D + 2 Θ12 PolyGamma@1, 1 + Θ1D2 +
Θ1 PolyGamma@2, 1 + Θ1D + Θ1 PolyGamma@1, Θ1D I- 6 - 6 Θ1 PolyGamma@1, 1 + Θ1D +
2 Θ12 PolyGamma@1, 1 + Θ1D2 - Θ1 PolyGamma@2, 1 + Θ1DMM -
PolyGamma@0, 1 + Θ1D IΘ3 I- 2 + Θ12 PolyGamma@2, Θ1D + Θ1 PolyGamma@1, 1 + Θ1D
I- 3 + Θ12 PolyGamma@2, Θ1DM + 2 Θ12 PolyGamma@2, 1 + Θ1DM + Θ1 PolyGamma@1, Θ1D
I- 2 + 3 Θ3 + 2 Θ1 H- 1 + 2 Θ3L PolyGamma@1, 1 + Θ1D + 2 Θ12 Θ3 PolyGamma@2, 1 + Θ1DMMMMM ‘
I2 n Θ3 IΘ1 PolyGamma@0, Θ1D2 - 2 Θ1 PolyGamma@0, Θ1D PolyGamma@0, 1 + Θ1D +
2
Θ1 PolyGamma@0, 1 + Θ1D2 - H- 1 + Θ1 PolyGamma@1, Θ1DL H1 + Θ1 PolyGamma@1, 1 + Θ1DLM M,
2
IΘ3 I- Θ1 PolyGamma@0, Θ1D3 + Θ1 PolyGamma@0, 1 + Θ1D3 + Θ1 PolyGamma@0, Θ1D2
H6 + 3 PolyGamma@0, 1 + Θ1D - 4 Θ1 PolyGamma@1, Θ1D - 2 Θ1 PolyGamma@1, 1 + Θ1DL 2 Θ1 PolyGamma@0, 1 + Θ1D2 H- 3 + 2 Θ1 PolyGamma@1, Θ1D + Θ1 PolyGamma@1, 1 + Θ1DL +
H- 1 + Θ1 PolyGamma@1, Θ1DL
I- 4 - 8 Θ1 PolyGamma@1, 1 + Θ1D - 2 Θ12 PolyGamma@1, 1 + Θ1D2 - Θ1 PolyGamma@2, 1 + Θ1D +
MLEBias = Mathematica unable to calculate
CSCK_MLE_Bias_Calculation_063012.nb
H****************************** FOUR
PARAMETER DISTRIBUTIONS ******************************L
H* TRANSFORMED BETA *L
f@yi D = HGamma@Θ1 + Θ4D  HGamma@Θ1D * Gamma@Θ4DLL *
HΘ3 * Hyi  Θ2L ^ HΘ3 * Θ4LL  Hyi * H1 + Hyi  Θ2L ^ Θ3L ^ HΘ1 + Θ4LL
`
`
MLEs = :Θ1 ® No Closed Form Solution, Θ2 ® No Closed Form Solution,
`
`
Θ3 ® No Closed Form Solution, Θ4 ® No Closed Form Solution>
ConLog = N  A
MLEBias = Mathematica unable to calculate
APPENDIX C: CSCK MLE BIAS CALCULATION FUNCTION:
The following function will output only the CSCK MLE bias for an inputted probability density function (f_) and set of
parameters (p_). It needs to be used in conjunction with the SMLE Mathematica
package:
b@f_, p_D := Module@8l, Gradient, Hessian, ThirdPartialDer,
ExpectHessian, ExpectThirdPartialDer, DerivativeExpectHessian, aijk, Amatrix,
Kinv, vecKinv, BIAS, Expect<, Expect@x_D := Integrate@x * f, 8yi , 0, ¥<,
Assumptions ® 8Θ1 Î Reals, Θ2 Î Reals, Θ3 Î Reals, Θ3 > 0, Θ2 > 0, Θ1 > 0<D;
SuperLog@OnD;
l = Log@Ûni=1 fD;
Gradient = D@l, 8p<D;
Hessian = D@l, 8p, 2<D;
ThirdPartialDer = D@l, 8p, 3<D;
ExpectHessian = Map@Expect@ðD &, HessianD;
ExpectThirdPartialDer = Map@Expect@ðD &, ThirdPartialDerD;
DerivativeExpectHessian = D@ExpectHessian, 8p<D;
aijk = DerivativeExpectHessian - ExpectThirdPartialDer  2;
Amatrix = Apply@Join, aijk~ Join~ 82<D;
Kinv = Inverse@- ExpectHessianD;
vecKinv = Flatten@Transpose@KinvDD;
BIAS = [email protected]; SuperLog@OffD;
BIASD
31