Download Problem set 3 solutions

Department of IOMS
Inference and Regression
Assignment 3
Professor William Greene
Phone: 212.998.0876
Office: KMC 7-90
Home page:www.stern.nyu.edu/~wgreene
Email: [email protected]
Course web page: www.stern.nyu.edu/~wgreene/Econometrics/Econometrics.htm
1. The random variable X has mean µx and standard deviation σx . The random variable Y has mean µy and
standard deviation σy . If X and Y are independent, find E[ XY ], Var[ XY ] and the standard deviation
of XY.
If they are independent, E[XY] = E[X]E[Y] = µxµy.
Var[XY]
= E[(XY)2] – {E[XY]}2
= E[X2Y2] – (E[X]E[Y])2
= E[X2]E[Y2] – (E[X])2(E[Y])2
= (σx2 + µx2)(σy2 + µy2) - µx2µy2
= σx2σy2 + µx2µy2 + σx2µy2 + σy2µx2 - µx2µy2
= σx2σy2 + σx2µy2 + σy2µx2
The standard deviation is the square root of this.
2. The number of auto vandalism claims reported per month at Sunny Daze Insurance Company (SDIC)
has mean 100 and standard deviation 20. The individual losses have mean $1,200/claim and standard
deviation $80/claim. The number of claims and the amounts of individual losses are independent.
Using the normal approximation, calculate the probability that SDIC’s aggregate auto vandalism losses
reported for a month will be less than $110,000.
Aggregate losses are claims times loss per claim.
Expected value is the product of the means, 100claims × $1,200/claim = $120,000
Variance based on question 1 is 202 × 802 + 202 ×1,2002 + 802 × 1002 = 642,560,000
Standard deviation = 25,349.
Prob[ losses < 110,000] = Prob[z < (110,000 – 120,000)/25,349] = Prob[z < .394= ] = .653.
3. Rice, problem 3, page 198.
X-bar is the mean of 16 observations from N(0,1). Find c such that prob(|x-bar|<c)=.5. Prob(|x-bar| <
c) implies that the probability that x-bar is less than –c is .25 (and the probability that is it is greater
than c is .25). X-bar is normally distributed with mean 1/sqr(16) = ¼ = .25. So,
Prob(x-bar < -c) = Prob[(xbar – 0)/.25 < -c/.25) = .25. From the normal table, -c/.25 = -.675. So,
4c = .675 or c = .16875.
4. Rice, problem 5, page 198.
It is possible to do this problem by brute force, using a change of variable and the density of Fn,m.
But, the result follows trivially from the definition of F.
Then,
Fn,m
1/Fn,m
= [chi-squared(n) / n] / [chi-squared(m) / m].
= [chi-squared(m) / m] / [chi-squared(n) / n]
5. Rice, problem 6, page 198.
T = tn
= Normal(0,1) / sqr[chi-squared(n)/n]
T2
= [Normal(0,1)]2 / chi-squared(n)/n.
The key now is to note that the square of a standard normal is chi-squared with 1 degree of
freedom. The definitioni of F1,n then follows immediately.
6. Rice, problem 18 page 316
a. You could not use the mean in the method of moments since E[X] does not depend on α. So, use
the variance. Equate s2 to 2/[9(3α+1)]. Solving,
2/s2 = 9(3α+1) or (2/9)/s2 = 3α + 1, or [2/(9s2) – 1]/3 = α
b. The log likelihood would be the sum of the logs of the density,
logL = nlogΓ(3α) - nlogΓ(α) – nlogΓ(2α) + (α-1)Σi logxi + (2α-1)Σilog(1-xi).
∂logL/∂α = 3nΨ(3α) - nΨ(α) - 2nΨ(2α) + Σilogxi - 2Σilog(1-xi) = 0.
c. (Ouch!). Differentiate it again. ∂2logL/∂α2 = 9nΨ′(3α) - nΨ′(α) – 4nΨ′(2α) = H
The asymptotic variance is -1/H. Since H does not involve xi, we don’t need to take the expected
value.
d. There are two sufficient statistics that are jointly sufficient for α. The density is an exponential
family, so we can see immediately that the sufficient statistics are Σilogxi and Σi log(1-xi).
The reason there are two statistics and one parameter is that this is a version of the beta
distribution which generally has two parameters, α and β and the two sufficient statistics shown.
This distribution forces β to equal 2α.
7. Rice, problem 50, page 323.
a. First find the expected value:
x2
1 ∞ 2
2
θ2 )dx
x exp( − ax 2 )dx, where a = 1/(2θ2 )
2 ∫0
∫0 θ2 exp(− x / (2=
θ
This is a gamma integral. There is a formula for this integral, but it is revealing
E[x] =
∞
to work through it. Make a change of variable to z = x 2 so x =
−1/ 2
z and
dx = (1/2)z . Make the change of variable to get
∞
1 ∞ 2
1 1 ∞
2
2
)dx
x exp(−ax =
z exp(−az )z −1/=
dz a ∫ z1/ 2 exp(−az )dz
2 ∫0
2
∫
0
θ
θ 2 0
Γ(3 / 2)
1
π
= a=
π =θ .
a −1/ 2 (1 / 2)Γ(1 / 2) = 2θ2
2
2
a 3/ 2
x
Since this is the mean, the method of moments estimator is θˆ =
.
π
2
b. The log likelihood is the sum of the logs of the densities
xi2
2θ 2
∂ log L
2n 1
N
=
− + 3 ∑ i =1 xi2 =
The likelihood equation is
0.
∂θ
θ θ
logL = ∑ i 1 =
log xi − 2n log θ − ∑ i 1
=
N
The solution is θˆ =
N
ΣiN=1 xi2
2n
c. Differentiate the log likelihood function again
∂ 2 log L 2n 3
n
= 2 − 4 ∑ i =1 xi2 .
2
∂θ
θ
θ
We need the expected value of this. Superficially, this needs a
solution for the expected square of x i . But, we have a trick available to us.
The expected value of the first derivative is zero. So, solving the moment
 2nθ2 . Use this to find the expected value
equation, we find E  ∑ i =1 xi2 =


2n 3
4n
of the second derivative. This is 2 − 4 2nθ2 =− 2 . The negative of the
θ
θ
θ
θ2
inverse of the second derivative gives the asymptotic variance,
.
4n
n
8. In the application of Bayesian estimation of a proportion of defectives that we examined in class and
that is detailed in ‘Notes-3,’ we used a uniform prior on [0,1] for the distribution of θ. Suppose we assume,
instead, an informative, beta prior with parameters α = 3 and β = 7. Repeat the analysis of the problem to
find the Bayesian estimator of θ given this prior. (Hint: The beta distribution is a conjugate prior for the
likelihood function given in the problem.)
Consider estimation of the probability that a production process will produce a defective product. In case 1,
suppose the sampling design is to choose N = 25 items from the production line and count the number of
defectives. If the probability that any item is defective is a constant θ between zero and one, then the
likelihood for the sample of data is
L( θ | data) = θ D(1 − θ) 25−D,
where D is the number of defectives, say, 8. The maximum likelihood estimator of θ will be
p = D/25 = 0.32,
and the asymptotic variance of the maximum likelihood estimator is estimated by
p(1 − p)/25 = 0.008704.
Now, consider a Bayesian approach to the same analysis. The posterior density is obtained by the following
reasoning:
p (θ, data)
=
p (data)
p (θ | data
=
)
p (θ, data)
=
p (θ, data)d θ
∫
θ
=
p (data | θ) p(θ)
p (data)
Likelihood(data | θ) p(θ)
p (data)
where p(θ) is the prior density assumed for θ. [We have taken some license with the terminology, since the
likelihood function is conventionally defined as L( θ | data) .] Inserting the
results of the sample first drawn, we have the posterior density:
θ D (1 − θ) N − D p (θ)
p (θ | data) =
.
D
N −D
∫ θ (1 − θ) p(θ)d θ
θ
Γ(3 + 7) 3−1
θ (1 − θ)7 −1 . The posterior is
Γ(3)Γ(3)
Γ(3 + 7) 3−1
θ D (1 − θ) N − D
θ (1 − θ)7 −1
Γ(3)Γ(7)
p (θ | data) =
3−1
7 −1
D
N − D Γ (3 + 7)
∫θ θ (1 − θ) Γ(3)Γ(7) θ (1 − θ) d θ
With the beta(3,7) prior,
=
p (θ)
=
θ D + 3−1 (1 − θ) N − D + 7 −1
∫θ
θ
=
D + 3−1
(1 − θ) N − D + 7 −1 d θ
Γ( D + 3 + N − D + 7) D + 3−1
(1 − θ) N − D + 7 −1 .
θ
Γ( D + 3)Γ( N − D + 7)
This is the density of a random variable with a beta distribution with parameters (α, β) = ( D+3,−D+7).
N
The mean of this random variable is (D+3)/(N+10) = 11/35 = 0.3143 (as opposed to 0.32, the MLE). The
posterior variance is [(D+3)/(N−D+7) ]/ [(N + 11) (N + 10)2] = 0.0000104.
9. Lifetimes for a new type of lightbulb are exponentially distributed with f(x) = (1/α)exp(-x/α), x> 0,
α> 0. The experiment consists of drawing a random sample of N lightbulbs and observing them until they
fail. However, the experiment ends at T hours. It is assumes that any bulbs that have not failed by T hours
will fail eventually. Suppose the observed sample of N observations consists of M observations, x1,…,xM
that have failed before the experiment ends, and K observations that have not failed, so N = M + K. Derive
the maximum likelihood estimator of α. (Hint, the log likelihood is the sum of the densities for the
observed data. For uncensored observations, the density is the exponential density shown above. For the
K censored observations, the density is probability that the observation fails, Prob[x>T | α].) After you
derive the MLE, show how to obtain the estimator of the asymptotic variance of the MLE.
For the M bulbs that fail, the log likelihood is the sum of the logs of the densities Σm -logα - x/α
For the K observations that fail, the probability is Prob(x > T) = exp(-T/α), so for them, the log
likelihood is K (-T/α).
The log likelihood is the sum of the two,
logL = -Mlogα - Σm xm/α - KT/α.
Differentiate wrt α to get
∂logL/∂α = -M/α + (1/α2)Σm xm + KT/α2 = 0. The solution is α = (Σmxm + TK)/M.