Download Solution

HW 2
Exercise 1 (Sample median I). Let Z be a real-valued random variable.
Recall that if m is the unique point such that P(Z ≤ m) = 12 , then it is
the median of Z. We say that Z is symmetric about m if for all c ≥
0, we have P(Z − m ≥ c) = P(Z − m ≤ −c) . Let X = (X1 , . . . , X2n+1 )
be a random sample from a symmetric distribution with unique median
zero and order statistics given by Y1 ≤ Y2 · · · ≤ Yn+1 ≤ · · · ≤ Y2n+1 .
The sample median is given by M (X) = Yn+1 .
(a) Show that −Yn+1 has the same distribution as Yn+1 . Hint, you can
do this without fancy order statistics knowledge.
(b) Assuming the expectations exist, show that EYn+1 = 0.
(c) Let ε > 0. Let B ∼ Bin(2n + 1, p), where p = P(X1 > ε). Show
that P(Yn+1 > ε) = P(B ≥ n + 1).
(d) Show that P(B ≥ n + 1) → 0 as n → ∞.
(e) Deduce that M (X) → 0 in probability.
Solution. (a) The symmetry assumption gives that
−X = (−X1 , . . . , −Xn ) =d X.
The order statistics for −X are given by
−Y2n+1 ≤ · · · ≤ −Yn+1 ≤ · · · ≤ −Y1
so that −Yn+1 is the sample median for −X, from which we deduce
that
−Yn+1 =d Yn+1 .
(b) From the previous part, we have that −EYn+1 = EYn+1 , from which
the desired result follows.
(c) We have that
{Yn+1 > ε} = {n + 1 of the Xi ’s are greater than ε} .
Since the Xi are independent, the probability of this event is the
probability of a binomial experiment with 2n + 1 trials, with n + 1
successes, and success probability p = P(Xi > ε). Note by the
uniqueness of the median in the assumption, we have that p < 21 .
(d) We have that
B
n+1
B
1
{B ≥ n + 1} =
≥
⊂
≥
.
2n + 1
2n + 1
2n + 1
2
Note that if Wi are independent Bernoulli random variables with
mean p, then W1 + · · · + W2n+1 =d B, so that the law of large
B
numbers gives that 2n+1
→ p < 12 in probability, from which the
desired result follows.
(e) A similar argument gives that P(Yn+1 < −ε) → 0, so that P(|Yn+1 | >
ε) = P(|M | > ε) → 0.
Exercise 2 (Sample median II). Let X = (X1 , . . . , X2n+1 ) be a random
sample from the normal distribution with unknown mean µ ∈ R and
known unit variance. Use the previous exercise to show that the sample
median is unbiased and consistent estimator for µ.
Solution. Consider
W := X − µ = (X1 − µ, . . . , X2n+1 − µ).
It is easy to verify that W satisfies the conditions of of the previous
exercise. Let Z givesthe order statistics for W , and Y give the order
statistics for X. Clearly, Zn+1 = Yn+1 − µ. Thus EYn+1 = µ. Also, we
know that Zn+1 → 0 in probability, so that Yn+1 → µ in probability.
Exercise 3 (Discrete uniform). Let X = (X1 , . . . , Xn ) be a random
sample where X1 is uniformly distributed in {1, 2, . . . , θ}, where θ ∈ Z+
is unknown. Let T = max {X1 , . . . , Xn }. Show that
T n+1 − (T − 1)n+1
T n − (T − 1)n
is an unbiased and consistent estimator for θ.
W =
Solution. An easy calculation shows that the pdf of T is given by
1
P(T = t) = n [tn − (t − 1)n ].
θ
Thus
θ
X
tn+1 − (t − 1)n+1
EW =
· P(T = t).
tn − (t − 1)n
t=1
This is a telescoping sum! We easily see that W is unbiased.
It is easy to show that T is consistent:
θ − 1 n
P(T 6= θ) =
→ 0.
θ
Re-write
T − ( T T−1 )n · (T − 1)
W =
.
1 − ( T T−1 )n
Since T converges to θ in probability, it is easy to show that ( T T−1 )n → 0
in probability, from which the result follows.
Exercise 4 (Normal distribution). Let X = (X1 , . . . , Xn ) be a random
sample from the normal distribution with mean µ and variance σ 2 . Find
the method of moment estimators in the case where both parameters are
unknown.
Solution. Clearly, the moment estimator for µ is given by X̄. The
moment estimator for µ2 = EX12 = σ 2 + µ2 is given by
n
1X 2
M2 =
X
n i=1 i
Thus the moment estimator for σ 2 is given by
T = M2 − [X̄]2 .
The short-cut formula gives that
n
1X
(Xi − X̄)2 .
T =
n i=1
Exercise 5 (A classic example of Fisher arising from genetics). Let
X = (X1 , . . . , Xn ) be a random sample such that X1 takes values on
the set {a, b, c, d} with respective probabilities given by
[ 2+θ
, 1−θ
, 1−θ
, 4θ ],
4
4
4
where θ ∈ (0, 1) is unknown. Suppose you observe data:
x = (a, b, c, d, a, a, a, b, b, d, c, a).
Find the mle for θ. You may need to use some sort of calculator.
Solution. Let na be the number a’s that occur. Similarly, define nb , nc ,
and nd . Note that na + nb + nc + nd = n. We have that
2 + θ na 1 − θ nb 2 − θ nc θ nd
P(X = x) =
·
·
·
4
4
4
4
If I counted properly, there are na = 5, nb = 3, nc = 2 = nd . Maximizing the log-likelihood, results in solving a polynomial equation. I got
θ̂ ≈ 0.355.
Remark 1. In an earlier version of this solution the multinomial term
n!
appeared in the likelihood P(X = x). This was an easy typo
na !nb !nc nd !
to make, and luckily does not effect the computation of the mle. What
is true is that:
P(Na = na , Nb = nb , Nc = nc , Nd = nd ) =
n!
2 + θ na 1 − θ nb 2 − θ nc θ nd
·
·
·
,
na !nb !nc nd !
4
4
4
4
where for example
Na =
n
X
1[Xi = a].
i=1
Exercise 6 (Gamma distribution). Recall that if Y has the Gamma
distribution with parameters α and β, then it has pdf given by
1
y α−1 e−y/β 1[y > 0]
g(y; α, β) = α
β Γ(α)
and EY = αβ. Let α > 0 be known and fixed. Let X = (X1 , . . . , Xn )
be a random sample from the Gamma distribution with parameters α
and β > 0. Suppose that Z is an unbiased estimator for β 3 . Show that
9β 6
Var(Z) ≥
.
nα
Solution. The Cramer-Rao bound gives that
[g 0 (β)]2
Var(Z) ≥
,
nI(β)
where g(β) = β 3 , so that g 0 (β) = 3β 2 , and I is the Fisher information
for a single sample, Y . We have that
∂2
I(β) = −E
log g(Y ; β) .
2
∂β
An easy calculation gives that
2Y
α
α
I(β) = E 3 −
= 2,
β
β
β
from which the result follows.