Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Inductive probability wikipedia , lookup
Bootstrapping (statistics) wikipedia , lookup
Taylor's law wikipedia , lookup
Foundations of statistics wikipedia , lookup
History of statistics wikipedia , lookup
Student's t-test wikipedia , lookup
Resampling (statistics) wikipedia , lookup
HW 2 Exercise 1 (Sample median I). Let Z be a real-valued random variable. Recall that if m is the unique point such that P(Z ≤ m) = 12 , then it is the median of Z. We say that Z is symmetric about m if for all c ≥ 0, we have P(Z − m ≥ c) = P(Z − m ≤ −c) . Let X = (X1 , . . . , X2n+1 ) be a random sample from a symmetric distribution with unique median zero and order statistics given by Y1 ≤ Y2 · · · ≤ Yn+1 ≤ · · · ≤ Y2n+1 . The sample median is given by M (X) = Yn+1 . (a) Show that −Yn+1 has the same distribution as Yn+1 . Hint, you can do this without fancy order statistics knowledge. (b) Assuming the expectations exist, show that EYn+1 = 0. (c) Let ε > 0. Let B ∼ Bin(2n + 1, p), where p = P(X1 > ε). Show that P(Yn+1 > ε) = P(B ≥ n + 1). (d) Show that P(B ≥ n + 1) → 0 as n → ∞. (e) Deduce that M (X) → 0 in probability. Solution. (a) The symmetry assumption gives that −X = (−X1 , . . . , −Xn ) =d X. The order statistics for −X are given by −Y2n+1 ≤ · · · ≤ −Yn+1 ≤ · · · ≤ −Y1 so that −Yn+1 is the sample median for −X, from which we deduce that −Yn+1 =d Yn+1 . (b) From the previous part, we have that −EYn+1 = EYn+1 , from which the desired result follows. (c) We have that {Yn+1 > ε} = {n + 1 of the Xi ’s are greater than ε} . Since the Xi are independent, the probability of this event is the probability of a binomial experiment with 2n + 1 trials, with n + 1 successes, and success probability p = P(Xi > ε). Note by the uniqueness of the median in the assumption, we have that p < 21 . (d) We have that B n+1 B 1 {B ≥ n + 1} = ≥ ⊂ ≥ . 2n + 1 2n + 1 2n + 1 2 Note that if Wi are independent Bernoulli random variables with mean p, then W1 + · · · + W2n+1 =d B, so that the law of large B numbers gives that 2n+1 → p < 12 in probability, from which the desired result follows. (e) A similar argument gives that P(Yn+1 < −ε) → 0, so that P(|Yn+1 | > ε) = P(|M | > ε) → 0. Exercise 2 (Sample median II). Let X = (X1 , . . . , X2n+1 ) be a random sample from the normal distribution with unknown mean µ ∈ R and known unit variance. Use the previous exercise to show that the sample median is unbiased and consistent estimator for µ. Solution. Consider W := X − µ = (X1 − µ, . . . , X2n+1 − µ). It is easy to verify that W satisfies the conditions of of the previous exercise. Let Z givesthe order statistics for W , and Y give the order statistics for X. Clearly, Zn+1 = Yn+1 − µ. Thus EYn+1 = µ. Also, we know that Zn+1 → 0 in probability, so that Yn+1 → µ in probability. Exercise 3 (Discrete uniform). Let X = (X1 , . . . , Xn ) be a random sample where X1 is uniformly distributed in {1, 2, . . . , θ}, where θ ∈ Z+ is unknown. Let T = max {X1 , . . . , Xn }. Show that T n+1 − (T − 1)n+1 T n − (T − 1)n is an unbiased and consistent estimator for θ. W = Solution. An easy calculation shows that the pdf of T is given by 1 P(T = t) = n [tn − (t − 1)n ]. θ Thus θ X tn+1 − (t − 1)n+1 EW = · P(T = t). tn − (t − 1)n t=1 This is a telescoping sum! We easily see that W is unbiased. It is easy to show that T is consistent: θ − 1 n P(T 6= θ) = → 0. θ Re-write T − ( T T−1 )n · (T − 1) W = . 1 − ( T T−1 )n Since T converges to θ in probability, it is easy to show that ( T T−1 )n → 0 in probability, from which the result follows. Exercise 4 (Normal distribution). Let X = (X1 , . . . , Xn ) be a random sample from the normal distribution with mean µ and variance σ 2 . Find the method of moment estimators in the case where both parameters are unknown. Solution. Clearly, the moment estimator for µ is given by X̄. The moment estimator for µ2 = EX12 = σ 2 + µ2 is given by n 1X 2 M2 = X n i=1 i Thus the moment estimator for σ 2 is given by T = M2 − [X̄]2 . The short-cut formula gives that n 1X (Xi − X̄)2 . T = n i=1 Exercise 5 (A classic example of Fisher arising from genetics). Let X = (X1 , . . . , Xn ) be a random sample such that X1 takes values on the set {a, b, c, d} with respective probabilities given by [ 2+θ , 1−θ , 1−θ , 4θ ], 4 4 4 where θ ∈ (0, 1) is unknown. Suppose you observe data: x = (a, b, c, d, a, a, a, b, b, d, c, a). Find the mle for θ. You may need to use some sort of calculator. Solution. Let na be the number a’s that occur. Similarly, define nb , nc , and nd . Note that na + nb + nc + nd = n. We have that 2 + θ na 1 − θ nb 2 − θ nc θ nd P(X = x) = · · · 4 4 4 4 If I counted properly, there are na = 5, nb = 3, nc = 2 = nd . Maximizing the log-likelihood, results in solving a polynomial equation. I got θ̂ ≈ 0.355. Remark 1. In an earlier version of this solution the multinomial term n! appeared in the likelihood P(X = x). This was an easy typo na !nb !nc nd ! to make, and luckily does not effect the computation of the mle. What is true is that: P(Na = na , Nb = nb , Nc = nc , Nd = nd ) = n! 2 + θ na 1 − θ nb 2 − θ nc θ nd · · · , na !nb !nc nd ! 4 4 4 4 where for example Na = n X 1[Xi = a]. i=1 Exercise 6 (Gamma distribution). Recall that if Y has the Gamma distribution with parameters α and β, then it has pdf given by 1 y α−1 e−y/β 1[y > 0] g(y; α, β) = α β Γ(α) and EY = αβ. Let α > 0 be known and fixed. Let X = (X1 , . . . , Xn ) be a random sample from the Gamma distribution with parameters α and β > 0. Suppose that Z is an unbiased estimator for β 3 . Show that 9β 6 Var(Z) ≥ . nα Solution. The Cramer-Rao bound gives that [g 0 (β)]2 Var(Z) ≥ , nI(β) where g(β) = β 3 , so that g 0 (β) = 3β 2 , and I is the Fisher information for a single sample, Y . We have that ∂2 I(β) = −E log g(Y ; β) . 2 ∂β An easy calculation gives that 2Y α α I(β) = E 3 − = 2, β β β from which the result follows.