Download 2016 May Theory - Solution 1. You have two components, whose

2016 May Theory - Solution
1. You have two components, whose lifetimes are X and Y , two independent Exponentially distributed random variables with mean λ. You put the two components in series,
meaning that your system works as long as both components work. Then, your system
stop working at some random time T , with T = min(X, Y ).
(a) Compute P (X > t), for any t ≥ 0.
(b) Compute the p.d.f. of T .
(c) Your remaining component has a remaining lifetime U (that is U = max(X, Y ) −
T ). Compute the joint p.d.f. of U and T , fU,T (u, t) for t, u ≥ 0.
(d) Are U and T independent?
2. Three random points A, B, and C, are chosen independently from the uniform distribution on a unit circle.
(a) State the distribution of the minimal distance of A and B along the circle (i.e.
the shorter arc length).
(b) Give the minimal distance of A and B along the circle is x, find the probability
that the center of the circle lies in the triangle ABC.
(c) Find the probability that the center of the circle lies in the triangle ABC.
3. A coin having probability p of coming up heads is continually flipped until both heads
and tails have appeared. Let X denote the total number of flips necessary.
(a) What is the probability that the last flip lands Head?
(b) Conditioning on the first flip being a head, what is the distribution of X − 1?
(c) Compute E(X).
4. Let X1 , · · · , Xn be independently and identically distributed random variables with
pdf
(
x2
2
exp{− πθ
if x ≥ 0
2}
πθ
f (x; θ) =
0
if x < 0,
where θ > 0 is an unknown parameter. Note that
2
π 3π
2
2
4
E(X1 ) = θ, E(X1 ) =
θ , E(X1 ) =
θ4 .
2
4
1
(a) Find the uniformly minimal variance unbiased estimator (UMVUE) for θ2 .
(b) Does the variance of the estimator in part (a) achieve the Cramér-Rao lower
bound? Justify your answer.
(c) Derive the uniformly most powerful (UMP) size α test of the null hypothesis
H0 : θ = θ0 against the alternative hypothesis H1 : θ = θ1 , wherePθ0 and θ1 are
given positive numbers with θ1 > θ0 . Express the test in terms of ni=1 Xi2 and a
quantile of χ2 distribution.
5. Suppose X1 , . . . , Xn are independent and identically distributed random variables with
the common pdf
f (x|θ) = e−(x−θ) , x ≥ θ,
where θ is an unknown parameter. Let X(1) = min1≤i≤n Xi and X(n) = max1≤i≤n Xi .
(a) Show that X(1) is sufficient for θ. Find the pdf and cdf of X(1) .
(b) Show that X(n) − X(1) is an ancillary statistic. Compute E(X(1) ).
(c) Consider a class of confidence intervals for θ in the form of (X(1) − a, X(1) − b)
with a ≥ b ≥ 0. Find the level 1 − α confidence interval that has the minimum
length.
6. Let X1 , X2 , . . . , Xn be independent and identically distributed
P (i.i.d.) random variables
with P (X1 = i) = pi for i = 1, 2, 3, where 0 < pi < 1 and 3i=1 pi = 1. Define
Yi =
n
X
I(Xj = i)
j=1
for i = 1, 2, 3, where I(Xj = i) is the indicator function that takes the value 1 if Xj = i
and the value 0 otherwise.
(a) Find the probability mass function (pmf) of Y1 . Find the joint pmf of (Y1 , Y2 ).
(b) Are Y1 and Y2 independent? Justify your answer.
(c) Find the maximum likelihood estimator (MLE) pd
1 p2 for the product p1 p2 ,
(d) Find the uniformly minimum variance unbiased estimator (UMVUE) for p1 p2 .
√
(e) Find the limiting distribution of n(p̂1 − p1 ) as n → ∞, where p̂1 = Y1 /n.
(f) Find the MLE’s for p1 , p2 , and p3 assuming that p1 = p2 .
2
Solutions:
1. (a) P (X > t) = e−t/λ .
(b) By the definition of T , P (T > t) = P (X > t, Y > t)) = e−2t/λ . So the the p.d.f.
of T is fT (t) = λ2 e−2t/λ .
(c) From the textbook, we can obtain the joint density function of (X(1) , X(2) ) is
fX(1) ,X(2) (x1 , x2 ) = λ22 e−(x1 +x2 )/λ . Using the the linear transformation U = X(2) −
X(1) , T = X(1) , we can get the joint density function of (U, T ) is fU,T (u, t) =
2 −u/λ −2t/λ
e
e
.
λ2
(d) Yes, they are independent.
2. (a) Uniform distribution on [0, π].
(b) Conditional on the distance between A and B is 0 ≤ x < π, the probability that
the center is in the triangle ABC is x/2π.
Rπ x 1
(c) The (unconditional) probability of the event is 0 2π
dx = 14 .
π
P
1
j
3. (a) P (last flip is Head) = P (T H)+P (T T H)+P (T T T H)+· · · = ∞
j=1 q p = pq 1−q =
q.
(b) Let H = first flip is Head, then X − 1|H ∼ Geometric(1 − p).
(c)
E(X) = E(X|H)P (H) + E(X|H c )P (H c )
1
1
)p + (1 + )(1 − p)
= (1 +
1−p
p
p
1−p
= 1+
+
.
1−p
p
P
4. (a) One-parameter full-rank exponential family. Y = ni=1 Xi2 is complete and sufficient for θ. And E(Y
) = nE(X 2 ) = nπ
θ2 . By Rao-Blackwell theorem, the
2
P
2
n
X2
i=1 i
estimator θ̂1 = 2Y
=
is UMVUE for θ2 . Alternatively, one can show that
nπ
nπ
2Y
achieves the C-R bound (as shown in part (b)), so it is UMVUE.
n
(b) Calculate
Var(θ̂1 ) =
4 π 2 θ4 n
2θ4
4
2
nV
ar(X
)
=
=
.
n2 π 2
n2 π 2 2
n
Note that
s1 (x, θ) =
∂
1
2
log f (x; θ) = − + 3 x2 .
∂θ
θ πθ
3
Then
4
I1 (θ) = Var(s1 ) = 2 6 Var(X 2 ).
π θ
2
2
Since V ar(X 2 ) = E(X 4 ) − [E(X 2 )]2 = 3π4 θ4 − π2 θ4 =
I1 (θ) =
π2 θ4
,
2
we have
4 π2 4
2
θ = 2.
2
6
π θ 2
θ
So the Cramér lower variance bound for θ2 is
(2θ)2
2θ4
(2θ)2
=
=
.
In (θ)
2n/θ2
n
Yes, its variance achieves the Cramér-Rao lower bound. Alternatively, decompose
the score function as
P
n
n
∂ X
2 X 2
n 2 ni=1 x2i
n
sn (x, θ) =
x = 3(
− θ2 ).
log f (xi ; θ) = − + 3
∂θ i=1
θ πθ i=1 i
θ
nπ
(c) Using the NP lemma, the uniformly most powerful (UMP) size α test of the null
hypothesis H0 : θ = θ0 against the alternative hypothesis H1 : θ = θ1 is: reject
(x;θ1 )
H0 if ff (x;θ
> c for some c. Since
0)
n
θ0n
1X 2
f (x; θ1 )
= n exp{
x}
f (x; θ0 )
θ1
π i=1 i
is increasing in x2 , the test is equivalent to: reject H0 if
α test, we need
α = P(
n
X
Xi2 ≥ c|θ = θ0 ) = P (Y ≥ c|θ = θ0 ) = P (
i=1
therefore χ2n,α =
2c
πθ02
and c =
Pn
i=1
Xi2 ≥ c. To get size
2
2
2
Y ≥ 2 c) = P (χ2n ≥ 2 c),
2
πθ0
πθ0
πθ0
πθ02 χ2n,α
.
2
5. (a) The joint pdf of data is
f (x|θ) = enθ e−
Pn
i=1
xi
I(X(1) ≥ θ),
where X(1) = min1≤i≤n Xi . By factorization theorem, X(1) is a sufficient statistic
for θ. The density of X(1) is fX(1) (x|θ) = ne−n(x−θ) I(x ≥ θ). The cdf is: if x < θ,
Rx
then FX(1) (x) = 0 if x < θ; if x ≥ θ, then FX(1) (x) = θ nen(θ−x) dx = 1 − en(θ−x) .
4
(b) f (x|θ) is a location family. Define Zi = Xi − θ, then Zi has the exponential
distribution with mean 1. Note that X(n) − X(1) = Z(n) − Z(1) whose distribution
does not depend on θ, so X(n) − X(1) is ancillary. Note X(1) − θ ∼ exp(1/n), so
E(X(1) ) = θ + n1 .
(c) Since P (X(1) − a ≤ θ ≤ X(1) − b) = P (b + θ ≤ X(1) ≤ a + θ) = FX(1) (a + θ) −
FX(1) (b + θ) = e−nb − e−na = 1 − α, so we have a = − n1 log[e−nb − (1 − α)]. The
length of CI is a − b = − n1 log[1 − (1 − α)enb ], which is increasing in b and takes
the minimum at b = 0. So the shortest length is given by a = − n1 log α and b = 0,
which leads to the CI [X(1) + n1 log α, X(1) ].
6. (a) The p.m.f. of Y1 is
fY1 (y1 ) = P (Y1 = y1 ) =
n!
py1 (1 − p1 )n−y1 ,
y1 !(n − y1 )! 1
y1 = 0, 1, . . . , n.
The p.m.f. of (Y1 , Y2 ) is
fY1 ,Y2 (y1 , y2 ) = P (Y1 = y1 , Y2 = y2 ) =
n!
py1 py2 (1−p1 −p2 )n−y1 −y2
y1 !y2 !(n − y1 − y2 )! 1 2
for y1 , y2 = 0, 1, . . . , n and y1 + y2 ≤ n.
(b) Y1 and Y2 are not independent, because fY1 ,Y2 (y1 , y2 ) 6= fY1 (y1 )fY2 (y2 ).
(c) The log likelihood function of (p1 , p2 ) is
l(p1 , p2 ) = Y1 log(p1 ) + Y2 log(p2 ) + (n − Y1 − Y2 ) log(1 − p1 − p2 ).
Its first derivatives with respect to p1 and p2 are
Y1 n − Y1 − Y2
∂l
=
−
,
∂p1
p1
1 − p1 − p2
∂l
Y2 n − Y1 − Y2
=
−
.
∂p2
p2
1 − p1 − p2
Letting these derivative equal zero, we will have
Y1
Y2
n − Y1 − Y2
=
=
.
p1
p2
1 − p1 − p2
This will give p̂1 =
Y1
n
and p̂2 =
Y2
.
n
5
So the MLE of the product is
Y1 Y2
n n
=
Y1 Y2
.
n2
(d) Note that
E(Y1 Y2 ) = Cov(Y1 , Y2 )+E(Y1 )E(Y2 ) = −np1 p2 +(np1 )(np2 ) = (n2 −n)p1 p2 = n(n−1)p1 p2 .
Alternatively,
E(Y1 Y2 ) = E
n
X
I(Xj = 1)
j=1
=
X
n
X
!
I(Xk = 2)
=
n X
n
X
E (I(Xj = 1)I(Xk = 2))
j=1 k=1
k=1
E (I(Xj = 1)I(Xk = 2)) = n(n − 1)P (Xj = 1)P (Xk = 2) = n(n − 1)p1 p2 .
j6=k
So Y1 Y2 /[n(n − 1)] is unbiased for p1 p2 . Note that the joint p.m.f of Xi ’s are
f (x1 , . . . , xn ) =
n
Y
I(Xi =1) I(Xi =2) I(Xi =3)
p3
p2
p1
i=1
Pn
I(X =1)
Pn
I(X =2)
Pn
(X =3)
= p1 i=1 i p2 i=1 i p3 i=1 i
= pY1 1 pY2 2 pY3 3 = pY1 1 pY2 2 (1 − p1 − p2 )n−Y1 −Y −2
= (1 − p1 − p2 )n exp{Y1 [ln p1 − ln(1 − p1 − p2 )] + Y2 [ln p2 − ln(1 − p1 − p2 )]},
which is a full-rank two-parameter exponential family, so (Y1 , Y2 ) are sufficient
Y1 Y2
is UMVUE.
statistics. By Rao-Blackwell. the estimator n(n−1)
(e) Note that I(X1 = 1), .√
. . , I(Xn = 1) is i.i.d Bernoulli random variable with mean
p1 . By CLT, we have n(p̂1 − p1 ) → N (0, p1 (1 − p1 )) as n → ∞.
(f) If p1 = p2 , the log likelihood function becomes
l(p1 ) = (Y1 + Y2 ) log(p1 ) + (n − Y1 − Y2 ) log(1 − 2p1 ).
Its first derivatives with respect to p1 is p̃1 = p̃2 =
6
Y1 +Y2
2n
and p̃3 = 1 − 2p̃2 =
Y3
.
n