Download solutions to x2 x3

Exercise X2 – Poisson Regression
Model: X1 , . . . , Xn are independent with Xi ∼ Poisson(λi ) where λi = exp(β0 +β1 zi )
and z1 , . . . , zn are known constants.
(a) Let θ = (β0 , β1 ). The joint pmf is
f (x | θ) =
n
Y
λxi e−λi
i
= e−
xi !
i=1
P
− i exp(β0 +β1 zi )
= e
P
i
λi
Q
P
( i xi !)−1 exp { i xi (β0 + β1 zi )}
Q
P
P
( i xi !)−1 exp {β0 i xi + β1 i zi xi }
which has the form of an n-variate 2pef with natural sufficient statistic T (X) = (
Note that the parameter space is Θ = R2 .
P
i
Xi ,
P
(b) The MLE will be the value of θ which solves Eθ T (X) = T (x) which is just an
abbreviation for the system of equations
P
P
E i Xi =
xi
P
Pi
E i zi Xi =
i zi xi .
Since EXi = exp(β0 + β1 zi ), this becomes
P
P
i exp(β0 + β1 zi ) =
i xi
P
P
i zi exp(β0 + β1 zi ) =
i zi xi
(‡)
(c) From part (a), the log-likelihood is
X
X
X
`(θ) = log f (x | θ) = −
exp(β0 + β1 zi ) + β0
xi + β 1
zi xi + constant .
i
i
i
The MLE will (typically) be a stationary point, that is, a solution of the system
X
X
∂`
= −
exp(β0 + β1 zi ) +
xi = 0
∂β0
i
i
X
X
∂`
= −
zi exp(β0 + β1 zi ) +
zi xi = 0 ,
∂β1
i
i
which is clearly equivalent to (‡) above.
Optional Remarks (not included in test material): You were not asked to solve
the system (‡) to obtain the MLE, but I will give a few remarks on this. The
P system
cannot be solved completely in closed form, but some progress can be made. If xi > 0,
dividing the second equation by the first leads to a single equation in one unknown
P β1 zi
P
zi e
zx
P β z = Pi i.
e1i
xi
The left side of this equation approaches max zi as β1 → ∞ and min zi as β1 → −∞.
So a finite solution for β1 exists so long as
P
zi xi
min zi < P
< max zi .
xi
i zi Xi ).
Exercise X3 – A Prior Distribution Which is a Mixture of Betas
We use the notation from lecture.
The likelihood is L(θ) = θt (1 − θ)n−t . The posterior satisfies
π(θ | x) ∝ L(θ) × π(θ)
= θt (1 − θ)n−t (p1 f1 (θ) + p2 f2 (θ))
α2 +t−1
α1 +t−1
θ
(1 − θ)β2 +n−t−1
θ
(1 − θ)β1 +n−t−1
+ p2
≡ g(θ) .
= p1
B(α1 , β1 )
B(α2 , β2 )
For convenience, we define g(θ) to be the expression on the last line above. Recall that
Z
1
θα−1 (1 − θ)β−1 dθ = B(α, β)
0
for α > 0, β > 0. Using this, we get the posterior by normalizing g to be a pdf:
π(θ | x) = R 1
0
g(θ)
g(θ0 ) dθ0
α2 +t−1
θ
(1 − θ)β2 +n−t−1
θα1 +t−1 (1 − θ)β1 +n−t−1
+ p2
B(α1 , β1 )
B(α2 , β2 )
B(α1 + t, β1 + n − t)
B(α2 + t, β2 + n − t)
p1
+ p2
B(α1 , β1 )
B(α2 , β2 )
p1
=
=
p1 B(α1 + t, β1 + n − t) θα1 +t−1 (1 − θ)β1 +n−t−1 p2 B(α2 + t, β2 + n − t) θα2 +t−1 (1 − θ)β2 +n−t−1
·
+
·
B(α1 , β1 )
B(α1 + t, β1 + n − t)
B(α2 , β2 )
B(α2 + t, β2 + n − t)
B(α2 + t, β2 + n − t)
B(α1 + t, β1 + n − t)
+ p2
p1
B(α1 , β1 )
B(α2 , β2 )
= p∗1 f1∗ (θ) + p∗2 f2∗ (θ)
where
pi B(αi + t, βi + n − t)
B(αi , βi )
p∗i =
p1 B(α1 + t, β1 + n − t) p2 B(α2 + t, β2 + n − t)
+
B(α1 , β1 )
B(α2 , β2 )
and fi is a Beta(αi + t, βi + n − t) pdf.