Survey

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Survey

Document related concepts

no text concepts found

Transcript

Exercise X2 – Poisson Regression Model: X1 , . . . , Xn are independent with Xi ∼ Poisson(λi ) where λi = exp(β0 +β1 zi ) and z1 , . . . , zn are known constants. (a) Let θ = (β0 , β1 ). The joint pmf is f (x | θ) = n Y λxi e−λi i = e− xi ! i=1 P − i exp(β0 +β1 zi ) = e P i λi Q P ( i xi !)−1 exp { i xi (β0 + β1 zi )} Q P P ( i xi !)−1 exp {β0 i xi + β1 i zi xi } which has the form of an n-variate 2pef with natural sufficient statistic T (X) = ( Note that the parameter space is Θ = R2 . P i Xi , P (b) The MLE will be the value of θ which solves Eθ T (X) = T (x) which is just an abbreviation for the system of equations P P E i Xi = xi P Pi E i zi Xi = i zi xi . Since EXi = exp(β0 + β1 zi ), this becomes P P i exp(β0 + β1 zi ) = i xi P P i zi exp(β0 + β1 zi ) = i zi xi (‡) (c) From part (a), the log-likelihood is X X X `(θ) = log f (x | θ) = − exp(β0 + β1 zi ) + β0 xi + β 1 zi xi + constant . i i i The MLE will (typically) be a stationary point, that is, a solution of the system X X ∂` = − exp(β0 + β1 zi ) + xi = 0 ∂β0 i i X X ∂` = − zi exp(β0 + β1 zi ) + zi xi = 0 , ∂β1 i i which is clearly equivalent to (‡) above. Optional Remarks (not included in test material): You were not asked to solve the system (‡) to obtain the MLE, but I will give a few remarks on this. The P system cannot be solved completely in closed form, but some progress can be made. If xi > 0, dividing the second equation by the first leads to a single equation in one unknown P β1 zi P zi e zx P β z = Pi i. e1i xi The left side of this equation approaches max zi as β1 → ∞ and min zi as β1 → −∞. So a finite solution for β1 exists so long as P zi xi min zi < P < max zi . xi i zi Xi ). Exercise X3 – A Prior Distribution Which is a Mixture of Betas We use the notation from lecture. The likelihood is L(θ) = θt (1 − θ)n−t . The posterior satisfies π(θ | x) ∝ L(θ) × π(θ) = θt (1 − θ)n−t (p1 f1 (θ) + p2 f2 (θ)) α2 +t−1 α1 +t−1 θ (1 − θ)β2 +n−t−1 θ (1 − θ)β1 +n−t−1 + p2 ≡ g(θ) . = p1 B(α1 , β1 ) B(α2 , β2 ) For convenience, we define g(θ) to be the expression on the last line above. Recall that Z 1 θα−1 (1 − θ)β−1 dθ = B(α, β) 0 for α > 0, β > 0. Using this, we get the posterior by normalizing g to be a pdf: π(θ | x) = R 1 0 g(θ) g(θ0 ) dθ0 α2 +t−1 θ (1 − θ)β2 +n−t−1 θα1 +t−1 (1 − θ)β1 +n−t−1 + p2 B(α1 , β1 ) B(α2 , β2 ) B(α1 + t, β1 + n − t) B(α2 + t, β2 + n − t) p1 + p2 B(α1 , β1 ) B(α2 , β2 ) p1 = = p1 B(α1 + t, β1 + n − t) θα1 +t−1 (1 − θ)β1 +n−t−1 p2 B(α2 + t, β2 + n − t) θα2 +t−1 (1 − θ)β2 +n−t−1 · + · B(α1 , β1 ) B(α1 + t, β1 + n − t) B(α2 , β2 ) B(α2 + t, β2 + n − t) B(α2 + t, β2 + n − t) B(α1 + t, β1 + n − t) + p2 p1 B(α1 , β1 ) B(α2 , β2 ) = p∗1 f1∗ (θ) + p∗2 f2∗ (θ) where pi B(αi + t, βi + n − t) B(αi , βi ) p∗i = p1 B(α1 + t, β1 + n − t) p2 B(α2 + t, β2 + n − t) + B(α1 , β1 ) B(α2 , β2 ) and fi is a Beta(αi + t, βi + n − t) pdf.