Download seminar

BSWG Seminar, Raleigh, NC
BSWG Seminar, Raleigh, NC
Overview
Prior Distributions for the Variable
Selection Problem
• The Variable Selection Problem(VSP)
• A Bayesian Framework
Sujit K Ghosh
• Choice of Prior Distributions
Department of Statistics
North Carolina State University
• Illustrative Examples
• Conclusions
http://www.stat.ncsu.edu/∼ghosh/
Email: [email protected]
Bayesian Statistics Working Group, NCSU
Disclaimer: This talk is not entirely based on my own research work
Sujit Ghosh, October 3, 2006
1
BSWG Seminar, Raleigh, NC
Sujit Ghosh, October 3, 2006
2
BSWG Seminar, Raleigh, NC
Suppose the true data generating process (DGP) is given by
The Variable Selection Problem
y = X 0 β 0 + ,
Consider the following canonical linear model:
y = Xβ + (1)
where β 0 = (β10 , . . . , βp00 )T , X 0 is n × p0 and WLOG assume that
X = (X 0 | X 1 )T and p ≥ p0 ≥ 1 (i.e., X 1 is n × (p − p0 ))
The LSE of β and σ 2 are given by
where ∼ Nn (0, σ 2 I) and β = (β1 , . . . , βp )T (X is an n × p matrix)
β̂
ˆ
σ2
• Under the above model, suppose also that only an unknown
subset of the coefficients βj ’s are nonzero
=
=
(X T X)− X T y
(3)
T
y (I − P X )y/(n − r)
• The problem of variable selection is to identify this unknown
subset.
where r =rank(X) ≤ min(n, p), P X = X T (X T X)− X T is the
projection matrix and (X T X)− is a g-inverse of X T X. Then
• Notice that the above canonical framework can be used to
address many other problems of interest including multivariate
polynomial regression and nonparametric function estimation
Lemma: E[β̂] = ((X T0 X 0 )− X T0 X 0 β 0 , 0)T and E[X β̂] = X 0 β 0 .
Further, E[σˆ2 ] = σ 2 for any g-inverse of X T X.
Sujit Ghosh, October 3, 2006
(2)
In particular, if rank(X 0 ) = p0 , then E[β̂] = (β 0 , 0)T .
3
Sujit Ghosh, October 3, 2006
4