Download Local Graphical Model Search - Duke Statistical

Local Graphical Model Search
Liang Zhang, Adrian Dobra and Mike West
Institute of Statistics & Decision Sciences
Duke University
Abstract
• MCMC/stochastic search in Graphical Models
• Global search versus “Local” learning around
a variable Y of scientific interest
Addressing the problem
Mixing times for
λ=0.9,0.99,0.999
Must continue to explore global picture to
understand local structure around Y
• “Large p” paradigm: (e.g., microarray data) difficulties with global search
• Novel “Local Graphical Model Search” here
• “Targeted” MCMC
Simulation:
• Motivating examples and illustrations
e.g. p=4
100 iterations
But: too much time/effort/weight on non-local
structure …
Targeted MH MCMC
• Local Edge: Incident at Y or between Y's neighbours
MCMC and Shotgun Stochastic Search (SSS)
methods for global graphical model search
1 edge in/out Metropolis methods
(Jones et al, 2005)
• MCMC: Add/delete randomly chosen
- LOCAL edge with probability λ (>0.5)
- Non-local edge with probability 1-λ
• High λ: "Targeted" proposal - favours local graph;
MC still globally irreducible
1000 iterations
10000 iterations
100
1000
10000
Pr(edge)
iterations
iterations
iterations
(Y, X1)
(0.36,0.63) (0.462,0.539) (0.488,0.513)
(Y, X2)
(0.37,0.63) (0.459,0.539) (0.488,0.513)
(Y, X3) (0.135,0.85) (0.381,0.614) (0.461,0.540)
Acc Rate (0.80,0.94)
(0.85,0.90)
(0.87,0.88)
Discussion
Regression Model Search versus Local Graphical
Model Search?
M: a local graphical model (Bernoulli edge inclusion)
R : a regression model (Bernoulli variable inclusion)
• Regression model search:
Local Interest
Focus on structure “around” target variable Y
e.g. p=3
23=8 states
• Local Graphical Model Search:
Transition matrix T and its
eigenvalues
• Regression model search challenged by collinearity
• Local Graphical Model search can effectively
explore neighbourhood structures
• Targeted Metropolis-Hasting can be effective:
theory and general development underway
• Problem: Dominance of global structure in
exploring many models when p is large
• Sparsity prior: Pr(edge in)=small, plays a role
• e.g.: p=100, Y related “weakly” to X1 and X2
Some key references
B. Jones, A. Dobra, C. Carvalho, C. Hans, C. Carter and M. West, (2005)
Experiments in stochastic computation for high-dimensional graphical models,
Statistical Science 20, 388-400.
Mixing time:
P. Giudici and P.J. Green (1999) Decomposable graphical Gaussian model
determination, Biometrika 86, 785—801.
C. Hans, A. Dobra and M. West (2005) Shotgun stochastic search in
regression with many predictors. ISDS Discussion Paper (submitted).
S.L. Lauritzen (1996) Graphical Models. Clarendon Press, Oxford.