Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Local Graphical Model Search Liang Zhang, Adrian Dobra and Mike West Institute of Statistics & Decision Sciences Duke University Abstract • MCMC/stochastic search in Graphical Models • Global search versus “Local” learning around a variable Y of scientific interest Addressing the problem Mixing times for λ=0.9,0.99,0.999 Must continue to explore global picture to understand local structure around Y • “Large p” paradigm: (e.g., microarray data) difficulties with global search • Novel “Local Graphical Model Search” here • “Targeted” MCMC Simulation: • Motivating examples and illustrations e.g. p=4 100 iterations But: too much time/effort/weight on non-local structure … Targeted MH MCMC • Local Edge: Incident at Y or between Y's neighbours MCMC and Shotgun Stochastic Search (SSS) methods for global graphical model search 1 edge in/out Metropolis methods (Jones et al, 2005) • MCMC: Add/delete randomly chosen - LOCAL edge with probability λ (>0.5) - Non-local edge with probability 1-λ • High λ: "Targeted" proposal - favours local graph; MC still globally irreducible 1000 iterations 10000 iterations 100 1000 10000 Pr(edge) iterations iterations iterations (Y, X1) (0.36,0.63) (0.462,0.539) (0.488,0.513) (Y, X2) (0.37,0.63) (0.459,0.539) (0.488,0.513) (Y, X3) (0.135,0.85) (0.381,0.614) (0.461,0.540) Acc Rate (0.80,0.94) (0.85,0.90) (0.87,0.88) Discussion Regression Model Search versus Local Graphical Model Search? M: a local graphical model (Bernoulli edge inclusion) R : a regression model (Bernoulli variable inclusion) • Regression model search: Local Interest Focus on structure “around” target variable Y e.g. p=3 23=8 states • Local Graphical Model Search: Transition matrix T and its eigenvalues • Regression model search challenged by collinearity • Local Graphical Model search can effectively explore neighbourhood structures • Targeted Metropolis-Hasting can be effective: theory and general development underway • Problem: Dominance of global structure in exploring many models when p is large • Sparsity prior: Pr(edge in)=small, plays a role • e.g.: p=100, Y related “weakly” to X1 and X2 Some key references B. Jones, A. Dobra, C. Carvalho, C. Hans, C. Carter and M. West, (2005) Experiments in stochastic computation for high-dimensional graphical models, Statistical Science 20, 388-400. Mixing time: P. Giudici and P.J. Green (1999) Decomposable graphical Gaussian model determination, Biometrika 86, 785—801. C. Hans, A. Dobra and M. West (2005) Shotgun stochastic search in regression with many predictors. ISDS Discussion Paper (submitted). S.L. Lauritzen (1996) Graphical Models. Clarendon Press, Oxford.