Block Hypergeometric Priors for Bayesian Variable Selection in Linear Models

Block Hypergeometric Priors for Bayesian Variable Selection in Linear Models

Cathy Lee

This summer I worked on the “Block Hypergeometric Priors for Bayesian Variable Selection in Linear Models” project with Dr. Merlise Clyde. My first goal was to derive an analytic expression for Bayes factors using the truncated Compound Confluent Hypergeometric prior from “Mixtures of g-Priors in Generalized Linear Models” by Li and Clyde and using the block g-prior structure proposed by Hans et al in “Block Hyper-g Priors in Bayesian Regression.” I completed my goal to study the theoretical properties of Bayes factors conditioned on error variance under block orthogonal designs. Specifically, I showed that such Bayes factors do not exhibit Conditional Lindley’s Paradox and do have Information Consistency (under certain restrictions on prior hyperparameters). I also showed that these Bayes factors are model selection consistent under certain cases, such as when the true model is not contained in the model being examined. I also evaluated the intrinsic consistency, estimation consistency, and prediction consistency, which relate to the form of the prior or expectation of the shrinkage factor. An examination of these properties was not part of the original proposal, but are important since priors are not chosen by their resulting Bayes factors properties alone. For unknown variance, I showed that it is not possible to derive an analytic expression for the marginal distribution of Y conditioned only on the model, and used Laplace approximation to find the marginal distribution of Y. I also used computer simulation to examine unconditional Bayes factors. In addition, the current R function for the hypergeometric functions encountered loss of precision errors, so I modified the C code for those functions and started to rebuild the package containing those functions. Next, I derived posteriors under the relaxed condition of non-orthogonal blocks, and modified the form of Y in order to construct a Markov Chain Monte Carlo (MCMC) sampling scheme to sample posterior values. I also implemented an adaptive random walk scheme to sample the complementary shrinkage factors. Simulations showed that the mean-squared error of the coefficients using the MCMC sampling scheme described above is approximately the same as the resulting MSE using the ordinary least squares method and the fixed shrinkage factor MCMC sampler.