© 2015 IEEE. Probabilistically quantifying uncertainty in parameters, predictions and decisions is a crucial component of broad scientific and engineering applications. This is however difficult if the number of parameters far exceeds the sample size. Although there are currently many methods which have guarantees for problems characterized by large random matrices, there is often a gap between theory and practice when it comes to measures of statistical significance for matrices encountered in real-world applications. This paper proposes a scalable framework that utilizes state-of-the-art methods to provide approximations to the marginal posterior distributions. This framework is used to approximate marginal posterior inclusion probabilities for Bayesian variable selection.