Center for Advanced Intelligence Project

Speaker: Prof. Alquier Pierre (ENSAE, France)

Title: Guarantees for Approximate Bayesian Inference

Abstract:
While Bayesian methods are extremely popular in statistics and machine learning, their application to massive datasets is often challenging, when possible at all. Indeed, the classical MCMC algorithms targeting the exact posterior are prohibitively slow when both the model dimension and the sample size are large or when the likelihood is not tractable. Fast algorithms were proposed, at the price of targeting approximations of the posterior. In this talk, I will discuss two approaches.

The first approach is an approximate Metropolis-Hastings algorithm (MH). In MH, a simulation from the transition kernel P requires a computation of the likelihood that might be expensive. We propose an approximation Q of P leading to a fast algorithm. We control explicity the total variation distance (TV) between the posterior and the distribution of the simulations. This algorithm was proposed in Alquier, Friel, Everitt and Boland (Statistics and Computing, 2016) under the name “Noisy-MCMC”. I will also mention recent results by Rudolf and Schweizer (2018) who relaxed the assumptions of our results by using the Wasserstein distance instead of TV.

The second approach is Variational Bayesian inference (VB). VB aims at approximating the posterior by a distribution in a tractable family. Thus, MCMC are replaced by an optimization algorithm which is orders of magnitude faster. VB methods have been applied in such computationally demanding applications as including collaborative filtering, NLP and text processing… However, despite nice results in practice, the theoretical properties of these approximations are usually not known. I will present conditions ensuring the asymptotic concentration of the variational approximation of the posterior around the true parameter. These results are taken from our recent works: Alquier and Ridgway (2017) and Chérief-Abdellatif and Alquier (2018).