Making Sense of Uncertainty: A New Approach to Bayesian Inference

Bayesian statistics is a powerful tool for making sense of data, helping us understand the uncertainty around our estimates. However, when our models are not perfect—an issue known as model misspecification—Bayesian posteriors may not accurately reflect the true uncertainty. This inconsistency can lead to different posteriors from different datasets drawn from the same true distribution, complicating reproducibility.

The Problem of Model Misspecification

Researchers have proposed a new criterion to address this issue: the probability that two credible sets from independent datasets overlap. This overlap probability assesses the reproducibility of uncertainty quantification. Unfortunately, the standard posterior often fails to meet this criterion under misspecification, highlighting a significant flaw in traditional Bayesian methods.

Introducing BayesBag: A Solution to the Problem

A promising solution is BayesBag, a method that averages posterior distributions from bootstrapped datasets. Motivated by Jeffrey conditionalization, BayesBag typically satisfies the overlap criterion. Additionally, the bagged posterior has an asymptotic normal distribution, supported by the Bernstein-Von Mises theorem.

Applications and Benefits

The effectiveness of BayesBag is demonstrated through simulation experiments and an application to crime rate prediction. This approach offers a practical and widely applicable way to improve the reproducibility of Bayesian inference under model misspecification.