Papers Published
Abstract:
We consider generalized Bayesian inference on stochastic processes and
dynamical systems with potentially long-range dependency. Given a sequence of
observations, a class of parametrized model processes with a prior
distribution, and a loss function, we specify the generalized posterior
distribution. The problem of frequentist posterior consistency is concerned
with whether as more and more samples are observed, the posterior distribution
on parameters will asymptotically concentrate on the "right" parameters. We
show that posterior consistency can be derived using a combination of classical
large deviation techniques, such as Varadhan's lemma, conditional/quenched
large deviations, annealed large deviations, and exponential approximations. We
show that the posterior distribution will asymptotically concentrate on
parameters that minimize the expected loss and a divergence term, and we
identify the divergence term as the Donsker-Varadhan relative entropy rate from
process-level large deviations. As an application, we prove new quenched and
annealed large deviation asymptotics and new Bayesian posterior consistency
results for a class of mixing stochastic processes. In the case of Markov
processes, one can obtain explicit conditions for posterior consistency,
whenever estimates for log-Sobolev constants are available, which makes our
framework essentially a black box. We also recover state-of-the-art posterior
consistency on classical dynamical systems with a simple proof. Our approach
has the potential of proving posterior consistency for a wide range of Bayesian
procedures in a unified way.