Enhanced gradient-based MCMC in discrete spaces

Published in Transactions on Machine Learning Research, 2022

We introduce several discrete Metropolis-Hastings samplers that are conceptually inspired by MALA, and demonstrate their strong empirical performance across a range of challenging sampling problems in Bayesian inference and energy-based modelling. Methodologically, we identify why discrete analogues to \emph{preconditioned} MALA are generally intractable, motivating us to introduce a new kind of preconditioning based on auxiliary variables and the “Gaussian integral trick”.

Recommended citation: Enhanced gradient-based MCMC in discrete spaces. Rhodes, B. and Gutmann, M. Transactions on Machine Learning Research (2022).

Telescoping Density-Ratio Estimation

Published in Advances in Neural Information Processing Systems - Spotlight (top 4% of submissions), 2020

We propose a new framework, Telescoping Density-ratio Estimation (TRE), that enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces.

Recommended citation: Rhodes, B., Xu, K., and Gutmann, M. (2020). Telescoping Density-Ratio Estimation. In Advances in Neural Information Processing Systems

Variational Noise-Contrastive Estimation

Published in The 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 2019

We propose a new method for estimating the parameters of energy-based, latent variable models. The core contribution is the derivation of a variational lower bound for the noise-contrastive estimation objective function.

Recommended citation: Rhodes, B. and Gutmann, M. U. (2019). Variational noise-contrastive estimation. InThe 22nd InternationalConference on Artificial Intelligence and Statistics, pages 2741–2750.

Dimension Formulae for Iterated Function Systems

Unpublished MSci dissertation. The majority of the dissertation is an exposition of prior books and papers, but the final 10 pages contain original material, culminating in Theorem 7.19 which gives a lower bound on the Hausdorff dimension of a certain class of planar fractals.

Download here