Zhang, Mingtian;
Hawkins-Hooker, Alex;
Paige, Brooks;
Barber, David;
(2023)
Moment Matching Denoising Gibbs Sampling.
In: Oh, A and Naumann, T and Globerson, A and Saenko, K and Hardt, M and Levine, S, (eds.)
Advances in Neural Information Processing Systems 36 (NeurIPS 2023).
(pp. pp. 1-17).
NeurIPS: San Diego, CA, USA.
Preview |
Text
8051_moment_matching_denoising_gibb.pdf - Published Version Download (5MB) | Preview |
Abstract
Energy-Based Models (EBMs) offer a versatile framework for modeling complex data distributions. However, training and sampling from EBMs continue to pose significant challenges. The widely-used Denoising Score Matching (DSM) method [41] for scalable EBM training suffers from inconsistency issues, causing the energy model to learn a ‘noisy’ data distribution. In this work, we propose an efficient sampling framework, (pseudo)-Gibbs sampling with moment matching, which enables effective sampling from the underlying clean model when given a ‘noisy’ model that has been well-trained via DSM. We explore the benefits of our approach compared to related methods and demonstrate how to scale the method to high-dimensional datasets.
| Type: | Proceedings paper |
|---|---|
| Title: | Moment Matching Denoising Gibbs Sampling |
| Event: | 37th Conference on Neural Information Processing Systems (NeurIPS 2023) |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.neurips.cc/paper_files/paper/2... |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10192386 |
Archive Staff Only
 |
View Item |
