UCL Discovery Stage
UCL home » Library Services » Electronic resources » UCL Discovery Stage

Generalized Energy Based Models

Arbel, M; Zhou, L; Gretton, A; (2021) Generalized Energy Based Models. In: Proceedings of the 9th International Conference on Learning Representations: ICLR 2021. ICLR Green open access

[thumbnail of 1585_generalized_energy_based_model.pdf]
Preview
Text
1585_generalized_energy_based_model.pdf - Published Version

Download (9MB) | Preview

Abstract

We introduce the Generalized Energy Based Model (GEBM) for generative modelling. These models combine two trained components: a base distribution (generally an implicit model), which can learn the support of data with low intrinsic dimension in a high dimensional space; and an energy function, to refine the probability mass on the learned support. Both the energy function and base jointly constitute the final model, unlike GANs, which retain only the base distribution (the "generator"). GEBMs are trained by alternating between learning the energy and the base. We show that both training stages are well-defined: the energy is learned by maximising a generalized likelihood, and the resulting energy-based loss provides informative gradients for learning the base. Samples from the posterior on the latent space of the trained model can be obtained via MCMC, thus finding regions in this space that produce better quality samples. Empirically, the GEBM samples on image-generation tasks are of much better quality than those from the learned generator alone, indicating that all else being equal, the GEBM will outperform a GAN of the same complexity. When using normalizing flows as base measures, GEBMs succeed on density modelling tasks, returning comparable performance to direct maximum likelihood of the same networks.

Type: Proceedings paper
Title: Generalized Energy Based Models
Event: ICLR 2021 - 9th International Conference on Learning Representations
Open access status: An open access version is available from UCL Discovery
Publisher version: https://openreview.net/forum?id=0PtUPB9z6qK
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery-pp.ucl.ac.uk/id/eprint/10167377
Downloads since deposit
561Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item