Karnewar, Animesh;
Wang, Oliver;
Ritschel, Tobias;
Mitra, Niloy;
(2023)
3INGAN: Learning a 3D Generative Model from Images of a Self-similar Scene.
In:
Proceedings - 2022 International Conference on 3D Vision, 3DV 2022.
(pp. pp. 342-352).
IEEE: Prague, Czech Republic.
Preview |
Text
2211.14902.pdf - Accepted Version Download (4MB) | Preview |
Abstract
We introduce 3 IN GAN, an unconditional 3D generative model trained from 2D images of a single self-similar 3D scene. Such a model can be used to produce 3D “remixes” of a given scene, by mapping spatial latent codes into a 3D volumetric representation, which can subsequently be rendered from arbitrary views using physically based volume rendering. By construction, the generated scenes remain view-consistent across arbitrary camera configurations, without any flickering or spatio-temporal artifacts. During training, we employ a combination of 2D, obtained through differentiable volume tracing, and 3D Generative Adversarial Network (GAN) losses, across multiple scales, enforcing realism on both its 2D renderings and its 3D structure. We show results on semi-stochastic scenes of varying scale and complexity, obtained from real and synthetic sources. We demonstrate, for the first time, the feasibility of learning plausible view-consistent 3D scene variations from a single exemplar scene and provide qualitative and quantitative comparisons against two recent related methods. Code and data for the paper are available at https://geometry.cs.ucl.ac.uk/group_website/projects/2022/3inGAN/.
| Type: | Proceedings paper |
|---|---|
| Title: | 3INGAN: Learning a 3D Generative Model from Images of a Self-similar Scene |
| Event: | 2022 International Conference on 3D Vision (3DV) |
| Dates: | 12 Sep 2022 - 16 Sep 2022 |
| ISBN-13: | 9781665456708 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/3DV57658.2022.00046 |
| Publisher version: | https://doi.org/10.1109/3DV57658.2022.00046 |
| Language: | English |
| Additional information: | This version is the author-accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | 3D GAN, 3INGAN, differentiable rendering, single 3D scene GAN. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10167430 |
Archive Staff Only
 |
View Item |
