eprintid: 10057489
rev_number: 26
eprint_status: archive
userid: 608
dir: disk0/10/05/74/89
datestamp: 2018-10-08 16:32:54
lastmod: 2021-09-28 22:29:07
status_changed: 2018-10-08 16:32:54
type: article
metadata_visibility: show
creators_name: Hedman, P
creators_name: Alsisan, S
creators_name: Szeliski, R
creators_name: Kopf, J
title: Casual 3D photography
ispublished: pub
divisions: UCL
divisions: B04
divisions: C05
keywords: Computing methodologies, Image-based rendering;
Computational photography; 3D Reconstruction, Virtual Reality
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: We present an algorithm that enables casual 3D photography. Given a set of input photos captured with a hand-held cell phone or DSLR camera, our algorithm reconstructs a 3D photo, a central panoramic, textured, normal mapped, multi-layered geometric mesh representation. 3D photos can be stored compactly and are optimized for being rendered from viewpoints that are near the capture viewpoints. They can be rendered using a standard rasterization pipeline to produce perspective views with motion parallax. When viewed in VR, 3D photos provide geometrically consistent views for both eyes. Our geometric representation also allows interacting with the scene using 3D geometry-aware effects, such as adding new objects to the scene and artistic lighting effects.

Our 3D photo reconstruction algorithm starts with a standard structure from motion and multi-view stereo reconstruction of the scene. The dense stereo reconstruction is made robust to the imperfect capture conditions using a novel near envelope cost volume prior that discards erroneous near depth hypotheses. We propose a novel parallax-tolerant stitching algorithm that warps the depth maps into the central panorama and stitches two color-and-depth panoramas for the front and back scene surfaces. The two panoramas are fused into a single non-redundant, well-connected geometric mesh. We provide videos demonstrating users interactively viewing and manipulating our 3D photos.
date: 2017-11
date_type: published
official_url: https://doi.org/10.1145/3130800.3130828
oa_status: green
full_text_type: other
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 1589418
doi: 10.1145/3130800.3130828
lyricists_name: Hedman, Lars
lyricists_id: LPJHE82
actors_name: Hedman, Lars
actors_id: LPJHE82
actors_role: owner
full_text_status: public
publication: ACM Transactions on Graphics
volume: 36
number: 6
article_number: 234
issn: 0730-0301
citation:        Hedman, P;    Alsisan, S;    Szeliski, R;    Kopf, J;      (2017)    Casual 3D photography.                   ACM Transactions on Graphics , 36  (6)    , Article 234.  10.1145/3130800.3130828 <https://doi.org/10.1145/3130800.3130828>.       Green open access   
 
document_url: https://discovery-pp.ucl.ac.uk/id/eprint/10057489/1/casual3d_siggasia_2017.pdf