Goo, JM;
Zeng, Z;
Boehm, J;
(2024)
Zero-Shot Detection of Buildings in Mobile LiDAR using Language Vision Model.
In:
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives.
(pp. pp. 107-113).
Copernicus Publications
Preview |
PDF
ISPRS_GOO.pdf - Published Version Download (10MB) | Preview |
Abstract
Recent advances have demonstrated that Language Vision Models (LVMs) surpass the existing State-of-the-Art (SOTA) in two-dimensional (2D) computer vision tasks, motivating attempts to apply LVMs to three-dimensional (3D) data. While LVMs are efficient and effective in addressing various downstream 2D vision tasks without training, they face significant challenges when it comes to point clouds, a representative format for representing 3D data. It is more difficult to extract features from 3D data and there are challenges due to large data sizes and the cost of the collection and labelling, resulting in a notably limited availability of datasets. Moreover, constructing LVMs for point clouds is even more challenging due to the requirements for large amounts of data and training time. To address these issues, our research aims to 1) apply the Grounded SAM through Spherical Projection to transfer 3D to 2D, and 2) experiment with synthetic data to evaluate its effectiveness in bridging the gap between synthetic and real-world data domains. Our approach exhibited high performance with an accuracy of 0.96, an IoU of 0.85, precision of 0.92, recall of 0.91, and an F1 score of 0.92, confirming its potential. However, challenges such as occlusion problems and pixel-level overlaps of multi-label points during spherical image generation remain to be addressed in future studies.
| Type: | Proceedings paper |
|---|---|
| Title: | Zero-Shot Detection of Buildings in Mobile LiDAR using Language Vision Model |
| Event: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.5194/isprs-archives-XLVIII-2-2024-107-2024 |
| Publisher version: | http://dx.doi.org/10.5194/isprs-archives-xlviii-2-... |
| Language: | English |
| Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
| Keywords: | LiDAR, Point Cloud, Building, Scene Understanding, Deep Learning, Foundation Model, Multi-Modal Model |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Civil, Environ and Geomatic Eng UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment > Bartlett School Env, Energy and Resources |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10195053 |
Archive Staff Only
 |
View Item |
