Ghoshal, Biraja;
Woof, William;
Mendes, Bernardo;
Al-Khuzaei, Saoud;
Guimaraes, Thales Antonio Cabral De;
Varela, Malena Daich;
Liu, Yichen;
... Pontikos, Nikolas; + view all
(2024)
Making Deep Learning Models Clinically Useful - Improving Diagnostic Confidence in Inherited Retinal Disease with Conformal Prediction.
In: Sudre, Carole H and Mehta, Raghav and Ouyang, Cheng and Qin, Chen and Rakic, Marianne and III, William M Wells, (eds.)
Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2024.
(pp. pp. 47-58).
Springer: Cham, Switzerland.
Preview |
Text
PaperID7_MICCAI2024_UNSURE2024.pdf - Accepted Version Download (877kB) | Preview |
Abstract
Deep Learning (DL), which involves powerful “black box” predictors, has achieved state-of-the-art performance in medical image analysis. However, these methods lack transparency and interpretability of point predictions without assessing the quality of their outputs. Knowing how much confidence there is in a prediction is essential for gaining clinicians’ trust in the technology and its use in medical decision-making. In this paper, we explore the use of Conformal Prediction (CP) methods to recommend statistically rigorous reliable prediction sets to a clinician, using multi-modal imaging for the genetic diagnosis of the 36 most common molecular causes of inherited retinal diseases (IRDs). These are monogenic conditions that represent a leading cause of blindness in children and working-age adults and require a costly and time-consuming genetic test for diagnosis. Three methods of CP were assessed: Least Ambiguous Adaptive Prediction Sets (LAPS), Adaptative Prediction Sets (APS), and Regularized Adaptive Prediction Sets (RAPS). Our IRD classifier (Eye2Gene), in combination with the three conformal predictors, was evaluated on an internal holdout subset and datasets from four external clinical centres. RAPS proved to be the best-performing method with single-digit set sizes and coverage above 90% at a confidence level of 80%. Implementing adaptive CP methods has the potential to reduce waiting time and costs of genetic diagnosis of IRDs by improving upon the current gene prioritisation systems, while simultaneously enabling safety-critical clinical environments by flagging clinicians for a second opinion.
| Type: | Proceedings paper |
|---|---|
| Title: | Making Deep Learning Models Clinically Useful - Improving Diagnostic Confidence in Inherited Retinal Disease with Conformal Prediction |
| Event: | Uncertainty for Safe Utilization of Machine Learning in Medical Imaging (UNSURE 2024) |
| ISBN-13: | 978-3-031-73157-0 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/978-3-031-73158-7_5 |
| Publisher version: | https://doi.org/10.1007/978-3-031-73158-7_5 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Deep Learning, Uncertainty, Conformal Prediction, IRD |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Institute of Ophthalmology |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10199286 |
Archive Staff Only
 |
View Item |
