Islam, M;
Li, Z;
Glocker, B;
(2023)
Robustness Stress Testing in Medical Image Classification.
In: Sudre, CH and Baumgartner, CF and Dalca, A and Mehta, R and Qin, C and Wells, WM, (eds.)
Uncertainty for Safe Utilization of Machine Learning in Medical Imaging: 5th International Workshop, UNSURE 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 12, 2023, Proceedings.
(pp. pp. 167-176).
Springer: Cham, Switzerland.
Preview |
Text
2308.06889.pdf - Accepted Version Download (2MB) | Preview |
Abstract
Deep neural networks have shown impressive performance for image-based disease detection. Performance is commonly evaluated through clinical validation on independent test sets to demonstrate clinically acceptable accuracy. Reporting good performance metrics on test sets, however, is not always a sufficient indication of the generalizability and robustness of an algorithm. In particular, when the test data is drawn from the same distribution as the training data, the iid test set performance can be an unreliable estimate of the accuracy on new data. In this paper, we employ stress testing to assess model robustness and subgroup performance disparities in disease detection models. We design progressive stress testing using five different bidirectional and unidirectional image perturbations with six different severity levels. As a use case, we apply stress tests to measure the robustness of disease detection models for chest X-ray and skin lesion images, and demonstrate the importance of studying class and domain-specific model behaviour. Our experiments indicate that some models may yield more robust and equitable performance than others. We also find that pretraining characteristics play an important role in downstream robustness. We conclude that progressive stress testing is a viable and important tool and should become standard practice in the clinical validation of image-based disease detection models.
| Type: | Proceedings paper |
|---|---|
| Title: | Robustness Stress Testing in Medical Image Classification |
| Event: | 5th International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging (UNSURE) |
| Location: | CANADA, Vancouver |
| Dates: | 12 Oct 2023 |
| ISBN-13: | 978-3-031-44335-0 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/978-3-031-44336-7_17 |
| Publisher version: | https://doi.org/10.1007/978-3-031-44336-7_17 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Med Phys and Biomedical Eng |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10187095 |
Archive Staff Only
 |
View Item |
