Kayser, Maxime Guillaume;
Menzat, Bayar;
Emde, Cornelius;
Bercean, Bogdan Alexandru;
Novak, Alex;
Morgado, Abdalá Trinidad Espinosa;
Papiez, Bartlomiej;
... Camburu, Oana-Maria; + view all
(2024)
Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting.
In:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
(pp. pp. 18891-18919).
Association for Computational Linguistics: Miami, FL, USA.
Preview |
Text
2024.emnlp-main.1051.pdf - Published Version Download (6MB) | Preview |
Abstract
The growing capabilities of AI models are leading to their wider use, including in safetycritical domains. Explainable AI (XAI) aims to make these models safer to use by making their inference process more transparent. However, current explainability methods are seldom evaluated in the way they are intended to be used: by real-world end users. To address this, we conducted a large-scale user study with 85 healthcare practitioners in the context of human-AI collaborative chest X-ray analysis. We evaluated three types of explanations: visual explanations (saliency maps), natural language explanations, and a combination of both modalities. We specifically examined how different explanation types influence users depending on whether the AI advice and explanations are factually correct. We find that text-based explanations lead to significant over-reliance, which is alleviated by combining them with saliency maps. We also observe that the quality of explanations, that is, how much factually correct information they entail, and how much this aligns with AI correctness, significantly impacts the usefulness of the different explanation types.
| Type: | Proceedings paper |
|---|---|
| Title: | Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting |
| Event: | Conference on Empirical Methods in Natural Language Processing 2024 |
| Dates: | Nov 2024 - Nov 2024 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.18653/v1/2024.emnlp-main.1051 |
| Publisher version: | https://doi.org/10.18653/v1/2024.emnlp-main.1051 |
| Language: | English |
| Additional information: | ACL materials are Copyright © 1963–2024 ACL. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10201412 |
Archive Staff Only
 |
View Item |
