Rahman, MS;
Ambler, G;
Choodari-Oskooei, B;
Omar, RZ;
(2017)
Review and evaluation of performance measures for survival prediction models in external validation settings.
BMC Medical Research Methodology
, 17
, Article 60. 10.1186/s12874-017-0336-2.
Preview |
Text
Ambler_s12874-017-0336-2.pdf - Published Version Download (2MB) | Preview |
Abstract
BACKGROUND: When developing a prediction model for survival data it is essential to validate its performance in external validation settings using appropriate performance measures. Although a number of such measures have been proposed, there is only limited guidance regarding their use in the context of model validation. This paper reviewed and evaluated a wide range of performance measures to provide some guidelines for their use in practice. METHODS: An extensive simulation study based on two clinical datasets was conducted to investigate the performance of the measures in external validation settings. Measures were selected from categories that assess the overall performance, discrimination and calibration of a survival prediction model. Some of these have been modified to allow their use with validation data, and a case study is provided to describe how these measures can be estimated in practice. The measures were evaluated with respect to their robustness to censoring and ease of interpretation. All measures are implemented, or are straightforward to implement, in statistical software. RESULTS: Most of the performance measures were reasonably robust to moderate levels of censoring. One exception was Harrell’s concordance measure which tended to increase as censoring increased. CONCLUSIONS: We recommend that Uno’s concordance measure is used to quantify concordance when there are moderate levels of censoring. Alternatively, Gönen and Heller’s measure could be considered, especially if censoring is very high, but we suggest that the prediction model is re-calibrated first. We also recommend that Royston’s D is routinely reported to assess discrimination since it has an appealing interpretation. The calibration slope is useful for both internal and external validation settings and recommended to report routinely. Our recommendation would be to use any of the predictive accuracy measures and provide the corresponding predictive accuracy curves. In addition, we recommend to investigate the characteristics of the validation data such as the level of censoring and the distribution of the prognostic index derived in the validation setting before choosing the performance measures.
| Type: | Article |
|---|---|
| Title: | Review and evaluation of performance measures for survival prediction models in external validation settings |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1186/s12874-017-0336-2 |
| Publisher version: | http://doi.org/10.1186/s12874-017-0336-2 |
| Language: | English |
| Additional information: | © The Author(s). 2017. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
| Keywords: | Science & Technology, Life Sciences & Biomedicine, Health Care Sciences & Services, Prognostic model, discrimination, calibration, Validation, Survival analysis, EXPLAINED VARIATION, REGRESSION-MODELS, ABILITY MEASURES, RISK, DISCRIMINATION, SIMULATION, ACCURACY, SURGERY |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Inst of Clinical Trials and Methodology > MRC Clinical Trials Unit at UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/1555341 |
Archive Staff Only
 |
View Item |
