UCL Discovery Stage
UCL home » Library Services » Electronic resources » UCL Discovery Stage

Cost-sensitive Boosting Algorithms: Do We Really Need Them

Nikolaou, N; Edakunni, N; Kull, M; Flach, P; Brown, G; (2016) Cost-sensitive Boosting Algorithms: Do We Really Need Them. Machine Learning , 104 (2-3) pp. 359-384. 10.1007/s10994-016-5572-x. Green open access

[thumbnail of Nikolaou2016_Article_Cost-sensitiveBoostingAlgorith.pdf]
Preview
Text
Nikolaou2016_Article_Cost-sensitiveBoostingAlgorith.pdf - Published Version

Download (1MB) | Preview

Abstract

We provide a unifying perspective for two decades of work on cost-sensitive Boosting algorithms. When analyzing the literature 1997–2016, we find 15 distinct cost-sensitive variants of the original algorithm; each of these has its own motivation and claims to superiority—so who should we believe? In this work we critique the Boosting literature using four theoretical frameworks: Bayesian decision theory, the functional gradient descent view, margin theory, and probabilistic modelling. Our finding is that only three algorithms are fully supported—and the probabilistic model view suggests that all require their outputs to be calibrated for best performance. Experiments on 18 datasets across 21 degrees of imbalance support the hypothesis—showing that once calibrated, they perform equivalently, and outperform all others. Our final recommendation—based on simplicity, flexibility and performance—is to use the original Adaboost algorithm with a shifted decision threshold and calibrated probability estimates.

Type: Article
Title: Cost-sensitive Boosting Algorithms: Do We Really Need Them
Location: Riva del Garda, ITALY
Open access status: An open access version is available from UCL Discovery
DOI: 10.1007/s10994-016-5572-x
Publisher version: https://doi.org/10.1007/s10994-016-5572-x
Language: English
Additional information: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Keywords: Boosting, Cost-sensitive, Class imbalance, Classifier calibration
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Physics and Astronomy
URI: https://discovery-pp.ucl.ac.uk/id/eprint/10059522
Downloads since deposit
2,808Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item