UCL Discovery Stage
UCL home » Library Services » Electronic resources » UCL Discovery Stage

Agile Effort Estimation: Have We Solved the Problem Yet? Insights from the Replication of the GPT2SP Study

Tawosi, V; Moussa, R; Sarro, F; (2024) Agile Effort Estimation: Have We Solved the Problem Yet? Insights from the Replication of the GPT2SP Study. In: 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). (pp. pp. 1034-1041). IEEE: Rovaniemi, Finland. Green open access

[thumbnail of GPT2SP_Replication_Report.pdf]
Preview
PDF
GPT2SP_Replication_Report.pdf - Accepted Version

Download (205kB) | Preview

Abstract

Replication studies in Software Engineering are indispensable for ensuring the reliability, generalizability, and transparency of research findings. They contribute to the cumulative growth of knowledge in the field and promote a scientific approach that benefits both researchers and practitioners. In this article, we report our experience replicating a recently published work proposing a Transformer-based approach for Agile Story Point Estimation' dubbed GPT2SP. GPT2SP was proposed with the intent of addressing the three limitations of a previous Deep Learning-based approach dubbed Deep-SE, and the results reported in the original study set GPT2SP as the new state-of-the-art. However, when we used the GPT2SP source code made publicly available by the authors of the original study, we found a bug in the computation of the evaluation measure and the reuse of erroneous results from previous work, which had unintentionally introduced biases in the GPT2SP's performance evaluation. In this study, we report on the results we obtained after fixing the issues present in the original study, which reveal that their results were in fact unintentionally inflated due to these issues and that despite advancements, challenges remain in providing accurate effort estimations for agile software projects.

Type: Proceedings paper
Title: Agile Effort Estimation: Have We Solved the Problem Yet? Insights from the Replication of the GPT2SP Study
Event: 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)
Dates: 12 Mar 2024 - 15 Mar 2024
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/SANER60148.2024.00111
Publisher version: http://dx.doi.org/10.1109/saner60148.2024.00111
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Deep learning, Accuracy, Source coding, Computer bugs, Estimation, Agile software development, Transformers
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery-pp.ucl.ac.uk/id/eprint/10196741
Downloads since deposit
1,152Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item