Alshahwan, Nadia;
Chheda, Jubin;
Finogenova, Anastasia;
Gokkaya, Beliz;
Harman, Mark;
Harper, Inna;
Marginean, Alexandru;
... Wang, Eddy; + view all
(2024)
Automated Unit Test Improvement using Large Language Models at Meta.
In: D'Amorim, M, (ed.)
Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering.
(pp. pp. 185-196).
ACM
Preview |
Text
RPS3.pdf - Accepted Version Download (667kB) | Preview |
Abstract
This paper describes Meta’s TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. We describe the deployment of TestGen-LLM at Meta test-a-thons for the Instagram and Facebook platforms. In an evaluation on Reels and Stories products for Instagram, 75% of TestGen-LLM’s test cases built correctly, 57% passed reliably, and 25% increased coverage. During Meta’s Instagram and Facebook test-a-thons, it improved 11.5% of all classes to which it was applied, with 73% of its recommendations being accepted for production deployment by Meta software engineers. We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement.
| Type: | Proceedings paper |
|---|---|
| Title: | Automated Unit Test Improvement using Large Language Models at Meta |
| Event: | FSE '24: 32nd ACM International Conference on the Foundations of Software Engineering |
| Location: | BRAZIL, Porto de Galinhas |
| Dates: | 15 Jul 2024 - 19 Jul 2024 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1145/3663529.3663839 |
| Publisher version: | https://doi.org/10.1145/3663529.3663839 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Unit Testing, Automated Test Generation, Large Language Models, LLMs, Genetic Improvement |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10199775 |
Archive Staff Only
 |
View Item |
