Nelson, Amy PK;
Gray, Robert J;
Ruffle, James K;
Watkins, Henry C;
Herron, Daniel;
Sorros, Nick;
Mikhailov, Danil;
... Nachev, Parashkev; + view all
(2022)
Deep forecasting of translational impact in medical research.
Patterns
, Article 100483. 10.1016/j.patter.2022.100483.
(In press).
Preview |
Text
2110.08904v1.pdf - Other Download (20MB) | Preview |
Abstract
The value of biomedical research--a $1.7 trillion annual investment--is ultimately determined by its downstream, real-world impact. Current objective predictors of impact rest on proxy, reductive metrics of dissemination, such as paper citation rates, whose relation to real-world translation remains unquantified. Here we sought to determine the comparative predictability of future real-world translation--as indexed by inclusion in patents, guidelines or policy documents--from complex models of the abstract-level content of biomedical publications versus citations and publication meta-data alone. We develop a suite of representational and discriminative mathematical models of multi-scale publication data, quantifying predictive performance out-of-sample, ahead-of-time, across major biomedical domains, using the entire corpus of biomedical research captured by Microsoft Academic Graph from 1990 to 2019, encompassing 43.3 million papers across all domains. We show that citations are only moderately predictive of translational impact as judged by inclusion in patents, guidelines, or policy documents. By contrast, high-dimensional models of publication titles, abstracts and metadata exhibit high fidelity (AUROC > 0.9), generalise across time and thematic domain, and transfer to the task of recognising papers of Nobel Laureates. The translational impact of a paper indexed by inclusion in patents, guidelines, or policy documents can be predicted--out-of-sample and ahead-of-time--with substantially higher fidelity from complex models of its abstract-level content than from models of publication meta-data or citation metrics. We argue that content-based models of impact are superior in performance to conventional, citation-based measures, and sustain a stronger evidence-based claim to the objective measurement of translational potential.
| Type: | Article |
|---|---|
| Title: | Deep forecasting of translational impact in medical research |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.patter.2022.100483 |
| Publisher version: | https://doi.org/10.1016/j.patter.2022.100483 |
| Language: | English |
| Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
| Keywords: | deep learning, representation learning, natural language processing, research impact, translational research |
| UCL classification: | UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Brain Repair and Rehabilitation UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10146602 |
Archive Staff Only
 |
View Item |
