Graph

PageRank

Imperfect

Mathematics

Operating system

Data mining

Statistics

Epistemology

World Wide Web

Machine learning

Philosophy

Artificial intelligence

Information retrieval

Data science

Quality (philosophy)

Citation

Warning system

Delphi

Computer science

Linguistics

Centrality

Delphi method

Theoretical computer science

Telecommunications

Nature Biotechnology

The scientific ecosystem relies on citation-based metrics that provide only imperfect, inconsistent and easily manipulated measures of research quality. Here we describe DELPHI (Dynamic Early-warning by Learning to Predict High Impact), a framework that provides an early-warning signal for ‘impactful’ research by autonomously learning high-dimensional relationships among features calculated across time from the scientific literature. We prototype this framework and deduce its performance and scaling properties on time-structured publication graphs from 1980 to 2019 drawn from 42 biotechnology-related journals, including over 7.8 million individual nodes, 201 million relationships and 3.8 billion calculated metrics. We demonstrate the framework’s performance by correctly identifying 19/20 seminal biotechnologies from 1980 to 2014 via a blinded retrospective study and provide 50 research papers from 2018 that DELPHI predicts will be in the top 5% of time-rescaled node centrality in the future. We propose DELPHI as a tool to aid in the construction of diversified, impact-optimized funding portfolios. Biotechnology-related papers predicted to be of long-term impact are identified in a machine learning framework (DELPHI) that analyzes relationships among a range of features from the scientific literature over time.

Learning on knowledge graph dynamics provides an early warning of impactful research