Virtual screening

Redundancy (engineering)

Pattern recognition (psychology)

Overfitting

Mathematics

Bioinformatics

Biology

Operating system

Image (mathematics)

Similarity (geometry)

Data mining

Machine learning

Mathematical analysis

Artificial intelligence

Generalization

Similarity measure

Measure (data warehouse)

Fingerprint (computing)

Computer science

Drug discovery

Artificial neural network

Training set

Classifier (UML)

Journal of Chemical Information and Modeling

Undetected overfitting can occur when there are significant redundancies between training and validation data. We describe AVE, a new measure of training-validation redundancy for ligand-based classification problems that accounts for the similarity amongst inactive molecules as well as active. We investigated seven widely-used benchmarks for virtual screening and classification, and show that the amount of AVE bias strongly correlates with the performance of ligand-based predictive methods irrespective of the predicted property, chemical fingerprint, similarity measure, or previously-applied unbiasing techniques. Therefore, it may be that the previously-reported performance of most ligand-based methods can be explained by overfitting to benchmarks rather than good prospective accuracy.

Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization