Measuring AI Progress in Drug Discovery: A Reproducible Leaderboard for the Tox21 Challenge

Abstract

Abstract Deep learning has fundamentally transformed fields such as computer vision and natural language processing since the early 2010s and has strongly influenced biomedical research. For drug discovery specifically, a key inflection point─akin to vision’s “ImageNet moment”─arrived in 2015, when deep neural networks surpassed traditional approaches in the Tox21 Data Challenge. This milestone accelerated the adoption of deep learning methods across the pharmaceutical industry, and today, most major companies have integrated these approaches into their research pipelines. Following the conclusion of the Tox21 Challenge, its data set was incorporated into several established benchmarks, including MoleculeNet and the Open Graph Benchmark. However, during these integrations, the data set was altered, and labels were imputed or manufactured, resulting in a loss of comparability across studies. Consequently, the extent to which bioactivity and toxicity prediction methods have improved over the past decade remains unclear. To this end, we introduce a reproducible leaderboard, hosted on Hugging Face with the original Tox21 Challenge data set, together with a set of baseline and representative methods. The current version of the leaderboard indicates that the original Tox21 winner from 2015─the ensemble-based DeepTox method─and the descriptor-based self-normalizing neural networks introduced in 2017 continue to perform competitively and rank among the top methods for toxicity prediction, raising questions about the extent of progress achieved over the past decade. As part of this work, we provide public access to all baselines and evaluated models for inference via standardized API end points on Hugging Face Spaces.