OpenTox Virtual Conference 2026

Predictive QSAR models are regularly used to fill data gaps for environmental and pharmaceutical data gaps. The unbiased selection of a single best machine learning regression model from a set of competing models can be quite challenging. While models selected based on cross-validation performance do not guarantee good predictions on external data, models selected solely on external validation performance do not ascertain precise predictions for other external sets. Therefore, we propose three quantitative metrics to guide modelers in selecting the best model using modeling set information only. Three quantitative datasets of varying sizes and complexities were considered. Each dataset was randomly split into a modeling set and an independent test set. The modeling set was further split thrice to generate three sets of training and validation sets. Various machine learning models were developed and validated against the validation sets. Our proposed metrics1 were computed for each model using only training and validation set performances. The novel metric values guided the selection of the best models, which also demonstrated expected performance on the independent test set. Furthermore, successful applications of our framework on two additional benchmark datasets demonstrated its wider generalizability.
Keywords: Machine Learning, Deep Learning, QSAR, Validation, Metrics
1) Banerjee A, Roy K. J. Chem. Inf. Model. 2026, 66, 10835-10846. https://doi.org/10.1021/acs.jcim.6c01964