Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics
Journal:
arXiv
Published Date:
Jun 27, 2025
Abstract
Recent benchmarks reveal that models for single-cell perturbation response
are often outperformed by simply predicting the dataset mean. We trace this
anomaly to a metric artifact: control-referenced deltas and unweighted error
metrics reward mode collapse whenever the control is biased or the biological
signal is sparse. Large-scale \textit{in silico} simulations and analysis of
two real-world perturbation datasets confirm that shared reference shifts, not
genuine biological change, drives high performance in these evaluations. We
introduce differentially expressed gene (DEG)-aware metrics, weighted
mean-squared error (WMSE) and weighted delta $R^{2}$ ($R^{2}_{w}(\Delta)$) with
respect to all perturbations, that measure error in niche signals with high
sensitivity. We further introduce negative and positive performance baselines
to calibrate these metrics. With these improvements, the mean baseline sinks to
null performance while genuine predictors are correctly rewarded. Finally, we
show that using WMSE as a loss function reduces mode collapse and improves
model performance.
