PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis
Journal:
arXiv
Published Date:
Aug 20, 2024
Abstract
We present a comprehensive framework for predicting the effects of
perturbations in single cells, designed to standardize benchmarking in this
rapidly evolving field. Our framework, PerturBench, includes a user-friendly
platform, diverse datasets, metrics for fair model comparison, and detailed
performance analysis. Extensive evaluations of published and baseline models
reveal limitations like mode or posterior collapse, and underscore the
importance of rank metrics that assess the ordering of perturbations alongside
traditional measures like RMSE. Our findings show that simple models can
outperform more complex approaches. This benchmarking exercise sets new
standards for model evaluation, supports robust model development, and advances
the potential of these models to use high-throughput and high-content genetic
and chemical screens for disease target discovery.
