Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction
Journal:
arXiv
Published Date:
Dec 18, 2024
Abstract
Predicting transcriptional responses to novel drugs provides a unique
opportunity to accelerate biomedical research and advance drug discovery
efforts. However, the inherent complexity and high dimensionality of cellular
responses, combined with the extremely limited available experimental data,
makes the task challenging. In this study, we leverage single-cell foundation
models (FMs) pre-trained on tens of millions of single cells, encompassing
multiple cell types, states, and disease annotations, to address molecular
perturbation prediction. We introduce a drug-conditional adapter that allows
efficient fine-tuning by training less than 1% of the original foundation
model, thus enabling molecular conditioning while preserving the rich
biological representation learned during pre-training. The proposed strategy
allows not only the prediction of cellular responses to novel drugs, but also
the zero-shot generalization to unseen cell lines. We establish a robust
evaluation framework to assess model performance across different
generalization tasks, demonstrating state-of-the-art results across all
settings, with significant improvements in the few-shot and zero-shot
generalization to new cell lines compared to existing baselines.
