Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models
Journal:
arXiv
Published Date:
Jul 11, 2025
Abstract
Current challenges in developing foundational models for volumetric imaging
data, such as magnetic resonance imaging (MRI), stem from the computational
complexity of training state-of-the-art architectures in high dimensions and
curating sufficiently large datasets of volumes. To address these challenges,
we introduce Raptor (Random Planar Tensor Reduction), a train-free method for
generating semantically rich embeddings for volumetric data. Raptor leverages a
frozen 2D foundation model, pretrained on natural images, to extract visual
tokens from individual cross-sections of medical volumes. These tokens are then
spatially compressed using random projections, significantly reducing
computational complexity while retaining semantic information. Extensive
experiments on ten diverse medical volume tasks verify the superior performance
of Raptor over state-of-the-art methods, including those pretrained exclusively
on medical volumes (+3% SuPreM, +6% MISFM, +10% Merlin, +13% VoCo, and +14%
SLIViT), while entirely bypassing the need for costly training. Our results
highlight the effectiveness and versatility of Raptor as a foundation for
advancing deep learning-based methods for medical volumes.
