MedRegion-CT: Region-Focused Multimodal LLM for Comprehensive 3D CT Report Generation
Journal:
arXiv
Published Date:
Jun 29, 2025
Abstract
The recent release of RadGenome-Chest CT has significantly advanced CT-based
report generation. However, existing methods primarily focus on global
features, making it challenging to capture region-specific details, which may
cause certain abnormalities to go unnoticed. To address this, we propose
MedRegion-CT, a region-focused Multi-Modal Large Language Model (MLLM)
framework, featuring three key innovations. First, we introduce Region
Representative ($R^2$) Token Pooling, which utilizes a 2D-wise pretrained
vision model to efficiently extract 3D CT features. This approach generates
global tokens representing overall slice features and region tokens
highlighting target areas, enabling the MLLM to process comprehensive
information effectively. Second, a universal segmentation model generates
pseudo-masks, which are then processed by a mask encoder to extract
region-centric features. This allows the MLLM to focus on clinically relevant
regions, using six predefined region masks. Third, we leverage segmentation
results to extract patient-specific attributions, including organ size,
diameter, and locations. These are converted into text prompts, enriching the
MLLM's understanding of patient-specific contexts. To ensure rigorous
evaluation, we conducted benchmark experiments on report generation using the
RadGenome-Chest CT. MedRegion-CT achieved state-of-the-art performance,
outperforming existing methods in natural language generation quality and
clinical relevance while maintaining interpretability. The code for our
framework is publicly available.
