CSTRL: Context-Driven Sequential Transfer Learning for Abstractive Radiology Report Summarization
Journal:
arXiv
Published Date:
Feb 21, 2025
Abstract
A radiology report comprises several sections, including the Findings and
Impression of the diagnosis. Automatically generating the Impression from the
Findings is crucial for reducing radiologists' workload and improving
diagnostic accuracy. Pretrained models that excel in common abstractive
summarization problems encounter challenges when applied to specialized medical
domains largely due to the complex terminology and the necessity for accurate
clinical context. Such tasks in medical domains demand extracting core
information, avoiding context shifts, and maintaining proper flow. Misuse of
medical terms can lead to drastic clinical errors. To address these issues, we
introduce a sequential transfer learning that ensures key content extraction
and coherent summarization. Sequential transfer learning often faces challenges
like initial parameter decay and knowledge loss, which we resolve with the
Fisher matrix regularization. Using MIMIC-CXR and Open-I datasets, our model,
CSTRL-Context-driven Sequential TRansfer Learning-achieved state-of-the-art
performance, showing 56.2% improvement in BLEU-1, 40.5% in BLEU-2, 84.3% in
BLEU-3, 28.9% in ROUGE-1, 41.0% in ROUGE-2 and 26.5% in ROGUE-3 score over
benchmark studies. We also analyze factual consistency scores while preserving
the medical context. Our code is publicly available at TBA.
