Learning to Diagnose Privately: DP-Powered LLMs for Radiology Report Classification
Journal:
arXiv
Published Date:
Jun 4, 2025
Abstract
Purpose: This study proposes a framework for fine-tuning large language
models (LLMs) with differential privacy (DP) to perform multi-abnormality
classification on radiology report text. By injecting calibrated noise during
fine-tuning, the framework seeks to mitigate the privacy risks associated with
sensitive patient data and protect against data leakage while maintaining
classification performance. Materials and Methods: We used 50,232 radiology
reports from the publicly available MIMIC-CXR chest radiography and CT-RATE
computed tomography datasets, collected between 2011 and 2019. Fine-tuning of
LLMs was conducted to classify 14 labels from MIMIC-CXR dataset, and 18 labels
from CT-RATE dataset using Differentially Private Low-Rank Adaptation (DP-LoRA)
in high and moderate privacy regimes (across a range of privacy budgets =
{0.01, 0.1, 1.0, 10.0}). Model performance was evaluated using weighted F1
score across three model architectures: BERT-medium, BERT-small, and
ALBERT-base. Statistical analyses compared model performance across different
privacy levels to quantify the privacy-utility trade-off. Results: We observe a
clear privacy-utility trade-off through our experiments on 2 different datasets
and 3 different models. Under moderate privacy guarantees the DP fine-tuned
models achieved comparable weighted F1 scores of 0.88 on MIMIC-CXR and 0.59 on
CT-RATE, compared to non-private LoRA baselines of 0.90 and 0.78, respectively.
Conclusion: Differentially private fine-tuning using LoRA enables effective and
privacy-preserving multi-abnormality classification from radiology reports,
addressing a key challenge in fine-tuning LLMs on sensitive medical data.
