ELM: Ensemble of Language Models for Predicting Tumor Group from Pathology Reports
Journal:
arXiv
Published Date:
Mar 24, 2025
Abstract
Population-based cancer registries (PBCRs) face a significant bottleneck in
manually extracting data from unstructured pathology reports, a process crucial
for tasks like tumor group assignment, which can consume 900 person-hours for
approximately 100,000 reports. To address this, we introduce ELM (Ensemble of
Language Models), a novel ensemble-based approach leveraging both small
language models (SLMs) and large language models (LLMs). ELM utilizes six
fine-tuned SLMs, where three SLMs use the top part of the pathology report and
three SLMs use the bottom part. This is done to maximize report coverage. ELM
requires five-out-of-six agreement for a tumor group classification.
Disagreements are arbitrated by an LLM with a carefully curated prompt. Our
evaluation across nineteen tumor groups demonstrates ELM achieves an average
precision and recall of 0.94, outperforming single-model and
ensemble-without-LLM approaches. Deployed at the British Columbia Cancer
Registry, ELM demonstrates how LLMs can be successfully applied in a PBCR
setting to achieve state-of-the-art results and significantly enhance
operational efficiencies, saving hundreds of person-hours annually.
