Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions
Journal:
arXiv
Published Date:
Feb 26, 2025
Abstract
Millions of melanocytic skin lesions are examined by pathologists each year,
the majority of which concern common nevi (i.e., ordinary moles). While most of
these lesions can be diagnosed in seconds, writing the corresponding pathology
report is much more time-consuming. Automating part of the report writing
could, therefore, alleviate the increasing workload of pathologists. In this
work, we develop a vision-language model specifically for the pathology domain
of cutaneous melanocytic lesions. The model follows the Contrastive Captioner
framework and was trained and evaluated using a melanocytic lesion dataset of
42,512 H&E-stained whole slide images and 19,645 corresponding pathology
reports. Our results show that the quality scores of model-generated reports
were on par with pathologist-written reports for common nevi, assessed by an
expert pathologist in a reader study. While report generation revealed to be
more difficult for rare melanocytic lesion subtypes, the cross-modal retrieval
performance for these cases was considerably better.
