Exploring AI-based System Design for Pixel-level Protected Health Information Detection in Medical Images
Journal:
arXiv
Published Date:
Jan 16, 2025
Abstract
De-identification of medical images is a critical step to ensure privacy
during data sharing in research and clinical settings. The initial step in this
process involves detecting Protected Health Information (PHI), which can be
found in image metadata or imprinted within image pixels. Despite the
importance of such systems, there has been limited evaluation of existing
AI-based solutions, creating barriers to the development of reliable and robust
tools. In this study, we present an AI-based pipeline for PHI detection,
comprising three key components: text detection, text extraction, and text
analysis. We benchmark three models, YOLOv11, EasyOCR, and GPT-4o, across
different setups corresponding to these components, evaluating the performance
based on precision, recall, F1 score, and accuracy. All setups demonstrate
excellent PHI detection, with all metrics exceeding 0.9. The combination of
YOLOv11 for text localization and GPT-4o for extraction and analysis yields the
best results. However, this setup incurs higher costs due to GPT-4o's token
generation. Conversely, an end-to-end pipeline that relies solely on GPT-4o
shows lower performance but highlights the potential of multimodal models for
complex tasks. We recommend fine-tuning a dedicated object detection model and
utilizing built-in OCR tools to achieve optimal performance and
cost-effectiveness. Additionally, leveraging language models such as GPT-4o can
facilitate thorough and flexible analysis of text content.
