MEDMKG: Benchmarking Medical Knowledge Exploitation with Multimodal Knowledge Graph
Journal:
arXiv
Published Date:
May 22, 2025
Abstract
Medical deep learning models depend heavily on domain-specific knowledge to
perform well on knowledge-intensive clinical tasks. Prior work has primarily
leveraged unimodal knowledge graphs, such as the Unified Medical Language
System (UMLS), to enhance model performance. However, integrating multimodal
medical knowledge graphs remains largely underexplored, mainly due to the lack
of resources linking imaging data with clinical concepts. To address this gap,
we propose MEDMKG, a Medical Multimodal Knowledge Graph that unifies visual and
textual medical information through a multi-stage construction pipeline. MEDMKG
fuses the rich multimodal data from MIMIC-CXR with the structured clinical
knowledge from UMLS, utilizing both rule-based tools and large language models
for accurate concept extraction and relationship modeling. To ensure graph
quality and compactness, we introduce Neighbor-aware Filtering (NaF), a novel
filtering algorithm tailored for multimodal knowledge graphs. We evaluate
MEDMKG across three tasks under two experimental settings, benchmarking
twenty-four baseline methods and four state-of-the-art vision-language
backbones on six datasets. Results show that MEDMKG not only improves
performance in downstream medical tasks but also offers a strong foundation for
developing adaptive and robust strategies for multimodal knowledge integration
in medical artificial intelligence.
