Vision-Language Models for Acute Tuberculosis Diagnosis: A Multimodal Approach Combining Imaging and Clinical Data
Journal:
arXiv
Published Date:
Mar 17, 2025
Abstract
Background: This study introduces a Vision-Language Model (VLM) leveraging
SIGLIP and Gemma-3b architectures for automated acute tuberculosis (TB)
screening. By integrating chest X-ray images and clinical notes, the model aims
to enhance diagnostic accuracy and efficiency, particularly in resource-limited
settings.
Methods: The VLM combines visual data from chest X-rays with clinical context
to generate detailed, context-aware diagnostic reports. The architecture
employs SIGLIP for visual encoding and Gemma-3b for decoding, ensuring
effective representation of acute TB-specific pathologies and clinical
insights.
Results: Key acute TB pathologies, including consolidation, cavities, and
nodules, were detected with high precision (97percent) and recall (96percent).
The model demonstrated strong spatial localization capabilities and robustness
in distinguishing TB-positive cases, making it a reliable tool for acute TB
diagnosis.
Conclusion: The multimodal capability of the VLM reduces reliance on
radiologists, providing a scalable solution for acute TB screening. Future work
will focus on improving the detection of subtle pathologies and addressing
dataset biases to enhance its generalizability and application in diverse
global healthcare settings.
