MedChat: A Multi-Agent Framework for Multimodal Diagnosis with Large Language Models
Journal:
arXiv
Published Date:
Jun 9, 2025
Abstract
The integration of deep learning-based glaucoma detection with large language
models (LLMs) presents an automated strategy to mitigate ophthalmologist
shortages and improve clinical reporting efficiency. However, applying general
LLMs to medical imaging remains challenging due to hallucinations, limited
interpretability, and insufficient domain-specific medical knowledge, which can
potentially reduce clinical accuracy. Although recent approaches combining
imaging models with LLM reasoning have improved reporting, they typically rely
on a single generalist agent, restricting their capacity to emulate the diverse
and complex reasoning found in multidisciplinary medical teams. To address
these limitations, we propose MedChat, a multi-agent diagnostic framework and
platform that combines specialized vision models with multiple role-specific
LLM agents, all coordinated by a director agent. This design enhances
reliability, reduces hallucination risk, and enables interactive diagnostic
reporting through an interface tailored for clinical review and educational
use. Code available at https://github.com/Purdue-M2/MedChat.
