Reliable Detection, Unreliable Clinical Decision-making: Artificial Intelligence Models in Burn Assessment.

BACKGROUND: Multimodal artificial intelligence (AI) systems are increasingly used for visual interpretation tasks. Whether commercially available models can be applied to burn surgery, and to what extent, remains unclear. We evaluated a clinical chain from visual recognition (distinguishing burns from nonburned skin) to treatment decision-making (debridement versus immediate skin grafting), including accuracy, output consistency, and skin tone subgroup performance. METHODS: Four multimodal large language models (GPT-5.4 Pro, Grok 4.1, Gemini 3.1 Pro, and Claude Opus 4.6) evaluated 50 cropped clinical photographs (25 burns and 25 nonburns) across 5 independent runs (n = 4000 total assessments). Models classified burn presence and treatment recommendation. Ground truth was established by expert consensus among plastic surgeons. Reproducibility was assessed using Fleiss κ. RESULTS: Burn detection was near-perfect (97.2%-100%, κ = 0.896-1.000) and equivalent across Fitzpatrick skin tone strata. Treatment accuracy in burn cases ranged from 24.8% to 72.8%, with 3 models performing below the majority-class baseline. Reproducibility for treatment decisions dropped to κ = 0.132-0.385 in 3 models, with inconsistent recommendations for identical images. Error patterns were systematic and opposing across models. CONCLUSIONS: AI systems reliably detected burns but failed to reliably recommend treatment. Reproducibility remained high for visual recognition but substantially lower for clinical decision-making. The clinical chain breaks at the transition from perception to decision, limiting the current applicability in burn surgery. The high sensitivity (94.4%-100%) and specificity (100%) for burn detection suggest that visual recognition may serve as a reliable foundation for future AI-assisted triage in burn care.