Vision-Based Automated Severity Rating of REM Sleep Behavior Disorder: From Heuristic Features to Foundation Models
Journal:
medRxiv
Published Date:
Jul 4, 2026
Abstract
Automated severity rating of rapid eye movement (REM) sleep behavior disorder (RBD) movements would enable multi-night monitoring to detect potentially injurious behaviors and provide objective endpoints for clinical trials. We compared a heuristic classifier using optical flow-derived features against V-JEPA2, a self-supervised video foundation model, for clip-level severity classification (3329 mild versus 284 moderate-to-severe) of in-laboratory video-polysomnography infrared recordings in 86 isolated RBD patients. V-JEPA2 with checkpoint fine-tuning and maximum optical flow-based frame sampling achieved the best performance across both evaluation conditions -- Macro F1 of 0.76 and 93% accuracy in the clip-level split, and 0.68 and 85% in the patient-level split -- outperforming heuristic and domain-specific pretrained models. Clip duration was the dominant heuristic predictor. Whole-night severity scores preserved patient-level ordering despite systematic overestimation, with V-JEPA2 achieving a mean absolute error of 25% versus 52% for the heuristic classifier. These findings establish a foundation for objective, home-deployable monitoring of RBD severity.
