Reinforcement Learning for Chronic Care Pathway Optimization: A Unified Framework across Three Clinical Goal Types
Journal:
medRxiv
Published Date:
Jul 6, 2026
Abstract
Objective: Chronic care requires sequential treatment under competing biomarker, safety, and cost constraints, yet clinical goal structures differ across diseases. We asked whether one physiology-informed reinforcement learning (RL) paradigm adapts to heterogeneous chronic-care goals without disease-specific policy architectures. Materials and Methods: We formalized a Type A/B/C clinical goal taxonomy (target cure, stable cruise, cycle completion) as a Physiology-Informed Markov Decision Process registry for gout, chronic kidney disease (CKD), and PCOS-mediated fertility treatment--each with PK/PD transitions, discrete actions, safety zones, and guideline doctor baselines. Unified BC->PPO training (GAE lambda=0.95) on 500 simulated trajectories per disease. Evaluation: paired seeds (N=50 primary; N=500 bootstrap 95% CIs), 10-seed robustness, ablation, literature sUA calibration, and out-of-distribution stress. McNemar/Wilcoxon with Benjamini-Hochberg FDR. Results: PCOS (Type C, primary): PPO 72.0% vs. doctor 54.0% at N=50 (+18 percentage points; FDR-significant); at N=500, PPO 69.8% [65.6, 73.8] vs. doctor 52.8% [48.8, 57.2]. Gout (Type A): PPO non-inferior--88.0% vs. 90.0% (McNemar p=1.0). CKD (Type B): doctor 32.0%, BC/PPO 38.0%. Offline CQL 92.0% on gout trajectories. PK recalibration RMSE 97.4 umol/L (r=0.809). Conclusions: Shared BC->PPO training generalizes across three goal types without cross-disease weight sharing. PCOS supports RL for bounded cycles; gout confirms guideline non-inferiority; CKD illustrates cruise-control difficulty. This framework offers a reproducible foundation for chronic pathway optimization pending prospective validation.
