Multi-Agent Deep Reinforcement Learning for Multiple Anesthetics Collaborative Control
Journal:
arXiv
Published Date:
Apr 7, 2025
Abstract
Automated control of personalized multiple anesthetics in clinical Total
Intravenous Anesthesia (TIVA) is crucial yet challenging. Current systems,
including target-controlled infusion (TCI) and closed-loop systems, either rely
on relatively static pharmacokinetic/pharmacodynamic (PK/PD) models or focus on
single anesthetic control, limiting personalization and collaborative control.
To address these issues, we propose a novel framework, Value Decomposition
Multi-Agent Deep Reinforcement Learning (VD-MADRL). VD-MADRL optimizes the
collaboration between two anesthetics propofol (Agent I) and remifentanil
(Agent II). And It uses a Markov Game (MG) to identify optimal actions among
heterogeneous agents. We employ various value function decomposition methods to
resolve the credit allocation problem and enhance collaborative control. We
also introduce a multivariate environment model based on random forest (RF) for
anesthesia state simulation. Additionally, a data resampling and alignment
technique ensures synchronized trajectory data. Our experiments on general and
thoracic surgery datasets show that VD-MADRL performs better than human
experience. It improves dose precision and keeps anesthesia states stable,
providing great clinical value.
