Learning optimal treatment strategies for intraoperative hypotension using deep reinforcement learning
Journal:
arXiv
Published Date:
May 27, 2025
Abstract
Traditional methods of surgical decision making heavily rely on human
experience and prompt actions, which are variable. A data-driven system
generating treatment recommendations based on patient states can be a
substantial asset in perioperative decision-making, as in cases of
intraoperative hypotension, for which suboptimal management is associated with
acute kidney injury (AKI), a common and morbid postoperative complication. We
developed a Reinforcement Learning (RL) model to recommend optimum dose of
intravenous (IV) fluid and vasopressors during surgery to avoid intraoperative
hypotension and postoperative AKI. We retrospectively analyzed 50,021 surgeries
from 42,547 adult patients who underwent major surgery at a quaternary care
hospital between June 2014 and September 2020. Of these, 34,186 surgeries were
used for model training and 15,835 surgeries were reserved for testing. We
developed a Deep Q-Networks based RL model using 16 variables including
intraoperative physiologic time series, total dose of IV fluid and vasopressors
extracted for every 15-minute epoch. The model replicated 69% of physician's
decisions for the dosage of vasopressors and proposed higher or lower dosage of
vasopressors than received in 10% and 21% of the treatments, respectively. In
terms of IV fluids, the model's recommendations were within 0.05 ml/kg/15 min
of the actual dose in 41% of the cases, with higher or lower doses recommended
for 27% and 32% of the treatments, respectively. The model resulted in a higher
estimated policy value compared to the physicians' actual treatments, as well
as random and zero-drug policies. AKI prevalence was the lowest in patients
receiving medication dosages that aligned with model's decisions. Our findings
suggest that implementation of the model's policy has the potential to reduce
postoperative AKI and improve other outcomes driven by intraoperative
hypotension.
