AI agents in clinical medicine: delegation, supervision, and the problem of correlated failure.

Journal: Irish journal of medical science
Published Date:

Abstract

Clinical artificial intelligence has to date generated outputs upon which a clinician then acted. Agentic systems execute the task itself, completing a sequence of steps and returning finished work. Adoption has proceeded rapidly across other sectors while clinical uptake has remained confined largely to pilots, an interval that affords medicine an a opportunity to govern the technology deliberately rather than retrospectively. The governing question is thereby transformed from whether a prediction is accurate to whether a task may be entrusted, a question medicine has addressed throughout its history of training juniors under supervision. We argue that the appropriate frame for agentic systems is consequently delegation rather than prediction, and that the entrustment frameworks developed within postgraduate education furnish a serviceable structure of graded autonomy, task-level scope, and named supervisory responsibility. We then identify the point at which the correspondence ceases to hold. Supervision of human colleagues presupposes that errors are independent and idiosyncratic, such that mistakes arise individually and are detected individually. A single agent deployed across an institution generates one error process reproduced at scale. The consequence is that agentic systems cannot be adequately supervised through case-level review, and require instead the population-level surveillance characteristic of quality improvement.

Authors

Keywords

No keywords available for this article.