Evidence-processing errors and their correction in an LLM-assisted systematic review: a retrospective methodological case study

Background: Errors in interpreting sources, identifying overlapping cohorts and handling corrections can change the results of systematic reviews. We investigated how documented evidence-processing errors affected one completed review assisted by large language models (LLMs), and how an auditable workflow linked their detection and correction to released outputs. Methods: We conducted a retrospective, non-blinded methodological case study of a registered prognostic review. Frozen correction and audit records were reconciled into distinct root-cause events. We described their recorded consequences for analysis membership, study weights, pooled results, certainty and publication files, and traced four illustrative error pathways to implemented controls. Six release-integrity domains provided supporting internal checks. The production review was also the development case; there was no external comparator or independent human reference. Results: Fifty root-cause events were documented; 12 had changed pooled results before correction. Forty-six were resolved and four remained disclosed non-critical limitations. Twenty-one events had at least one recorded scientific consequence. The four pathways concerned survival time origin, cohort dependence, outcome direction and certainty assessment. Corrections changed analysis membership, weighting or interpretation; in the certainty case, the risk-of-bias domain changed while the final certainty grade remained at its lowest level. The application comprised 454 reports and 445 studies, with 41 of 49 registered analyses fitted. Public records link the reported defects, corrections, rebuilds and release checks. Conclusions: In this review, consequential errors arose in the relationships between source judgments and their downstream use. Explicit estimand rules, adjudication-driven dependence mapping, source-based outcome coding and documented downstream rebuilds made corrections inspectable. The case provides operational rules and documented failure examples for other review teams to examine and test; it does not establish error rates or comparative effectiveness. Systematic review registration: Underlying clinical review: PROSPERO CRD420261449181. The retrospective methods evaluation was not prospectively registered.