Comparative effectiveness of ambient documentation tools in primary care tool-specific variations in efficiency, documentation burden, and productivity over time.

OBJECTIVES: To evaluate the comparative effectiveness of ambient documentation tools (ADTs) with distinct architectures (Tablet-Based Virtual Human-Assisted Ambient [Tool A], EHR-Integrated Ambient [Tool B], and Standalone Ambient [Tool C]) on provider efficiency, documentation burden, and productivity in primary care. MATERIALS AND METHODS: We conducted a real-world comparative-effectiveness study of 163 primary care providers in a large integrated health system (January 2024-June 2025), analyzing 59 130 provider-days. Providers who did not adopt an ADT (n = 13) were excluded from comparative models. Exposure groups included Tool A (n = 65), Tool B (n = 68), and Tool C (n = 17). Primary outcomes were pajama time (hours of after-hours EHR activity per provider-day), visit closure within 2 days (proportion of encounters signed within 48 hours), proportion of manual note composition, and a composite opportunity score reflecting EHR efficiency (scaled 0-1). Outcomes were derived from daily provider-level EHR logs. Analyses used intention-to-treat and per-protocol frameworks with provider-clustered SEs and month fixed effects. RESULTS: Compared with Tool B, Tool A was associated with increased pajama time (+0.022 hours of after-hours EHR activity per provider-day), higher manual note composition (+0.046 absolute proportion), lower timely visit closure (-0.120 [95% CI, -0.126,-0.115]), and reduced opportunity scores (-0.035[-0.044,-0.027]). Tool C reduced after-hours work versus Tool B (-0.007 hours per provider-day) and relative to both the reference tool and pre-exposure baseline levels (-0.055 hours per provider-day), but was associated with lower timely visit closure (-0.028[-0.036, -0.020]); opportunity scores were not significantly changed (-0.007[-0.018, 0.003]). Effects for Tool C correspond to approximately 30 fewer annual after-hours hours per provider. CONCLUSION: Tool choice materially shapes real-world outcomes, and tool-specific comparative evaluation is essential to guide evidence-based procurement and deployment decisions.