Integrating human and artificial intelligence for robust postmarketing safety surveillance systems: reflections from the FDA Sentinel Innovation Center.

OBJECTIVES: To describe considerations for integration of human and artificial intelligence for creating a postmarketing surveillance system capable of timely and reliably identifying causal effects of medications on safety endpoints. MATERIALS AND METHODS: The FDA has prioritized more extensive Electronic Health Records (EHR) integration along with generative artificial intelligence and machine learning (Gen AI/ML) into the national active surveillance program for medical products-the Sentinel Initiative. Based on our experience of leading these efforts, we provide perspectives on the opportunities and challenges of Gen AI/ML integration into Sentinel. RESULTS: Using specific examples, we outline the role of Gen AI and ML in a causal inference framework for scalable information extraction, assessment of fitness-for-purpose of data sources, diagnosing residual confounding, and enhancing confounding adjustment. Critically, we outline steps and checkpoints along the way where human involvement remains indispensable. DISCUSSION AND CONCLUSION: In public health applications where stakes are high, use of Gen AI/ML needs to be carefully considered with appropriate guardrails ensuring human expert involvement.