Leveraging Traditional and Generative Artificial Intelligence for Programmatic Decision Making in Faculty Development.

INTRODUCTION: Traditional program evaluation in medical education often faces challenges with large data volumes and manual analysis, which can delay data-driven decisions. This article introduces the artificial intelligence (AI)-enhanced program evaluation (AIEPE) framework to address this gap, proposing a novel, integrated approach using both traditional and generative AI to improve efficiency and insight. METHOD: The AIEPE framework is a five-phase, iterative system that augments human evaluators. It begins with Multi-Modal Data Aggregation (Phase 1). Next, a Dual-Stream AI-Powered Analysis (Phase 2) uses traditional AI for sentiment and thematic analysis while generative AI synthesizes narrative summaries. These insights inform AI-Powered Strategic Formulation (Phase 3), which assists in prioritizing issues and drafting action plans. The final phases, Human-in-the-Loop Decision-Making (Phase 4) and Programmatic Implementation and Iteration (Phase 5), ensure continuous improvement with human oversight. RESULTS: Applying the AIEPE framework to a faculty development program evaluation, we analyzed 3361 qualitative comments. Sentiment analysis revealed 87.6% positive comments and thematic analysis identified key topics. The dual-stream approach provided a deeper understanding, linking a high no show rate in an AI course to faculty feeling unprepared. This led to a top-priority recommendation to redesign the curriculum. DISCUSSION: The AIEPE framework transforms program evaluation from a labor-intensive process into a dynamic engine for continuous improvement. It empowers academic leaders to shift from data collection to architects of data-informed strategy. Although the framework significantly increases efficiency and depth of insight, it highlights the critical need for human oversight to mitigate challenges such as bias and over reliance on technology.