Using Natural Language Processing to Evaluate the Quality of Supervisor Narrative Comments in Competency-Based Medical Education.

Journal: Academic medicine : journal of the Association of American Medical Colleges
Published Date: May 1, 2024

Abstract

PURPOSE: Learner development and promotion rely heavily on narrative assessment comments, but narrative assessment quality is rarely evaluated in medical education. Educators have developed tools such as the Quality of Assessment for Learning (QuAL) tool to evaluate the quality of narrative assessment comments; however, scoring the comments generated in medical education assessment programs is time intensive. The authors developed a natural language processing (NLP) model for applying the QuAL score to narrative supervisor comments.

Authors

  • Maxwell Spadafore
    Third-year medical student, University of Michigan Medical School, Ann Arbor, Michigan; maxspad@umich.edu; ORCID: http://orcid.org/0000-0001-5927-1428. Assistant dean for assessment, evaluation, and quality improvement and associate professor of internal medicine and learning health sciences, University of Michigan Medical School, Ann Arbor, Michigan; ORCID: http://orcid.org/0000-0002-3374-2989.
  • Yusuf Yilmaz
    Department of Gastroenterology, School of Medicine, Recep Tayyip Erdoğan University, Rize, Turkey.
  • Veronica Rally
  • Teresa M Chan
    T.M. Chan is associate professor, Division of Emergency Medicine, Department of Medicine, assistant dean, Program for Faculty Development, Faculty of Health Sciences, and adjunct scientist, McMaster Education Research, Innovation, and Theory (MERIT) program, McMaster University, Hamilton, Ontario, Canada; ORCID: https://orcid.org/0000-0001-6104-462X .
  • Mackenzie Russell
  • Brent Thoma
    B. Thoma is associate professor, Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada, and clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0003-1124-5786 .
  • Sim Singh
  • Sandra Monteiro
    Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Canada.
  • Alim Pardhan
  • Lynsey Martin
  • Seetha U Monrad
  • Rob Woods

Keywords

External Resources

Popular Topics
Recent Journals