Using Natural Language Processing to Automatically Assess Feedback Quality: Findings From 3 Surgical Residencies.

Journal: Academic medicine : journal of the Association of American Medical Colleges
PMID: 33951682

Abstract

PURPOSE: Learning is markedly improved with high-quality feedback, yet assuring the quality of feedback is difficult to achieve at scale. Natural language processing (NLP) algorithms may be useful in this context as they can automatically classify large volumes of narrative data. However, it is unknown if NLP models can accurately evaluate surgical trainee feedback. This study evaluated which NLP techniques best classify the quality of surgical trainee formative feedback recorded as part of a workplace assessment.

Authors

  • Erkin Ötleş
    E. Ötleş is Medical Scientist Training Program fellow, Department of Industrial and Operations Engineering, University of Michigan Medical School, Ann Arbor, Michigan.
  • Daniel E Kendrick
    D.E. Kendrick is assistant professor, Department of Surgery, University of Minnesota Medical School, Minneapolis, Minnesota.
  • Quintin P Solano
    Q.P. Solano is a third-year medical student, University of Michigan Medical School, Ann Arbor, Michigan.
  • Mary Schuller
    M. Schuller is senior project manager, Department of Surgery, University of Michigan Medical School, Ann Arbor, Michigan.
  • Samantha L Ahle
    S.L. Ahle is a resident, Department of Surgery, Yale School of Medicine, New Haven, Connecticut.
  • Mickyas H Eskender
    M.H. Eskender is a resident, Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
  • Emily Carnes
    E. Carnes is research assistant, Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
  • Brian C George
    B.C. George is assistant professor and director, Center for Surgical Training and Research, Department of Surgery, University of Michigan Medical School, Ann Arbor, Michigan.

Keywords

External Resources

Popular Topics
Recent Journals