Using artificial intelligence to assess personal qualities in college admissions.

Journal: Science advances
Published Date: Oct 12, 2023

Abstract

Personal qualities like prosocial purpose and leadership predict important life outcomes, including college success. Unfortunately, the holistic assessment of personal qualities in college admissions is opaque and resource intensive. Can artificial intelligence (AI) advance the goals of holistic admissions? While cost-effective, AI has been criticized as a "black box" that may inadvertently penalize already disadvantaged subgroups when used in high-stakes settings. Here, we consider an AI approach to assessing personal qualities that aims to overcome these limitations. Research assistants and admissions officers first identified the presence/absence of seven personal qualities in = 3131 applicant essays describing extracurricular and work experiences. Next, we fine-tuned pretrained language models with these ratings, which successfully reproduced human codes across demographic subgroups. Last, in a national sample ( = 309,594), computer-generated scores collectively demonstrated incremental validity for predicting 6-year college graduation. We discuss challenges and opportunities of AI for assessing personal qualities.

Authors

  • Benjamin Lira
    University of Pennsylvania, Philadelphia, PA, USA.
  • Margo Gardner
    University of Colorado-Boulder, Boulder, CO, USA.
  • Abigail Quirk
    University of Pennsylvania, Philadelphia, PA, USA.
  • Cathlyn Stone
    University of Colorado-Boulder, Boulder, CO, USA.
  • Arjun Rao
    Institute for Theoretical Computer Science, Graz University of Technology, 8010 Graz, Austria.
  • Lyle Ungar
    University of Pennsylvania, USA.
  • Stephen Hutt
    University of Denver, Denver, CO, USA.
  • Louis Hickman
    Purdue University, USA.
  • Sidney K D'Mello
    Department of Psychology, University of Notre Dame, Notre Dame, Indiana, 46556, USA.
  • Angela L Duckworth
    University of Pennsylvania, Philadelphia, PA, USA.

Keywords

External Resources

Popular Topics
Recent Journals