Readability Assessment and Comparison of Large Language Model-Generated Summaries of Trial Descriptions on ClinicalTrials.gov.

Journal: Studies in health technology and informatics
Published Date: Aug 7, 2025

Abstract

This study evaluated the readability of ClinicalTrials.gov trial information using traditional readability measures (TRMs) and compared it to summaries generated by large language models (LLMs), specifically ChatGPT and a fine-tuned BART-Large-CNN (FBLC). The study involved: 1) assessing required reading levels (RRL) with TRMs, 2) generating sample LLM-based summaries, and 3) evaluating summary quality based on scores provided by two independent reviewers. The results show that the original ClinicalTrials.gov trial descriptions were scored above the recommended readability level. In contrast, ChatGPT-generated summaries had significantly lower RRLs and higher quality scores. We conclude that ChatGPT shows great promise of creating readable, high-quality summaries. Future research is warranted to assess whether LLMs could be a viable solution to improve the readability of ClinicalTrials.gov to facilitate comprehension by laypersons.

Authors

  • Tzu-Chun Wu
    Department of Biomedical Informatics, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States.
  • Hanniel Shih
    University of Cincinnati, Cincinnati, OH, United States.
  • Anunita Nattam
    University of Cincinnati, Cincinnati, Ohio, United States.
  • Himaja Chintalapalli
    University of Cincinnati, Cincinnati, Ohio, United States.
  • David A Hanauer
    Department of Pediatrics, University of Michigan Medical School, Ann Arbor, MI, USA; School of Information, University of Michigan, Ann Arbor, MI, USA. Electronic address: hanauer@med.umich.edu.
  • Kai Zheng
    University of California, Irvine, Irvine, CA, USA.
  • Danny T Y Wu
    Department of Biomedical Informatics, College of Medicine, University of Cincinnati, Cincinnati, OH.

Keywords

External Resources

Popular Topics
Recent Journals