Token Probabilities to Mitigate Large Language Models Overconfidence in Answering Medical Questions: Quantitative Study.

Journal: Journal of medical Internet research
Published Date: Aug 29, 2025

Abstract

BACKGROUND: Chatbots have demonstrated promising capabilities in medicine, scoring passing grades for board examinations across various specialties. However, their tendency to express high levels of confidence in their responses, even when incorrect, poses a limitation to their utility in clinical settings.

Authors

  • Raphaël Bentegeac
    Department of Public Health, Lille University, Lille University Hospital Center, avenue du Professeur Emile Laine, Lille, 59037, France.
  • Bastien Le Guellec
    Department of Neuroradiology, CHU Lille, Salengro Hospital, Lille 59000, France; Université Lille, INSERM, CHU Lille, Institut Pasteur de Lille, U1167-RID-AGE - Facteurs de Risque et Déterminants Moléculaires des Maladies Liées au Vieillissement, Lille 59000, France; INSERM, U1172-LilNCog-Lille Neuroscience & Cognition, Université de Lille, Lille 59000, France. Electronic address: bastien.leguellec@chu-lille.fr.
  • Grégory Kuchcinski
    Department of Neuroradiology, CHU Lille, Salengro Hospital, Lille 59000, France; INSERM, U1172-LilNCog-Lille Neuroscience & Cognition, Université de Lille, Lille 59000, France.
  • Philippe Amouyel
    From the Department of Neuroradiology (B.L.G., A.L., C.B., J.P.P., G.K.), Department of Public Health (B.L.G., P.A., A.H.), and INclude Health Data Warehouse (C.G., L.S.), CHU Lille-Université Lille, Rue Emile Laine, 59000 Lille, France; Department of Radiology, UC Davis Health, Sacramento, Calif (L.H.B.); Université Lille, INSERM, CHU Lille, Institut Pasteur de Lille, U1167-RID-AGE - Facteurs de risque et déterminants moléculaires des maladies liées au vieillissement, Lille, France (P.A., A.H.); INSERM, U1172-LilNCog-Lille Neuroscience & Cognition, Université Lille, Lille, France (J.P.P., G.K.); and UAR 2014-US 41-PLBS-Plateformes Lilloises en Biologie & Santé, Université Lille, Lille, France (J.P.P., G.K.).
  • Aghiles Hamroun
    Université Lille, INSERM, CHU Lille, Institut Pasteur de Lille, U1167-RID-AGE - Facteurs de Risque et Déterminants Moléculaires des Maladies Liées au Vieillissement, Lille 59000, France; Public Health - Epidemiology Department, CHU Lille, Maison Régionale de la Recherche Clinique, Lille 59000, France.

Keywords

External Resources

Popular Topics
Recent Journals