Evaluation of artificial intelligence (AI) chatbots for providing sexual health information: a consensus study using real-world clinical queries.

Journal: BMC public health
PMID: 40375254

Abstract

INTRODUCTION: Artificial Intelligence (AI) chatbots could potentially provide information on sensitive topics, including sexual health, to the public. However, their performance compared to nurses and across different AI chatbots, particularly in the field of sexual health, remains understudied. This study evaluated the performance of three AI chatbots - two prompt-tuned (Alice and Azure) and one standard chatbot (ChatGPT by OpenAI) - in providing sexual health information on questions that experienced sexual health nurses could correctly answer.

Authors

  • Phyu M Latt
    Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia. Electronic address: phyu.latt@monash.edu.
  • Ei T Aung
    Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.
  • Kay Htaik
    Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.
  • Nyi N Soe
    Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.
  • David Lee
    Department of Media Engineering, Catholic University of Korea, 43-1, Yeoggok 2-dong, Wonmmi-gu, Bucheon-si, Gyeonggi-do 14662, Korea. leedabid@catholic.ac.kr.
  • Alicia J King
    School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.
  • Ria Fortune
    Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.
  • Jason J Ong
    School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia; Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; Department of Clinical Research, London School of Hygiene and Tropical Medicine, London, United Kingdom.
  • Eric P F Chow
    School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia; Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, Australia.
  • Catriona S Bradshaw
    Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.
  • Rashidur Rahman
    Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.
  • Matthew Deneen
    Alfred Health, Melbourne, Australia.
  • Sheranne Dobinson
    Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.
  • Claire Randall
    Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.
  • Lei Zhang
    Division of Gastroenterology, Union Hospital, Tongji Medical College Medical College, Huazhong University of Science and Technology, Wuhan, China.
  • Christopher K Fairley
    School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia; Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.

Keywords

External Resources

Popular Topics
Recent Journals