Responses of large language model-based chatbots to simulated psychotic ideation across repeated interactions.

Following the development of widely accessible large language model-based chatbot systems, concerns have emerged that such systems may reinforce psychotic ideation through affirmation of delusional content. However, systematic evaluations of chatbot responses to psychotic scenarios remain limited. We hypothesized that chatbot responses would become increasingly affirmatory over repeated interactions with the user. To test this, we evaluated the responses of ChatGPT, DeepSeek and Replika to simulated psychosis scenarios. Conversations were conducted across seven consecutive days in two languages. Responses were coded for chatbot stance toward psychotic content, recommendation to seek clinical assessment, presence of follow-up questions, and additional advice. Mixed-effects and linear regression models were used, including cluster-robust linear probability models for binary outcomes. As simulated symptom severity increased across days, ChatGPT responses became less affirmatory, DeepSeek responses remained relatively stable, and Replika responses became more affirmatory; latter two trajectories differed significantly from ChatGPT. Response variability decreased for ChatGPT and Replika but increased for DeepSeek across days. The effects of language and symptom domain were non-significant. Recommendation rates differed significantly between systems. DeepSeek and Replika provided potentially harmful advice, including discontinuation of antipsychotic treatment, in 21% and 37% of conversations, respectively. These findings suggest that affirmation of psychotic ideation significantly varies across different chatbots and does not appear to be an unavoidable consequence of large language model deployment. Furthermore, they underscore the need for developing explicit safety standards governing chatbot responses to psychotic content.