Towards Understanding Bipolar Disorder Through Social Media and Transformer Models: Challenges and Insights

Social media presents a promising avenue for monitoring mental health, yet detecting bipolar disorder (BD) remains significantly underexplored. The complexity arises from the overlap of linguistic patterns associated with depression and anxiety, making accurate identification challenging. This study aims to benchmark the performance of various transformer models trained on Reddit posts, to distinguish BD from other mental health conditions. Using a high-performing generative AI model (GPT-4o) as a benchmark, the analysis reveals that certain open small models (ex. MISTRAL, LLAMA) excel in capturing subtle linguistic cues linked to BD, achieving an F1 score of up to 0.86 with high precision and recall. However, BD was frequently misclassified as depression (23%–51%), normal (2%–41%), and anxiety (1%–7%), underscoring the need for improved approaches. The study highlights the importance of domain-specific data and the development of more nuanced models to enhance BD detection accuracy, paving the way for more effective mental health monitoring and timely interventions.