Sentiment Simulation using Generative AI Agents
Journal:
arXiv
Published Date:
May 28, 2025
Abstract
Traditional sentiment analysis relies on surface-level linguistic patterns
and retrospective data, limiting its ability to capture the psychological and
contextual drivers of human sentiment. These limitations constrain its
effectiveness in applications that require predictive insight, such as policy
testing, narrative framing, and behavioral forecasting. We present a robust
framework for sentiment simulation using generative AI agents embedded with
psychologically rich profiles. Agents are instantiated from a nationally
representative survey of 2,485 Filipino respondents, combining sociodemographic
information with validated constructs of personality traits, values, beliefs,
and socio-political attitudes. The framework includes three stages: (1) agent
embodiment via categorical or contextualized encodings, (2) exposure to
real-world political and economic scenarios, and (3) generation of sentiment
ratings accompanied by explanatory rationales. Using Quadratic Weighted
Accuracy (QWA), we evaluated alignment between agent-generated and human
responses. Contextualized encoding achieved 92% alignment in replicating
original survey responses. In sentiment simulation tasks, agents reached
81%--86% accuracy against ground truth sentiment, with contextualized profile
encodings significantly outperforming categorical (p < 0.0001, Cohen's d =
0.70). Simulation results remained consistent across repeated trials
(+/-0.2--0.5% SD) and resilient to variation in scenario framing (p = 0.9676,
Cohen's d = 0.02). Our findings establish a scalable framework for sentiment
modeling through psychographically grounded AI agents. This work signals a
paradigm shift in sentiment analysis from retrospective classification to
prospective and dynamic simulation grounded in psychology of sentiment
formation.
