Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacy
Journal:
arXiv
Published Date:
Mar 31, 2025
Abstract
This paper investigates the use of Large Language Models (LLMs) to synthesize
public opinion data, addressing challenges in traditional survey methods like
declining response rates and non-response bias. We introduce a novel technique:
role creation based on knowledge injection, a form of in-context learning that
leverages RAG and specified personality profiles from the HEXACO model and
demographic information, and uses that for dynamically generated prompts. This
method allows LLMs to simulate diverse opinions more accurately than existing
prompt engineering approaches. We compare our results with pre-trained models
with standard few-shot prompts. Experiments using questions from the
Cooperative Election Study (CES) demonstrate that our role-creation approach
significantly improves the alignment of LLM-generated opinions with real-world
human survey responses, increasing answer adherence. In addition, we discuss
challenges, limitations and future research directions.
