Redefining Research Crowdsourcing: Incorporating Human Feedback with LLM-Powered Digital Twins
Journal:
arXiv
Published Date:
May 29, 2025
Abstract
Crowd work platforms like Amazon Mechanical Turk and Prolific are vital for
research, yet workers' growing use of generative AI tools poses challenges.
Researchers face compromised data validity as AI responses replace authentic
human behavior, while workers risk diminished roles as AI automates tasks. To
address this, we propose a hybrid framework using digital twins, personalized
AI models that emulate workers' behaviors and preferences while keeping humans
in the loop. We evaluate our system with an experiment (n=88 crowd workers) and
in-depth interviews with crowd workers (n=5) and social science researchers
(n=4). Our results suggest that digital twins may enhance productivity and
reduce decision fatigue while maintaining response quality. Both researchers
and workers emphasized the importance of transparency, ethical data use, and
worker agency. By automating repetitive tasks and preserving human engagement
for nuanced ones, digital twins may help balance scalability with authenticity.
