Exploring the Boundaries of Content Moderation in Text-to-Image Generation
Journal:
arXiv
Published Date:
Sep 9, 2024
Abstract
This paper analyzes the community safety guidelines of five text-to-image
(T2I) generation platforms and audits five T2I models, focusing on prompts
related to the representation of humans in areas that might lead to societal
stigma. While current research primarily focuses on ensuring safety by
restricting the generation of harmful content, our study offers a complementary
perspective. We argue that the concept of safety is difficult to define and
operationalize, reflected in a discrepancy between the officially published
safety guidelines and the actual behavior of the T2I models, and leading at
times to over-censorship. Our findings call for more transparency and an
inclusive dialogue about the platforms' content moderation practices, bearing
in mind their global cultural and social impact.
