TWIG: Two-Step Image Generation using Segmentation Masks in Diffusion Models
Journal:
arXiv
Published Date:
Apr 21, 2025
Abstract
In today's age of social media and marketing, copyright issues can be a major
roadblock to the free sharing of images. Generative AI models have made it
possible to create high-quality images, but concerns about copyright
infringement are a hindrance to their abundant use. As these models use data
from training images to generate new ones, it is often a daunting task to
ensure they do not violate intellectual property rights. Some AI models have
even been noted to directly copy copyrighted images, a problem often referred
to as source copying. Traditional copyright protection measures such as
watermarks and metadata have also proven to be futile in this regard. To
address this issue, we propose a novel two-step image generation model inspired
by the conditional diffusion model. The first step involves creating an image
segmentation mask for some prompt-based generated images. This mask embodies
the shape of the image. Thereafter, the diffusion model is asked to generate
the image anew while avoiding the shape in question. This approach shows a
decrease in structural similarity from the training image, i.e. we are able to
avoid the source copying problem using this approach without expensive
retraining of the model or user-centered prompt generation techniques. This
makes our approach the most computationally inexpensive approach to avoiding
both copyright infringement and source copying for diffusion model-based image
generation.
