Low-Biased General Annotated Dataset Generation
Journal:
arXiv
Published Date:
Dec 14, 2024
Abstract
Pre-training backbone networks on a general annotated dataset (e.g.,
ImageNet) that comprises numerous manually collected images with category
annotations has proven to be indispensable for enhancing the generalization
capacity of downstream visual tasks. However, those manually collected images
often exhibit bias, which is non-transferable across either categories or
domains, thus causing the model's generalization capacity degeneration. To
mitigate this problem, we present a low-biased general annotated dataset
generation framework (lbGen). Instead of expensive manual collection, we aim at
directly generating low-biased images with category annotations. To achieve
this goal, we propose to leverage the advantage of a multimodal foundation
model (e.g., CLIP), in terms of aligning images in a low-biased semantic space
defined by language. Specifically, we develop a bi-level semantic alignment
loss, which not only forces all generated images to be consistent with the
semantic distribution of all categories belonging to the target dataset in an
adversarial learning manner, but also requires each generated image to match
the semantic description of its category name. In addition, we further cast an
existing image quality scoring model into a quality assurance loss to preserve
the quality of the generated image. By leveraging these two loss functions, we
can obtain a low-biased image generation model by simply fine-tuning a
pre-trained diffusion model using only all category names in the target dataset
as input. Experimental results confirm that, compared with the manually labeled
dataset or other synthetic datasets, the utilization of our generated
low-biased dataset leads to stable generalization capacity enhancement of
different backbone networks across various tasks, especially in tasks where the
manually labeled samples are scarce.
