FashionDPO:Fine-tune Fashion Outfit Generation Model using Direct Preference Optimization
Journal:
arXiv
Published Date:
Apr 17, 2025
Abstract
Personalized outfit generation aims to construct a set of compatible and
personalized fashion items as an outfit. Recently, generative AI models have
received widespread attention, as they can generate fashion items for users to
complete an incomplete outfit or create a complete outfit. However, they have
limitations in terms of lacking diversity and relying on the supervised
learning paradigm. Recognizing this gap, we propose a novel framework
FashionDPO, which fine-tunes the fashion outfit generation model using direct
preference optimization. This framework aims to provide a general fine-tuning
approach to fashion generative models, refining a pre-trained fashion outfit
generation model using automatically generated feedback, without the need to
design a task-specific reward function. To make sure that the feedback is
comprehensive and objective, we design a multi-expert feedback generation
module which covers three evaluation perspectives, \ie quality, compatibility
and personalization. Experiments on two established datasets, \ie iFashion and
Polyvore-U, demonstrate the effectiveness of our framework in enhancing the
model's ability to align with users' personalized preferences while adhering to
fashion compatibility principles. Our code and model checkpoints are available
at https://github.com/Yzcreator/FashionDPO.
