Dynamic Fusion Strategies for Federated Multimodal Recommendations
Journal:
arXiv
Published Date:
Oct 11, 2024
Abstract
Delivering deeply personalized recommendations necessitates understanding
user interactions with diverse multimedia features, but achieving this within
the constraints of Federated Recommendation Systems (FedRec) is severely
hampered by communication bottlenecks, user heterogeneity, and the complexity
of privacy-preserving multimodal fusion. To this end, we propose FedMR, a novel
multimodal FedRec framework centered around the Mixing Feature Fusion Module
(MFFM). FedMR employs a two-stage process: (1) Server-side centralized
multimedia content processing provides rich, shared item context using
pre-trained models, mitigating limitations from client sparsity and resource
constraints efficiently. (2) Client-Side Personalized Refinement, where the
MFFM dynamically adapts these server-provided multimodal representations based
on client-specific interaction patterns, effectively tailoring recommendations
and resolving heterogeneity in user preferences towards different modalities.
Extensive experiments validate that FedMR seamlessly enhances existing ID-based
FedRecs, effectively transforming them into high-performing federated
multimodal systems.
