Query-based Knowledge Transfer for Heterogeneous Learning Environments
Journal:
arXiv
Published Date:
Apr 12, 2025
Abstract
Decentralized collaborative learning under data heterogeneity and privacy
constraints has rapidly advanced. However, existing solutions like federated
learning, ensembles, and transfer learning, often fail to adequately serve the
unique needs of clients, especially when local data representation is limited.
To address this issue, we propose a novel framework called Query-based
Knowledge Transfer (QKT) that enables tailored knowledge acquisition to fulfill
specific client needs without direct data exchange. QKT employs a data-free
masking strategy to facilitate communication-efficient query-focused knowledge
transfer while refining task-specific parameters to mitigate knowledge
interference and forgetting. Our experiments, conducted on both standard and
clinical benchmarks, show that QKT significantly outperforms existing
collaborative learning methods by an average of 20.91\% points in single-class
query settings and an average of 14.32\% points in multi-class query scenarios.
Further analysis and ablation studies reveal that QKT effectively balances the
learning of new and existing knowledge, showing strong potential for its
application in decentralized learning.
