DPZV: Elevating the Tradeoff between Privacy and Utility in Zeroth-Order Vertical Federated Learning
Journal:
arXiv
Published Date:
Feb 27, 2025
Abstract
Vertical Federated Learning (VFL) enables collaborative training with
feature-partitioned data, yet remains vulnerable to privacy leakage through
gradient transmissions. Standard differential privacy (DP) techniques such as
DP-SGD are difficult to apply in this setting due to VFL's distributed nature
and the high variance incurred by vector-valued noise. On the other hand,
zeroth-order (ZO) optimization techniques can avoid explicit gradient exposure
but lack formal privacy guarantees. In this work, we propose DPZV, the first ZO
optimization framework for VFL that achieves tunable DP with performance
guarantees. DPZV overcomes these limitations by injecting low-variance scalar
noise at the server, enabling controllable privacy with reduced memory
overhead. We conduct a comprehensive theoretical analysis showing that DPZV
matches the convergence rate of first-order optimization methods while
satisfying formal ($\epsilon, \delta$)-DP guarantees. Experiments on image and
language benchmarks demonstrate that DPZV outperforms several baselines in
terms of accuracy under a wide range of privacy constraints ($\epsilon \le
10$), thereby elevating the privacy-utility tradeoff in VFL.
