Variable Bregman Majorization-Minimization Algorithm and its Application to Dirichlet Maximum Likelihood Estimation
Journal:
arXiv
Published Date:
Jan 13, 2025
Abstract
We propose a novel Bregman descent algorithm for minimizing a convex function
that is expressed as the sum of a differentiable part (defined over an open
set) and a possibly nonsmooth term. The approach, referred to as the Variable
Bregman Majorization-Minimization (VBMM) algorithm, extends the Bregman
Proximal Gradient method by allowing the Bregman function used in the
divergence to adaptively vary at each iteration, provided it satisfies a
majorizing condition on the objective function. This adaptive framework enables
the algorithm to approximate the objective more precisely at each iteration,
thereby allowing for accelerated convergence compared to the traditional
Bregman Proximal Gradient descent. We establish the convergence of the VBMM
algorithm to a minimizer under mild assumptions on the family of metrics used.
Furthermore, we introduce a novel application of both the Bregman Proximal
Gradient method and the VBMM algorithm to the estimation of the
multidimensional parameters of a Dirichlet distribution through the
maximization of its log-likelihood. Numerical experiments confirm that the VBMM
algorithm outperforms existing approaches in terms of convergence speed.
