Theoretical Framework for Tempered Fractional Gradient Descent: Application to Breast Cancer Classification
Journal:
arXiv
Published Date:
Apr 26, 2025
Abstract
This paper introduces Tempered Fractional Gradient Descent (TFGD), a novel
optimization framework that synergizes fractional calculus with exponential
tempering to enhance gradient-based learning. Traditional gradient descent
methods often suffer from oscillatory updates and slow convergence in
high-dimensional, noisy landscapes. TFGD addresses these limitations by
incorporating a tempered memory mechanism, where historical gradients are
weighted by fractional coefficients $|w_j| = \binom{\alpha}{j}$ and
exponentially decayed via a tempering parameter $\lambda$. Theoretical analysis
establishes TFGD's convergence guarantees: in convex settings, it achieves an
$\mathcal{O}(1/K)$ rate with alignment coefficient $d_{\alpha,\lambda} = (1 -
e^{-\lambda})^{-\alpha}$, while stochastic variants attain
$\mathcal{O}(1/k^\alpha)$ error decay. The algorithm maintains $\mathcal{O}(n)$
time complexity equivalent to SGD, with memory overhead scaling as
$\mathcal{O}(d/\lambda)$ for parameter dimension $d$. Empirical validation on
the Breast Cancer Wisconsin dataset demonstrates TFGD's superiority, achieving
98.25\% test accuracy (vs. 92.11\% for SGD) and 2$\times$ faster convergence.
The tempered memory mechanism proves particularly effective in medical
classification tasks, where feature correlations benefit from stable gradient
averaging. These results position TFGD as a robust alternative to conventional
optimizers in both theoretical and applied machine learning.
