Adaptive Diversity Induced Reweighting for long-tailed classification.
Journal:
Neural networks : the official journal of the International Neural Network Society
Published Date:
May 19, 2025
Abstract
Real-world large-scale data often exhibit a long-tailed distribution, making a classifier difficult to generalize well on tail categories. A straightforward and effective approach to tackling this issue is reweighting. Traditionally, the weight of a category is designed according to its cardinality, i.e., the number of samples it contains. Recently, some researchers claim that reweighting based on the spanned space size is more effective. They have made great progress in measuring the spanned space by analyzing sample distribution. Different from them, we aim to approximate the instinct space of a class spanned by infinite samples, which can instruct the optimizer to learn a more generalized classifier. To achieve this purpose, we maximize the space spanned by the limited training samples by sufficiently and subtly adjusting their distribution. Therefore, this paper introduces a novel metric, diversity, to measure the size of the maximized spanned space of a category. To quantify the class's diversity, we derive an easy and non-parametric formula from a new perspective of estimating its optimal prototype. Subsequently, an Adaptive Diversity Induced Reweighting (ADIR) loss is designed by allocating each class a weight inversely proportional to its diversity. Experiments conducted on four long-tailed datasets demonstrate that the proposed ADIR outperforms other state-of-the-art reweighting methods. Moreover, statistically balanced datasets can be imbalanced in diversity, where our ADIR can also achieve better results.
