A regularized orthogonal activated inverse-learning neural network for regression and classification with outliers.
Journal:
Neural networks : the official journal of the International Neural Network Society
Published Date:
Feb 24, 2024
Abstract
A novel regularized orthogonal activated inverse-learning (ROAIL) neural network is proposed and investigated for reducing the impact of outliers in regression and classification fields. The proposed ROAIL network does not require extensive iterative computations. Instead, it can achieve the desired results with a single step of computation, allowing for the efficient acquisition of network weights. By extending the Gegenbauer polynomials to a multi-variate version, and integrating the ℓ regularization and Welsch loss function into the orthogonal activated inverse-learning framework, two forms of ROAIL are obtained, i.e., ℓ norm ROAIL (ℓ-ROAIL) and Welsch-ROAIL (W-ROAIL). ℓ-ROAIL neural network is proposed to minimize the empirical and structural risk simultaneously since taking the structural risk as a part of loss function can effectively reduce the complexity of the model and thus improve the generalization ability. W-ROAIL neural network further improves the robustness of the ℓ-ROAIL neural network by replacing the original two-norm in loss function with Welsch function. The Welsch function can determine the weights of each sample according to its output error, and influence of outliers could be weakened since the weights of outliers would be reduced. Both regression and classification experiments show that W-ROAIL neural network has strong ability to suppress the influence of outliers.
