Statistical guarantees for regularized neural networks.

Journal: Neural networks : the official journal of the International Neural Network Society
Published Date:

Abstract

Neural networks have become standard tools in the analysis of data, but they lack comprehensive mathematical theories. For example, there are very few statistical guarantees for learning neural networks from data, especially for classes of estimators that are used in practice or at least similar to such. In this paper, we develop a general statistical guarantee for estimators that consist of a least-squares term and a regularizer. We then exemplify this guarantee with ℓ-regularization, showing that the corresponding prediction error increases at most logarithmically in the total number of parameters and can even decrease in the number of layers. Our results establish a mathematical basis for regularized estimation of neural networks, and they deepen our mathematical understanding of neural networks and deep learning more generally.

Authors

  • Mahsa Taheri
    Department of Mathematics, Ruhr-University Bochum, Universitätsstraße 150, 44801 Bochum, Germany. Electronic address: mahsa.taheri@rub.de.
  • Fang Xie
    Shandong Luoxin Pharmaceutical Group Stock Co. Ltd, Linyi, Shandong, China.
  • Johannes Lederer
    Department of Mathematics, Ruhr-University Bochum, Universitätsstraße 150, 44801 Bochum, Germany. Electronic address: johannes.lederer@rub.de.