Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery
Journal:
arXiv
Published Date:
Feb 3, 2025
Abstract
Multi-index models provide a popular framework to investigate the
learnability of functions with low-dimensional structure and, also due to their
connections with neural networks, they have been object of recent intensive
study. In this paper, we focus on recovering the subspace spanned by the
signals via spectral estimators -- a family of methods that are routinely used
in practice, often as a warm-start for iterative algorithms. Our main technical
contribution is a precise asymptotic characterization of the performance of
spectral methods, when sample size and input dimension grow proportionally and
the dimension $p$ of the space to recover is fixed. Specifically, we locate the
top-$p$ eigenvalues of the spectral matrix and establish the overlaps between
the corresponding eigenvectors (which give the spectral estimators) and a basis
of the signal subspace. Our analysis unveils a phase transition phenomenon in
which, as the sample complexity grows, eigenvalues escape from the bulk of the
spectrum and, when that happens, eigenvectors recover directions of the desired
subspace. The precise characterization we put forward enables the optimization
of the data preprocessing, thus allowing to identify the spectral estimator
that requires the minimal sample size for weak recovery.
