Adaptive token selection for scalable point cloud transformers.

Journal: Neural networks : the official journal of the International Neural Network Society
Published Date:

Abstract

The recent surge in 3D data acquisition has spurred the development of geometric deep learning models for point cloud processing, boosted by the remarkable success of transformers in natural language processing. While point cloud transformers (PTs) have achieved impressive results recently, their quadratic scaling with respect to the point cloud size poses a significant scalability challenge for real-world applications. To address this issue, we propose the Adaptive Point Cloud Transformer (AdaPT), a standard PT model augmented by an adaptive token selection mechanism. AdaPT dynamically reduces the number of tokens during inference, enabling efficient processing of large point clouds. Furthermore, we introduce a budget mechanism to flexibly adjust the computational cost of the model at inference time without the need for retraining or fine-tuning separate models. Our extensive experimental evaluation on point cloud classification tasks demonstrates that AdaPT significantly reduces computational complexity while maintaining competitive accuracy compared to standard PTs. The code for AdaPT is publicly available at https://github.com/ispamm/adaPT.

Authors

  • Alessandro Baiocchi
    Sapienza University of Rome, Department of Computer, Control and Management Engineering, Via Ariosto 25, Rome, 00185, Italy.
  • Indro Spinelli
    Department of Information Engineering, Electronics and Telecommunications (DIET), Sapienza University of Rome, Via Eudossiana 18, 00184 Rome, Italy.
  • Alessandro Nicolosi
    Leonardo Labs, Piazza Monte Grappa, 4, Rome, 00195, Italy.
  • Simone Scardapane
    Department of Information Engineering, Electronics and Telecommunications (DIET), "Sapienza" University of Rome, Via Eudossiana 18, 00184 Rome, Italy.