Multiplex visibility graphs to investigate recurrent neural network dynamics.

Journal: Scientific reports
Published Date:

Abstract

A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods.

Authors

  • Filippo Maria Bianchi
    Department of Information Engineering, Electronics and Telecommunications (DIET), "Sapienza" University of Rome, Via Eudossiana 18, 00184 Rome, Italy. Electronic address: filippomaria.bianchi@uniroma1.it.
  • Lorenzo Livi
    Department of Computer Science, College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter EX4 4QF, United Kingdom.
  • Cesare Alippi
    Department of Electronics, Information, and Bioengineering, Politecnico di Milano, 20133 Milan, Italy.
  • Robert Jenssen
    Department of Physics and Technology, University of Tromsø - The Arctic University of Norway, Tromsø, Norway; Norwegian Centre for Integrated Care and Telemedicine, University Hospital of North Norway, Tromsø, Norway.