KITINet: Kinetics Theory Inspired Network Architectures with PDE Simulation Approaches
Journal:
arXiv
Published Date:
May 23, 2025
Abstract
Despite the widely recognized success of residual connections in modern
neural networks, their design principles remain largely heuristic. This paper
introduces KITINet (Kinetics Theory Inspired Network), a novel architecture
that reinterprets feature propagation through the lens of non-equilibrium
particle dynamics and partial differential equation (PDE) simulation. At its
core, we propose a residual module that models feature updates as the
stochastic evolution of a particle system, numerically simulated via a
discretized solver for the Boltzmann transport equation (BTE). This formulation
mimics particle collisions and energy exchange, enabling adaptive feature
refinement via physics-informed interactions. Additionally, we reveal that this
mechanism induces network parameter condensation during training, where
parameters progressively concentrate into a sparse subset of dominant channels.
Experiments on scientific computation (PDE operator), image classification
(CIFAR-10/100), and text classification (IMDb/SNLI) show consistent
improvements over classic network baselines, with negligible increase of FLOPs.
