Bidirectional Linear Recurrent Models for Sequence-Level Multisource Fusion
Journal:
arXiv
Published Date:
Apr 11, 2025
Abstract
Sequence modeling is a critical yet challenging task with wide-ranging
applications, especially in time series forecasting for domains like weather
prediction, temperature monitoring, and energy load forecasting. Transformers,
with their attention mechanism, have emerged as state-of-the-art due to their
efficient parallel training, but they suffer from quadratic time complexity,
limiting their scalability for long sequences. In contrast, recurrent neural
networks (RNNs) offer linear time complexity, spurring renewed interest in
linear RNNs for more computationally efficient sequence modeling. In this work,
we introduce BLUR (Bidirectional Linear Unit for Recurrent network), which uses
forward and backward linear recurrent units (LRUs) to capture both past and
future dependencies with high computational efficiency. BLUR maintains the
linear time complexity of traditional RNNs, while enabling fast parallel
training through LRUs. Furthermore, it offers provably stable training and
strong approximation capabilities, making it highly effective for modeling
long-term dependencies. Extensive experiments on sequential image and time
series datasets reveal that BLUR not only surpasses transformers and
traditional RNNs in accuracy but also significantly reduces computational
costs, making it particularly suitable for real-world forecasting tasks. Our
code is available here.
