Edit Flows: Flow Matching with Edit Operations
Journal:
arXiv
Published Date:
Jun 10, 2025
Abstract
Autoregressive generative models naturally generate variable-length
sequences, while non-autoregressive models struggle, often imposing rigid,
token-wise structures. We propose Edit Flows, a non-autoregressive model that
overcomes these limitations by defining a discrete flow over sequences through
edit operations-insertions, deletions, and substitutions. By modeling these
operations within a Continuous-time Markov Chain over the sequence space, Edit
Flows enable flexible, position-relative generation that aligns more closely
with the structure of sequence data. Our training method leverages an expanded
state space with auxiliary variables, making the learning process efficient and
tractable. Empirical results show that Edit Flows outperforms both
autoregressive and mask models on image captioning and significantly
outperforms the mask construction in text and code generation.
