MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
MARCH introduces a novel architecture that scales recurrent state-space models beyond fixed-size dimensions by periodically caching cumulative recurrent-state checkpoints as "state anchors" with content-conditioned anchor keys The model maintains a growing memory bank that provides a controllable trade-off between historical resolution and memory cost, addressing the key limitation of recurrent models where earlier associations get overwritten At each token, MARCH produces an anchor query to att
Analysis
TL;DR
- MARCH introduces a novel architecture that scales recurrent state-space models beyond fixed-size dimensions by periodically caching cumulative recurrent-state checkpoints as "state anchors" with content-conditioned anchor keys
- The model maintains a growing memory bank that provides a controllable trade-off between historical resolution and memory cost, addressing the key limitation of recurrent models where earlier associations get overwritten
- At each token, MARCH produces an anchor query to attend all causally available state anchors, computing output via attention-style aggregation over historical anchors
- After standard pretraining, MARCH consistently outperforms multiple linear attention variants across commonsense reasoning, LongBench, and in-context retrieval benchmarks
- The approach demonstrates that content-routed state caching substantially strengthens recurrent long-range memory while preserving the computational efficiency of recurrent architectures
Why It Matters
This work addresses a fundamental tension in long-context AI: Transformers excel at recall-intensive tasks but suffer from quadratic training complexity and linear KV-cache growth during inference, while recurrent models are computationally efficient but lose early contextual information. MARCH offers a compelling middle ground that could enable more efficient long-context models without sacrificing retrieval performance, which is critical for applications requiring extended reasoning over large documents or conversations.
Technical Details
- State Anchor Mechanism: MARCH periodically caches cumulative recurrent-state checkpoints as state anchors, each associated with a compact, content-conditioned anchor key, enabling the memory bank to grow proportionally with context length
- Content-Routed Attention: At each decoding step, the model generates an anchor query that attends to all causally available state anchors, performing attention-style aggregation over the historical anchor memory while maintaining the native recurrent computation path
- Scalable Architecture: The architecture effectively scales state-space models beyond fixed-size dimensions, providing a tunable trade-off between historical resolution (more anchors = better recall) and memory/computational cost
- Benchmarks: Evaluated on commonsense reasoning, LongBench, and in-context retrieval tasks, consistently outperforming multiple linear attention variants after standard pretraining
- Efficiency Preservation: Unlike full attention mechanisms, MARCH preserves the computational efficiency of recurrent architectures during autoregressive inference, avoiding the linear KV-cache growth problem of Transformers
Industry Insight
- The content-routed state caching paradigm could become a key design principle for next-generation long-context models, offering a practical alternative to both full Transformer attention and pure recurrent architectures
- As applications demanding extended context windows (e.g., full-document analysis, long-form code generation, multi-turn reasoning) grow, architectures like MARCH that decouple memory capacity from computational overhead will likely see increased adoption
- The controllable trade-off between historical resolution and memory cost provides engineers with a practical tuning knob, making recurrent-based long-context models more viable for production deployment where inference efficiency remains a critical constraint
Disclaimer: The above content is generated by AI and is for reference only.