A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth
RLT closes the architectural gap in decoder-only LLMs by carrying the decoder's final hidden state and layerwise sliding-window attention cache from token t into token t+1, with no reset at prompt-response boundaries The architecture pairs a causal encoder with a recurrent decoder, using a gated merge between encoder representations and the previous decoder output, achieving unbounded temporal depth (48t structural depth after t tokens in the reference 48-layer config) Three core design principl
Analysis
TL;DR
- RLT closes the architectural gap in decoder-only LLMs by carrying the decoder's final hidden state and layerwise sliding-window attention cache from token t into token t+1, with no reset at prompt-response boundaries
- The architecture pairs a causal encoder with a recurrent decoder, using a gated merge between encoder representations and the previous decoder output, achieving unbounded temporal depth (48t structural depth after t tokens in the reference 48-layer config)
- Three core design principles: latent reasoning with unbounded temporal depth, model-hardware co-design with encoder parallelism and sequential decoder transitions, and model-RL algorithm co-design where pretraining, SFT, sampling, and RL replay share a single state transition
- The report is explicitly a design specification with no measured efficiency, reasoning quality, or scaling results — all remain open validation targets
- For RL replay, states are rebuilt from scratch under current parameters rather than reused from old rollouts, with behavior log-probabilities recorded during sampling preserved as importance-ratio denominators
Why It Matters
This proposal challenges the dominant decoder-only paradigm by introducing genuine recurrent state continuity across the entire sequence, which could enable more coherent multi-turn reasoning and longer-horizon latent computation without architectural resets. For practitioners working on agentic AI systems, the model-RL co-design and exact prefix snapshot protocol offer a principled framework for multi-turn interaction that avoids gradient path fragmentation — a common pain point in current fine-tuning pipelines.
Technical Details
- Architecture: RLT pairs a causal encoder (48 layers) with a recurrent decoder (48 layers), using tied weights. Each token executes 96 logical blocks. The decoder state Ht = (st, CtD) carries the final output st and layerwise SWA key-value cache CtD across all tokens, including across the prompt-response boundary with no reset
- Token computation: A gated merge combines encoder representation et with previous decoder output s_{t-1}, followed by causal SWA over decoder activations, cross-attention to encoder-derived memory M≤t, and an FFN. Memory groups can be shared (G=1) or layer-specific (G=L_D)
- Training: Full-sequence next-token prediction during pretraining with full backpropagation through time. During SFT, loss is masked to assistant targets but state updates are never masked — assistant losses backpropagate through user and tool tokens. The report warns that partial detaching is risky due to cross terms in the state-to-state Jacobian through the decoder KV cache
- RL co-design: The sampler records behavior log-probabilities under actual sampling distributions (including temperature and truncation). The trainer rebuilds encoder memory, recurrent output, and every SWA cache from the sequence start under current parameters before scoring each action; old rollout states are never reused
- Serving: Multi-turn exact prefix snapshots include encoder cache, memory, complete decoder state, position metadata, window convention, and model version. Weight updates invalidate old states, and editing a prefix forces recomputation from an earlier checkpoint
Industry Insight
- The unbounded temporal depth claim (state path growing as 48t blocks) could unlock more persistent "thought" trajectories in agentic workflows, but the explicit warning that gates and contraction may suppress long paths means practitioners should treat the depth-as-reasoning guarantee as aspirational until empirical validation arrives
- The model-RL algorithm co-design directly addresses a critical bottleneck in RLHF/RLAIF for conversational agents: the mismatch between prefill and decode kernels. By unifying state transitions across pretraining, SFT, sampling, and RL replay, RLT eliminates the gradient discontinuities that currently plague multi-turn fine-tuning
- The hardware implications are significant but conservative — encoder parallelism offers immediate wins, while the sequential decoder remains a scalability challenge. Teams should prioritize kernel development for batching across independent sequences rather than expecting parallel scan solutions
Disclaimer: The above content is generated by AI and is for reference only.