Baidu's 'Unlimited OCR' processes dozens of document pages in one pass by treating memory like human forgetting
Baidu introduces "Unlimited OCR," a model capable of processing dozens of document pages in a single inference pass by maintaining constant memory usage and speed regardless of text length. The core innovation is Reference Sliding Window Attention (R-SWA), which fixes the KV cache size by limiting attention to the last 128 generated tokens while preserving full access to visual reference tokens. Built on the DeepSeek OCR foundation, the model achieves 93% accuracy on OmniDocBench v1.5, outperfor
Analysis
TL;DR
- Baidu introduces "Unlimited OCR," a model capable of processing dozens of document pages in a single inference pass by maintaining constant memory usage and speed regardless of text length.
- The core innovation is Reference Sliding Window Attention (R-SWA), which fixes the KV cache size by limiting attention to the last 128 generated tokens while preserving full access to visual reference tokens.
- Built on the DeepSeek OCR foundation, the model achieves 93% accuracy on OmniDocBench v1.5, outperforming the baseline by six percentage points, particularly in table structure recognition.
- R-SWA prevents the degradation of visual features caused by standard sliding windows and reduces kernel latency, offering a 12.7% speed increase in Base mode compared to previous systems.
- While currently limited by a 32,000-token context window, the architecture demonstrates significant potential for extending language model memory efficiency to other reference-based tasks like speech recognition and translation.
Why It Matters
This development addresses a critical scalability bottleneck in end-to-end OCR systems, where traditional language model decoders suffer from linear memory growth and slowing generation speeds as document length increases. By decoupling visual reference retention from output history management, Baidu enables efficient processing of long documents without the computational overhead of page-by-page looping, setting a new standard for high-throughput document understanding.
Technical Details
- Reference Sliding Window Attention (R-SWA): Unlike standard sliding window attention that shifts all tokens, R-SWA treats visual tokens as static references that remain fully accessible, while the KV cache for generated text acts as a fixed-length queue (last 128 tokens). This prevents the blurring of image features and keeps memory usage constant.
- Architecture: The model utilizes a frozen DeepEncoder to compress 1024x1024 pixel images into 256 tokens, paired with a Mixture-of-Experts (MoE) decoder containing 3 billion parameters (500 million active). All standard attention layers in the decoder are replaced with R-SWA.
- Training Data & Setup: Trained on approximately two million document samples (90% single-page, 10% multi-page synthetic stitched documents) using Paddle OCR annotations. Training occurred over 4,000 steps on eight Nvidia A800 GPUs with sequences packed to 32,000 tokens.
- Performance Metrics: Achieves 93% overall accuracy on OmniDocBench v1.5 and 93.92% on v1.6. In long-horizon tests exceeding 40 pages, the edit distance remains below 0.11. Throughput reaches 5,580 tokens/second in Base mode, surpassing the DeepSeek OCR baseline.
Industry Insight
- Efficiency Over Scale: The success of R-SWA suggests that future multimodal models should prioritize architectural innovations that manage context windows efficiently rather than simply increasing parameter counts, especially for tasks involving long-form visual-text alignment.
- Cross-Domain Applicability: The technique of separating static reference inputs from dynamic output histories is likely to be adopted in other domains requiring long-context processing, such as continuous speech recognition or translation of lengthy documents, reducing inference costs significantly.
- Data Strategy Implications: As OCR becomes more efficient, the ability to rapidly digitize and parse large volumes of physical documents will accelerate the creation of high-quality training corpora, potentially intensifying debates around copyright and data sourcing for large language models.
Disclaimer: The above content is generated by AI and is for reference only.