Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation

Background

Neural operators have emerged as powerful tools for learning mappings between function spaces, enabling fast inference for scientific simulations. However, they often suffer from spectral bias, a tendency to preferentially learn low-frequency solution components while struggling with high-frequency details. This limitation degrades performance for complex physical systems like turbulence, where high-frequency dynamics are critical. Traditional approaches treat inference as a monolithic, single-pass process, offering no mechanism to iteratively correct residual errors, particularly in the high-frequency spectrum.

Key Points

The IRNO framework proposes a fundamental architectural shift from a single-pass to an iterative refinement paradigm.

• Core Mechanism: It augments a pre-trained base operator with a learned refinement module. The process is structured as a coarse-to-fine decomposition: the base operator provides an initial coarse prediction, and the refinement module is applied iteratively to correct residuals.
• Theoretical Foundation: The refinement process is modeled as a fixed-point iteration. Under local assumptions, the authors establish a contraction mapping property for the induced operator, guaranteeing convergence to a unique fixed point. This provides a theoretical stability guarantee analogous to classical iterative solvers.
• Targeted Training: To directly combat spectral bias, a progressive spectral loss is introduced during training. This loss function adaptively increases the penalty on high-frequency components as the refinement steps progress, explicitly guiding the model to focus on correcting high-frequency errors in later iterations.
• Empirical Validation: Testing across complex physical systems (turbulent flow, active matter) shows:
  • Consistent performance gains, with error reduction up to 56.05% on turbulent flow.
  • Spectral error analysis on active matter reveals that IRNO reduces normalized error ratios relative to the base operator across all frequency bands, with the most pronounced improvement in high frequencies (error reduced to 1.48-2.04% of the base operator's error).
  • The model's improvements generalize beyond the trained iteration count, indicating stable learned representations.

Significance

IRNO represents a significant methodological advance in scientific machine learning by bridging the gap between classical numerical analysis and deep learning.

• It introduces a principled iterative framework into neural operators, moving beyond one-shot prediction toward a process that mimics the residual correction of traditional solvers.
• The progressive spectral loss is a novel training objective that directly targets the known weakness of spectral bias, moving beyond generic loss functions.
• The demonstration of robust high-frequency accuracy and stability has critical implications for deploying neural surrogates in applications where detail resolution is paramount, such as predicting small-scale features in turbulence or collective behaviors in active matter. The provided code facilitates adoption and further research.