Feature Interaction Modeling for Physics-Informed Neural Networks and Neural Operators
Embeds factorization machine (FM) feature interaction modules into physics-informed neural networks (PINNs) and neural operator learning to improve expressiveness for parameterized PDE solution manifolds Proposes FM-PINN, which explicitly captures spatio-temporal variable interactions using a second-order Taylor expansion motivation, improving approximation accuracy for smooth high-order PDEs Develops FM-Operator and FM-DeepONet by grouping spatial coordinates, time, physical parameters, and bou
Analysis
TL;DR
- Embeds factorization machine (FM) feature interaction modules into physics-informed neural networks (PINNs) and neural operator learning to improve expressiveness for parameterized PDE solution manifolds
- Proposes FM-PINN, which explicitly captures spatio-temporal variable interactions using a second-order Taylor expansion motivation, improving approximation accuracy for smooth high-order PDEs
- Develops FM-Operator and FM-DeepONet by grouping spatial coordinates, time, physical parameters, and boundary/initial conditions into independent feature sets and modeling cross-group interactions
- Demonstrates substantial accuracy gains on shock-dominated nonlinear conservation laws and problems with sharp gradients or discontinuities
- Shows no consistent advantage on smooth operator learning benchmarks, indicating the approach is specifically suited for challenging, non-smooth PDE regimes
Why It Matters
This work bridges a critical gap in physics-informed machine learning by introducing explicit feature interaction modeling into PINNs and neural operators, which traditionally rely on implicit neural representations that struggle with complex variable couplings. For practitioners working on PDE-constrained problems involving shocks, discontinuities, or strong cross-field dependencies—common in fluid dynamics, climate modeling, and engineering simulations—this approach offers a principled architectural enhancement that can significantly improve solution accuracy without sacrificing physical consistency.
Technical Details
- FM-PINN: Integrates factorization machine second-order interaction terms into the PINN architecture, enabling explicit modeling of spatio-temporal variable couplings. The design is motivated by the second-order Taylor expansion of multivariate functions, providing a theoretically grounded approach to capturing variable interactions that standard MLPs approximate only implicitly.
- FM-Operator and FM-DeepONet: Extends the FM interaction paradigm to neural operator frameworks by partitioning inputs into independent feature sets—spatial coordinates, time, physical parameters, and initial/boundary conditions—and explicitly modeling cross-group interactions through factorized pairwise terms.
- Target problem class: Parameterized partial differential equations (PDEs), with emphasis on nonlinear conservation laws, shock-dominated equations, and problems exhibiting sharp gradients or discontinuities.
- Benchmark findings: Numerical tests show substantial accuracy improvements on challenging shock-dominated equations, while smooth operator learning benchmarks see no consistent advantage, suggesting the FM interaction modules primarily address hardness arising from non-smooth solution manifolds.
- Theoretical motivation: The factorization machine's pairwise interaction term serves as an efficient approximation of second-order cross-variable dependencies, avoiding the O(d²) explicit computation through learned latent vectors.
Industry Insight
- The selective effectiveness of FM modules—strong gains on non-smooth problems but no advantage on smooth benchmarks—suggests practitioners should adopt a diagnostic-driven approach: deploy FM-enhanced architectures primarily for shock-capturing, discontinuity-rich, or strongly coupled multi-physics problems rather than as a universal upgrade.
- Grouping features into semantically meaningful sets (spatial, temporal, parametric, boundary) and modeling cross-group interactions represents a generalizable design principle that could be extended to other scientific ML domains involving heterogeneous input modalities, such as climate-chemistry coupling or multi-scale material modeling.
- As physics-informed neural operators face increasing scrutiny on real-world deployment, this work highlights that architectural inductive biases—rather than raw capacity—are key to handling the hardest regimes in PDE solution manifolds, pointing toward hybrid designs that combine implicit neural representations with explicit interaction modeling.
Disclaimer: The above content is generated by AI and is for reference only.