Multimodal Auto-regressive Transformer Surrogate for Modeling Variable Operations and Quantifying Uncertainty in Geological Carbon Storage
A multimodal auto-regressive transformer surrogate model was developed to model variable well perforation and injection strategies in geological carbon storage under geological uncertainty The architecture processes three input modalities (3D geomodel, relative permeability parameters, and control variables) through separate encoders, fused via self-attention, with a temporal decoder generating auto-regressive predictions Trained on 4000 GEOS flow simulations, the model achieves a median saturat
Analysis
TL;DR
- A multimodal auto-regressive transformer surrogate model was developed to model variable well perforation and injection strategies in geological carbon storage under geological uncertainty
- The architecture processes three input modalities (3D geomodel, relative permeability parameters, and control variables) through separate encoders, fused via self-attention, with a temporal decoder generating auto-regressive predictions
- Trained on 4000 GEOS flow simulations, the model achieves a median saturation MAE of 0.028 and relative errors of 0.2-5% on a held-out test set
- The surrogate successfully captures operational control switches (from rate to bottom-hole-pressure control), a critical real-world behavior
- Integrated into a hierarchical MCMC data assimilation framework, the model achieves substantial uncertainty reduction for key metaparameters, particularly fault permeabilities
Why It Matters
This work demonstrates the practical application of transformer-based surrogate modeling to a high-stakes climate technology domain—geological carbon storage—where accurate uncertainty quantification is essential for operational decision-making. The ability to rapidly predict subsurface behavior under variable injection strategies enables real-time optimization and risk assessment, bridging the gap between computationally expensive physics-based simulations and the fast inference needed for operational control.
Technical Details
- Architecture: Multimodal auto-regressive transformer with three separate encoders for (1) 3D geomodel, (2) scalar relative permeability parameters, and (3) control variables; modalities fused via self-attention in a transformer encoder; temporal decoder uses encoder-decoder cross-attention for auto-regressive prediction
- Application domain: Modified SEAM CO2 geomodel featuring a faulted system with three stacked aquifers and two injection wells perforated in stages from bottom to top
- Training data: 4000 GEOS flow simulations used for training; test set involves randomly sampled geomodels and control variables
- Predictions: Saturation and pressure at monitoring locations, total injected and mobile CO2 mass, and saturation footprints
- Data assimilation: Hierarchical Markov chain Monte Carlo (MCMC) procedure applied to a synthetic true model under three operational strategies, demonstrating posterior consistency with true model results
Industry Insight
- Transformer surrogates are proving effective for replacing expensive physics simulators in subsurface energy applications; practitioners should consider multimodal architectures when dealing with heterogeneous input types (spatial, scalar, temporal controls)
- The successful capture of control-mode switching (rate to BHP) highlights the importance of training surrogates on diverse operational regimes, not just steady-state conditions, to ensure robustness in real-world deployment
- Uncertainty quantification via surrogate-assisted MCMC offers a computationally tractable path for operational data assimilation in carbon storage, suggesting that similar frameworks could be adapted for other subsurface applications like hydrogen storage or geothermal energy
Disclaimer: The above content is generated by AI and is for reference only.