M3-Former: Multimodal Transformer with Mixture-of-Experts for Long-Term Vessel Trajectory Prediction
M3-Former is a multimodal trajectory prediction framework that integrates LLM-encoded semantic priors (vessel static attributes and navigational intent) with dynamic trajectory features via self-attention alignment A dual-granularity Mixture-of-Experts architecture is introduced, with sequence-level experts modeling global navigation trends and token-level experts refining fine-grained maneuvering behaviors A novel Steering-Weighted Cross-Entropy loss addresses the long-tail distribution of spar
Analysis
TL;DR
- M3-Former is a multimodal trajectory prediction framework that integrates LLM-encoded semantic priors (vessel static attributes and navigational intent) with dynamic trajectory features via self-attention alignment
- A dual-granularity Mixture-of-Experts architecture is introduced, with sequence-level experts modeling global navigation trends and token-level experts refining fine-grained maneuvering behaviors
- A novel Steering-Weighted Cross-Entropy loss addresses the long-tail distribution of sparse turning samples, improving accuracy in critical maneuvering scenarios
- Evaluated on a real-world Danish AIS dataset, M3-Former outperforms SOTA baselines across 1–4 hour prediction horizons, reducing ADE by 4.4% and FDE by 5.1% at the 4-hour mark
- Semantic fusion effectively reduces long-term trajectory drift, while the dual-granularity MoE enhances robustness in complex waterways and route-branching scenarios
Why It Matters
This work demonstrates a compelling integration of large language models into spatiotemporal trajectory prediction, showing that semantic priors can meaningfully reduce error accumulation over long forecasting horizons—a persistent challenge in autonomous navigation systems. The dual-granularity MoE design offers a scalable architectural pattern that could generalize beyond maritime domains to any long-horizon prediction task requiring both global planning and local refinement.
Technical Details
- LLM-Enhanced Semantic Fusion: Vessel static attributes and navigational intent are encoded by a pre-trained LLM and aligned with dynamic trajectory features in a unified multimodal representation space through self-attention mechanisms
- Dual-Granularity MoE Architecture: Sequence-level experts capture global route planning and long-term navigation trends, while token-level experts handle fine-grained local motion variations and maneuvering behaviors
- Steering-Weighted Cross-Entropy Loss: A custom loss function that reweights turning samples to mitigate the long-tail distribution problem, directly targeting improved prediction accuracy during critical maneuvering events
- Benchmark & Evaluation: Tested on a real-world Danish AIS dataset with prediction horizons from 1 to 4 hours; metrics include Average Displacement Error (ADE) and Final Displacement Error (FDE), with ablation studies confirming the contribution of each component
Industry Insight
- The semantic-guided hierarchical prediction paradigm established here is directly transferable to other long-horizon trajectory prediction domains such as autonomous driving, drone navigation, and robotics, where combining high-level intent with low-level dynamics remains an open challenge
- The dual-granularity MoE approach offers a practical blueprint for scaling prediction models without proportionally increasing computational cost, as experts can be sparsely activated based on input context
- Maritime AI safety applications—particularly collision avoidance and port automation—stand to benefit immediately from the demonstrated improvements in long-term prediction accuracy, especially in complex waterway environments where traditional models struggle with route-branching scenarios
Disclaimer: The above content is generated by AI and is for reference only.