Researchers introduce the Focal Modulated Attention Encoder (FATE), a novel transformer architecture designed to accurately forecast multivariate time-series data by explicitly capturing complex spatiotemporal correlations. FATE outperforms existing state-of-the-art methods across
Addressing the critical need for accurate forecasting in complex systems like climate change monitoring, this research presents FATE (Focal Modulated Attention Encoder), a new transformer architecture. Unlike conventional models that struggle with sequential dependencies and parallelization in long-horizon multivariate datasets, FATE employs a novel tensorized focal modulation mechanism. This mechanism explicitly captures spatiotemporal correlations within the time-series data. The model also introduces interpretable modulation scores that highlight the most critical environmental features influencing predictions. Benchmarked across seven diverse real-world datasets (including weather and traffic data), FATE consistently outperformed all state-of-the-art methods, demonstrating its strong generalization capability for complex multivariate forecasting tasks.