IMPROVING MULTI-VARIATE TIME SERIES FORECASTING WITH DYNAMIC MULTI-HEAD ATTENTION ADJACENCY MATRIX

Abstract

Time series data is prevalent in many fields, such as finance, weather forecasting, and economics. Predicting future values of a time series can offer valuable insights for decision-making, such as identifying trends, detecting anomalies, and improving resource allocation. Existing research, including neural network-based models and transformer-based models, has demonstrated high performance in learning temporal information. However, capturing spatial information within time series data remains a significant challenge. In this project, we explored whether the attention mechanism can be effectively integrated into non-transformer-based models to enhance their ability to learn spatial information. To achieve this goal, we propose a novel framework that uses a dynamically learned adjacency matrix based on related work called the Multi-variate Time-series Graph Neural Network(MTGNN). Instead of using a correlation-based learned adjacency matrix, the adjacency matrix and graph modules are replaced with a dynamically learned adjacency matrix with multi-attention. This framework shows that a dynamically learned attention adjacency matrix can perform as well as other frameworks when learning spatial information.