Let $G_t=\left(V,E,A,X_t\right)$, $t=1,2.\cdot \cdot \cdot T$, where $G_t$is a dynamic complex network, V and E are sets of nodes and edges respectively, $\left|V\right|=N$ is the number of nodes

in the graph, $A={\left(a_{ij}\right)}_{N\times N}$ is the adjacency matrix. If $v_i$, $v_j\in V$, $\left(v_i,v_j\right)\in E$, $a_{ij}=1$. Others $a_{ij}=0$. $X_t\in R$ is the matrix of characteristic attributes. It represents the traffic flow information of the traffic network in time period t, such as traffic volume, traffic speed, and traffic density. When constructing a traffic network, the nodes of the network can be determined according to the data flow acquisition method, for example, if the traffic flow data is obtained through multiple sensors distributed on the road network, the sensors are viewed as network nodes to construct the traffic network. If the traffic flow data is obtained through vehicle GPS, the intersection can be viewed as a network node to construct the traffic network, i.e., the network can be constructed by using the master method, or by using the pairwise method to construct the traffic network [26] .

Historical observations of the transportation network are known for the past p time periods $\left\{G_{t-p+1},G_{t-p+2},\cdot \cdot \cdot ,G_t\right\}$ are known to predict the transportation network data for the future q time periods $\left\{G_{t+1},G_{t+2},\cdot \cdot \cdot ,G_{t+q}\right\}$. The traffic flow prediction problem can be defined as:

$\left[G_{t+1},G_{t+2},\cdot \cdot \cdot ,G_{t+q}\right]=f\left(G_{t-p+1},G_{t-p+2},\cdot \cdot \cdot ,G_t\right)$ (1)

where f(g) is a learnable prediction function. When q = 1 is a single-step prediction and q ≥ 2 is a multi-step prediction.

In this paper, we propose a traffic prediction model based on autoencoder structure, whose core modules include Causal Convolutional Network (CCN), Multi-Head Self-Attention (MHSA) and Graph Convolutional Network (GCN), the overall framework of the model is shown in Figure 1 , and the specific steps of traffic flow prediction include:

Step 1: Input The traffic network is constructed by preprocessing the attribute features of the traffic network nodes (i.e., traffic speed).

Step 2: Encoder It consists of two causal convolution-pooling layers sequentially connected, each consisting of a causal convolution, an activation function SiLU, and a Dropout layer, which achieves dimensionality compression by pooling.

Step 3: MHSA For the features extracted by the encoder, the attention weights are adaptively assigned, the attention coefficients are computed, the importance of different time steps is learned, and the information of the time step that is most important for the current task is highlighted to capture long-term dependencies.

Step 4: GCN The output of the multi-attention mechanism is used as the input to the graph convolution, and the graph structure information is utilized to further extract features, and the feature representations of the nodes are updated by computing the information transfer between the nodes.

Step 5: Decoder Sequential connection of two causal convolution-pooling layers, recovering dimensionality by Up-pooling and using jump connections from encoder, allows the decoder to reuse shallow features extracted by the encoder in the feature reconstruction phase, preserving more detailed information of the original data. Specifically, the jump connections are implemented by addition: the second layer of the encoder is added to the first layer of the decoder, and the resulting output is fed into the second layer of the decoder; meanwhile, the first layer of the encoder is added to the second layer of the decoder. The output is passed through the fully connected layer to obtain the final prediction.

Autoencoder is a neural network model that learns an efficient representation of data through an unsupervised learning approach [27] . It usually consists of an encoder and a decoder. The encoder usually consists of a series of neural network layers that achieve data compression by gradually reducing the number of neurons, i.e., mapping the input data to a low-dimensional potential space. Decoder is constructed in contrast to an encoder and reconstructs the output similar to the input data by gradually increasing the number of neurons, i.e., mapping the potential space representation back to the original data space. The autoencoder construction is shown in Figure 2 .

Let X be the input data. Z is the coded representation of the input data (potential space). $\hat{X}$ is the reconstructed output (decoded data). W_e and W_d are the encoder and decoder weight matrices. b_e and b_d are bias vectors. The encoder and decoder functions are mathematically defined as:

$Z=f\left(X\right)=\sigma \left(W_{e}X+b_e\right)$ (2)

$\hat{X}=g\left(Z\right)=\sigma \left(W_{d}Z+b_d\right)$ (3)

where σ( ) is activation function.

Autoencoder solves two functions f and g. So that the input data and reconstructed output errors are minimized:

$f,g=\arg \underset{f,g}{\min }{\Vert X-\hat{X}\Vert }^2$ (4)

Causal Convolution Network [28] is a special convolution operation in convolutional neural network that follows a causal convolution operation in the time or sequence dimension. When processing sequence data, it ensures that when calculating the output of a certain moment, only the input information of that moment and its predecessors will be used, and not the input information of future moments, which avoids the problem of information leakage, and conforms to the causal logic of processing sequence data in practical applications. A schematic diagram of one-dimensional CCN computation is shown in Figure 3 .

Given an input one-dimensional time series $X=\left[x_1,x_2,x_3,\cdot \cdot \cdot ,x_n\right]$ and $F=\left[f_1,f_2,f_3,\cdot \cdot \cdot ,f_k\right]$ is the convolution kernel. The causal convolution operation F is defined as:

${\left(F\ast X\right)}_t={\displaystyle \underset{k=1}{\overset{K-1}{\sum }}{f}_k{x}_t}-K+k$ (5)

where * denotes the dilation convolution operator.

Transportation time series data has both time dimension and spatial dimension information, so it is processed with two-dimensional causal convolution. Let the input feature map be $X={\left(X_{i,j}\right)}_{T\times S}$, where T is the length of the time dimension and S is the length of the space dimension. The convolution kernel is $K={\left(K_{i,j}\right)}_{K_t\times K_s}$, where K_t is the convolutional kernel time dimension and K_s is the convolutional kernel space dimension. The bias is b ∈ R. To ensure causality, zero-padding is performed from the beginning of the time dimension. Suppose that the time dimension is populated with $p_t=K_{t-1}$ zeros. The filled feature map is

$\tilde{X}={\left({\tilde{x}}_{i,j}\right)}_{\left(T+K_{t-1}\right)}^S$. The two-dimensional causal convolution operation is:

${\left(K\ast X\right)}_{\left(t,s\right)}={\displaystyle \underset{i=1}{\overset{K_t}{\sum }}{\displaystyle \underset{j=1}{\overset{K_s}{\sum }}{k}_{i,j}}}{\tilde{x}}_{t+i,s+j}+b$ (6)

where * is the convolution operation. When $t+1>T+K_{t-1}$ or s + j > S, ${\tilde{x}}_{t+i,s+j}=0$.

The convolution operation is followed by a nonlinear transformation through activation functions, and the commonly used activation functions include Sigmoid, ReLU, SiLU, Tanh, and so on. In order to avoid overfitting the model on the training data, which leads to poor performance on the test data, the data is processed by the Dropout layer after the activation function. A portion of neurons and their connections are randomly dropped during the training process so that the model cannot rely on a specific combination of neurons, thus reducing the co-adaptation between neurons, forcing the network to learn more robust feature representations, improving the model’s generalization ability, and reducing the risk of overfitting.

The traffic flow of a node is not only affected by the historical traffic flow of that node, but also by the traffic flow of its neighboring nodes. Therefore, a spatio-temporal attention mechanism can be used to capture the dynamic spatio-temporal correlation of traffic flow, where the spatial attention mechanism capture the dynamic spatial correlation between a node and its neighboring nodes, and the temporal attention mechanism capture the dynamic temporal correlation between nodes at different times.

The self-attention mechanism portrays the interdependence between sequence elements by calculating the correlation (i.e., weight) of each element in the sequence with all other elements. Multi-Head Self-Attention (MHSA) [29] capture the distribution of attention in time series data in different subspaces from multiple perspectives by introducing multiple heads of attention, thus enabling a better understanding of the interactions between variables in complex series data, and improving the model representation and generalization capabilities.

The steps to achieve MHSA include:

Step 1: Input transformations The input sequence X is passed through three different linear transformations to obtain the query vector Q, the key vector K and the value vector V: Q = XW_Q, K = XW_K, V = XW_V, where W_Q, W_K, W_V are the learnable weight matrices of Q, K, V.

Step 2: Parallel computation of multiple attentions For each head, the attention score was computed separately using dot product attention:

${\text{Head}}_i=\text{Attention}\left(Q_i,K_i,V{}_i\right)=\text{SoftMa}x\frac{Q_i{K}_i^T}{\sqrt{d_k}}{V}_i$ (7)

where d_k is the embedding dimension.

Step 3: Splicing the attention heads and linearly transforming them output The spliced vectors are passed through a linear transformation to get the final multi-head attention data:

$\text{MultiHead}\left(Q,K,V\right)=\text{Concat}\left({\text{head}}_1,\cdot \cdot \cdot ,{\text{head}}_h\right)W_o$ (8)

where W_o is the output weight matrix and h is the number of attention heads. The structure of MHSA is shown in Figure 4 .

Graph convolutional network [30] is a neural network structure for deep learning. Compared with the traditional network model CNN for deep learning, graph convolutional network is able to handle spatio-temporal data with non-Euclidean structure. Traffic road network is a typical complex network structure, and its network characteristics are inscribed with non-Euclidean structure data, using graph convolutional network can extract the topological relationship between nodes and their surrounding nodes in the complex network, so as to accurately extract the spatial characteristics of the traffic data, and lay the foundation for improving the accuracy of traffic speed prediction.

Graph convolution usually includes null-domain convolution and spectral-domain convolution, the core of null-domain convolution is the aggregation process of neighboring node feature and inference to get the prediction, such as GraphSage network (2017). Spectral domain convolution is network learning by neighbor matrix and node attribute features, such as Graph Convolutional Network (GCN). In this paper, we use GCN to construct a deep learning network. In the spectral domain map convolution, we convolve the graph G with the convolution kernel g_θ:

$g_{\theta }\ast {}_{G}X=g_{\theta }\left(L\right)X=g_{\theta }\left(U\Lambda U^T\right)X=U{g}_{\theta }\left(\Lambda \right)U^{T}X$ (9)

where ^∗G is the graph convolution operation. Laplace $L=D^{-\frac{1}{2}}\left(D-A\right)D^{-\frac{1}{2}}=I_N-D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}$, $D\in R^{N\times N}$ is the degree matrix, $I_N\in R^{N\times N}$ is the unit matrix and X is the attribute feature matrix of the graph. The eigenvalue decomposition of L is $L=U\Lambda U^T$, where Λ is the diagonal matrix consisting of the eigenvalues of L and $U=\left\{u_1,u_2,\cdot \cdot \cdot u_N\right\}$ consists of the eigenvectors of L.

Due to the large structure of the traffic road network graph, the computation of the Laplace transform on the graph usually involves complex operations such as eigen-decomposition of the Laplace matrix of the graph, which is computationally expensive. To avoid higher computational complexity, Chebyshev polynomials are used to obtain an efficient approximation, so the graph convolution can be rewritten as:

$g_{\theta }\ast {}_{G}X\approx {\displaystyle \underset{k=0}{\overset{K-1}{\sum }}{\theta }_k}{T}_k\left(\tilde{L}\right)X$ (10)

where θ ∈ R^K is a vector of polynomial coefficients and K is the size of the graph convolution kernel. $T_k\left(\tilde{L}\right)\in R^{N\times N}$ is a term of order k of the Chebyshev polynomial, which satisfies the recurrence formula: $T_k\left(X\right)=2X{T}_{k-1}\left(X\right)-T_{k-2}\left(X\right)$,

T₀(X) = 1, T₁(X) = X, $\tilde{L}=\frac{2L}{{\lambda }_{\max }}-I_N$, ${\lambda }_{\max }$ is the largest eigenvalue of L.

In this paper, Huberloss is chosen as the loss function for training, and the use of MSE for small errors and MAE for large errors is able to overcome the drawbacks of both by smoothing out the training process as well as minimizing the effect of outliers. The mathematical formula of Huberloss is defined as follows:

$L_{\delta }\left(y,\hat{y}\right)=\{\begin{array}{l}\frac{1}{2}{\left(y-\hat{y}\right)}^2,\left|y-\hat{y}\right|\le \delta \\ \delta \left|y-\hat{y}\right|-\frac{1}{2}{\delta }^2,\text{others}\end{array}$ (11)

where y is the true value, yˆ is the predicted value, δ is the threshold value. (In this paper, δ = 1.)

In this paper, the validity of the model is tested using the PEMSD7M and Traffic SH datasets. PEMSD7M ( http://pems.dot.ca.gov ) were collected and provided by the Performance Measurement System (PeMS) of the California Department of Transportation. The dataset was selected from 228 sensor nodes from the transportation roadway network, and monitoring data was selected as a sample for the period from May 1, 2017 to August 31, 2017, with data collected at a frequency of every 5 minutes. TrafficSH selected 896 sensor nodes in Shanghai from March 5, 2022 to April 5, 2022, and the frequency of data collection was every 30 minutes. The data collected by each sensor each time includes features in three dimensions: flow rate, average speed and average occupancy. For the missing values present in the datasets, this paper employs missing value filling using periodic features of traffic flow. Normalization is done using Z-Score.

70% of the datasets was selected as the training set, 15% as the test set, and 15% as the validation set. Multi-step prediction is performed using historical 12-step traffic speed data, including future 3, 6, 12-step prediction. The model use AdamW optimizer with a learning rate of 0.001. The batch-size is 32 and the training period is 500 with an early stop mechanism. All experiments were performed on a machine equipped with NVIDIA GeForce 3080 Ti GPU and 128GB of RAM. Models were implemented using python 3.10.0.

Commonly used assessment metrics for traffic flow forecasting include Mean Absolute Error(MAE), Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error(MAPE). The definitions of the three assessment metrics are presented as follows:

1) Mean Absolute Error (MAE)

$\text{MAE}=\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}\left|y_i-{\hat{y}}_i\right|}$ (12)

where y_i is the true value, ${\hat{y}}_i$ is the predicted value and n is the number of the records. The following formulas have the same meaning. The range of the MAE error is in the interval [0, +∞). The MAE value is taken to be 0 when the predicted and true values match exactly, and furthermore, the large the error, the large the value.

2) Root Mean Squared Error (RMSE)

$\text{RMSE}=\sqrt{\frac{1}{n}{{\displaystyle \underset{i=1}{\overset{n}{\sum }}\left(y_i-{\hat{y}}_i\right)}}^2}$ (13)

The range of the RMAE error is in the interval [0, +∞). Similar to the MAE, the RMSE value is taken to be 0 when the predicted and true values match exactly, and the large the error, the large the value.

3) Mean Absolute Percentage Error (MAPE)

$MAPE=\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}\left|\frac{y_i-{\hat{y}}_i}{y_i}\right|}\times 100\%$ (14)

The range of the MAPE error is in the interval [0, +∞). MAPE is presented as a percentage, with a value of 0% indicating a perfect model and a value greater than 100% indicating a poor model.

In this paper, we use History Average Model (HA), Feedforward Neural Network (FNN), Spatiotemporal Graph Convolutional Network (STGCN), Diffusion Convolutional Recurrent Neural Network (DCRNN), and Graph Multi-Attention Network (GMAN) for the comparison of the prediction performance with the proposed model (Ours).

HA [2] : A simple and basic time series prediction model, usually used to forecast data with cyclical or seasonal patterns. Its core assumption is that the data has a cyclical pattern of change in time. Based on this assumption, the model predicts future values by calculating the average value over the same time period in history.

FNN [8] : It is characterized by a unidirectional transfer of information from the input layer to the output layer, with a number of hidden layer in between, and the neurons in each layer are nonlinearly transformed by weighted sums and activation functions.

STGCN [16] : A deep learning model combining Graph Convolutional Network (GCN) and Convolutional Neural Network (CNN) is specialized to deal with spatio-temporal relationships in graph-structure data. It captures spatial dependencies by mapping time-series data onto graphs using graph convolution, while combining it with temporal convolution to handle temporal dependencies.

DCRNN [24] : It models the dynamics of traffic flow as a directed weighted graph and proposes diffusion convolution operations to capture spatial dependencies. The upstream and downstream traffic impacts are captured flexibly by incorporating bi-directional random wandering in the convolution.

GMAN [25] : The model uses an encoder-decoder structure, where the encoder and decoder consist of multiple spatio-temporal attention blocks, each consisting of a spatial attention mechanism, a temporal attention mechanism, and a gated fusion mechanism.

Figure 5 shows the comparison of traffic speed prediction between the five baseline models and the model proposed in this paper (Ours), with the 24 hours prediction results from a randomly selected sensor visualized. Table 1 shows the prediction performance metrics of each model, where black bold indicates the metrics with the best results, and underlining marks the metrics with the second highest results.

From Table 1 , it can be seen that the HA and FNN models cannot well explore the mutual influence of traffic speeds between different road sections and at different time points, and lack of consideration of the spatial structure of the complex road network and the temporal correlation of traffic flow, and the prediction performances of these two models on both datasets are poorer. The STGCN, DCRNN, and GMAN models extract the features of the data in both the temporal and spatial dimension, and achieve better results on the two datasets. Both datasets with better results. Compared with the baseline model, the performance of the proposed model in this paper performs optimally. On the dataset TrafficSH, the values of three error metrics such as MAE, RMSE and MAPE are minimized for the multi-step prediction of the proposed model in this paper. On the dataset PEMSD7M, the value of other metrics are also minimized, except for the MAE of 6-step and 12-step, which are ranked second. The results fully demonstrate that the prediction model in this paper is able to capture the spatio-temporal characteristics of traffic flow simultaneously and has excellent prediction performance for traffic speed.

We validate the effect of each component of the model on the prediction effect using three ablation experiments, where Model A indicates replacing the causal convolution with a normal two-dimensional convolution, and Model B indicates using the causal convolution and removing the temporal self-attention mechanism. Model C indicates using causal convolution and removing graph convolution. Table 2 shows the performance comparison of the ablation experiments.

According to Table 2 , it can be seen that after replacing causal convolution with ordinary 2D convolution, the error metrics of multi-step prediction have increased, which indicates that compared with causal convolution, ordinary 2D convolution is unable to capture the temporal features of traffic flow. After Model B removes the temporal self-attention mechanism, the model is unable to identify important time points and subsequences in the traffic sequence, making the model prediction accuracy decrease. Graph convolution is more important to capture the spatial or structural dependence of the data, and the removal of graph convolution by Model C leads to a decrease in the model’s ability to extract local features, which in turn affects the prediction accuracy.