Seen in the full project workflow, Figure 1 , this study utilized the Extended Reconstructed Sea Surface Temperature version 5 (ERSSTv5) dataset, which provides global sea surface temperature (SST) anomalies. ERSSTv5 is a widely used dataset known for its accuracy in long-term climate studies, offering monthly SST data with a spatial resolution of 2˚ × 2˚ from 1854 to present ( [15] ). This study used a time range of January 1949 to December 2014, of length 792 months. This dataset was chosen due to its comprehensive coverage and its relevance widespread in ENSO—related climate modeling studies and analyzing SST anomalies, which are critical in understanding and predicting ENSO events.

To ensure quality analysis, the image data was centered around the most relevant area for ENSO-related activity, the tropical Pacific region from the ERSSTv5 dataset. This region is crucial for detecting and understanding the sea surface temperature (SST) anomalies that are indicative of El Niño and La Niña events, as it encompasses the key zones where these phenomena typically manifest to the greatest extent ( [15] , Figure 2 ). This focus allowed for a more detailed and accurate analysis of the SST patterns associated with ENSO variability.

The SST anomaly was then standardized using scikit-learn’s StandardScalar, which normalizes feature values by subtracting the sample mean and dividing by the sample standard deviation:

$z_i=\frac{x_i-m}{s}.$ (1)

In addition to SST data, the Oceanic Niño Index (ONI) was selected as a key indicator for identifying ENSO phases. The ONI is derived from SST anomalies in the Niño 3.4 region ( Figure 2 , [16] ), covering the central equatorial Pacific [2] . It is the standard metric used to classify ENSO events, with thresholds set at ±0.5˚C deviations from the average. Given that the goal of the study was to accurately forecast ENSO events, the target dataset was lagged 4 months ahead of time for the model to learn precursory patterns indicative of ensuing ENSO events.

M-band orthogonal discrete wavelet transforms (DWT) are utilized to decompose each image in the dataset into multiple frequency bands, enhancing the model’s ability to analyze various levels of detail. An M-band DWT is determined by a filter bank consisting of M filters ( $M\ge 2$), including a low-pass filter $\alpha$ and $M-1$ high-pass filters $\beta \left(j\right)$ for $j=1,\cdots ,M-1$ ( [5] [14] ). This decomposition allows for the extraction of both fine and coarse features, capturing multi-scale spatial and temporal characteristics essential for ENSO prediction.

In this study, the M-band DWT used prior to deep learning models to capture patterns across different scales. The multi-scale approach facilitated by the M-band DWT will later allow the use of a combined detail model with parallel computing, to overall significantly boost the computational efficiency and accuracy of the predictive model ( [17] ).

An M-band orthogonal wavelet is considered N-regular if it has N vanishing moments; hence wavelets with more vanishing moments tend to be smoother with longer filters.

$2\underset{i=1}{\overset{n}{{\displaystyle \sum }}}{\alpha }_i=\sqrt{M}$ (2)

$\underset{i=1}{\overset{n}{{\displaystyle \sum }}}{i}^k{\beta }_i^{\left(j\right)}=0\text{for}k=0,\cdots ,N-1;j=1,\cdots ,M-1$ (3)

$\alpha ={\beta }^{\left(j\right)}=1\text{for}j=1,\cdots ,M-1$ (4)

$\alpha {\beta }^{\left(j\right)}=0\text{for}j=1,\cdots ,M-1$ (5)

${\beta }^{\left(i\right)}{\beta }^{\left(j\right)}=0\text{for}i,j=1,\cdots ,M-1\text{and}i\ne j$ (6)

For adequate integer values of k, an M-band wavelet can be used to create an Mk × Mk wavelet transform matrix to decompose channels of a dimension.

An example 3-band DWT matrix is

$W=\left[\begin{array}{cccccccccccc}{\alpha }_1& {\alpha }_2& {\alpha }_3& {\alpha }_4& {\alpha }_5& {\alpha }_6& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& {\alpha }_1& {\alpha }_2& {\alpha }_3& {\alpha }_4& {\alpha }_5& {\alpha }_6& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& {\alpha }_1& {\alpha }_2& {\alpha }_3& {\alpha }_4& {\alpha }_5& {\alpha }_6\\ {\alpha }_4& {\alpha }_5& {\alpha }_6& 0& 0& 0& 0& 0& 0& {\alpha }_1& {\alpha }_2& {\alpha }_3\\ {\beta }_1& {\beta }_2& {\beta }_3& {\beta }_4& {\beta }_5& {\beta }_6& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& {\beta }_1& {\beta }_2& {\beta }_3& {\beta }_4& {\beta }_5& {\beta }_6& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& {\beta }_1& {\beta }_2& {\beta }_3& {\beta }_4& {\beta }_5& {\beta }_6\\ {\beta }_4& {\beta }_5& {\beta }_6& 0& 0& 0& 0& 0& 0& {\beta }_1& {\beta }_2& {\beta }_3\\ {\gamma }_1& {\gamma }_2& {\gamma }_3& {\gamma }_4& {\gamma }_5& {\gamma }_6& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& {\gamma }_1& {\gamma }_2& {\gamma }_3& {\gamma }_4& {\gamma }_5& {\gamma }_6& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& {\gamma }_1& {\gamma }_2& {\gamma }_3& {\gamma }_4& {\gamma }_5& {\gamma }_6\\ {\gamma }_4& {\gamma }_5& {\gamma }_6& 0& 0& 0& 0& 0& 0& {\gamma }_1& {\gamma }_2& {\gamma }_3\end{array}\right]$

The filters for said matrix being

$\alpha =\left[\begin{array}{c}0.3384\\ 0.5308\\ 0.7233\\ 0.2390\\ 0.0465\\ -0.1459\end{array}\right],\beta =\left[\begin{array}{c}-0.1174\\ 0.5443\\ -0.0187\\ -0.6991\\ -0.1361\\ 0.4270\end{array}\right],\gamma =\left[\begin{array}{c}0.4036\\ -0.6285\\ 0.4606\\ -0.4036\\ -0.0786\\ 0.2465\end{array}\right].$

It can be proven that all the DWT matrices we are using are orthogonal, i.e., $W=W$ for all such DWT matrices W ( [18] ).

The 2D DWT of an channel X is defined as the matrix WXW. This decomposition can be seen as an M × M block matrix where the top-left submatrix is the approximation (low frequency) component, and the other submatrices are detail (high frequency) components.

For example, a 2D 3-band DWT decomposes a channel X into a block matrix

$WXW=\left[\begin{array}{ccc}A& D_{12}& D_{13}\\ D_{21}& D_{22}& D_{23}\\ D_{31}& D_{32}& D_{33}\end{array}\right]$

where A is the approximation, or low frequency, component and $D_{ij}$ are detail, or high frequency, components ( [19] ).

Figure 3 is a 88 × 180 (clipped for multiples of 4) pixel image of the mean SST anomalies of the world under Extended Reconstructed Sea Surface Temperatures Version 5 in October of 1949.

To pre-process the data, a 4-band 4-regular wavelet filter bank derived from [18] was used across each image of the 792 months in the SST anomaly time series dataset. As such, an approximation and 15 detail components were generated, each now of pixel size 22 × 45 ( Figure 4 ).

Figure 4. Results of wavelet decomposition using the 4-band 4-regular wavelet. The cold tongue in the Tropical Pacific in the approximation (top left, subfigure 4(a)) remains noticeable as seen in the original, Figure 3 . Because we applied a 4-band DWT to the image, the image is decomposed into 16 sub-images. The pixel values of detail components are very small compared to the approximation, so the subfigure 4(b) is the same as the subfigure 4(a) but with a smaller color gradient.

In addition to its usage in pre-processing the image data, the DWT was also applied to the target ONI index dataset for the purpose of denoising ( [20] ).

Given a signal $S\in {ℝ}^n$,

W = 4-band 4-regular wavelet filter bank:

For each detail coefficient $d_i\left(i=1,2,3\right)$:

$\tilde{S}=W\times S=\left(\begin{array}{c}a\\ d_1\\ d_2\\ d_3\end{array}\right)$ (7)

$a,d_1,d_2,d_3\in {ℝ}^{\frac{n}{4}}$

$\text{Threshold}{\delta }_i={\sigma }_i\sqrt{2\text{log}\left(\frac{n}{4}\right)}\text{for}i=1,2,3$ (8)

where ${\sigma }_i$ is the standard deviation of the elements of $d_i$.

$d_i^{\text{*}}=\left(\begin{array}{l}{d}_{11}^{\text{*}}\hfill \\ ⋮\hfill \\ d_{1\frac{n}{4}}^{\text{*}}\hfill \end{array}\right)\text{where}{d}_i^{\text{*}}=\{\begin{array}{ll}{d}_{ij},\hfill & \text{if}\left|d_{ij}\right|\ge {\delta }_i,\hfill \\ 0,\hfill & \text{if}\left|d_{ij}\right|<{\delta }_i\hfill \end{array}\text{for}i=1,2,3\text{and}j=1,\cdots ,\frac{n}{4}.$ (9)

${\tilde{S}}^{*}=\left(\begin{array}{c}a\\ d_1^{*}\\ d_2^{*}\\ d_3^{*}\end{array}\right)$ (10)

$S^{\text{*}}=W^t\times {\tilde{S}}^{\text{*}}$ (where $S^{\text{*}}$ is the denoised signal) (11)

This approach was beneficial for removing any unexpected anomalies in the data that may have been caused by fluctuations in various other variables in the global climate—seen in Figure 5 .

Autoencoders, particularly those paired with LSTM networks, are effective for dimensionality reduction in time-series data to capture critical patterns stored in the SST anomaly data. Autoencoders will output multiple nodes or channels based on trends observed in the original dataset, each with different configurations that the autoencoder decides will best match information patterns observed in the original dataset ( [21] ). By selecting the nodes with highest Pearson correlation to the target dataset, training data can be further denoised and optimized for retaining the most ENSO-correlated patterns in the SST anomaly dataset.

This study utilized a hybrid model, with both original and flattened versions of the time-series image data to capture both spatial and generalized trends, respectively—which were then used through a concatenate layer in the CNN.

In traditional autoencoder architecture, the encoder compresses the input data into a lower-dimensional representation (latent space), capturing essential features, while the LSTM network learns temporal patterns. This compressed representation is then passed through a decoder, which reconstructs the data back to its original form, preserving the temporal sequences and features.

This method is particularly valuable in applications where feature extraction from high-dimensional time-series data is very useful, as the non-image data will capture key generalized patterns associated with ENSO trends, hence flattened input data was used to create multiple nodes of encoded data in the form of vectors. Reconstruction from the decoder thus only maintains the temporal dimensions as the final output nodes are all in vector form.

Convolutional autoencoders, designed for image data, combine convolutional layers with autoencoder architecture to perform both dimensionality reduction and data reconstruction while preserving spatial information.

In this setup, instead of flattening the input image data, the encoder directly applies convolutional filters to the input image, extracting features and reducing the spatial dimensions ( Figure 6 ). The decoder then reconstructs the image from the compressed representation, maintaining the same spatial dimensions as the original time series image dataset. This technique is particularly effective for processing and denoising image sequences, allowing the model to learn more highly ENSO-correlated particular spatial features for time-series forecasting.

In our case, 128 channels were reconstructed from the non-image autoencoder, and 4 channels from the convolutional autoencoder. Each was trained with 200 epochs and earlystopping to prevent overfitting.

Neural Networks are composed of three layers—the Input layer, hidden layers, and output layers. Each are made of neurons, which receive an input from the previous layer and perform a weighted sum with trainable weights w and adds a trainable bias value $b_i$. Finally, the neuron applies a specified activation function f to produce an output value $z_n$, which serves as the input for the neurons in the following layer. Such layers that receive and produce an input are called Dense layers. All outputs can be defined by these two equations:

$y_i=b_i+\underset{j=1}{\overset{n}{{\displaystyle \sum }}}{w}_j{x}_j,$ (12)

$z_i=f\left(y_i\right).$ (13)

A Convolutional Neural Network (CNN) is a type of neural network that is often used in the case of learning features from image input data ( [22] ). For an input variable x consisting of c channels, each P × P pixels, the CNN applies N convolutional filters, each of size Q × Q. The output $y_{i,j,n}$ at each layer is calculated as:

$y_{i,j,n}=\underset{s=0}{\overset{Q-1}{{\displaystyle \sum }}}\underset{t=0}{\overset{Q-1}{{\displaystyle \sum }}}{w}_{s,t,n}\cdot x_{i+s,j+t}+b_n.$ (14)

It can be seen that the initial input layer has dimensions of (None, 12, 8, 20, 4) in the model framework, Figure 7 . The dimension 12 arises from the usage of sequences. Prior to model training, the dataset was reformatted into a dataframe-like sequence, such that there are moving windows of 12 months at a time (Jan 1949 to Dec 1949, Feb 1949 to Jan 1951, etc.), each correlating to a predicted ONI value 4 months ahead of time. This procedure was carried out for both the image and non-image data.

The model framework begins with processing image data through time-distributed convolutional layers. These layers allow the model to capture spatial features independently for each time step, an important method when handling sequential data as it preserves temporal information necessary for making predictions based on a series of inputs. The Rectified Linear Unit (ReLU) activation function is employed in these convolutional layers to introduce non-linearity, allowing the model to handle complex patterns in the data. Specifically, the ReLU function filters out negative values using the equation:

$f\left(x\right)=\text{max}\left(x,0\right)$ (15)

This function outputs the maximum between the input value and zero, effectively retaining positive values and discarding negative ones, which helps the model learn any relevant features.

Such a manner of processing is necessary as the model will be able to learn much more information for a prediction if inputted 12 months of data at a once compared to just one. Following the convolutional operations, max-pooling layers reduce the dimensionality while retaining key features for following layers.

After the convolution and pooling steps, the downsampled image data is passed through a Convolutional Long Short-Term Memory (ConvLSTM2D) layer ( [23] ). This layer is critical for capturing both spatial and temporal dependencies in the data, which is especially important for time-series prediction tasks. The ConvLSTM2D layer uses trainable weights within a 3 × 3 convolutional window and applies the hyperbolic tangent (tanh) activation function, which scales input values to a range between −1 and 1. The tanh activation function is particularly suited for the ConvLSTM layer as it helps keep the temporal dependencies needed across sequences while also controlling the signal amplitude.

$\text{tanh}\left(x\right)=\frac{{\text{e}}^x-{\text{e}}^{-x}}{{\text{e}}^x+{\text{e}}^{-x}}$ (16)

Concurrently, the non-image data, representing the sequence of features correlated with the target dataset, is processed through a dense layer. This dense output is reshaped and repeated to align with the dimensions of the image features before being concatenated with the pooled image features, combining the processed image and nonimage output before the final dense and dropout layers.

Dropout regularization is applied to these dense layers to prevent overfitting. By randomly dropping a fraction of the neurons during training, the dropout regularization helps ensure that the model does not become too dependent on any single feature, improving model precision. The dropout rate is set to 0.2 in order to maintain a balance between reducing overfitting and retaining model complexity ( [24] ). The final output is then obtained through a softmax activation function in the Dense layer, which transforms the combined features into a single predicted value.

Recall that while the detail components of the discrete wavelet transform are generally not used, they are still components of the original dataset, and hence hold potentially useful information for further optimizing the model. In this study, the focus shifted to refining the final predictive model using a parallel computing approach with multiple regression analysis tools, specifically the Support Vector Regression (SVR) ( [25] ). After the wavelet transform decomposed the signal into approximation and detail components, the regression step was crucial for combining these components into a unified predictive model.

Support Vector Regression (SVR) was chosen for its high capacity in managing high-dimensional and complex data. SVR aims to approximate the underlying patterns in the data while minimizing prediction errors. A distinguishing feature of SVR is its margin of tolerance, defined by the $ϵ$-insensitive loss function. This margin allows the model to disregard small deviations within a specified range ( $ϵ$), focusing on more significant errors that exceed this threshold. This approach enhances the final model’s ability to handle noisy data and outliers effectively ( [26] ).

Given the already relatively well-performing model from just the wavelet approximation component of the dataset, we decided to optimize the regression to fit to the residuals between the true and approximation-predicted values:

$r\left(t\right)=y_{\text{true}}\left(t\right)-y_0\left(t\right)$ (17)

To optimize these residuals, SVR was applied to the detail components $D_j\left(t\right)$ for $j=1,2,\cdots ,15$. The SVR model was used to predict the residual $r\left(t\right)$ based on a linear combination of the detail components:

$\hat{r}\left(t\right)=\underset{j=1}{\overset{15}{{\displaystyle \sum }}}{w}_j{D}_j\left(t\right)$

Here, $w_j$ are the weights learned by the SVR model for each detail component $D_j\left(t\right)$. The final predictor $\hat{y}\left(t\right)$ was obtained by adding the optimized residuals back to the initial approximation:

$\hat{y}\left(t\right)=y_0\left(t\right)+\hat{r}\left(t\right)$

To test out reliability of different SVR models, two different kernels were tested: the Radial Basis Function (RBF) kernel and the linear kernel.

The RBF kernel specializes in its ability to map data into a higher-dimensional space where linear relationships can be more easily identified. Such a property enhances its ability particularly in capturing complex, non-linear relationships between the residuals and the detail components, notably in cases where the data is not linearly separable ( [27] ). The RBF kernel is defined as:

$K\left(x_i,x_j\right)=\text{exp}\left(-\gamma \underset{k=1}{\overset{n}{{\displaystyle \sum }}}{\Vert x_{i,k}-x_{j,k}\Vert }^2\right)$ (18)

where $x_{i,k}$ and $x_{j,k}$ are the k-th feature values of the i-th and j-th data points, respectively, and $\gamma$ controls the kernel’s spread. The optimization process in SVR with an RBF kernel involves adjusting these kernel parameters to fit the data. The predicted residuals $\hat{r}\left(t\right)$ in this case are indirectly influenced by the kernel parameters rather than directly learned weights.

A SVR linear kernel was also utilized, which indeed behaves similarly to multivariate linear regression in that both methods attempt to fit a linear relationship between the input features (in this case, the detail components $D_j\left(t\right)$) and the output (the residuals $r\left(t\right)$). In both methods, the goal is to find a set of weights $w_j$ that best fit the data.

$K\left(x_i,x_j\right)=\underset{k=1}{\overset{n}{{\displaystyle \sum }}}{x}_{i,k}\cdot x_{j,k}$ (19)

where $x_{i,k}$ and $x_{j,k}$ are the k-th feature values of the i-th and j-th data points, respectively.

In both SVR with a linear kernel and multivariate linear regression, the goal is to find a set of weights $w_j$ that best fit the data. However, recall that SVR differs in its approach towards error minimization through a different kind of loss function that includes a margin of tolerance, allowing it to be more suited to noisy data such as SST anomalies which experience great variation from factors other than just ENSO events.

In experiments for both kernels, fitting the regressions was done with predicted values from the detail component models on the validation set, while forecast accuracy was measured using the test dataset.

Given the added dimension due to sequences, a DataGenerator class was used to input the image, non-image and target datasets into the model for both training and testing, along with a “on epoch end” shuffler to reduce overfitting. Data was split into subsets of proportions 0.6, 0.2, and 0.2 for the training, validation and test sets, respectively.

It is to be noted that the image autoencoder outputs four nodes, or four different reconstructed datasets each emphasizing different information from the original SST anomaly dataset. As such, tests were run with both each of the four nodes individually (where the channel dimension in the CNN would be 1), compared to the performance seen in the CNN with a channel dimension of 4, with all nodes integrated into the datasets.

In addition, control experiments were run without the wavelet transform and the non-image autoencoded data (non-image data was set to a array of just ones) to see the magnitude of improvement gained with the non-image data, wavelet transform and SVR (seen in Table 1 ).

Metrics calculated to measure accuracy of models include: 1) Pearson correlation for evaluating how well trends are followed, 2) Root Mean Square Error (RMSE) and 3) Mean Absolute Error for error from true values, and 4) coefficient of determination (R²) for overall performance. In addition, three other metrics were used to determine capacity of event forecasting in general ( [28] ):

5) Critical Success Index (CSI):

$\text{CSI}=\frac{\text{TP}}{\text{TP}+\text{FN}+\text{FP}}$

CSI ranges from 0 to 1, with values closer to 1 indicating better performance. A higher CSI means that the model correctly identifies more positive instances while lesser false positives and false negatives ( [29] ).

6) Hit Rate (HR):

$\text{HR}=\frac{\text{TP}}{\text{TP}+\text{FN}}$

HR ranges from 0 to 1, with values closer to 1 indicating better performance. A higher HR means the model correctly identifies a greater proportion of actual positive instances.

7) False Alarm Ratio (FAR):

$\text{FAR}=\frac{\text{FP}}{\text{TP}+\text{FP}}$

FAR ranges from 0 to 1, with values closer to 0 indicating better performance. A lower FAR means fewer false alarms, i.e., the model makes fewer incorrect positive predictions.

A ONI threshold of 0.5 was used, as such is what ( [30] ) classifies as what is considered a El Niño or La Niña event.

The monthly averaged Sea Surface Temperature Anomaly data from the Extended Reconstructed Sea Surface Temperature Data can be found from this OpenDAP link: http://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCDC/.ERSST/.version5/.anom/T/ . The 3-month running mean standardized Oceanic Nino Index Data can be found from NOAA’s website: https://origin.cpc.ncep.noaa.gov/products/analysis_monitoring/ensostuff/ONI_v5.php . Data is available from 1950 to present; this study used April 1950 to April 2015.

Here is the 4-band 4-regular wavelet filter bank used—derived from [14] .