Self-organizing map (SOM) is an unsupervised artificial neural network model first proposed by [6]. This method repeatedly learns and trains meteorological data based on neural network principles to extract dominant circulation modes of synoptic systems. The detailed SOM classification procedure follows [19]. Following [19] and [15], sea level pressure (SLP), which is closely related to near-surface meteorological variables, is adopted to represent atmospheric synoptic conditions in this study. The SLP field is processed into a spatial anomaly field by subtracting the regional daily mean value from the grid-point SLP values. A total of 920 daily summer SLP spatial anomaly fields during 2005-2014 are used as training samples and input into the two-dimensional SOM network. After iterative training, all daily samples are objectively classified into distinct synoptic weather types, and the specific training process is shown in Figure 1.

Figure 1. Diagram of the SOM’s training process.

Optimal SOM grid dimensions depend on the complexity of local synoptic regimes: too few categories fail to capture full synoptic variability and evolution, while excessive categories produce overly similar, non-representative patterns. Two core metrics quantify SOM classification quality: quantization error (QE) and topographic error (TE) ([6]). QE denotes the average Euclidean distance between samples within a cluster and the reference vector of their best-matching neuron, analogous to within-group variance in clustering. TE represents the percentage of samples whose first and second best-matching neurons are non-adjacent, quantifying inter-cluster dissimilarity. Lower QE and TE values indicate superior classification performance. Multiple grid dimension experiments are conducted (Figure 2), with QE and TE ranked ascendingly (smaller values correspond to higher ranks). The 5 × 5 grid yields the minimum summed rank of QE and TE, hence selected as the final SOM grid size.

Figure 2. Ranked order of QE and TE values for alternative SOM grid dimensions (sorted from low to high).

SOM training assigns each daily SLP field sample to a single node on the two-dimensional map, enabling calculation of the occurrence frequency of each weather pattern (defined as the fraction of total samples, 920 summer days 2005-2014, belonging to a given pattern). A skillful climate model must accurately reproduce both the spatial morphology and occurrence frequency of observed synoptic patterns. The evaluation and ensemble workflow proceeds as follows:

Daily ERA5 sea level pressure reanalysis data are used as observational reference. After spatial anomaly calculation, the data are fed into the SOM network to generate 25 benchmark synoptic patterns (5 × 5 layout), daily classification labels and corresponding occurrence frequencies for observed circulations. Daily sea level pressure outputs of the 16 CMIP6 models are preprocessed consistently with ERA5: leap-day February 29 is discarded, bilinear interpolation is performed to a uniform 1.0˚ × 1.0˚ resolution, and daily spatial anomalies are calculated by subtracting regional averages. Standardized circulation samples are projected onto the ERA5-trained SOM network to obtain daily synoptic classifications and simulated pattern frequencies, followed by frequency significance tests described by [15]. Correlation coefficients of synoptic pattern frequencies between ERA5 and each model are computed for the whole summer, June, July and August. The average of the four coefficients serves as the comprehensive metric to rank the 16 models; higher values denote better performance in reproducing circulation frequencies. Three equally-weighted ensemble schemes are established based on rankings: MME3 (top 3 models), MME5 (top 5 models) and MME16 (all 16 models). Daily circulation and precipitation fields of each ensemble are derived by arithmetic averaging. Synoptic frequency biases and spatial precipitation discrepancies between the three ensembles and ERA5 are compared. Combined with pattern-wise precipitation maps and precipitation Q-Q plots, quantitative analysis is carried out on the improvements of regional and heavy rainfall simulation from different ensembles and well-performing individual models.

Figure 3. SLP spatial anomaly fields of SOM synoptic patterns derived from 2005-2014 ERA5 data (unit: hPa; rectangular box denotes study domain).

The 5 × 5 SOM grid generates 25 dominant SLP anomaly synoptic patterns over the Yangtze River Basin from 2005-2014 ERA5 data (Figure 3). Neighboring SOM nodes feature similar SLP anomaly distributions, while distant nodes exhibit stark contrasts, primarily driven by variations in the intensity and position of subtropical high-pressure systems over the ocean and low-pressure systems over land. Nodes (1,1), (2,1), (3,1), (1,2), (2,2) (top-left): Northwest low pressure, southeast subtropical high pattern. Nodes (5,5), (4,5), (3,5), (5,4), (4,4) (bottom-right): Northwest high pressure, southeast low-pressure pattern. Central nodes represent transitional circulation regimes.

After SOM synoptic classification, the occurrence frequency of the 25 synoptic weather types can be further calculated. [11] demonstrated that calculating the correlation coefficient between CMIP6 model outputs and ERA5 daily reanalysis data allows quantitative assessment of models’ ability to reproduce the observed climatological features of synoptic patterns. A two-tailed Student’s t-test for Pearson correlation coefficients was used to test the statistical significance of correlations between synoptic pattern frequencies derived from each model and ERA5. The paired sample size corresponds to the 25 synoptic weather types classified via SOM, resulting in a degree of freedom of df = N-2 = 23 and a significance threshold of alpha = 0.05; correlations with p < 0.05 are identified as statistically significant. The full t-statistics and p-values for each model are provided in Table 2.

Table 2.Two-tailed t-test results for Pearson correlation between synoptic pattern frequencies of individual CMIP6 models and ERA5.

As illustrated in Figure 4, the summer occurrence frequency curves of synoptic patterns from ERA5 are broadly consistent with those from most CMIP6 models except CNRM-CM6-1, corresponding to statistically significant positive correlations. The correlation coefficient between CNRM-CM6-1 and ERA5 for SOM synoptic pattern occurrence frequencies is −0.101. Collectively, these results indicate that all 15 remaining CMIP6 climate models can reasonably capture the observed characteristics of synoptic pattern occurrence frequencies, while CNRM-CM6-1 exhibits poor simulation performance.

Figure 4. Time series of synoptic pattern occurrence frequencies for ERA5 and the 16 CMIP6 models (summer season).

Figure 5. Correlation coefficients (sorted descending) between ERA5 and each CMIP6 model’s synoptic pattern occurrence frequencies for (a) full summer, (b) June, (c) July and (d) August.

It is worth noting that the summer weather patterns over the Yangtze River Basin vary from month to month. Accordingly, this study calculates the correlation coefficients for the occurrence frequency of summer-mean weather patterns, and further discusses the correlation coefficients for individual months. Figure 5 ranks the 16 CMIP6 climate models according to their correlation coefficients, to evaluate their performance in simulating observed weather-pattern occurrence frequencies for summer as a whole and for each individual month. As shown in the figure, the rankings of the 16 CMIP6 models differ considerably between summer-mean and monthly conditions. For the full summer season, the top-five performing models are BCC-CSM2-HR, INM-CM5-H, MPI-ESM1-2-HAM, MPI-ESM1-2-HR, and HadGEM3-GC31-MM. For June, the five best-performing models for simulating weather-pattern occurrence frequency are IPSL-CM6A-MR1, MPI-ESM1-2-HR, IPSL-CM6A-LR, HadGEM3-GC31-LL, and CanESM5. For July, the top-five models are NorESM1-F, CESM2-FV2, IPSL-CM6A-LR, IPSL-CM6A-MR1, and MIROC6. For August, the five bestperforming models are HadGEM3-GC31-MM, INM-CM5-H, HadGEM3-GC31-LL, MPI-ESM1-2-HR, and MPI-ESM1-2-HAM.

Table 3.Correlation coefficients and performance ranks between each CMIP6 model and ERA5 synoptic pattern occurrence frequencies.

Table 3 lists the correlation coefficients and corresponding ranks of SOM-derived weather-pattern occurrence frequencies between the 16 CMIP6 models and ERA5 reanalysis dataset. The BCC-CSM2-HR model exhibits the best performance in simulating summer-mean weather-pattern occurrence frequency, yet its skills degrade remarkably for individual months, leading to a low overall rank. Although INM-CM5-H reproduces August weather-pattern features reasonably well, it shows poor performance in June and July, which also results in a low comprehensive rank. HadGEM3-GC31-MM achieves the highest simulation skill for August weather-pattern frequency, but its performance deteriorates in June-July, yielding a low overall rank as well.

These results reveal that an individual climate model can show substantial discrepancies in simulating monthly weather-pattern occurrence frequencies across summer months. Such discrepancies are likely overlooked in previous model evaluations based on monthly-mean datasets, which highlights the necessity of model assessment using daily-scale outputs. Accordingly, the average of four correlation coefficients (summer-mean, June, July and August) is adopted to produce a comprehensive rank for selecting optimal models for multi-model ensemble. The top-five ranked models are MPI-ESM1-2-HR, IPSL-CM6A-MR1, IPSL-CM6A-LR, MPI-ESM1-2-HAM and FGOALS-f3-L. Multi-model ensembles are constructed by averaging the outputs of the top-3, top-5 and all 16 models, denoted as MME3, MME5 and MME16, respectively. Table 4 presents the correlation coefficients of weatherpattern occurrence frequencies between these ensemble products and ERA5. As shown in Table 4, MME3 and MME5 yield higher correlation coefficients than individual models and produce satisfactory ensemble performance, whereas MME16 shows low correlation coefficients and poor ensemble skills. This indicates that multi-model ensemble can improve the simulation of summer weather-pattern characteristics. Nevertheless, owing to large inter-monthly variations in individual model performance, the ensemble skill does not monotonically increase with the number of participating models. Selecting appropriate optimal models for ensemble is beneficial to enhance simulation capacity.

Table 4.Correlation coefficients of weatherpattern occurrence frequencies between multi-model ensembles (MME3, MME5, MME16) and ERA5 reanalysis dataset.

Figure 6 presents the occurrence frequencies of SOM synoptic weather types derived from ERA5 daily reanalysis data for summers 2005-2014, as well as the frequency biases between three multi-model ensembles (MME3, MME5, MME16) and ERA5. A one-sample two-tailed Student’s t-test is conducted on the frequency bias of each synoptic weather type. The test samples consist of all 920 daily summer records from 2005 to 2014, with a degree of freedom of df = N-1 = 919. Only nodes with p < 0.05 are marked as statistically significant overestimated or underestimated frequencies in Figure 6.

As shown in Figure 6, the frequency biases between simulations from the three ensemble schemes and ERA5 observations are all constrained within ±2%, with consistent spatial bias patterns. The ensembles overestimate the occurrence of synoptic types located at the top-left and bottom-right of the SOM grid, while underestimating the central transitional weather types. [11] (2011) noted that the central SOM nodes generally correspond to noisy synoptic regimes. Multi-model ensembles mitigate the oversimulation of these noisy weather types, thereby improving overall model performance in reproducing synoptic circulation characteristics. Nevertheless, the number of weather types with statistically significant frequency biases varies across ensemble configurations: MME16 exhibits significant deviations for 22 synoptic patterns, MME5 reduces this number to 20, and MME3 has the fewest (19). This result demonstrates that MME3 produces the smallest simulation errors for observed synoptic pattern frequencies, which is consistent with the correlation coefficient analysis presented in Section 4.2. Accordingly, subsequent precipitation bias analyses focus primarily on comparisons between MME3 outputs and ERA5 observations.

Figure 6. (a) ERA5 summer synoptic pattern occurrence frequencies (2005-2014); frequency differences between ERA5 and (b) MME3, (c) MME5, (d) MME16 (unit: %; red = significant overestimation, blue = significant underestimation).

Figure 7 compares observed precipitation against outputs from the three top individual models and MME3, alongside MME3 precipitation biases relative to ERA5. All three single models broadly reproduce observed precipitation spatial patterns, with MPI-ESM1-2-HR best capturing the two major precipitation maxima over the basin, followed by IPSL-CM6A-MR1 and IPSL-CM6A-LR. Averaging the three models into MME3 further aligns simulated precipitation with observations. MME3 precipitation is slightly lower than ERA5 across the domain, with absolute biases confined to 0 - 2 mm/day, demonstrating that ensemble outputs reasonably replicate observed precipitation spatial structure.Precipitation biases vary drastically across distinct synoptic patterns, illustrated via three representative SOM nodes (1,1), (3,3) and (5,5) in Figure 8:

Pattern (1,1): Northern precipitation maxima, southern minima. All three single models overpredict northern rainfall and underpredict southern rainfall; ensemble averaging retains this spatial bias yet reduces its magnitude.Pattern (3,3): Disparate spatial bias structures among individual models produce mutual offsetting, drastically reducing overall precipitation bias in MME3 and delivering marked simulation improvements.Pattern (5,5): Southern precipitation maxima, northern minima. MME3 overestimates northern rainfall and underestimates southern rainfall, yielding minimal overall improvement for this regime.

Figure 7. (a) ERA5 observed summer precipitation; (b) MME3 ensemble precipitation; (c) precipitation bias (MME3 minus ERA5); (d)-(f) precipitation from the three individual top models MPI-ESM1-2-HR, IPSL-CM6A-MR1 and IPSL-CM6A-LR (unit: mm/day).

Figure 8. Mean daily ERA5 precipitation for representative synoptic patterns (a)-(c) (1,1), (3,3), (5,5); precipitation biases relative to ERA5 for (d)-(f) MME3, (g)-(i) MPI-ESM1-2-HR, (j)-(l) IPSL-CM6A-MR1, (m)-(o) IPSL-CM6A-LR (unit: mm/day).

Figure 9. Q-Q plots of total regional precipitation (model vs ERA5) for (a) all synoptic pattern samples, (b) Pattern (1,1), (c) Pattern (3,3), (d) Pattern (5,5).

Quantile-quantile (Q-Q) plots of total regional precipitation for all samples and the three representative patterns (Figure 9) quantify ensemble performance across rainfall magnitudes. Points falling close to the 1:1 diagonal indicate accurate simulation:

All synoptic samples (Figure 9(a)): MPI-ESM1-2-HR and IPSL-CM6A-MR1 perform well for light-to-moderate rainfall but diverge from observations above 2000 mm total precipitation; IPSL-CM6A-LR systematically overestimates rainfall across all magnitudes. MME3 shows minor biases for light rain yet substantially improves simulation of moderate to extreme heavy rainfall.Pattern (1,1) (Figure 9(b)): All single models overpredict rainfall at all magnitudes; MME3 progressively converges to observed heavy rainfall values with increasing precipitation intensity.Pattern (3,3) (Figure 9(c)): MPI-ESM1-2-HR and IPSL-CM6A-MR1 outperform MME3 due to large positive biases in IPSL-CM6A-LR.Pattern (5,5) (Figure 9(d)): Similar to Pattern (1,1), MME3 delivers clear improvements for heavy rainfall magnitudes.

Collectively, model performance varies across synoptic regimes and rainfall intensities; MME3 provides the most substantial improvements in simulating heavy rainfall volumes.