A Comparison between the Rainfall Observation and Global Rainfall Data Including Satellite and Reanalysis ()
1. Introduction
The rainfall estimation is a key process to fill the gaps for a larger scale, which is done by using satellite and reanalysis data, but this data needs to be validated and evaluated for each region. The study focuses on the performance evaluation of satellite and reanalysis estimated rainfall data over Oman by using the ground observation as a control sample. Oman is affected by the rainfall in different patterns, which vary with seasons. The wintertime is a well-known season for low-pressure systems developing in the Mediterranean basin. The characteristics of the precipitation during wintertime, as well as the first transition period (March to May), are different from the summertime precipitation patterns, such that the winter and the following months are associated with stratiform clouds associated with continuous rainfall on some occasions, and it is described as torrential rainfall covering a larger area. However, the summertime and the second transitional period (October to November) have localised rainfall events due to the development of thunderstorms over the mountains. The so-called orographic rainfall is well-known in this time of the year, but unlike the stratiform-related rainfall, this type is very localised with a high intensity rate. The Easterly systems also affect the country with torrential rainfall, caused by upper troughs elongated from the Indian sub-continent. The final precipitation system is the monsoon winds, which affect the southern part of Oman, especially along the coastal areas of Dhofar governorate. The moist, tropical airmasses interact with the mountain chain, which plays a key role in the lifting and condensation process. This mechanism is associated with a drizzly and continuous shower from late June to late September. These differences in the rainfall mechanisms explain the challenges faced by the reanalysis and satellite rainfall estimation.
The challenges in estimating the rainfall using the remote sensing and reanalysis data include the spatial and temporal patterns of the rainfall, which can be significant in terms of the estimation, which also varies with seasons [1] [2]. It is shown that the reanalysis data respond differently to wet and dry biases, and they face difficulties in simulating the observed rainfall [3].
However, it is stated that some reanalysis models generally can capture the daily rainfall and the monthly cycle of the rainfall patterns in some regions worldwide with an overestimation, yet they underestimate the rainfall in other parts of the same region [4], and also overestimate low precipitation and underestimate the heavy precipitation [2] [5].
In contrast, the satellite-based rainfall estimation is much better in different regions compared to the reanalysis data [4]. Also, it is noticed that the performance of the satellite-based rainfall is much better during weather events associated with heavy rainfall, unlike the rainfall estimation by reanalysis models [6], which is supported by [7], which showed that the satellite-based rainfall is better in the rainy seasons when the reanalysis data showed a poor performance. However, in some regions, as [8] [9] stated, the satellite and reanalysis rainfall estimation is not helpful as the underestimation is significant, which agrees that the performance of the global gridded rainfall data can significantly vary with space and time. The rainfall coverage and precipitation mechanisms can be critical, as [10] shows by comparing the rainfall detection in the Sahara desert to regions where the East African monsoon affects.
2. Study Area
The study focuses on some rain gauges in Oman at different elevations. Oman is located in the south-eastern part of the Arabian Peninsula. The rain gauges cover different places, including deserts, mountains, and coastal areas, as shown in Figure 1. The network has more than 75 rain gauges, but the study focuses on the best-performing rain gauges during the period of interest and on the topographic diversity, as each place has its unique features. The Al Hajar Mountains chain has many stations that help to study different rainfall mechanisms in different seasons.
Figure 1. The location of used rain gauges operated by the Directorate General of Meteorology in Oman.
3. Dataset
There are 11 datasets that the study relied on to make the comparison, including satellite and reanalysis data. In addition to that, the study used different temporal and spatial resolutions for the same datasets to check the skills of different models in estimating the rainfall for a given location by comparing the data with the ground observation from the Oman Directorate General of Meteorology (DGMET). The Global rainfall datasets have been created by using different algorithms and input data. All the global data is reanalysed data, which includes ground observation, reanalysis data, along with a modelling process, except IMERG, which is a satellite rainfall estimation. Table 1 summarises the description of the datasets used by the study to evaluate the rainfall estimation. Datasets used in this study have different temporal and spatial resolutions from 2012 to 2021, but some datasets cover a shorter period.
Table 1. Description of the datasets used for evaluating precipitation estimates.
Dataset |
Spatial
Resolution |
Temporal Resolution |
Range |
Reference |
Rainfall
Observation |
NA |
Hourly |
2012-2021 |
DGMET |
PERSIANN |
0.25 × 0.25 |
Daily |
2012-2019 |
CHRS Data Portal [11] |
MERRA-2 |
0.5 × 0.625 |
Daily |
2012-2021 |
The Modern-Era Retrospective Analysis for Research and Applications [12] |
Monthly |
2012-2021 |
IMERG |
0.5 × 0.5 |
Daily |
2012-2021/10/27 |
NASA Global Precipitation Measurement (GPM)
Integrated Multi-Satellite Retrievals for GPM [13] |
0.2 × 0.2 |
Daily |
2012-2021/10/27 |
GPCP |
2.5 × 2.5 |
Monthly |
2012-2021 |
NOAA Climate Data Record Program [14] |
GPCC |
0.5 × 0.5 |
Monthly |
2012-2019 |
gpcc_normals_v2022 [15] |
0.25 × 0.25 |
Monthly |
2012-2019 |
gpcc_normals_v2022 [15] |
1 × 1 |
Monthly |
2012-2021 |
Earth System Science Data [16] |
2.5 × 2.5 |
Monthly |
2012-2021 |
gpcc_normals_v2022 [15] |
1 × 1 |
Daily |
2012-2018/04/30 |
gpcc_normals_v2022 [15] |
ERA5 |
0.25 × 0.25 |
Hourly |
2012-2021 |
ERA5 Global Reanalysis [17] |
CRU |
1 × 1 |
Monthly |
2012-2019 |
CRU TS Monthly High-Resolution Gridded Multivariate
Climate Dataset [18] |
2.5 × 2.5 |
Monthly |
2012-2019 |
CPC |
0.5 × 0.5 |
Daily |
2012-2019 |
Assessing Objective Techniques for Gauge‐Based Analyses of Global Daily Precipitation [19] |
CMORPH |
0.5 × 0.5 |
Monthly |
2012-2021/10 |
CMORPH Global High-Resolution Precipitation [20] |
0.5 × 0.5 |
Daily |
2012-2021/10/26 |
NCEP |
2.5 × 2.5 |
Daily |
2012-2021 |
NCEP/NCAR 40-Year Reanalysis [21] |
4. Methodology
The study evaluated the accuracy of satellite and reanalysis data by comparing them with ground rain gauges, as illustrated in Figure 1. As the observation dataset is tabular data and the global data is gridded data, the data extraction was made by using the rain gauges coordinated to have a representative comparison. After the extraction, the data were processed to match the available data from the rain gauges because the observation had some missing data. Once the pre-processing has been done, the statistical techniques are applied to the final dataset. The study compared the global datasets with the observation by using the correlation coefficient CC, Root Mean Square Error (RMSE), and Mean Bias Error (MBE). For each statistical technique, the analysis is divided into multiple steps. The first step is to have an analysis of the annual data to check the performance on an annual scale. The evaluation in this step applied CC., RMSE, and MBE. The second step is to evaluate the performance of the global data in seasonal scale to provide a comprehensive analysis covering different precipitation regimes. The final step covered the best values from each model to investigate the skills of the models for each time interval (annual, seasonal). This step is to illustrate to what extent the models can be described as the best. The percentage of rain gauges that have the best statistical analysis can elaborate and justify the final statement of the study. The Correlation coefficient is used to describe the matching degree between the global datasets and the ground stations. This step can illustrate how close the global data are to the directed measurements on an annual and seasonal basis. It is described by Equation (1):
(1)
where:
R is the correlation coefficient;
X is the first dataset;
Y is the second dataset;
n is the number of elements in each dataset.
Root Mean Square Error (RMSE) is used to check how close the estimated rainfall is to the observations, which is described by Equation (2):
(2)
where:
RMSE is the Root Mean Square Error;
N is the number of observations.
The Mean Bias Error (MBE) is used to investigate whether the global reanalysis models and satellite instruments overestimated or underestimated the observation, which can be described by Equation (3):
(3)
where:
MBE is the Mean Bias Error;
N is the number of observations.
Python 3.9 was used to extract the data from the reanalysis and satellite datasets. The extraction depended on the nearest pixel to the rain gauge location. It is known that each pixel has a single value, and that value was taken to make the comparison with the observation.
5. Results
5.1. CC, RMSE, and MBE of the Annual Scale
5.1.1. Correlation Coefficient (CC)
Figure 2 shows that throughout the study period, NCEP had the lowest correlation coefficient, with the highest value not exceeding 0.2. The data illustrate that IMERG 0.5˚ has the highest CC with the rain gauges along the Northern coast of Oman, with values exceeding 0.7, and shows a good CC in the mountainous areas along the Al Hajar Mountains and the adjacent areas, as well as the stations in Dhofar, except the Salalah coastal area. It is noticed that all datasets have the lowest CC with the rain gauges over Al Wusta Governorate, except GPCC, which gives a better CC. Furthermore, the degree of matching shows a spatial variation in the daily rainfall data for annual intervals. Regarding the performance over the Al Hajar Mountains, the PERSIANN model shows a low CC, unlike its performance in the southern coast of Dhofar.
Figure 2. The correlation coefficient of the daily rainfall data on an annual scale. The annual mean of accumulation has been calculated for 10 years.
The study shows that the datasets in the monthly scale do a better job compared to the daily scale of those that have daily and monthly rainfall data, as shown in Figure 3. It is clear that GPCC (0.25 & 0.5) has a good performance with the highest CC exceeding 0.75. The performance of those two datasets within the Al Hajar Mountains has CC ranged between 0.5 and 0.6. However, GPCC (1 & 2.5) are as good as the versions with a higher spatial resolution. The satellite data IMERG (0.2, 0.5) have as good performance as GPCC (0.25 & 0.5) within the Al Hajar Mountains, but the performance in Al Wusta Governorate is not as good as GPCC (0.25 & 0.5). Also, it is shown that NCEP and CMORPH perform better at the monthly scale than at the daily scale. CMORPH and CPC show as good performance as satellite data and GPCC (0.25 & 0.5) within the Al Hajar Mountains. In the annual scale, it is shown that CRU (0.5, 1.0, 2.5), along with NCEP, has the lowest CC. Additionally, ERA-5 has a good performance over the Al Hajar Mountains and the desert in southern Oman.
![]()
Figure 3. The correlation coefficient of the monthly rainfall data on an annual scale. The annual mean of accumulated rainfall was calculated from a 10-year rainfall record.
5.1.2. Root Mean Square Error (RMSE)
RMSE from daily data for the annual scale is much better than the RMSE from the monthly data. Most of the datasets have an RMSE of 10 to 20 mm. The degree of agreement is great between the estimations and the observation, but as mentioned previously, the extreme value in the Ras Al Hadd region in the northeast coast of Oman makes the visualization harder, as Figure 4 shows. One of the challenges is that fog formation in some areas can be adequate to observe some precipitation, which complicates the modeling of this process.
Figure 4. The RMSE of rainfall data at the annual scale over 10 years of data. It represents the quantitative differences between the observation and estimated rainfall.
The RMSE of the monthly data shows that the difference between the observation and the global data is significant, which exceeds 100 mm. The GPCC 1 & 2.5 show a significant difference, which is illogical and can explain the lower CC. from the annual scale. The problem is that all datasets have an RMSE greater than 100 mm, making the results difficult to visualize, as illustrated in Figure 5.
Figure 5. The RMSE of the monthly rainfall data on an annual scale. It represents the quantitative differences between the observation and estimated rainfall.
5.1.3. Mean Bias Error (MBE)
The performance of the reanalysis and satellite data is significantly better than the performance in the monthly data. IMERG in both versions overestimated rainfall at all locations, with a maximum of 20 mm in Muscat and Musandam, and showed a higher overestimation along the Al Hajar Mountains, unlike MERRA-2, which did well in terms of MBE. ERA-5 and others, except NCEP, have quite a good estimation. The NCEP almost overestimated the rainfall by 4 mm. Figure 6 summarizes the performance in terms of MBE.
Figure 6. The MBE of the daily rainfall data on an annual scale. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
The MBE of the annual scale based on the monthly data shows that all models and satellites underestimated the observation, which reached −40 mm if the extreme values were ignored. The effect of extreme values is clear in the visualisation, similar to the RMSE. In some stations, the underestimation reached 25 mm or less than that. Fortunately, all datasets have a range of underestimation between 0 and 40 mm, except the satellite data, which overestimated the rainfall by less than 10 mm in some locations, as shown in Figure 7.
Figure 7. The MBE of monthly rainfall data on an annual scale. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
5.2. CC, RMSE, and MBE of Wintertime
5.2.1. Correlation Coefficient (CC)
When the study divided the data into seasonal intervals, it showed different CC. It is shown that the NCEP has a better performance compared to the general case, with an increase in the best CC to reach 0.35, but it still struggles with the estimation. One of the models that improves in wintertime is MERRA-2 with a good CC in northwest Oman and the coastal areas along the Oman Sea. On the other hand, IMERGE (0.2, 0.5) shows a lower CC compared to the general case, with a best CC below 0.8. GPCC shows a good performance in winter compared to the performance on an annual scale. Moving to the CMORPH, the performance is worse in winter compared to the annual scale, with a highest CC not exceeding 0.6. Figure 8 summarises the CC of each dataset with the rain gauges.
Figure 8. The correlation coefficient of the daily rainfall data in the wintertime. The data represent December, January, and February.
In winter, the monthly data shows that ERA-5 has a good performance over the country, which reached CC of 0.8 in the south and 0.5 to 0.6 in the north, as well as the Al Hajar Mountains (Figure 9). ERA-5 has a much better performance compared to the performance of the daily data on a Seasonal scale. In addition to that, the satellite data shows a good performance in the north. In contrast, the performance in the south is not as good as the performance in the north. Noticeably, the CRU versions have a better CC compared to what they have in annual scale, especially the version with the highest spatial resolution (CRU 0.5), which has a good performance in the Northern coast of Oman. A similar situation is applied to GPCC with different spatial resolution, such that the highest spatial resolution version has a better CC, but the performance in the annual scale is much better for GPCC 0.25 & 0.5.
![]()
Figure 9. The correlation coefficient of the monthly rainfall data in the wintertime. Some datasets were generated with different spatial resolutions.
5.2.2. Root Mean Square Error (RMSE)
Based on the daily data, winter shows a significantly different overview of RMSE from all datasets compared to springtime and summertime. All datasets have an RMSE of greater than 20 mm over all locations, as shown in Figure 10. The rainfall estimation can be acceptable in terms of the frequency of occurrence, but the issue of quantity estimation makes the difference significantly high.
Figure 10. The RMSE of rainfall data in wintertime. It represents the quantitative differences between the observation and estimated rainfall.
The monthly data in wintertime shows that RMSE gets smaller to within 50 mm with some extreme values. All datasets have an issue with the Ras Al Hadd location, which is the blue dot over the edge of the northeast coast of Oman, but the exceeding RMSE is also observed in CPC and CRU 2.5. As mentioned previously, the extreme values make the visualization much harder. The RMSE of the mean of monthly accumulation shows a significant difference, which raises a concern in using the monthly data in terms of the rainfall quantity, as shown in Figure 11.
Figure 11. The RMSE of the monthly rainfall data in the wintertime. It represents the quantitative differences between the observation and estimated rainfall.
5.2.3. Mean Bias Error (MBE)
Based on the daily data, wintertime rainfall estimation has the same issues as spring and summer from NCEP and satellite data, but IMERG 0.5 is much better compared to summer and spring estimations. The rest of the datasets worked well. The results show that most of the models aggregated with the rain gauges, as Figure 12 shows.
Figure 12. The MBE of daily rainfall data in the wintertime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
Moving to the monthly data, performance during wintertime is much better, with an underestimation of less than 15 mm across all datasets. IMERG 0.2 & 0.5 overestimated the north coast rainfall, as it is noticed in springtime, and CPC overestimated the rainfall over the south, as well as some locations in the inner places to the south of the Al Hajar Mountains, as shown in Figure 13. The performance over the Al Hajar Mountains is much better compared to the summertime.
Figure 13. The MBE of monthly rainfall data in the wintertime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
5.3. CC, RMSE, and MBE of Springtime
5.3.1. Correlation Coefficient (CC)
In springtime, it seems that all daily datasets do a better job compared to wintertime. Noticeably, they did a good job along the Al Hajar Mountains, where the CC. exceeds 0.6 in both IMERG (0.2, 0.5). GPCC shows a very good CC in the northern part, but it is the worst in the southern part of Oman compared to its performance in the north. Also, CMORPH shows a much better performance compared to winter and annual terms with CC. exceeding 0.6 in general. Also, ERA-5 shows a good performance compared to its performance in wintertime and annual intervals. The so-called spring season is locally defined as the first transitional period starting from March and ending in early June. This period is well-known for westerly low systems, which are associated with widespread stratiform clouds (Figure 14).
Figure 14. The correlation coefficient of the daily rainfall data in the springtime or the first transitional period covering the period from March to early June.
The monthly data in springtime shows interesting features from different datasets. For instance, the spatial variation in CC in PERSIANN is significant, which shows locations with CC of 0.9, but the nearby locations have as low CC as 0.3. CRU versions have a much better performance compared to their performance in the annual scale and wintertime. By and large, all datasets have a good performance along the Al Hajar Mountains and nearby places, with CC ranging between 0.5 and 0.9 except CRU 1.0. CMORPH and NCEP, which show a good improvement in springtime. The other interesting feature is that the datasets have the lowest CC in the driest locations in Al Wusta Governorate. Figure 15 illustrates the CC of all datasets in springtime.
Figure 15. The correlation coefficient of the monthly rainfall data in the springtime from models with different spatial resolutions.
5.3.2. Root Mean Square Error (RMSE)
The performance of the daily datasets in spring is much better compared to the RMSE on an annual scale, as shown in Figure 16. From GPCC, CMORPH, CPC ERA-5, it shows that the Al Hajar Mountains have a higher RMSE compared to the rest of the areas, unlike the satellite data, which has lower RMSE values along the mountainous areas. The results show the challenge of estimating the rainfall over the mountainous areas due to the characteristics of cloud formation and thunderstorm development.
Figure 16. The RMSE of rainfall data in the springtime. It represents the quantitative differences between the observation and estimated rainfall.
The monthly data in springtime is almost the same as what the annual scale shows. The RMSE is worse than what wintertime shows in Figure 17.
Figure 17. The RMSE of the monthly rainfall data in the springtime. It represents the quantitative differences between the observation and estimated rainfall.
5.3.3. Mean Bias Error (MBE)
According to the daily data, Figure 18 shows that the satellite data struggled to estimate rainfall during springtime compared to the annual scale. Also, it has the same challenge over the Al Hajar Mountains as on the annual scale. The NCEP did worse in the springtime compared to the annual scale. The rest of the datasets worked well in the estimation with a lower degree of struggling compared to the mentioned datasets in the beginning.
Moving to the monthly data, the performance during wintertime is much better with an underestimation of less than 15 mm from all the datasets. IMERG 0.2 & 0.5 overestimated the north coast rainfall, as it is noticed in springtime, and CPC overestimated the rainfall over the south, as well as some locations in the inner places to the south of the Al Hajar Mountains. The performance over the Al Hajar Mountains is much better compared to the summertime, which can be due to the difference in the characteristics of the precipitation in both seasons. Figure 19 shows the MBE of each dataset.
Figure 18. The MBE of daily rainfall data in the springtime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
Figure 19. The MBE of monthly rainfall data in the springtime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
5.4. CC, RMSE, and MBE of Summertime
5.4.1. Correlation Coefficient (CC)
In summertime, the performance of daily data significantly varies in the spatial aspect, which can be seen in Figure 20. It is clear that CC shows that IMERG 0.2 is relatively the best over the Al Hajar Mountains, but the other version of IMERG, which has coarser spatial resolution, did not estimate the rainfall well. Also, IMERG 0.2 did well in the southern part during the summertime when the monsoon affects the region. In general, the estimated rainfall was low in this season, as indicated by CC. This season revealed that the reanalysis models struggle with the small-scale thunderstorms over the Al Hajar Mountains and the low clouds of the monsoon season in the southern part of Oman.
Figure 20. The correlation coefficient of the daily rainfall data in the summertime. It covers the period from June to late September.
The monthly data of summertime shows slightly different results compared to springtime. For instance, the performance along the Al Hajar Mountains is not as good as what they have in springtime, but the best performance is observed in the satellite data from IMERG 0.2 & 0.5. Also, GPCC versions and ERA-5 show a good performance over the mountainous regions. Most of the global data show good performance in the south, except for CPC, GPCC_2.5, IMERG (0.2, 0.5), and NCEP, as shown in Figure 21. The monthly data shows that the reanalysis models and satellite measurements have a better performance over the mountains compared to the daily data.
Figure 21. The correlation coefficient of the monthly rainfall data in the summertime from models with different spatial resolutions.
5.4.2. Root Mean Square Error (RMSE)
The daily data from summertime show significantly different performance in terms of RMSE across all datasets compared to Springtime, as shown in Figure 22. All datasets have an RMSE of 10 - 15 mm at the vast majority of locations, except MERRA-2, which has a higher RMSE in the south. All models struggled on the northeast coast, which can be due to issues with the rain gauge.
Figure 22. The RMSE of rainfall data in the summertime. It represents the quantitative differences between the observation and estimated rainfall.
The monthly data of summertime shows that RMSE gets smaller to be within 50 mm with some extreme values, as Figure 23 shows. All datasets have an issue with the Ras Al Hadd location, which is the blue dot over the edge of the northeast coast of Oman, but the exceeding RMSE is also observed in CPC and CRU 2.5. As mentioned previously, the extreme values make the visualisation much harder. The smaller RMSE values can be due to a lack of rainfall in this season.
Figure 23. The RMSE of the monthly rainfall data in the summertime. It represents the quantitative differences between the observation and estimated rainfall.
5.4.3. Mean Bias Error (MBE)
The daily scale shows that satellite data in summertime overestimated the rainfall as it did in the other seasons. The greatest magnitude of overestimation is along the Al Hajar Mountains and its adjacent areas, unlike NCEP, which overestimated the rainfall, but the Al Hajar Mountains suffered the least. MERRA-2 overestimated the rainfall by almost 1 mm over more than 85% of the locations. GPCC shows a good performance over most locations, as shown in Figure 24.
Figure 24. The MBE of the daily rainfall data in the summertime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
The monthly data in summertime shows that some datasets overestimated the rainfall in some locations, such as the satellite data, GPCC 0.25 & 0.5. CRU 0.5 and CPC. Figure 25 shows that the datasets struggled with the Al Hajar Mountains, which have the greatest underestimation compared to the rest. However, the range did not exceed 20 mm in the vast majority of datasets, but the extreme values make things harder to visualize.
Figure 25. The MBE of monthly rainfall data in the summertime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
5.5. CC, RMSE, and MBE of Autumntime
5.5.1. Correlation Coefficient (CC)
Unlike summertime, the performance of all data on a daily scale is much better, as CC illustrates in Figure 26. In addition, MERRA-2 and IMERG (0.2, 0.5) perform best over the Al Hajar Mountains, with CC ranging from 0.6 to 0.7, while CPC is best in the south. Also, it is noted that all datasets show good CC over the north and northwest of Oman, ranging from 0.5 to 0.6, except ERA-5, which struggled in most areas. However, all datasets have a low CC over the desert. The rainfall distribution in this season is more to the north along the coastal areas and the mountain chain, as well as the southern part, which is occasionally affected by tropical cyclones.
Figure 26. The correlation coefficient of the daily rainfall data in autumntime or the second transitional period, which covers the period from late September to early December.
According to the monthly data, Figure 27 shows that spatial variation in PERSIANN, which was observed in spring, can be noticed in autumn with almost a similar performance. The effect of spatial resolution differences of GPCC datasets is significant in this season, such that the versions with a higher spatial resolution have a better CC compared to GPCC (1 & 2.5). This feature is also observed in CRU datasets. MERRA-2 did a good job in the north, but the performance over the south is not. The satellite data did well in general compared to the reanalysis data, which struggled in the driest areas. However, all datasets have a high CC in the northwest part of Oman.
Figure 27. The correlation coefficient of the monthly rainfall data in autumntime from models with different spatial resolution.
5.5.2. Root Mean Square Error (RMSE)
The daily data of autumntime shows a significantly different overview of RMSE from all datasets compared to springtime and summertime. All datasets have an RMSE of greater than 20 mm over all locations, which is similar to the wintertime. The characteristics of the rainfall over arid and semi-arid areas are complex to model. Figure 28 shows that the RMSE exceeded 20 mm.
The monthly data of autumntime shows greater extreme values compared to the rest, with the RMSE of most of the stations being within 200 mm, as illustrated in Figure 29 which shows the lowest RMSE calculated from the data.
Figure 28. The RMSE of rainfall data in autumntime. It represents the quantitative differences between the observation and estimated rainfall.
Figure 29. The RMSE of monthly rainfall data in autumntime. It represents the quantitative differences between the observation and estimated rainfall.
5.5.3. Mean Bias Error (MBE)
Based on the daily scale, the satellite data has the same issue as in springtime, with a smaller magnitude of overestimation. Also, NCEP has the same issue as in springtime. The rest of the datasets worked well in terms of MBE, as shown in Figure 30.
Figure 30. The MBE of daily rainfall data in the autumntime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount.
The MBE of autumn monthly data shows that all the data underestimated the observations, which reached 50 mm if the extreme values are ignored. It is noticed that the datasets have suffered the most in the Salalah region, which can possibly be interpreted by the effect of coastal areas on the pixels. The effect of extreme values is clear in the visualisation, similar to the RMSE as shown in Figure 31.
Figure 31. The MBE of monthly rainfall data in the autumntime. The negative values indicate that the global data underestimate the rainfall, but the positive values show an overestimated amount
5.6. Best Performing Global Models Based on CC, RMSE, MBE
The defined thresholds for CC, RMSE, and MBE are based on the overall performance of all datasets over Oman. It is found that most locations have a low CC with corresponding values from both satellite and reanalysis datasets. Therefore, the defined threshold of CC is ≥0.35 which seems low, but the CC values in arid and mountainous areas are relatively low [22] [23].
5.6.1. Annual Statistical Analysis
The study defined a threshold to identify the best global dataset to estimate the rainfall in the locations of interest. Table 2 shows the defined threshold for the three statistical tests, including CC, RMSE, and MBE. As shown in the table below, the best performance from the daily data in terms of the CC is the satellite data (IMERG 0.5). The CC of greater than 0.35 has been calculated for 62.5% of the locations. However, the RMSE is not as good as the CC or the MBE. The best performing dataset in terms of RMSE and MBE is GPCC, which has the best estimations in terms of the degree of bias. All locations have the estimation within the range between 10 mm and −10 mm. It is shown from the statistical analysis that the CPC data has a good CC and MBE. In terms of daily data on an annual scale, it is shown that PERSIANN has the smallest percentage of locations within the CC range, and CMORPH has the smallest percentage of locations within the RMSE range. Finally, IMERG 0.2 has the smallest percentage of locations within the MBE range. Moving to the monthly data, which is shown in the same table, the best performing dataset in terms of CC is GPCC (0.25 & 0.5), with almost 71% of the locations having a CC of 0.35 or greater than that. These two datasets also have a high percentage of locations with an estimation within the range, which is 93.75%. However, the RMSE is not as good as the CC and MBE. IMERG (0.2 & 0.5) comes second with more than 68% of locations with a CC of 0.35 or greater. Also, they have a relatively good percentage of stations within the threshold, which represents 87.5% and 84.375%, respectively. However, the analysis shows that PERSIANN and ERA5 have the best RMSE percentage as well as the MBE. In other words, PERSIANN and ERA-5 have 53.125% of the locations within the defined threshold of RMSE and all the locations within the defined threshold of MBE. The analysis of monthly data at the annual scale also shows that CRU 0.5 has the smallest percentage of locations within the CC range, and CRU 1.0 has the smallest percentage within the RMSE range. Regarding MBE, the dataset with the smallest percentage of locations within the range is MERRA-2.
Table 2. The percentage of best locations based on CC, RMSE, and MBE from daily and monthly data for the annual scale.
Dataset |
CC (R) ≥ 0.35 (%) |
RMSE ≤ 4 & ≥−4 (Daily)
RMSE ≤ 15 & ≥−15 (Monthly) (%) |
MBE ≤ 10 & MBE ≥ −10 (%) |
Persiann (Daily) |
9.375 |
53.125 |
100 |
Persiann (Monthly) |
40.625 |
31.25 |
96.875 |
MERRA-2 (Daily) |
21.875 |
40.625 |
100 |
MERRA-2 (Monthly) |
21.875 |
28.125 |
71.875 |
IMERG (0.2) (Daily) |
28.125 |
34.375 |
53.125 |
IMERG (0.2) (Monthly) |
68.75 |
31.25 |
87.5 |
IMERG (0.5) (Daily) |
62.5 |
34.375 |
46.875 |
IMERG (0.5) (Monthly) |
68.75 |
40.625 |
84.375 |
GPCC (Daily) |
37.5 |
71.875 |
100 |
GPCC (0.25) (Monthly) |
71.875 |
43.75 |
93.75 |
GPCC (0.5) (Monthly) |
71.875 |
40.625 |
93.75 |
GPCC (1.0) (Monthly) |
NA |
NA |
NA |
GPCC (2.5) (Monthly) |
NA |
NA |
NA |
ERA5 (Daily) |
12.5 |
53.125 |
100 |
ERA5 (Monthly) |
37.5 |
28.125 |
71.875 |
CPC (Daily) |
18.75 |
46.875 |
96.875 |
CPC (Daily) |
53.125 |
37.5 |
96.875 |
CMORPH (Daily) |
15.625 |
25 |
100 |
CMORPH (Monthly) |
43.75 |
28.125 |
75 |
NCEP (Daily) |
NA |
37.5 |
100 |
NCEP (Monthly) |
25 |
15.625 |
90.625 |
GPCP (Monthly) |
46.875 |
28.125 |
75 |
CRU (0.5) (Monthly) |
18.75 |
25 |
93.75 |
CRU (1.0) (Monthly) |
NA |
12.5 |
93.75 |
CRU (2.5) (Monthly) |
NA |
21.875 |
81.25 |
5.6.2. Winter Statistical Analysis
As shown in Table 3, the best performance from the daily data in terms of CC is GPCC. The CC, which is greater than 0.35, has been calculated in 43.75% of the locations, and the RMSE and MBE are also very high. The dataset is the best if all tests are considered in wintertime, and it is the best-performing dataset in terms of RMSE as well. The performance of MERRA-2 is better in wintertime compared to the annual scale, with the second highest number of locations within the RMSE threshold, and all locations are within the MBE threshold. ERA-5 has the same performance as MERRA-2, but the locations vary. CMORPH has a better performance in winter compared to the annual scale, even if it has the smallest number of locations with RMSE within the threshold. The analysis of the daily data in wintertime also shows that IMERG 0.5 has the smallest number of locations within the CC threshold; however, the RMSE and MBE are good compared to other datasets.
Finally, IMERG 0.2 has the smallest number of locations within the threshold of MBE. Moving to the monthly data in wintertime, which shows in Table 3 that the best performing datasets in terms of CC are ERA-5 and CPC, with 46.875% of the locations within the CC threshold. However, CPC is much better than ERA-5 in RMSE and MBE. PERSIANN has the greatest number of locations within the RMSE threshold, which represent 65.625% of locations, and it also did well in terms of MBE. NCEP has the second-best performance in wintertime in terms of RMSE and MBE, which is better than its performance on the annual scale. The analysis in the same table shows that CRU 1.0 has the smallest number of stations within the range of CC and RMSE, but ERA-5 has the smallest number of locations within the threshold of MBE.
Table 3. The percentage of best locations based on CC, RMSE, and MBE from daily and monthly data during winter.
Dataset |
CC (R) ≥ 0.35 (%) |
RMSE ≤ 4 & ≥−4 (Daily)
RMSE ≤ 15 & ≥−15 (Monthly) (%) |
MBE ≤ 10 & MBE ≥ −10 (%) |
Persiann (Daily) |
9.375 |
65.625 |
100 |
Persiann (Monthly) |
31.25 |
65.625 |
96.875 |
MERRA-2 (Daily) |
37.5 |
71.875 |
100 |
MERRA-2 (Monthly) |
18.75 |
50 |
84.375 |
IMERG (0.2) (Daily) |
3.125 |
56.25 |
81.25 |
IMERG (0.2) (Monthly) |
40.625 |
53.125 |
93.75 |
IMERG (0.5) (Daily) |
6.25 |
53.125 |
84.375 |
IMERG (0.5) (Monthly) |
50 |
50 |
90.625 |
GPCC (Daily) |
43.75 |
81.25 |
100 |
GPCC (0.25) (Monthly) |
43.75 |
56.25 |
96.875 |
GPCC (0.5) (Monthly) |
40.625 |
56.25 |
96.875 |
GPCC (1.0) (Monthly) |
NA |
NA |
NA |
GPCC (2.5) (Monthly) |
NA |
NA |
NA |
ERA5 (Daily) |
18.75 |
71.875 |
100 |
ERA5 (Monthly) |
46.875 |
50 |
84.375 |
CPC (Daily) |
15.625 |
53.125 |
93.75 |
CPC (Daily) |
46.875 |
53.125 |
96.875 |
CMORPH (Daily) |
12.5 |
46.875 |
100 |
CMORPH (Monthly) |
25 |
50 |
84.375 |
NCEP (Daily) |
NA |
62.5 |
96.875 |
NCEP (Monthly) |
15.625 |
53.125 |
93.75 |
GPCP (Monthly) |
31.25 |
50 |
84.375 |
CRU (0.5) (Monthly) |
9.375 |
31.25 |
93.75 |
CRU (1.0) (Monthly) |
6.25 |
28.125 |
90.625 |
CRU (2.5) (Monthly) |
9.375 |
65.625 |
100 |
5.6.3. Springtime Statistical Analysis
The springtime shows different results, as illustrated in Table 4. The daily data show that GPCC has the highest percentage of locations within the threshold (65.625%), as well as the lowest RMSE and MBE, making it the best-performing global rainfall dataset for estimating rainfall in Oman. ERA-5 is the second-best performing dataset with more than 50% of the locations within the range of CC, RMSE, and MBE. In general, the performance in spring is much better than the performance in wintertime and annual scale. In contrast, NCEP has the smallest number of locations with the defined threshold of CC, RMSE, and MBE. Moving to the monthly data, which reveals the best and the worst performing datasets based on the statistical tests. It is shown in Table 4 that the best performing dataset is GPCC 0.25, which has the highest number of locations with CC within the threshold, representing 84.375%. GPCC 0.25 is also amongst the best performing datasets along with GPCC 0.5 and CRU 0.5. CRU 1.0 has the greatest percentage of locations that have MBE within the range, but GPCC 0.5 has the greatest percentage of locations within the RMSE threshold, and it is amongst the best in terms of MBE. Finally, the monthly data shows that CRU 2.5 has the smallest number of locations with the defined range of all statistical tests.
Table 4. The percentage of best locations based on CC, RMSE, and MBE from daily and monthly data during springtime.
Dataset |
CC (R) ≥ 0.35 (%) |
RMSE ≤ 4 & ≥−4 (Daily) RMSE ≤ 15 & ≥−15 (Monthly) (%) |
MBE ≤ 10 & MBE ≥ −10 (%) |
Persiann (Daily) |
37.5 |
59.375 |
100 |
Persiann (Monthly) |
59.375 |
37.5 |
84.375 |
MERRA-2 (Daily) |
43.75 |
31.25 |
100 |
MERRA-2 (Monthly) |
40.625 |
28.125 |
46.875 |
IMERG (0.2) (Daily) |
62.5 |
37.5 |
21.875 |
IMERG (0.2) (Monthly) |
71.875 |
28.125 |
65.625 |
IMERG (0.5) (Daily) |
59.375 |
31.25 |
28.125 |
IMERG (0.5) (Monthly) |
71.875 |
31.25 |
65.625 |
GPCC (Daily) |
65.625 |
62.5 |
100 |
GPCC (0.25) (Monthly) |
84.375 |
40.625 |
81.25 |
GPCC (0.5) (Monthly) |
81.25 |
46.875 |
84.375 |
GPCC (1.0) (Monthly) |
68.75 |
21.875 |
46.875 |
GPCC (2.5) (Monthly) |
65.625 |
28.125 |
46.875 |
ERA5 (Daily) |
53.125 |
56.25 |
100 |
ERA5 (Monthly) |
62.5 |
28.125 |
46.875 |
CPC (Daily) |
40.625 |
53.125 |
96.875 |
CPC (Daily) |
68.75 |
37.5 |
87.5 |
CMORPH (Daily) |
62.5 |
28.125 |
100 |
CMORPH (Monthly) |
62.5 |
28.125 |
53.125 |
NCEP (Daily) |
6.25 |
28.125 |
71.875 |
NCEP (Monthly) |
40.625 |
18.75 |
81.25 |
GPCP (Monthly) |
62.5 |
28.125 |
50 |
CRU (0.5) (Monthly) |
71.875 |
43.75 |
87.5 |
CRU (1.0) (Monthly) |
21.875 |
31.25 |
90.625 |
CRU (2.5) (Monthly) |
18.75 |
25 |
71.875 |
5.6.4. Summertime Statistical Analysis
The summertime data show that performance is not as good as in previous periods, as shown in Tables 2-5 for CC. IMERG 0.2 has the greatest percentage of locations within the CC threshold, which reached 37%, but it has the lowest number of locations with MBE within the threshold. However, ERA-5 has the greatest number of locations within the RMSE threshold as well as MBE, which are 78.125% and 100%, respectively, but it is among the datasets with the smallest number of locations within the defined threshold of CC. Also, GPCC has the second-best performance in terms of RMSE, and it has the same performance as PERSIANN and ERA-5 in MBE, but it is among the datasets that have the smallest number of locations with CC within the threshold. PERSIANN is also amongst the best performing datasets in terms of RMSE and MBE. In contrast, CPC has the smallest number of locations within the defined range of CC, and IMERG 0.2 has the smallest number of locations within the defined range of RMSE as well as MBE. The monthly data shows that IMERG 0.2 has the best performance in terms of CC, unlike the daily data, with 50 % of the locations within the threshold, and its performance in MBE is among the best. On the other hand, PERSIANN is the best in terms of RMSE and among the best in MBE. Regarding MBE, CPC, IMERG, and PERSIANN are the best, with 90% of the locations within the MBE range. In contrast, GPCC 2.5 and CRU 2.5 have the smallest number of locations with the defined threshold of CC, and CRU 0.5 has the smallest number of locations with the defined threshold of RMSE. Finally, GPCC 1.0 and GPCP have the smallest number of locations with the defined range of MBE.
Table 5. The percentage of best locations based on CC, RMSE, and MBE from daily and monthly data during the summertime.
Dataset |
CC (R) ≥ 0.35 (%) |
RMSE ≤ 4 & ≥−4 (Daily) RMSE ≤ 15 & ≥−15 (Monthly) (%) |
MBE ≤ 10 & MBE ≥ −10 (%) |
Persiann (Daily) |
12.5 |
71.875 |
100 |
Persiann (Monthly) |
40.625 |
78.125 |
90.625 |
MERRA-2 (Daily) |
6.25 |
68.75 |
100 |
MERRA-2 (Monthly) |
12.5 |
56.25 |
71.875 |
IMERG (0.2) (Daily) |
37.5 |
53.125 |
56.25 |
IMERG (0.2) (Monthly) |
50 |
46.875 |
90.625 |
IMERG (0.5) (Daily) |
3.125 |
56.25 |
56.25 |
IMERG (0.5) (Monthly) |
46.875 |
50 |
87.5 |
GPCC (Daily) |
12.5 |
75 |
100 |
GPCC (0.25) (Monthly) |
34.375 |
56.25 |
87.5 |
GPCC (0.5) (Monthly) |
34.375 |
53.125 |
87.5 |
GPCC (1.0) (Monthly) |
37.5 |
56.25 |
68.75 |
GPCC (2.5) (Monthly) |
3.125 |
NA |
NA |
ERA5 (Daily) |
18.75 |
78.125 |
100 |
ERA5 (Monthly) |
31.25 |
56.25 |
75 |
CPC (Daily) |
3.125 |
68.75 |
96.875 |
CPC (Daily) |
34.375 |
53.125 |
90.625 |
CMORPH (Daily) |
12.5 |
56.25 |
100 |
CMORPH (Monthly) |
28.125 |
59.375 |
71.875 |
NCEP (Daily) |
NA |
71.875 |
100 |
NCEP (Monthly) |
15.625 |
50 |
81.25 |
GPCP (Monthly) |
28.125 |
59.375 |
68.75 |
CRU (0.5) (Monthly) |
12.5 |
43.75 |
81.25 |
CRU (1.0) (Monthly) |
NA |
46.875 |
81.25 |
CRU (2.5) (Monthly) |
3.125 |
46.875 |
71.875 |
5.6.5. Autumntime Statistical Analysis
The daily data in Table 6 shows that the satellite data (IMERG 0.2, 0.5) have the best performance in terms of CC, with 40% of the locations within the threshold. Regarding the RMSE, it is shown that CPC and GPCC have the best results, which is 70% of the locations within the threshold, and they are among the best in MBE results. It is also shown that PERSIANN, MERRA-2, CMORPH, and NCEP are the best in terms of MBE. In contrast, the ERA-5 has the lowest percentage of locations with CC within the threshold, and IMERG 0.5 has the lowest percentage of locations with RMSE within the range. Regarding MBE, IMERG0.2 has the fewest locations within the defined MBE range. Moving to the monthly data, IMERG 0.2, GPCC 0.25, and GPCC 0.5 have the best performance in terms of CC, with 68% of the locations within the threshold. GPCC 0.25 has the best RMSE results and is among the best in terms of MBE. CRU 0.5 and CRU 1.0 have the best results of MBE, but they have the lowest percentage of locations within the defined threshold of RMSE. Finally, it is shown that GPCC 2.5 has the lowest percentage of locations within the CC threshold range, and MERRA-2 has the lowest percentage of locations within the defined MBE range.
Table 6. The percentage of best locations based on CC, RMSE, and MBE from daily and monthly data during autumntime.
Dataset |
CC (R) ≥ 0.35 (%) |
RMSE ≤ 4 & ≥−4 (Daily) RMSE ≤ 15 & ≥−15 (Monthly) (%) |
MBE ≤ 10 & MBE ≥ −10 (%) |
Persiann (Daily) |
25 |
59.375 |
100 |
Persiann (Monthly) |
46.875 |
53.125 |
84.375 |
MERRA-2 (Daily) |
37.5 |
59.375 |
100 |
MERRA-2 (Monthly) |
43.75 |
46.875 |
75 |
IMERG (0.2) (Daily) |
40.625 |
59.375 |
68.75 |
IMERG (0.2) (Monthly) |
68.75 |
59.375 |
87.5 |
IMERG (0.5) (Daily) |
40.625 |
56.25 |
71.875 |
IMERG (0.5) (Monthly) |
68.75 |
53.125 |
87.5 |
GPCC (Daily) |
18.75 |
71.875 |
96.875 |
GPCC (0.25) (Monthly) |
68.75 |
65.625 |
90.625 |
GPCC (0.5) (Monthly) |
68.75 |
62.5 |
93.75 |
GPCC (1.0) (Monthly) |
3.125 |
NA |
NA |
GPCC (2.5) (Monthly) |
6.25 |
NA |
NA |
ERA5 (Daily) |
12.5 |
65.625 |
96.875 |
ERA5 (Monthly) |
40.625 |
46.875 |
81.25 |
CPC (Daily) |
25 |
71.875 |
90.625 |
CPC (Daily) |
62.5 |
50 |
93.75 |
CMORPH (Daily) |
31.25 |
53.125 |
100 |
CMORPH (Monthly) |
34.375 |
46.875 |
78.125 |
NCEP (Daily) |
9.375 |
62.5 |
100 |
NCEP (Monthly) |
46.875 |
59.375 |
87.5 |
GPCP (Monthly) |
53.125 |
46.875 |
75 |
CRU (0.5) (Monthly) |
34.375 |
40.625 |
93.75 |
CRU (1.0) (Monthly) |
25 |
40.625 |
93.75 |
CRU (2.5) (Monthly) |
21.875 |
56.25 |
81.25 |
6. Discussion and Conclusion
The analysis shows that all dataset’s behaviours vary with time and space as well as the temporal scale, which means that the daily data has a different performance compared to the monthly data. The results show that summertime is a challenging season, which is supported by the rainfall mechanism during this period. In other words, the local convection dominates the rainfall mechanisms along the Al Hajar Mountains. To solve this issue, a high spatial resolution is required. Additionally, the models may not be able to resolve rainfall in such complex terrain, and moisture convergence at the sub-grid scale is highly effective [24]. Also, orographic rainfall strongly depends on local thermal forcing, which requires high resolution to capture. Furthermore, such complex and small-scale rainfall mechanisms need parameterisation schemes that introduce uncertainty. Based on the correlation coefficient from the daily data, it is shown that the global data poorly estimates the rainfall except for the satellite data IMERG0.2. However, the ERA-5 has a good CC with the rain gauges in the south during the same season, which is dominantly affected by the Indian monsoon. For the same case, [10] shows that the detection of the rainfall in the Sahara Desert is poorly performed, which has the same rainfall mechanism as along the Al Hajar Mountains, whereas the performance is much better in areas where the East African monsoon affects. Also, [23] showed that the ERA-5 has a large uncertainty in desert areas. However, the monthly accumulation estimation shows an improvement in the CC values, which means that the rainfall temporal distribution is not the same as the observation, or the daily data underestimates the rainfall, but the accumulations get the monthly total closer to the observation. The daily data in springtime is much better compared to the summertime. It is shown that the reanalysis data have a good CC along the Al Hajar Mountains and the north in general. During this season, Oman is affected by mesoscale systems, including upper troughs or low-pressure systems. This rainfall mechanism leads to torrential rainfall on a larger scale, which is easier for models and satellite instruments to detect or estimate. The reanalysis data can estimate rainfall well during extreme events associated with torrential rainfall, as ERA-5 did in Italy [25]. In wintertime, the global daily data show that GPCC is the best in terms of CC, along with MERRA-2. This season has a similar characteristic to springtime, with some differences in severity and intensity, which are lower in winter than in springtime. In contrast, the monthly data shows a much better performance for the wintertime compared to the daily data. By and large, the performance varies with different time intervals, which has been discussed previously. Regarding the estimation accuracy, it is shown that most of the global datasets underestimate the rainfall on an annual scale, except for the satellite estimation, which overestimates the daily data. It is found that, from the daily data in annual scale, PERSIANN, ERA-5, NCEP, and GPCC are the best in terms of MBE, but they did not do well in other statistical tests. In general, the global data estimation of the rainfall during the autumn and winter is much better than in the other seasons based on the daily data. Also, it is shown that the estimation of the satellite instruments during summertime is poor compared to the reanalysis data, such as MERRA-2, GPCC, and CMORPH, which are the best performing reanalysis data in terms of the number of locations with the defined MBE threshold in this study, but the monthly data show worse estimations from all the datasets. The accuracy of rainfall estimation from satellite can be affected by the characteristics of mountainous terrain clouds, which can be a challenge for satellite instruments to detect the rainfall using microwave signals. Also, the Infar-red radiation is affected by the cloud top temperature [22]. The evaluation of global models and satellite instruments in estimating the rainfall over areas with a lack of rain gauges can affect the statistical significance of the performance, such as the Al Wusta region, where the rain gauges are scattered. To conclude, the study shows that the performance of the global reanalysis models and satellite instruments varies with time and space, as well as the temporal resolution. Furthermore, it is shown that the rainfall estimation during summertime is a challenge, especially for the monthly data. However, some datasets have a better performance than the others, as mentioned previously. It is recommended to have another project to assess the effect of including more stations to expand the network of rain gauges for comparison. Also, it is recommended to verify the quality of rainfall data from rain gauges.
Acronyms
MERRA-2: Modern-Era Retrospective Analysis for Research and Applications, Version 2
ERA-5: ECMWF Reanalysis, Version 5
PERSIANN: Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks
IMERG: Integrated Multi-Satellite Retrievals for GPM
NCEP: National Centre for Environmental Prediction
CRU: Climatic Research Unit
CPC: Climate Prediction Centre
GPCC: Global Precipitation Climatology Centre
GPCP: Global Precipitation Climatology Project
CMORPH: CPC Morphing
CC: Correlation Coefficient
RMSE: Root Mean Square Error
MBA: Mean Bias Error
DGMET: Directorate General of Meteorology