Researchers frequently encounter missing data challenges, particularly during the data collection phase. According to the literature [8] - [10] , the causes of missing data can generally be classified into two categories: intentional and unintentional.

Intentional Missing Data

During data collection, specific responses or conditions may render the collection of certain features unnecessary. For instance, the answer to a preliminary question might make subsequent questions irrelevant for some respondents. In such cases, missingness occurs by design, reflecting the logical structure of the data acquisition process.

Unintentional Missing Data

This category encompasses missingness arising from real-world challenges such as errors in manual data entry, sensor failures, system malfunctions, or difficulties in accessing certain measurements. Unintentional missingness is often random and more problematic, as it can compromise data quality if not properly addressed.

A pattern refers to a vector, case, or observation characterized by n features, each supporting continuous, discrete, or symbolic values. Mathematically, a pattern can be represented as:

$X=\left[x_1,x_2,\cdots ,x_n\right]$

Figure 2 illustrates two examples: (a) a complete pattern without missing values and (b) a pattern containing missing data, where black cells represent missing entries [6] [11] .

Missing data patterns can vary considerably. Figure 3 shows six prototypical missing data structures commonly encountered, following the notation from [12] .

The six principal patterns are described as follows:

Missing data mechanisms describe the relationship between missingness and the observed or unobserved values in the dataset. Following the seminal work of Little and Rubin [4] , three principal mechanisms are distinguished:

Missing Completely at Random (MCAR)

The probability of a value being missing is independent of both observed and unobserved data. MCAR typically occurs in datasets with less than 5% missingness [4] and implies that the observed cases are representative of the full sample. Formally:

$P\left(\text{Missing}|\text{Complete Data}\right)=P\left(\text{Missing}\right)$ (1)

Missing at Random (MAR)

The probability of missingness depends only on the observed data but not on the unobserved (missing) values themselves. Under MAR, correct modeling of the observed data can mitigate missingness bias. Formally:

$P\left(\text{Missing}|\text{Complete Data}\right)=P\left(\text{Missing}|\text{Observed Data}\right)$ (2)

Not Missing at Random (NMAR)

The probability of missingness depends on the unobserved (missing) data itself. NMAR poses the greatest challenge because traditional imputation methods generally assume MCAR or MAR. Formally:

$P\left(\text{Missing}|\text{Complete Data}\right)=P\left(\text{Missing}|\text{Observed Data},\text{Missing Data}\right)$ (3)

Among these mechanisms, NMAR is particularly problematic and often requires specialized modeling approaches. Ignoring the mechanism underlying missingness can lead to biased estimates and compromised inference. In particular, listwise deletion under MCAR remains unbiased but sacrifices efficiency, while under MAR or NMAR, it introduces systematic biases.

Among the most intuitive strategies, mean imputation replaces missing entries with the mean of the corresponding variable. Its simplicity and computational efficiency make it attractive, particularly for preliminary analyses or low-stakes applications [14] . However, the method has serious drawbacks: it artificially reduces data variance, weakens feature correlations, and may bias estimates, especially under non-MCAR missingness mechanisms. Zhang [15] emphasized that mean, median, and frequent-value imputations introduce systematic errors in variance and covariance estimation, leading to unreliable statistical inferences. Farhangfar et al. [16] confirmed that simple imputations should be restricted to cases where missingness is rare (less than 10%) and where model robustness to data quality is not a critical requirement. Despite its limitations, mean imputation remains widely used in industrial settings where interpretability and speed outweigh concerns about statistical rigor.

Introduced by Rubin et al. [4] , Multiple Imputation (MI) has become a gold standard for addressing missing data, especially under MAR assumptions. MI accounts for uncertainty by creating multiple complete datasets, analyzing each separately, and combining the results according to Rubin’s rules [17] . This approach preserves variability and improves inferential validity compared to single imputation.

Recent advances include:

While MI is computationally demanding, its ability to model the imputation uncertainty makes it preferable for high-stakes analyses, such as medical research and policy evaluations.

Regression imputation imputes missing values based on predictive models built from observed data [3] . Linear or logistic regression models are typically used to estimate missing entries, preserving inter-variable dependencies. However, this method is not without challenges:

To address these limitations, Bayesian Regression Imputation (BRI) [20] models the full posterior distribution of missing values, thereby incorporating uncertainty into the imputation process. Regression-based methods remain powerful when feature relationships are strong and correctly specified but require careful model validation.

The K-Nearest Neighbors (KNNs) approach, originally proposed by Cover et al. [21] , imputes missing values by leveraging local similarity among instances. It operates through: Measuring distances between observations, identifying K closest neighbors and averaging their feature values to impute missing data.

KNN imputation is non-parametric, meaning it does not assume any particular data distribution. However, its computational cost grows with dataset size [22] . Recent developments, such as Approximate KNN using Locality-Sensitive Hashing (LSH) [23] , significantly reduce the computational burden, making KNN viable even for larger datasets.

KNN remains particularly attractive for imputation in biomedical and environmental datasets where local correlations dominate.

The Expectation-Maximization (EM) algorithm [24] provides a statistically grounded approach to imputation under MCAR or MAR assumptions. It iteratively alternates between:

Convergence is declared once changes between iterations fall below a threshold. While EM offers elegant theoretical properties, its practical limitations include:

Stochastic versions (SEM) and Variational EM (VEM) [25] introduce approximations to enhance convergence speed. EM remains a reference method for structured missingness problems, particularly in psychometrics and clinical studies.

Multi-Layer Perceptrons (MLPs) are neural network models capable of modeling complex non-linear relationships, making them well-suited for imputing missing data [6] . Theoretical underpinnings of MLP-based imputation lie in their universal approximation capabilities, enabling them to effectively capture intricate data structures and interdependencies. However, the performance and applicability of MLPs heavily depend on the dataset size and complexity of missingness patterns. Training separate networks for distinct missingness configurations significantly increases computational complexity, demanding considerable computational resources and careful hyperparameter tuning. Furthermore, small datasets exacerbate the risk of overfitting, necessitating regularization techniques like dropout, L2 regularization, and early stopping. Empirical successes have been documented in domains such as breast cancer diagnosis [26] , where data complexity justified the computational overhead, and environmental monitoring [27] , where the nonlinear relationships between variables benefited from MLPs’ modeling strength. The advancement of self-supervised learning frameworks is anticipated to mitigate existing limitations, further enhancing MLP-based imputation effectiveness in diverse scenarios.

Certain machine learning algorithms inherently accommodate missing data without prior imputation by leveraging algorithm-specific mechanisms. Decision trees (e.g. J48, C4.5) use surrogate splits to manage missing features, theoretically allowing robust predictions despite incomplete inputs. Naïve Bayes models handle missingness through marginal probability estimations, assuming conditional independence among features, which simplifies missing data treatment. Random forests inherently accommodate missing data during bootstrapping by selecting optimal splits that minimize prediction error despite incomplete observations [28] . Support Vector Machines (SVMs), particularly Sequential Minimal Optimization (SMO) variants, exhibit robustness by optimizing decision boundaries based solely on observed dimensions [29] . Nonetheless, these methods have practical limitations; for instance, extensive missingness in critical features can reduce predictive accuracy, and surrogate splits in decision trees can sometimes obscure model interpretability. Such algorithms remain particularly advantageous for rapid prototyping and real-time decision-making, where preprocessing and imputation are impractical or computationally prohibitive.

Recent advancements in deep learning have significantly expanded capabilities in missing data imputation, especially for high-dimensional and complex datasets. Methods such as Generative Adversarial Imputation Networks (GAINs) [30] leverage adversarial training to generate realistic imputations, theoretically benefiting from the generative modeling of data distributions. BRITS [31] employs recurrent neural networks to effectively capture temporal dependencies, ideal for time-series datasets with sequential missingness patterns. Transformer-based approaches, including SAITS [32] and MissFormer [33] , utilize self-attention mechanisms capable of modeling extensive long-range dependencies within sequences or structured tabular data. Despite these strengths, deep learning methods inherently require extensive computational resources, careful hyperparameter tuning, and robust training frameworks to prevent model instability or mode collapse, especially in adversarial contexts. Empirical comparisons suggest SAITS and BRITS outperform traditional imputation methods in sequential contexts, while MissFormer demonstrates considerable potential in general tabular datasets due to its direct handling of heterogeneous missing data types.

Graph Neural Networks (GNNs) exploit relational structures inherent in many datasets to improve imputation performance. Theoretical foundations for graph-based imputation methods, such as GRIN [34] and TSI-GNN [35] , involve leveraging node relationships and neighborhood information to infer missing values effectively. By modeling spatiotemporal dynamics and relational contexts explicitly, GNNs provide superior performance in structured domains such as social networks, sensor networks, and biological systems. However, significant limitations include the necessity for well-defined relational data, the computational cost associated with constructing and training on large graphs, and the sensitivity of GNN models to noise and incorrect relational assumptions. Practical implementations should carefully consider the dataset’s suitability, particularly evaluating relational structure strength, node interdependencies, and scalability concerns.

Modern ensemble tree algorithms, including XGBoost [36] , LightGBM [37] , and CatBoost [38] , natively manage missing data effectively through built-in mechanisms such as optimal split-direction learning and missing value masking. Theoretically, these algorithms mitigate missingness by systematically identifying splits that minimize prediction error, thus maintaining robustness without explicit data imputation. Although effective, these methods can reduce model interpretability due to their implicit handling of missing values, and high levels of missingness might sometimes decrease model performance compared to explicit imputation strategies. Comparative evaluations indicate these methods are especially beneficial in large-scale, sparse datasets, offering computational efficiency and robust performance when preprocessing and explicit imputation are impractical. Clearly articulating these scenarios facilitates better methodological choices tailored to specific application contexts.

Handling missing data remains a critical challenge in contemporary data analysis pipelines across various scientific and industrial domains. The primary objective of this study is to present a comprehensive review of classical and modern techniques for recovering missing data, emphasizing machine learning-based and data-driven methodologies. The choice and effectiveness of imputation methods fundamentally depend on the specific patterns and mechanisms underlying the missing data. Structured patterns, such as monotone or planned missingness, naturally support targeted imputation strategies like regression-based methods or Expectation-Maximization (EM), benefiting from clear and predictable feature dependencies. Conversely, arbitrary or general patterns characterized by random and irregular missing entries necessitate robust and flexible approaches, including Random Forests or advanced deep-learning models (e.g. GAIN, MissFormer).

Additionally, understanding the missing data mechanisms—Missing Completely at Random (MCAR), Missing at Random (MAR), or Not Missing at Random (NMAR)—is essential for method selection. Under MCAR conditions, simpler methods such as mean imputation or listwise deletion remain unbiased yet may sacrifice statistical efficiency. For MAR data, techniques that leverage observed correlations, such as Multiple Imputation by Chained Equations (MICEs), provide unbiased and efficient imputations. NMAR scenarios, representing the most challenging conditions, necessitate specialized modeling frameworks such as selection or pattern-mixture models, explicitly addressing dependencies related to the unobserved values. Thus, accurately identifying and characterizing missing data patterns and mechanisms is crucial for selecting appropriate imputation methods, ultimately ensuring their effectiveness and validity.

The success of missing data recovery strategies is closely tied to the initial feature selection process. Selecting informative and relevant features not only mitigates the adverse impact of missingness but also enables the effective application of algorithms that natively tolerate missing values. In contrast, retaining non-informative or sparsely populated features can degrade model performance and lead to biased inferences, particularly in high-dimensional settings. Feature selection thus must be adapted to the missing data patterns and mechanisms at play, reinforcing its strategic importance in the imputation pipeline. Simple statistical imputation methods—such as mean, median, or mode imputation—continue to be widely used due to their simplicity and computational efficiency. However, as confirmed in this study and supported by the literature, these methods tend to artificially reduce data variability and neglect inter-feature dependencies. Consequently, they often introduce biases and degrade model reliability, especially when the missingness rate exceeds 10% - 20%. Advanced statistical approaches, such as Expectation-Maximization (EM) and Multiple Imputation (MI), provide better estimates under the Missing at Random (MAR) assumption by modeling the underlying data distribution. Nevertheless, these methods are computationally intensive and show limitations when faced with high-dimensional datasets or Not Missing at Random (NMAR) conditions, which are increasingly encountered in real-world data.

Recent advances in machine learning have significantly enhanced the capacity to handle missing data. Techniques based on Generative Adversarial Networks (e.g. GAIN), Variational Autoencoders, Transformer-based architectures (e.g. SAITS, MissFormer), and Graph Neural Networks (e.g. GRIN, TSI-GNN) have demonstrated superior performance compared to classical methods. These models are particularly adept at:

Such capabilities make deep learning-based methods highly suitable for critical application domains such as healthcare analytics, financial risk prediction, and Internet of Things (IoT) monitoring.

In fuzzy systems, missing inputs exacerbate uncertainty propagation, thereby impairing the reasoning process and decision reliability. Hybrid strategies that combine fuzzy logic with machine learning-based imputation or fuzzy clustering show promise in overcoming these limitations, especially in sensitive fields like medical diagnosis and environmental monitoring.

The selection of a missing data handling technique should be driven by multiple factors, including:

Traditional statistical methods remain appropriate for small, low-dimensional datasets with moderate missingness rates. Conversely, modern deep learning and graph-based models offer scalable and robust alternatives for complex, large-scale data environments.

To provide a consolidated view, Table 1 presents a comparative summary highlighting the key distinctions between classical statistical imputation methods and modern deep learning-based techniques, encompassing their respective strengths, limitations, and typical application domains.

Two datasets were employed to evaluate the performance of various missing data handling techniques:

Heart Disease Dataset: Sourced from https://www.kaggle.com/c/heart-disease-uci, this dataset comprises 303 instances and 14 features. It is fully complete, containing no missing values (0% missingness).

Kidney Disease Dataset: Also obtained from https://www.kaggle.com/datasets/akshayksingh/kidney-disease-dataset , this dataset consists of 400 instances and 26 features. It is an inherently incomplete dataset, containing real missing values. The dataset includes two diagnostic classes: “sick” (59%) and “not sick” (41%).

All datasets were preprocessed through normalization prior to applying missing data techniques.

To systematically evaluate imputation methods, artificial missingness was introduced into the complete heart disease dataset at varying levels: 10%, 20%, 30%, 40%, and 50%. Missing values were inserted completely at random (MCAR mechanism) across the dataset to simulate realistic scenarios. This procedure generated five experimental settings, each corresponding to a different proportion of missing data.

For the heart disease dataset, five different imputation strategies were applied to handle missing data: Mean Imputation, K-Nearest Neighbors (KNNs) Imputation, Multiple Imputation by Chained Equations (MICEs), Regression Imputation, Expectation-Maximization (EM) Imputation.

For the KNN imputation method, the optimal number of neighbors (k) was determined by minimizing the Mean Error Rate (MER) for each missingness scenario.

For the kidney disease dataset, the following classifiers were used to assess the impact of missing data handling: J48 Decision Tree, Naive Bayes Classifier, Random Forest, REP Tree, Support Vector Machine (via Sequential Minimal Optimization, SMO) and the XGBoost (Extreme Gradient Boosting) algorithm was included in the experimental evaluation. XGBoost is a modern ensemble method that natively supports missing values by learning optimal split directions during training, eliminating the need for explicit imputation. Its high scalability and robustness make it a strong candidate for structured data classification tasks.

Each model was trained and evaluated after applying appropriate missing data-handling techniques.

Performance evaluation was conducted using the following criteria:

$\text{RMSE}=\sqrt{\frac{1}{n}\underset{i=1}{\overset{n}{{\displaystyle \sum }}}{\left(y_i-{\hat{y}}_i\right)}^2}$ (4)

$\kappa =\frac{p_o-p_e}{1-p_e}$ (5)

where $p_o$ is the observed agreement and $p_e$ is the expected agreement by chance.

To determine the optimal number of neighbors K for KNN-based imputation, an empirical evaluation was conducted across varying levels of missingness (10% - 50%) in the Heart disease dataset. Figure 4 illustrates the evolution of the error rate as a function of K for each missingness level. As expected, the error rate generally decreases with increasing K, although fluctuations are observed due to dataset variance. Notably, the lowest error rate is achieved for K = 16 when 10% of the data is missing, and for K = 7 when the missingness reaches 50%. This analysis highlights the importance of tuning K for optimal performance in KNN imputation.

Table 2 presents the Root Mean Square Error (RMSE) values for five imputation methods (mean, KNN, MICE, regression, and EM) applied to the heart disease dataset under five missing data scenarios. The Expectation-Maximization (EM) method consistently achieved the lowest RMSE across all levels of missingness, confirming its robustness in preserving data integrity. In contrast, regression-based imputation exhibited the poorest performance, particularly at higher missingness levels (e.g. RMSE = 0.67 at 50%). These findings indicate that more sophisticated probabilistic or iterative methods are better suited for handling significant data gaps.

The classification performance of five algorithms (J48, Naïve Bayes, Random Forest, REP Tree, SMO and XGBoost) was assessed using the kappa statistic before and after applying mean imputation. As shown in Table 3 , mean imputation substantially improved agreement scores across all classifiers. The SMO and XGBoost algorithm achieved the highest kappa value after imputation ( $\kappa =0.88$, $K=0.9$), indicating strong predictive consistency.

To further assess classification performance, Receiver Operating Characteristic (ROC) analysis was conducted. Table 4 shows the Area Under the ROC Curve for the six classifiers, with and without imputation. Across all models, performance improved after imputation. SMO and XGBoost again outperformed other methods, achieving an ROC higher than 95%. The lowest performance was observed for REP Tree, which showed limited sensitivity to imputation.

The experimental analysis provided quantitative insights into how different missing data handling strategies influence downstream tasks such as imputation accuracy and classification performance across datasets of varying complexity and size. In this section, we systematically discuss the empirical trends, emphasizing comparative granularities among similar methodological approaches.

Results obtained from the Heart Disease dataset ( Table 2 ) highlight that the Expectation-Maximization (EM) approach consistently outperformed alternative imputation methods-including mean, KNN, MICE, and regression-based approaches-especially at lower levels of missingness (below 40%). EM yielded the lowest RMSE values, reflecting its capability to effectively model underlying data distributions. For example, at missingness levels of 10% and 20%, EM achieved RMSE scores of 0.16 and 0.20, respectively, demonstrating notable superiority. However, as missingness increased beyond 40%, the performance advantage of EM significantly diminished, allowing simpler imputation methods such as mean imputation to become competitively viable. This shift underscores how substantial missingness erodes data structures, thus diminishing the relative benefit of sophisticated probabilistic models.

Moreover, a detailed analysis of KNN hyperparameters ( Figure 4 ) revealed that the optimal number of neighbors (K) is sensitive to the proportion of missing data, emphasizing the necessity of adaptive tuning for instance-based methods.

In classification experiments conducted on the Kidney Disease dataset, machine learning algorithms inherently robust to missing values, such as Random Forest, SMO, and XGBoost, demonstrated superior performance compared to models trained on datasets imputed with simpler techniques, such as mean imputation. Notably, both SMO and XGBoost achieved the highest Kappa scores and ROC AUC values ( Table 3 and Table 4 ). Specifically, SMO achieved a Kappa score of 0.88 and ROC AUC of 96.5%, closely matched by XGBoost, which exhibited consistently robust performance across varying missingness scenarios. These results underscore the robustness of gradient boosting and kernel-based models in internally mitigating the detrimental effects of missing data without requiring explicit imputation procedures.

Crucially, while the results provide clear quantitative assessments of method performance, they also reveal a lack of detailed comparative insights among similar deep-learning architectures within the same scenarios. Future empirical analyses should aim to incorporate finer-grained comparisons between deep-learning approaches (e.g. GAIN versus MissFormer), elucidating how architectural differences influence their performance in handling identical missing data situations. Such granularity will facilitate deeper methodological understanding and enable researchers to make more informed decisions tailored precisely to their analytical contexts.

A consistent observation across both evaluated datasets is that no single imputation or modeling strategy universally excels. EM demonstrates strong efficacy in moderate missingness scenarios but loses its advantage with increasing missingness. Conversely, robust models like SMO, Random Forest, and XGBoost offer stable performance irrespective of explicit imputation, although their exact benefits depend on data complexity and missingness patterns.

The experimental findings stress the necessity of context-sensitive method selection. For datasets exhibiting moderate missingness and structured inter-feature dependencies, combining EM-based imputation with kernel-based (SMO) or gradient boosting (XGBoost) classification strategies is recommended. In contrast, for high missingness or datasets lacking strong feature correlations, simpler or hybrid imputation approaches may suffice. Incorporating performance metrics such as RMSE, ROC AUC, and Kappa scores into decision-making processes will further enhance the principled selection of suitable methodologies.

Finally, to support method selection decisions, Table 5 summarizes the evaluated imputation and modeling strategies, detailing their strengths, limitations, and suitable application contexts. This comparative overview underscores the importance of aligning method choices with the statistical properties of the dataset, analytical objectives, and practical constraints specific to each application.

This comparative summary highlights the importance of aligning method selection with the statistical properties of the data, the analytical objectives, and the operational constraints of the target application.