Missingness is considered completely random when it does not depend on any other variables. This condition, known as Missing Completely at Random (MCAR), occurs when the probability of missing data is unrelated to both observed and unobserved variables [29] - [31] . In this scenario, the missingness is purely due to chance, with no influence from any characteristics of the data [4] . MCAR is the most stringent assumption and rarely occurs in practice. However, when data are MCAR, analyses using only complete cases will be unbiased, although potentially inefficient [1] .

Let $R$ be the missingness indicator and $Y$ be the complete data. MCAR can be defined as: $P\left(R|Y\right)=P\left(R\right)$.

For example, in a survey, some participants accidentally skip questions regardless of their characteristics or responses to other items.

The probability of missing data depends on other observed variables but not on the missing data itself. This condition, known as Missing at Random (MAR), occurs when the likelihood of missingness is related to observed variables but not to unobserved ones [18] . MAR represents a less stringent assumption compared to Missing Completely at Random (MCAR) and is often more realistic in practice. In summary, data are considered MAR when the probability of missingness is influenced by observed variables while remaining independent of unobserved variables [18] . MAR allows for relationships between observed variables and the probability of missingness. Many modern imputation methods, such as multiple imputation, assume MAR [17] .

Let $Y_{obs}$ be the observed data and $Y_{mis}$ be the missing data. MAR can be defined as: $P\left(R|Y\right)=P\left(R|Y_{obs}\right)$.

For example, in a longitudinal study on income, older participants are more likely to withhold information about their earnings, but this likelihood is not related to the actual income amount after accounting for age. For example, men might be less likely to answer questions about emotions in a survey.

Missingness is categorized as Missing Not at Random (MNAR) when the probability of missing data is influenced by the values of the missing data itself or by unobserved variables. In this case, the missingness is directly related to the unobserved values [4] . MNAR is the most challenging type of missing data to handle. Standard imputation methods can yield biased results, necessitating specialized techniques or sensitivity analyses to address the issue effectively [32] .

MNAR occurs when the probability of missing data depends on unobserved variables or the missing values themselves [4] . This is the most challenging type of missingness to address and often requires specialized techniques. MNAR occurs when: $P\left(R|Y\right)\ne P\left(R|Y_{obs}\right)$.

An example of MNAR is in a mental health survey, participants with severe depression may be less likely to complete questions about their symptoms, with this likelihood directly related to the severity of their undisclosed condition. Similarly, individuals with high incomes might be less inclined to report their income in a survey.

In practice, it is often impossible to definitively determine whether data are MAR or MNAR based solely on observed data [33] . Therefore, researchers often must rely on subject-matter knowledge and conduct sensitivity analyses to assess the robustness of their findings under different missing data assumptions [17] .

Traditional methods for handling missing data have been widely used due to their simplicity and ease of implementation [1] . Traditional imputation methods are widely used to handle missing data in datasets. These methods aim to replace missing values with plausible estimates based on the available data. The most common traditional imputation techniques are discussed as follows.

Missing categorical values were replaced with the mode, while missing numerical values were replaced with the mean of the corresponding feature. This method involves replacing missing values with the mean or median of the observed values for that variable [4] . Although simple, this approach can result in biased estimates and an underestimation of standard errors. For numerical data, missing values are replaced with the mean or median of the non-missing values in the same column, while for categorical data, the mode (most frequent value) is utilized.

This iterative approach uses mean/mode imputation to initialize the dataset [16] . Then, a random forest is trained to predict the missing values in each feature, iteratively refining the imputed values until convergence (defined by a lack of improvement in the imputed matrix). Convergence criteria included a maximum of 20 iterations and 100 trees. The difference between successive imputed matrices ( $M_{imp\_new}$ and $M_{imp\_old}$) for numerical ( $N$) and categorical ( $F$) features was measured as:

${\delta }_N=\frac{{{\displaystyle \sum }}_{j\in N}{\left(M_{new}^{imp}-M_{old}^{imp}\right)}^2}{{{\displaystyle \sum }}_{j\in N}{\left(M_{new}^{imp}\right)}^2}$ (1)

(2)

where $F_{NA}$ represents the number of missing values in categorical variables.

Last Observation Carried Forward (LOCF) and Next Observation Carried Backward (NOCB) are both methods used for handling missing data in longitudinal studies. LOCF fills in missing data points by carrying forward the last observed value. For example, if a participant’s value is missing at a follow-up, the last recorded value is used to fill in that gap [34] .

The advantages of LOCF method are: 1) Simplicity: Easy to implement and understand. 2) Preservation of Sample Size: Retains all participants in the analysis, which can be important in clinical trials. On the other hand, the disadvantages of LOCF method are: 1) Assumption of Stability: Implies that the last observation is a good estimate for future values, which may not hold true. 2) Potential Bias: Can introduce bias if the last observed value is not representative of the participant’s state at the time of the missing data. 3) Underestimation of Variability: Fails to account for natural fluctuations in the data, potentially leading to misleading conclusions.

NOCB fills in missing data points by carrying the next observed value backward. For example, if a participant’s value is missing before a subsequent observation, the next recorded value is used to fill in the gap.

The advantages of NOCB method are: 1) Preservation of Trends: Can better reflect changes over time if later observations are more representative of the participant’s condition. 2) Potentially Reduces Bias: Addresses some issues associated with LOCF by using future data, which may be more accurate. On the other hand, the disadvantages of NOCB method are: 1) Assumption of Continuity: Assumes that the value observed in the future can be reliably applied to the past, which may not always be valid. 2) Temporal Distortion: Can introduce bias if there are systematic changes between the missing data point and the next observation. 3) More Complex: Generally considered less intuitive and harder to justify in some contexts than LOCF.

Hot deck imputation involves replacing missing values with observed values from similar respondents or cases [35] . This method can help preserve the distribution of the data but may be challenging to implement for large datasets. This method replaces missing values with values from a similar donor record (a record with non-missing values) in the dataset.

The main steps of Hot Deck Imputation are: 1) Define a set of matching criteria (e.g., age, gender, income) based on the variables with available data. 2) For each missing value, find a donor record that matches the criteria. 3) Replace the missing value with the corresponding value from the donor record. Advantages: Simple, can be effective for handling missing values in categorical variables.

This method generates $n$ imputed datasets [36] . Parameter estimates and standard errors are calculated for each dataset, and then pooled to obtain overall estimates ( $\bar{P}$) and variances ( $\bar{V}$):

$\bar{P}=\frac{1}{n}\underset{i=1}{\overset{n}{{\displaystyle \sum }}}{P}_i$

$\bar{V}=\frac{1}{n}\underset{i=1}{\overset{n}{{\displaystyle \sum }}}{V}_i$

Between-dataset variability ( $\bar{B}$) is also calculated:

$\bar{B}=\frac{1}{n-1}\underset{i=1}{\overset{n}{{\displaystyle \sum }}}{\left(P_i-\bar{P}\right)}^2$

This approach uses a denoising autoencoder, pre-imputed with kNN ( $k=5$), to learn feature relationships and impute missing values [37] . The encoder and decoder are defined by:

$f_{enc}\left(X\right)=s\left(X\cdot W^{\text{T}}+b\right)$

$f_{dec}\left(Y\right)=s\left(Y\cdot {\bar{W}}^{\text{T}}+\bar{b}\right)$

where $Y=f_{enc}\left(X\right)$ and $Z=f_{dec}\left(Y\right)$ are the hidden and output vectors, respectively; $W$ and $b$ are the encoder weights and bias; and $\bar{W}$ and $\bar{b}$ are the decoder weights and bias. Training minimizes the reconstruction error between $X$ and $Z$.

This enhanced version uses an undercomplete autoencoder architecture. To avoid overfitting to the initial kNN imputation, the kNN imputation is updated every 10 epochs, varying the $k$ value within a predefined range. Furthermore, the missing values to be imputed are randomly selected at the start of each epoch. A custom loss function combines mean squared error (MSE) for numerical features and binary cross-entropy (BCE) for categorical features:

$\text{MSE}=\frac{1}{N}\underset{i=1}{\overset{N}{{\displaystyle \sum }}}{\left({\hat{Y}}_i-Y_i\right)}^2$

$\text{BCE}=-\frac{1}{N}\underset{i=N+1}{\overset{N+C}{{\displaystyle \sum }}}{y}_i\cdot \log \left(p\left(y_i\right)\right)+\left(1-y_i\right)\cdot \log \left(1-p\left(y_i\right)\right)$

$\text{Loss}=\text{RMSE}+\text{BCE}$

Multiple imputation (MI) is a powerful technique for handling missing data that addresses the limitations of single imputation methods. Unlike single imputation, which replaces missing values with a single estimate, MI generates multiple complete datasets by imputing missing values multiple times, each time using different plausible values [38] . This approach accounts for the uncertainty introduced by missing data, leading to more accurate and robust analyses.

The main steps of MI method are: 1) Imputation: Multiple complete datasets are created by imputing the missing values using a statistical model that accounts for the relationships between variables. The model is typically based on the observed data and assumes a specific distribution for the missing values. Each imputed dataset is generated using different random draws from the conditional distribution of the missing values, reflecting the uncertainty associated with the missing data. 2) Analysis: Each of the imputed datasets is analyzed separately using the chosen statistical methods. This results in multiple sets of estimates for the parameters of interest. 3) Pooling: The results from each imputed dataset are combined using appropriate methods to obtain a single set of estimates and standard errors that reflect the uncertainty introduced by missing data. The most common pooling methods include averaging the estimates and variances across the imputed datasets.

The advantages of Multiple Imputation are: 1) Accounts for Uncertainty: MI explicitly acknowledges the uncertainty associated with missing values by generating multiple plausible estimates. This results in more realistic confidence intervals and p-values [39] . 2) Reduces Bias: By generating multiple imputed datasets, MI reduces the bias introduced by single imputation methods, especially when the missing data is not missing at random. 3) More Accurate Estimates: MI generally produces more accurate estimates of parameters and statistical tests than single imputation methods. 4) Provides Insights into Missing Data: The variability of estimates across imputed datasets can provide insights into the sensitivity of the analysis to the missing data.

The challenges of Multiple Imputation are: 1) Computational Complexity: MI can be computationally intensive, especially for large datasets and complex models. 2) Model Selection: Choosing the appropriate imputation model is crucial. The model should accurately reflect the relationships between variables and the distribution of the missing data. 3) Software Requirements: Specialized software is often required to perform multiple imputation, as it involves generating and analyzing multiple datasets.

The choice of imputation method depends on the specific characteristics of the data, the nature of the missing data, and the goals of the analysis. It is important to consider the potential biases and limitations of each method before applying it to your data. Table 2 summarizing common imputation methods, their use cases, advantages, disadvantages, Python packages, and suitability for classification or regression problems.

Regression imputation uses the relationship between variables to predict missing values based on observed data [9] . This method can account for relationships between variables but may overestimate the strength of these relationships.

This method identifies the k-nearest neighbors to the missing value based on the similarity of other features [40] . The missing value is then replaced with the average (for numerical data) or the most frequent value (for categorical data) among those neighbors.

The main steps of KNN Imputation are: 1) Calculate the distance between the data point with the missing value and all other points in the dataset. 2) Identify the k-nearest neighbors based on these distances. 3) For numerical data, compute the average of the corresponding values in the k-nearest neighbors and use it as the imputed value. For categorical data, select the most frequent value among the neighbors.

Missing values were imputed using the average (numerical features) or mode (categorical features) of the $k$ nearest neighbors in feature space, using Euclidean distance. For example, $k=4$. Formally, for a sample $S\left(X,Y,0\right)$ with four nearest neighbors $N_4=\left\{\left(X_i,Y_i,1\right)|i=1,2,3,4\right\}$, the imputed value $Y$ is calculated as:

$Y=\{\begin{array}{ll}\arg {\max }_z\left\{{\displaystyle {\sum }_{\left(X_i,Y_i,1\right)\in N_4}1\left(Y_i=z\right)}\right\}\hfill & \text{if}Y\text{is}\text{categorical}\hfill \\ \frac{1}{4}{\displaystyle {\sum }_{i=1}^4{Y}_i}\hfill & \text{if}Y\text{is}\text{numerical}\hfill \end{array}$

where $z\in \left\{0,1\right\}$ and $1\left(Y_i=z\right)$ is an indicator function. The Euclidean distance between points $x$ and $y$ is defined as:

$d\left(x,y\right)=\sqrt{{\displaystyle {\sum }_{i=1}^n{\left(x_i-y_i\right)}^2}}$

where $n$ is the number of features.

KNN imputation identifies the $k$ most similar cases to the ones with missing data and uses their values for imputation [41] [42] . This method can capture complex relationships in the data but may be computationally expensive for large datasets. K-Nearest Neighbors (KNN) imputation is a popular method for handling missing data, leveraging the similarities between observations. Here is a closer look at how KNN imputation works, its advantages, and its limitations.

The advantages of KNN Imputation are: 1) Flexibility: KNN can be applied to both numerical and categorical data, making it versatile. 2) Local Information: By considering the closest observations, KNN can capture local data patterns, potentially leading to more accurate imputations. 3) Non-parametric: KNN does not assume a specific data distribution, which can be advantageous in real-world datasets.

The limitations of KNN Imputation are: 1) Computationally Intensive: KNN can be slow, especially with large datasets, since it requires distance calculations for each observation. 2) Curse of Dimensionality: As the number of features increases, the concept of “closeness” can become less meaningful, making it harder to identify true neighbors. Sensitive to Outliers: The presence of outliers can skew distance calculations, leading to poor imputation results.

These methods use tree-based models to predict missing values based on other variables [16] . They can handle both categorical and continuous variables and capture non-linear relationships. Decision Trees and Random Forests are powerful machine learning techniques that can also be used to input missing values into datasets. Here’s a breakdown of how these methods work for imputation, along with their advantages and disadvantages.

The advantages of Decision Trees Imputation: 1) Captures Non-linear Relationships: Decision trees can model complex relationships between features, potentially leading to more accurate imputations. 2) Interpretable: The model is relatively easy to interpret, as you can visualize how decisions are made. The disadvantages of Decision Trees Imputation: 1) Overfitting: Decision trees can easily overfit to the training data, especially if not properly pruned. 2) Sensitivity to Noise: Outliers can affect the structure of the tree, impacting the imputation results.

The advantages of Random Forests Imputation: 1) Improved Accuracy: Random forests generally provide better accuracy than single decision trees due to their ensemble nature, reducing overfitting and variance. 2) Robust to Outliers: The averaging mechanism makes random forests less sensitive to outliers compared to individual trees. 3) Handles Large Datasets: Random forests can effectively manage large datasets with high dimensionality. The disadvantages of Random Forests Imputation: 1) Complexity: The model is less interpretable than a single decision tree, as it’s harder to visualize how predictions are made. 2) Computationally Intensive: Training multiple trees can be resource-intensive, especially for large datasets.

Support Vector Machines (SVM) are primarily known for classification and regression tasks. However, they can also be utilized to input missing data. SVM-based imputation methods use support vector regression to predict missing values [43] [44] . These methods can be effective for high-dimensional data but may require careful tuning of hyperparameters.

The advantages of SVM-Based Imputation are: 1) Effective for Non-linear Relationships: SVM can capture complex, non-linear relationships in the data by using different kernel functions (e.g., polynomial, radial basis function). 2) Robustness to Overfitting: SVM includes regularization parameters that help prevent overfitting, making it suitable for high-dimensional datasets. 3) Flexibility: SVM can be applied to both classification (categorical variables) and regression (continuous variables) tasks.

The limitations of SVM-Based Imputation are: 1) Computational Complexity: SVM can be computationally intensive, particularly for large datasets, due to the optimization required for finding the best hyperplane. 2) Parameter Sensitivity: The performance of SVM can be sensitive to the choice of kernel and hyperparameters (e.g., C and gamma), requiring careful tuning. 3) Requires Sufficient Data: SVM models generally require a substantial amount of complete data to build an accurate model, which may not always be available.

Autoencoders are neural networks that can learn compressed representations of data and have been applied to missing data imputation [45] - [47] . They can capture complex patterns in the data but may require large amounts of training data.

GANs have been adapted for missing data imputation by learning to generate realistic imputed values [48] [49] . This approach can produce high-quality imputations but may be challenging to train and tune. The Generative Adversarial Imputation Network (GAIN) [50] employs a generative adversarial network (GAN) architecture. Unlike standard GANs, the discriminator in GAIN does not classify the entire generated output as real or fake; instead, it classifies each individual variable as either imputed or observed. Convergence is achieved when the generator produces imputations indistinguishable from the true data distribution. A “hint” mechanism augments the discriminator’s input with partial information about the missing values (M), represented by the hint vector H. This hint is typically a proportion of M (e.g., 90% identical). The generator then learns to impute the remaining values. The original work demonstrates that insufficient hints lead to multiple optimal generator outputs.

Formally, given a random vector Z, the generator produces an imputed dataset ${\tilde{X}}_i$, from which ${\bar{X}}_i$ is derived (Equation (1)). The discriminator loss function, $L_D$, is defined as:

$L_D\left(M,\bar{M},H\right)=\underset{i:H_i=0}{{\displaystyle \sum }}\left[M_i\cdot \log \left({\bar{M}}_i\right)+\left(1-M_i\right)\cdot \log \left(1-M_i\right)\right]$

where $M$ is the true mask vector, $\bar{M}$ is the generated mask vector, and $H$ is the hint vector. The summation is restricted to indices where $H_i=0$ to prevent overfitting to the hint. The discriminator is trained to minimize this loss:

${\min }_D-\underset{i=1}{\overset{batchsize}{{\displaystyle \sum }}}{L}_D\left(M_i,{\bar{M}}_i,H_i\right)$

The generator loss function comprises two terms: $L_G$, which measures the generator’s ability to deceive the discriminator, and $L_M$, which quantifies the accuracy of the imputation for observed values:

$L_G\left(M,\bar{M},H\right)=-\underset{i:H_i=0}{{\displaystyle \sum }}\left(1-M_i\right)\cdot \log \left(M_i\right)$

$L_M\left(\tilde{X},\bar{X}\right)=-\underset{i=1}{\overset{d}{{\displaystyle \sum }}}{M}_i\cdot Diff\left({\tilde{X}}_i,{\bar{X}}_i\right)$

where $d$ is the data dimensionality and $Diff$ is defined as:

$Diff\left(\tilde{X},\bar{X}\right)=\{\begin{array}{ll}{\left({\tilde{X}}_i-{\bar{X}}_i\right)}^2\hfill & \text{if}{X}_i\text{is}\text{numerical}\hfill \\ -X_i\log \left({\bar{X}}_i\right)\hfill & \text{if}{X}_i\text{is}\text{binary}\hfill \end{array}$

The generator is trained to minimize the combined loss:

${\min }_G\underset{i=1}{\overset{batchsize}{{\displaystyle \sum }}}{L}_G\left(M_i,{\bar{M}}_i,H_i\right)+\alpha L_{M_i}\left({\tilde{X}}_i,{\bar{X}}_i\right)$

where $\alpha$ is a scaling parameter.

Autoregressive Integrated Moving Average (ARIMA) models are a class of statistical models used for analyzing and forecasting time series data. ARIMA models can be employed to impute missing values in time series data by leveraging temporal dependencies [51] [52] . They are particularly useful when the data exhibits trends and seasonality. An ARIMA model is denoted as ARIMA (p, d, q), where “p” represents the order of the autoregressive (AR) component, “d” represents the degree of difference required to make the time series stationary, and “q” represents the order of the moving average (MA) component. The AR component models the relationship between the current observation and previous observations; the MA component models the relationship between the current observation and past forecast errors, and differencing (d) removes trends and makes the series stationary. A general ARIMA (p, d, q) model can be represented by the following equation:

$\varphi \left(B\right){\left(1-B\right)}^d{y}_t=\theta \left(B\right){ϵ}_t$

where $y_t$ is the time series at time $t$, $B$ is the backshift operator ( $B{y}_t=y_{t-1}$), $\varphi \left(B\right)$ is the autoregressive polynomial of order $p$, $\theta \left(B\right)$ is the moving average polynomial of order $q$, and ${ϵ}_t$ is white noise. The choice of $p,d$, and $q$ values is crucial for model fitting and depends on the characteristics of the specific time series being analyzed. Techniques like autocorrelation and partial autocorrelation functions (ACF and PACF) are often used to identify suitable model orders. Table 3 shows the summary of advanced imputation techniques.