Feature engineering is a crucial step in data preprocessing for machine learning. It involves identifying and transforming the variables most significantly impacting model outcomes and decision-making. This process converts raw data into meaningful features suitable for machine learning algorithms. Feature engineering includes selecting, extracting, and transforming the most relevant features from a dataset to enhance model performance and accuracy.

Figure 2 presents a cross-correlation heatmap highlighting key patterns and relationships within the data. This visualization aids in improving the model’s learning capability by revealing essential interdependence. After data cleaning and labeling, machine learning teams typically conduct thorough exploratory data analysis (EDA) to assess the data quality and fitness for modeling. Visual tools such as histograms, scatter plots, box plots, line graphs, and bar charts are vital in verifying data integrity. These visualizations help scientists detect data patterns, spot anomalies, test hypotheses, and validate assumptions.

The performance of machine learning models is greatly affected by the quality of the features employed in their training. Feature engineering involves a range of techniques that facilitate the generation of new features by combining or transforming existing ones. These approaches are essential.

Exploratory data analysis does not require formal modeling; instead, data science teams can utilize visualizations to gain meaningful insights from the data. The data collection stage involves compiling all relevant information necessary for machine learning, which can be a labor-intensive task due to the often-fragmented nature of data across various sources, such as personal computers, data warehouses, cloud storage, applications, and devices.

Connecting to these diverse data sources can present considerable challenges. Furthermore, the volume of data is increasing at an extraordinary pace, leading to large datasets that demand comprehensive analysis. Additionally, the formats and types of data can differ significantly depending on their source, complicating the integration of various data types, such as video and tabular data. These figures suggest that four major features determine the outcomes. We will consider both cases when all features are considered ( Figure 3 ) and only four features ( Figure 4 ).

Outlier detection and removal is a technique used to identify and eliminate anomalies from a dataset. The main goal of this process is to improve the accuracy of data representation, which can significantly impact model performance. The extent of this impact varies—some models are highly sensitive to outliers, while others remain largely unaffected.

For instance, linear regression is particularly susceptible to outliers, making it essential to address them before training a model. Various methods can be employed to manage outliers, including:

Removal: This approach involves deleting records containing outliers from the dataset. However, if outliers are present in multiple variables, this can lead to substantial data loss.

Replacing Values: Here, outliers are treated as missing values and replaced with imputed values deemed appropriate.

Capping: This method substitutes extreme values with a predetermined threshold, or a value based on the variable’s distribution.

Discretization: This process converts continuous variables into discrete categories by segmenting the range into intervals or bins.

The following boxplots illustrate the selected features. In descriptive statistics, a boxplot (also called a box-and-whisker plot) is a valuable tool for exploration data analysis. It provides a visual representation of data distribution and skewness by displaying quartiles, percentiles, and averages. A boxplot encapsulates the five-number summary of a dataset, consisting of:

Minimum Score: The lowest value in the dataset, excluding outliers, represented by the left whisker’s endpoint.

Lower Quartile (Q1): The first quartile, marking the value below which 25% of the data falls.

Median (Q2): The midpoint of the dataset, splitting it into two equal halves. It signifies that half of the values are above and half are below this point.

Upper Quartile (Q3): The third quartile, representing the value below which 75% of the data lies.

Maximum Score: The highest value in the dataset, excluding outliers, shown at the right whisker’s endpoint.

Whiskers: These extend from the box, covering the lower and upper 25% of the data, excluding extreme outliers.

Interquartile Range (IQR): The range between the first and third quartiles, representing the middle 50% of the data.

Boxplots efficiently summarize data using a simple visual format, making it easy to identify quartiles, medians, and outliers briefly. Outliers can significantly affect data analysis and machine learning models, leading to biased predictions, misleading conclusions, and distorted statistical measures.

To mitigate these effects, statistical methods such as the Interquartile Range (IQR) are used to quantify dispersion. The IQR measures the spread of the middle 50% of the data and is calculated as the difference between the 75th percentile (Q3) and the 25th percentile (Q1). The following figures show the boxplots of major features and outliers. Figure 5 shows Q1, Q2, Q3 quartiles and IQR (shadowed). Outliers are beyond the upper bound.

Figure 6 shows the boxplot for the Glucose records with outlier at 0 and Figure 7 demonstrates the Boxplot for the Pregnancy records.

Figure 8 presents the boxplot of the Age data. Although it indicates outliers beyond the upper bound, we will not exclude them in this case, as the cohort includes patients aged 60 - 70.

The Interquartile Range (IQR) is a statistical measure of dispersion that represents the spread of the middle 50% of a dataset. It is calculated by subtracting the 25th percentile (Q1) from the 75th percentile (Q3). To detect outliers using the IQR method, two boundaries are established:

These boundaries help identify potential outliers in a dataset. Any data point below the lower bound (Q1 – 1.5 × IQR) is considered an outlier, as it significantly deviates from the rest of the data and may warrant further review or removal. Similarly, any data point above the upper bound (Q3 + 1.5 × IQR) is considered an outlier, as it is substantially higher than most data points and may require special attention.

One of the main advantages of the IQR method is its resilience to skewed data distributions. Since it identifies outliers based on percentiles, it is less sensitive to extreme values. Additionally, the IQR method is straightforward to implement and interpret, providing a clear range within which most data points should fall. This makes it an effective tool for data analysis and quality control.

Outliers can arise from various factors and often result from either genuine variability in the data or errors in data collection, measurement, or recording. Common causes of outliers include:

The Z-score is used to standardize variables, giving insight into how far a particular observation is from the meaning. Specifically, the Z-score indicates how many standard deviations a data point is away from the mean. The process of transforming a feature into Z-scores is known as standardization.

If the Z-score of a data point exceeds 3, it suggests the data point is significantly different from the others and could be an outlier. Z-scores can be both positive and negative; the farther the score is from 0, the higher the likelihood that the data point is an outlier. Generally, a Z-score greater than 3 is considered extreme.

For the Z-score method to be effective in identifying outliers, the data should follow a normal distribution. Removing outliers using the Z-score method involves the following steps:

$Z=\frac{X-\mu }{\sigma }$

where: X = data point, µ = mean of the dataset, σ = standard deviation of the dataset The procedure includes the following steps:

Choose a threshold (commonly 2 or 3)

A common choice is Z > 3 or Z < −3, meaning the data point is 3 standard deviations away from the mean. If the dataset is small or you want to be less strict, you might use Z > 2.5 or even Z > 2. Remove outliers: Any data point with a Z-score beyond the chosen threshold is considered an outlier and can be removed.

The Z-score indicates the number of standard deviations a data point is from the mean. It is calculated as: Z-score removal of outliers requires normal distribution.

Figure 9 shows the application of z-scoring to data sets which after the z-transform becomes normally distributed. While Z-scores can be computed for any distribution, they are particularly useful when the data is normally distributed, because:

Z-scores don’t require a normal distribution but are most useful when the data is (approximately) normal. Figure 10 shows the detection of outliers using z-scores. Figure 11 , Figure 12 show the feature distributions: Figure 11 represents skewed normal distributions and can be used to eliminate outliers, while Figure 12 does not represent normal distribution and cannot use a-scores to eliminate outliers.

Figure 13 shows the BMI and Glucose features after outliers have been removed using z-scores. It is important to emphasize that outliers can disproportionately impact models, particularly those sensitive to extreme values (e.g., linear regression, k-means clustering). In classification tasks, outliers can lead to misclassification by distorting decision boundaries. A model trained on data with outliers may struggle to generalize effectively to unseen data. Tree-based methods, such as Random Forest and XGBoost, exhibit reduced sensitivity to outliers. However, in the context of machine learning applied to public health or healthcare, it is crucial to remove outliers to avoid degrading model accuracy.

Having obesity significantly increases the likelihood of developing diabetes, a condition characterized by excessive glucose (sugar) in the bloodstream. It also accelerates the progression of diabetes. A high BMI (in the overweight or obese range) raises the risk of insulin resistance, which can lead to type 2 diabetes. Excess fat, particularly around the abdomen, disrupts insulin function, making it harder for cells to absorb glucose from the blood [5] .

Research indicates that individuals with a BMI of 30 or higher face a much greater risk of developing type 2 diabetes compared to those with a normal BMI (18.5 - 24.9). Those in the overweight range (BMI 25 - 29.9) are more susceptible to prediabetes, a state where blood sugar levels are elevated but not yet high enough for a diabetes diagnosis. Losing just 5% - 10% of body weight can significantly lower this risk.

Here’s how it works: The pancreas regulates blood glucose levels by producing insulin, a hormone responsible for moving glucose out of the bloodstream. Under normal conditions, insulin helps transport glucose to muscles for immediate energy use or stores it in the liver for future needs.

However, in cases of diabesity (a combination of obesity and diabetes), cells become resistant to insulin, preventing glucose from entering. Additionally, the liver’s glucose storage becomes saturated with fat. With nowhere else to go, glucose remains in the bloodstream, prompting the pancreas to produce more insulin to counteract this resistance.

Over time, the pancreas becomes overworked and starts producing less insulin, leading to diabetes, which can quickly worsen if insulin resistance persists.

Individuals with obesity are approximately six times more likely to develop type 2 diabetes than those at a healthy weight. However, not everyone with obesity will necessarily develop diabetes. Other contributing factors include:

Some people with obesity may produce enough insulin without overburdening the pancreas, while others may have a limited insulin production capacity, making them more susceptible to diabesity.

Figure 14 demonstrates the cross plot between BMI values of patients with diabetes (pink color) and healthy ones (green color).

The term “glucose” comes from the Greek word meaning “sweet.” It is a type of sugar derived from the foods you consume, serving as a primary energy source for your body. When glucose circulates in your bloodstream to reach your cells, it is known as blood sugar. Insulin, a hormone, helps transfer glucose into cells for energy and storage. In the case of diabetes, blood glucose levels become higher than normal. This can happen either because your body doesn’t produce enough insulin to regulate glucose or because your cells don’t respond effectively to insulin.

Prolonged high blood sugar can damage the heart and blood vessels, increasing the risk of heart disease, high blood pressure, and stroke. It can also lead to kidney disease or failure (diabetic nephropathy), damage to the retina (diabetic retinopathy), causing vision loss or blindness, and nerve damage (diabetic neuropathy), which can result in pain, tingling, or numbness, especially in the feet. Additionally, high blood sugar raises the risk of cognitive decline and Alzheimer’s disease, contributes to fatty liver disease, and worsens insulin resistance.

The glucose in your bloodstream primarily comes from carbohydrate-rich foods, such as bread, potatoes, and fruit. As shown in Figure 15 , when glucose levels exceed 125 - 130 mg/dL, the risk of developing diabetes significantly increases.

As we eat, food travels down the esophagus into the stomach, where acids and enzymes break it down into smaller components, releasing glucose in the process. This glucose then moves to the intestines, where it is absorbed into the bloodstream. Insulin plays a crucial role in helping glucose enter cells. After eating, blood sugar levels naturally rise and gradually decrease a few hours later as insulin facilitates glucose absorption.

Hyperglycemia, or high blood glucose, occurs when blood sugar levels exceed 200 mg/dL two hours after eating or 125 mg/dL while fasting. For patients with diabetes, regular blood sugar testing is essential. Exercise, a balanced diet, and medication can help maintain healthy blood glucose levels and prevent complications. Figure 16 presents a violin plot showing that higher glucose levels are associated with diabetes.

Chronic high blood glucose can lead to nerve damage (diabetic neuropathy), causing pain, tingling, or numbness, especially in the feet. It also increases the risk of cognitive decline and Alzheimer’s disease, contributes to fatty liver disease, and worsens insulin resistance. The glucose in the bloodstream primarily comes from carbohydrate-rich foods, such as bread, potatoes, and fruit. Figure 15 illustrates that when glucose levels exceed 125 - 130 mg/dL, the risk of diabetes increases significantly.

Diabetes is mostly diagnosed in individuals over the age of 45. The risk increases with age due to factors such as reduced insulin sensitivity, weight gain, and decreased physical activity. However, an increasing number of younger people are being diagnosed with diabetes, largely due to lifestyle changes. More than 90% of individuals with diabetes have Type 2.

For middle-aged individuals, it’s essential to focus on weight management, blood sugar control, and regular monitoring for complications.

For older adults, the risk of complications like heart disease, nerve damage, and kidney problems rises, necessitating careful management and monitoring.

Diabetes can go undiagnosed for years, as symptoms like excessive thirst, blurred vision, and tingling in the hands and feet may develop gradually and go unnoticed.

Middle age marks a significant rise in diabetes diagnoses. Approximately 15% of middle-aged Americans are diagnosed with Type 2 diabetes, which is nearly five times the rate of those under 45. Incidence increases even more as individuals age. Nearly 25% of older adults in the U.S. have been diagnosed with Type 2 diabetes, with undiagnosed cases possibly accounting for an additional 5%. This means that over one in four senior Americans live with Type 2 diabetes.

Age is a significant risk factor for Type 2 diabetes. The older a patient is, the more likely they are to develop the condition. This is also true for preteens and teenagers, as the rates of diabetes in this age group have increased sharply in recent years. Type 2 diabetes is a disease caused by a combination of genetic factors and lifestyle choices. Figure 17 displays a cross-plot of age and outcomes, with diabetes shown in red and non-diabetes in green.

Being overweight, having high blood pressure, and leading a sedentary lifestyle all increase the risk of Type 2 diabetes. Managing diabetes at different ages involves different approaches:

Younger individuals: Focus on diet, exercise, and insulin management (for Type 1).

Middle-aged individuals: Emphasize weight management, blood sugar control, and monitoring for complications.

Older adults: The risk of complications such as heart disease, nerve damage, and kidney issues increase, requiring careful management.

Diabetes can go undiagnosed for years. Symptoms such as excessive thirst, blurry vision, and tingling in the hands and feet may develop gradually and be overlooked.

Middle age is when the number of diabetes diagnoses starts to rise significantly. Approximately 15% of middle-aged Americans are diagnosed with Type 2 diabetes, nearly five times the rate among those under 45. The rate increases even further as individuals enter their senior years. Nearly 25% of older adults in the U.S. have been diagnosed with Type 2, with undiagnosed cases possibly accounting for an additional 5%. This means that more than one in four senior Americans lives with Type 2 diabetes [6] .

The disease also is affecting ever more teens and even children. Researchers believe childhood obesity and lack of exercise are among the reasons behind that trend.

Pregnancy can have a significant impact on diabetes, and diabetes can also affect pregnancy. There are two main scenarios to consider:

Gestational Diabetes Mellitus (GDM)—This type of diabetes develops during pregnancy, usually in the second or third trimester, and often resolves after childbirth. It occurs when the body cannot produce enough insulin to meet the increased needs of pregnancy.

Pre-existing Diabetes (Type 1 or Type 2)—Women who have diabetes before pregnancy need to manage their blood sugar levels carefully to prevent complications for both them and the baby.

Gestational diabetes is diabetes that a woman can develop during pregnancy. When you have diabetes, your body cannot use the sugars and starches (carbohydrates) it takes in as food to make energy. As a result, your body collects extra sugar in your blood.

We don’t know all the causes of gestational diabetes. Some with gestational diabetes are overweight before getting pregnant or have diabetes in the family. From 1 in 50 to 1 in 20 pregnant women has gestational diabetes. It is more common in Native American, Alaskan Native, Hispanic, Asian, and Black women, but it is found in white women, too. Figure 18 shows the Pregnancies data distribution for patients with diabetes (red) and without (green).

The literature shows the effectiveness of generative AI in clinical decision-making, as well as highlighting its challenges. For example, AI systems provided an effective method for the management of gastrointestinal disease including early detection and diagnosis, and the author suggested using various methods of generative AI for different gastrointestinal health information in future research [11] . Furthermore, AI has already demonstrated great capability in breast cancer management and patient outcomes and . For example, AI systems have shown promise in mammography screening and radiotherapy treatment . In addition to the benefits of AI, the researchers noted some of its challenges such as patient confidentiality, problems with moral principles, and regulation issues . They emphasized the need to conduct specific studies that can prove the accuracy of AI techniques in healthcare delivery .

Despite these challenges, AI has the potential of making a dramatic change in the delivery of healthcare services by empowering health practitioners to provide the best patient care [14] . Similarly, AI should be held in high regard because of its ability to identify not only specific diseases but all diseases and healthcare professionals should pay attention to its effectiveness [15] . Other studies highlighted the potential of Generative AI into clinical practice, specifically in the prevention of cardiovascular disease [16] .

Moreover, AI has proven to be effective in clinical decision-making despite limitations. Similarly, AI techniques have the potential of accurately classifying diabetes to improve the health status of patients.

In this paper, we developed a new generative AI algorithm that consists of the following steps: 1) PM GenAI explores the data and generates new data by sampling from the posterior distribution of the model weights. This allows it to create diverse outputs based on the learned distributions, rather than relying on a fixed, deterministic model, 2) Uncertainty Quantification: By modeling uncertainty in parameters, PM GenAI produces a distribution of outputs instead of a single prediction. This is particularly useful for tasks that benefit from multiple plausible outcomes, such as data generation or decision-making under uncertainty, 3) Design the neural network architecture and specify prior distributions for the model parameters. Use techniques such as variational inference or Markov Chain Monte Carlo (MCMC) to approximate the posterior distribution of the weights, 4) Once the posterior is approximated (typically via backpropagation), sample weights from the distribution and perform feedforward passes through the network to generate predictions or synthetic data points that reflect the learned uncertainty.

The algorithm divides the data into test and training. In the training data set it defines the trend and statistics: Mean vector (μ), the center of the distribution, covariance matrix (Σ): the shape, spread, and orientation. The PDF of a Gaussian component is:

$P\left(x|{\mu }_k,{\Sigma }_k\right)=\frac{1}{{\left(2\pi \right)}^{1/d}{\left|{\Sigma }_k\right|}^{1/2}}\exp \left(-\frac{1}{2}{\left(x-{\mu }_k\right)}^{\text{T}}{\Sigma }_k^{-1}\left(x-{\mu }_k\right)\right)$ (6)

The covariance matrix is:

${\Sigma }_k=\frac{1}{N_k}{\displaystyle {\sum }_{i=1}^N{\gamma }_{ik}\left(x_i-{\mu }_k\right){\left(x_i-{\mu }_k\right)}^{\text{T}}}$ (7)

In this expression:

$N_k={\displaystyle {\sum }_{i=1}^N{\gamma }_{i\kappa }}$ (8)

PM GenAI generates augmented data that resembles real data, which is especially valuable in scenarios with limited labeled samples largely improving the quality of data sets. By incorporating uncertainty, the algorithm also enables models to be more robust to outliers and noise, ultimately improving generalization. In summary, using PM GenAI in deep learning for data generation enables the incorporation of uncertainty into models, generate realistic synthetic data, and make more reliable and accurate predictions.

Learning Rate: Controls how much the model updates its weight during training. A high learning rate may lead to convergence issues, while a low one can make training slow (we use Adam or Adagrad optimizers to automatically adjust the learning rate).

Batch Size: Defines the number of training samples used in one forward and backward pass. Common choices include 32, 64, and 128. We use 128 and then split it into mini batches.

Number of Epochs: Specifies how many times the entire dataset is passed through the neural network. We usually do not use more than 100 epochs.

Optimizer (e.g., SGD, Adam, RMSprop): Determines how the model updates weights during training. We usually use an Adam optimizer.

Loss Function (e.g., MSE, Cross-Entropy): Measures the difference between predicted and actual outputs. We use Cross-Entropy because we solve the classification problem. MSE us used in regression algorithms,

Number of Layers and Neurons per Layer: Defines the depth and complexity of the network. We start with 8 neurons (the number of features). It is highly advisable this way of building the neural network to prevent overfitting if many neurons are selected.

Dropout Rate: Controls the percentage of neurons randomly dropped during training to prevent overfitting (we use dropout rate 0.3-0.4 to prevent over or underfitting)

Weight Initialization: Determines how weights are initialized before training (e.g., Xavier, He initialization). Choosing the right weight initialization method depends on the activation function and network depth. We use Xavier initialization to work well with classification problems. Proper weight initialization leads to more stable training and improved model performance.

Activation Functions (e.g., ReLU, Sigmoid): Define how neurons process inputs. We use ReLU for hidden layers and Sigmoid for the output layer typical to classification problems.

L1/L2 Regularization (Weight Decay): Helps prevent overfitting by penalizing large weights. It is implemented in optimizers like AdamW and SGD.

K-fold cross-validation. We used K-fold cross-validation as a resampling technique to evaluate the performance of the method. It evaluates how well a model generalizes unseen data by splitting the dataset into multiple subsets. K-fold cross-validation includes:

1) The dataset is divided into K equally sized folds (subsets).

2) The model is trained on K-1 folds and tested on the remaining one.

3) The process is repeated K times, with each fold serving as the test set once.

4) The final model performance is obtained by averaging the results across all K iterations.

5) Stratified K-Fold ensures class distribution is preserved in each fold (useful for imbalanced datasets and can be used in several applications including public health).

Statistical analysis of the results shows that the comparison of previous methods ( Figure 21 ) and the PM GenAI accuracy yields:

P ~ 0.004 < 0.05

We may conclude that PM GenAI significantly improves the performance of the classification algorithms for diabetes.

Figure 22 depicts the Confusion matrix generated by PM GenAI. Table 3 is a complete classification report demonstrating 97% accuracy.