The prediction of diabetes is an important topic in medical research and has been the subject of numerous studies. In Table 1 , we have highlighted major findings by researchers who have adopted several statistical procedures in the bid to predict diabetes. Some of these techniques include principal component analysis, logistic regression, decision trees, support vector machines (SVM), discriminant analysis and artificial neural networks (ANN) [6] .

Table 2 highlights some of the pros and cons of using these identified statistical techniques based on findings.

A multivariate logistic regression equation can be used to screen for diabetes. In November of 2002, Diabetes Care published an article describing the development and validation of this empirical equation. The research design and methods were unique as subjects were from international sources. A predictive equation was developed with data collected from 1032 Egyptian subjects with no history of diabetes. The equation incorporated age, gender, BMI, postprandial time, and random capillary plasma glucose as independent covariates for prediction of undiagnosed diabetes. These covariates were based on a fasting plasma glucose level of ≥126 mg/dl and/or a plasma glucose level 2 hr after a 75-g oral glucose load ≥200 mg/dl. The equation was validated using data collected from an independent sample of 1065 American subjects [14] . The predictive equation was calculated with the following logistic regression parameters:

P = 1/(1 − e^−x), where x = −10.0382 + [0.0331*(age in years) + 0.0308*(random plasma glucose in mg/dl) + 0.2500*(postprandial time assessed as 0 to ≥8 hr) + 0.5620*(if female) + 0.0346*(BMI)].

The cut-off point for the prediction of previously undiagnosed diabetes was defined as a probability value ≥ 0.20. The equation’s sensitivity was 65%, specificity 96%, and positive predictive value (PPV) 67%. When applied to a new sample, the equation’s sensitivity was 62%, specificity 96%, and PPV 63% . The equation improved on recommended methods of screening for undiagnosed diabetes and could be easily implemented in a handheld programmable calculator to predict previously undiagnosed diabetes . Machine learning, using several different models, has been attempted without further model improvement [15] .

The sensitivity of a test is defined as the proportion of people with disease who will have a positive result. A test with a high sensitivity is useful for “ruling out” a disease if a person tests negative [17] .

The specificity of a test is the proportion of people without the disease who will have a negative result. A test with a high specificity is useful for “ruling in” a disease if a person tests positive [17] .

The predictive value of a test is determined by the test’s sensitivity and specificity and by the prevalence of the condition for which the test is used. Both PPV and NPV vary with changing prevalence of disease. It will therefore be wrong for clinicians to directly apply published predictive values of a test to their own populations when the prevalence of disease in their population is different from the prevalence of disease in the population in which the published study was carried out [17] . Figure 1 shows a graphical representation of how Sensitivity, Specificity, Negative predictive value (NPV), Positive predictive value (PPV), and Accuracy are calculated.

For this work, publicly available data from Kaggle was used, which was initially sourced from the National Institute of Diabetes and Digestive and Kidney Diseases. The dataset contains 390 instances with 14 predictor variables on patients, including their overall cholesterol level, HDL cholesterol level, cholesterol to HDL ratio, glucose levels, systolic and diastolic blood pressure, age, height, weight, BMI, waist and hip measurements, and diabetes status. All variables and their meanings are presented in Table 3 . These values represent the ground truth labels in the dataset used for model training and evaluation. Performance metrics were chosen to provide a comprehensive evaluation of model performance, including sensitivity (recall), specificity (true negative rate), precision (positive predictive value), negative predictive value, and overall accuracy. These metrics were selected to reflect the clinical importance of both correctly identifying diabetic patients and minimizing false positives. In addition, a scatter plot matrix, and a correlation matrix were plotted to check for multicollinearity and to identify whether any strongly correlated features were present in the dataset that could impact model performance.

The dependent variable in this analysis was the binary variable Outcome, representing the presence or absence of diabetes, while the 13 independent variables were the diagnostic measures mentioned above. Before analysis, the data were cleaned to ensure their suitability for the study.

The National Institute of Diabetes and Digestive and Kidney Diseases collects data from different individuals through various programs and sources, such as surveys, the Diabetes Prevention Program, the National Diabetes Information Clearinghouse, and clinical trials. For example, the National Health and Nutrition Examination Survey (NHANES), conducted by the Centers for Disease Control and Prevention (CDC) in collaboration with the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), collects information on diabetes prevalence, risk factors, and treatment. In contrast, the National Diabetes Prevention Program (NDPP) collects data on participants’ demographics, medical history, physical activity, and dietary habits to evaluate the program’s effectiveness. The National Diabetes Information Clearinghouse (NDIC) collects and compiles data on diabetes incidence, prevalence, risk factors, and complications from scientific literature and government reports. Finally, NIDDK conducts and supports clinical trials to test new treatments and interventions for diabetes and related conditions, collecting data on participants’ demographics, medical history, symptoms, and treatment outcomes to evaluate the safety and effectiveness of the intervention.

In this paper, we aim to predict the susceptibility of patients to diabetes using multivariate statistical techniques. Logistic Regression (LR) and Discriminant Analysis (DA) were chosen due to their established use in the field of medicine and engineering with categorical or discrete response variables. They are also robust to outliers and missing data, making them suitable for real-world datasets that may be incomplete or contain errors.

The first phase of our research involves using DA and LR models to determine the dimensions of the cleaned Kaggle diabetes dataset that can reliably and accurately classify subjects into groups (i.e., diabetic or non-diabetic). LR will be used to identify a linear combination of independent variables (IVs) that best predicts membership in the diabetic or non-diabetic group, as measured by a categorical dependent variable (DV). This will involve deriving an equation for predicting the likelihood of diabetes in identified subjects. Additionally, we will fit the DA model to our dataset to derive the two discriminant functions.

In the second phase of our research, we will perform Principal Component Analysis (PCA) on our dataset and fit the LR and DA models again on the PCA-reduced dataset. We will then check the model fit and accuracy of the LR and DA models on both the original and PCA-reduced datasets to draw insights into the prediction accuracies [8] . Finally, we will use the K-fold cross-validation technique to compare the performance of the LR and DA models [18] . We used SAS software and Python IDE as the statistical tools for coding. These statistical methods will help us determine the most accurate procedures for predicting diabetes susceptibility and provide insights into the usefulness of PCA as a tool for data reduction in medical studies [7] . Principal Component Analysis (PCA) was applied as an exploratory technique for dimensionality reduction. The variance explained by each component was examined through the proportion of the corresponding eigenvalues. The scree plot was used to determine the optimal number of principal components to retain for the model. Each principal component is a linear combination of the original features and the components are orthogonal (uncorrelated), allowing them to represent distinct directions of variance in the data. These components were used for model training to evaluate whether dimensionality reduction would improve classification performance. The component loadings were examined using the Maximum Likelihood Estimates from logistic regression and the Linear Discriminant Function coefficients.

Model development for both Linear Discriminant Analysis (LDA) and Logistic Regression was performed using SAS software. The LDA model was developed based on the statistical analysis of linear discriminant functions, which aim to find a linear combination of features that best separates the two classes (diabetic and non-diabetic). The Logistic Regression model was developed as a linear model that estimates the probability of diabetes occurrence based on the input features through a logistic function. Both models were trained on the original set of 14 features and, separately, on a reduced set of principal components. Training was performed on both the original feature set and the reduced PCA-based feature set to enable direct comparison of model performance between the full and PCA-based representations.

To ensure the validity of our analysis, several data pre-processing steps were conducted before assessing the adequacy of our model. Missing data, outliers, and unreadable characters in our dataset were checked for and removed. Also, special characters, such as commas and semicolons were fixed, to ensure the data was properly formatted.

Next, several tests (in SAS) were conducted to ensure that our dataset met the necessary conditions for conducting multivariate statistical analysis, specifically Logistics Regression (LR) and Discriminant Analysis (DA). As multivariate analysis relies on assumptions of normality and homogeneity of variance, we performed univariate normality checks on randomly selected variables in the dataset to verify that the data followed a normal distribution.

Due to the complexity involved in testing multivariate normality, a univariate normality procedure was opted for, instead. Investigations showed that the dataset met the necessary normality assumptions, as confirmed by the Q-Q plots and distribution curves presented in Figure 2 . These steps ensured that our dataset was suitable for further statistical analysis.

Next, the test of homoscedasticity (homogeneity of variance) was conducted in SAS. This is given in the scatter matrix provided in Figure 3 . No known pattern or trend can be seen, implying that the test of homoscedasticity is not violated.

The first step involved conducting a correlation analysis amongst each of the independent variables. From the results obtained and presented in Table 4 , there are no such variables with very high correlation among the independent variables. Although, high correlation was recorded between Waist and Weight (0.8478), Hip and Weight (0.8270) and Waist and Hip (0.8352). We will still proceed with our analysis.

As can be seen from the result of the simple statistics conducted on the Diabetes dataset (please see Table 5 ), the mean age of the patients is 47 years, which implies that the population sampled is an aging population with a standard deviation of approximately 16.4 years. The average height and weight of the population are 66 inches and 177.4 lbs respectively (corresponding standard deviations were measured as 3.9 inches and 40.4 lbs respectively). The systolic/diastolic blood pressure averaged 137/83. According to the American Heart Association, the systolic to diastolic blood pressure is ideal around 120/80, although this could vary depending on age, sex and health condition. The glucose level average for all the patients was 107.3 mg/dL but the dispersion or spread was relatively high (standard deviation = 53.80 mg/dL). The descriptive statistics are also shown in the visualization plot (error plot) shown in Figure 4 .

The choice of Logistic Regression (LR) was motivated by the fact that the dependent variables in our dataset were categorical (binary). We fitted the pre-processed dataset to a Logistic Regression model and checked the output file for the final results in SAS.

To assess the validity and goodness of fit of our model, the −2LogL (where L is the likelihood function), AIC (Akaike Information Criterion), and SC (Schwarz’s Bayesian Criterion) values were evaluated and shown in Table 6 . Lower values of −2LogL, AIC, and SC typically indicate better model fitting, and we will discuss this in more detail in another section.

The Chi-square test statistics yielded p-values of ≤0.001, which is less than the level of significance of 0.05, indicating that the model is statistically significant.

Overall, these results suggest that our fitted Logistic Regression model fits our dataset well and provides a statistically significant explanation for the relationship between our independent and dependent variables. A subsequent section will provide further details on the model performance and interpretation. The Logistic Regression model below is formulated according to the SAS output file shown:

$\begin{array}{c}\ln \left(\frac{p}{1-p}\right)=-11.4431+0.0121\ast \text{cholesterol}+0.0372\ast \text{glucose}\\ -0.0278\ast \text{hdlchol}-0.1552\ast \text{cholhdlratio}+0.0315\ast \text{Age}\\ +0.0392\ast \text{Height}-0.0174\ast \text{Weight}+0.1258\ast \text{BMI}\\ +0.00639\ast \text{Systolicbp}+0.00868\ast \text{Diastolicbp}\\ +0.1098\ast \text{Waist}-0.08333\ast \text{Hip}-2.6863\ast \text{Waisthipratio}\end{array}$

where p is the probability of a patient having diabetes based on the maximum likelihood estimate approach (MLE). From Table 7 , the p-value of glucose being less than 0.05 indicates that the glucose level is a predominant predictor variable from the LR model. The medical implication of this result is that higher glucose levels of patients could be a pointer to a higher risk of diabetes.

Additionally, the Odds ratio estimates other than 1 indicates that the independent variables have some form of relationship with the outcome variables. The Odds ratio estimate is shown in Table 8 .

Given the results above, the following inferences can be deduced:

1) The odds ratio of 1.134 for BMI suggests that the odds of having diabetes are about 13.4% higher in obese individuals compared to non-obese individuals.

2) The odds ratio of 1.032 for Age suggests that the odds of having diabetes are about 3.2% higher in older individuals compared to younger individuals.

3) The odds ratios of 1.006 and 1.009, respectively, for Systolic and Diastolic blood pressure suggest that the odds of having diabetes are almost the same (about 0.6% and 0.9% higher) in groups with high and low blood pressure.

4) The odds ratio of 1.038 for glucose levels suggests that the odds of having diabetes are about 3.8% higher in individuals with higher glucose levels compared to those with lower glucose levels.

Overall, the odds ratios provide insight into the relationship between each independent variable and the dependent variable (diabetes) in the logistic regression model. These interpretations can be used to better understand the factors that are associated with the presence of diabetes in the population under study.

Next, we performed a Discriminant Analysis (DA) on the same pre-processed Kaggle diabetes dataset that was used for the Logistic Regression (LR) model. DA is a similar statistical procedure to LR, and it was carried out using the SAS software.

The results obtained from the DA model were analyzed, and the confusion matrix for the statistical technique is presented in Table 9 , also obtained from SAS.

This additional analysis using DA provides further insights into the relationship between the independent variables and the dependent variable (diabetes) in our dataset [11] . By comparing the results of the DA model to those of the LR model, we can better understand the robustness and generalizability of our findings.

The confusion table (refer to Table 9 ) shows that there is a total of 390 patients (330 of which are patients without diabetes while 60 are patients with diabetes). Of the 330 patients without diabetes, 310 patients were correctly predicted as non-diabetic while 20 were wrongly classified as diabetic), we can state that the model gave correct prediction for approximately 94% of the patients without diabetes. Additionally, the model wrongly predicted 11 diabetes patients as “non-diabetic” and appropriately predicted 49 patients as actually having diabetes. We can say that the model prediction here is around 82%.

The overall accuracy of a binary classification is a measure of how often the model correctly predicts the class of new observations. It is calculated as follows:

$\text{Overall accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FP}+\text{TN}+\text{FN}}$,

where TP = True Positive, TN = True Negative, FP = False Positive and FN = False Negative.

The overall prediction of the DA model can therefore be computed as follows:

TN = 310, TP = 49, FN = 20 and FP = 11. The prior probabilities were 50:50 (an equally likely chance scenario)

$\frac{310+49}{310+49+20+11}\times 100=92\%$

That means that the DA model correctly predicted outcomes (class labels) for 92% of the sample population provided.

Performance metrics were computed from the confusion matrix presented in Table 10 , using the following standard formulas: Metrics Formula Value Overall Accuracy (TP + TN)/(TP + TN + FP + FN) 92.05%, Sensitivity (Recall) TP/(TP + FN) 71.01%, Specificity TN/(TN + FP) 96.57%, Precision (PPV) TP/(TP + FP) 81.67%, Negative Predictive Value (NPV) TN/(TN + FN) 93.94%.

The high overall accuracy and specificity suggests that the DA model effectively identifies non-diabetic individuals [10] . At the same time, the lower sensitivity implies a higher chance of false negatives in identifying diabetic individuals. The high negative predictive value means that the model effectively identifies individuals who do not have diabetes. In contrast, the positive predictive value indicates that the model can correctly identify 8 out of every 10 individuals who have diabetes.

The dataset was subjected to Principal Component Analysis (PCA) to reduce the number of attributes to the optimal number of variables that capture the variability in the data [19] . From the analysis done in SAS, it was observed that five principal components are sufficient to run the model, as their eigenvalues were greater than 1. The results indicate that the Logistics Regression and Discriminant Analysis procedures can be run using these five principal components, which explain approximately 78.1% of the variance in the dataset. Table 11 displays the eigenvalues and the proportion of variance explained by each principal component.

The scree plot elbow also establishes that five principal components are statistically sufficient to explain the variance (please see Figure 5 ).

We fitted Logistic Regression to the PCA-reduced dataset. We checked the output file for the final results and the goodness of fit by inspecting the -2LogL values, the AIC (Akaike Information Criterion), SC (Schwarz’s Bayesian Criterion). We checked the difference in the values as the intercepts, prin1, prin2, prin3, prin4 and prin5 using the stepwise procedure (forward selection). Figure 6 presents screenshots results of the step by step process utilized. The p-values (≤0.001) are less than 0.05 (the significance level), indicating that the model is statistically significant at every level of variable addition. The result section is given below:

As seen in Figures 6(a)-(f) and Table 12 , the values of the parameters (AIC, SC, and −2logL) remained unchanged. Theoretically, this implies that the addition of new variables (principal components) did not directly improve the model, but this is not to say that the principal components are not outrightly irrelevant.

The Logistic Regression model below is formulated according to the maximum likelihood estimates obtained from SAS and presented in Table 13 .

$\begin{array}{c}\ln \left(\frac{p}{1-p}\right)=-2.5661+0.5689\times \text{Prin}1+0.7415\times \text{Prin}2-0.3527\times \text{Prin}3\\ -0.3473\times \text{Prin}4+0.6877\times \text{Prin}5\end{array}$

Using the PCA-reduced dataset, the DA model produced classification results ( Table 14 ), and two discriminant functions ( Table 15 ) which were derived in SAS.

$\begin{array}{c}{D}_0\left(\text{No}\text{Diabetes}\right)=-0.03684-0.09219\times \text{Prin}1-0.14026\times \text{Prin}2\\ +0.07197\times \text{Prin}3+0.05707\times \text{Prin}4-0.12280\times \text{Prin5}\end{array}$

$\begin{array}{c}{D}_1\left(\text{Diabetes}\text{present}\right)=-1.11450+0.50703\times \text{Prin}1+0.77140\times \text{Prin}2\\ -0.39586\times \text{Prin}3-0.31388\times \text{Prin}4+0.67540\times \text{Prin}5\end{array}$

The values of the goodness of fit (−2logL, AIC, and SC) were examined for the LR model outcomes on the original diabetes dataset and after the PCA transformation. The results, presented in Table 16 , showed that these values remained unchanged for both procedures, indicating that PCA did not enhance the model’s performance. PCA is typically used to improve model accuracy in datasets with highly correlated independent variables. However, in this study, PCA did not significantly affect the model performance, likely because only a few variables had high correlation values. Additionally, comparing the AIC and SC values before and after PCA transformation, which were 336.872 and 340.838, respectively, further confirmed that the procedure did not improve the model fit. In general, to achieve a better model fit, the values of −2logL, AIC, and SC should decrease after the PCA transformation, with smaller values indicating better performance.

Table 17 shows the comparison of the results obtained for the DA procedure before the PCA was done and after. It was observed that the overall model accuracy reduced from 92% to 81% including specificity and sensitivity.

In order to compare the accuracy of the Logistic Regression and Discriminant Analysis and check which procedure is better, cross-validation procedure was

done on the original diabetes dataset using Python Integrated Development Environment (code snippets for this procedure are presented in Figure 7 and Figure 8 respectively). From the sklearn.model selection library, Kfold and cross_val_score are imported.

The result obtained for the k-fold cross validation shows a mean score of 92.67% for the LR model and 92.39% for the DA model. Both statistical techniques gave very good results.

The hazard risk for diabetes mellitus is rising [20] . Accurate prediction analysis can significantly benefit clinical practice. Other studies have indicated large waist circumference, high HgbA1C, and high Fatty Liver Index (FLI) all indicate the greater possibility of contracting diabetes [21] . Our dataset has one similar variable, the waist-to-hip ratio and others like glucose and BMI. Our list of variables may be more reflective of collection site limited data. Regardless, we are able to provide a valid prediction model.

Uncontrolled glucose levels in the bloodstream can increase damage to the body. Serious damage to the eyes, kidneys, and vasculature (heart) require extensive medical intervention. Predicting the likelihood of becoming a diabetic can help direct educational resources to those most in need of reversing this trend. Education focuses on a healthy diet and frequent exercise to improve long-term health outcomes [22] . Computational intensity of neural networks and the need for large datasets, limit complex model usefulness in a clinical practice environment. More likely, these techniques can be found in data analyst environments. Given the small size of our dataset, LR and DA adequately supported prediction analysis without extensive time and costs. Having a data analyst on the healthcare team could be beneficial.

There are at least four critical assumptions to consider when choosing LR and DA. We rely on assumptions about the data. Violating these assumptions can lead to inaccurate results. The assumptions are:

1) Linearity of data

2) Normality of residuals

3) Homogeneity of residuals variance

4) Independence of residuals error terms

Before applying a technique, it is crucial to ensure that the assumptions are met. Regression beta coefficients and R2 can be used to tell how well the LR model fits to the data.

There are potential problems to address:

1) Non-linearity

2) Heteroscedasticity

3) Presence of influential values

These problems can be checked with diagnostic plots in R software. Moderate multicollinearity may not be problematic. However, severe multicollinearity is a problem because it can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. The result is that the coefficient estimates are unstable and difficult to interpret. Multicollinearity saps the statistical power of the analysis, can cause the coefficients to switch signs, and makes it more difficult to specify the correct model (Regression Analysis blog). High predictor correlation or multicollinearity can impact the interpretability of the regression model. However, a reasonable prediction can still be made, if attention is directed toward eliminating inflated standard errors and preventing overfitting.