Classification models are a type of machine learning algorithms that are used to predict the category or class that a given input belongs to. In classification, the output variable (or target) is a discrete label or class. The goal of a classification model is to learn the relationship between input features and the target class, so that it can predict the class for new, unseen data. This study applies supervised learning approaches where the model is trained on labeled data (i.e., data that includes both features and the corresponding class labels). Once trained, the model can make predictions on new, unseen data by assigning a class label to each new input. For each model fitted, a confusion matrix is developed and various metrics including accuracy, recall, specificity and precision are computed to determine which classifier is best suited for prediction of malaria.

This study considers five widely used supervised classification models to cover both linear and non-linear decision boundaries as well as both instance-based and ensemble approaches, namely logistic regression, Decision trees, Random forest, Support vector machine, and Naïve bayes model as described in [18] and [19] . Other potential models initially considered were XGBoost and multilayer neural networks. However, we deferred these to future work to keep the workflow transparent and interpretable for public‐health stakeholders in the current application. A brief description of each classifier is given here:

A confusion matrix is a table often used to describe the performance of a classifier on a set of test data for which the true values are known. In this study confusion matrices were obtained for binary classifiers. Regarding malaria incidence, the classifier may label the subject status as: Yes—subject has malaria or No—Subject does not have malaria, while the known/true status of the individual may be Infected/Not infected.

The matrix shows the number of true positives, false positives, true negatives, and false negatives, where:

The confusion matrix helps to identify model errors and specify the types of errors (false positives or false negatives) are more prone to. This can be especially important in applications where one type of error is more costly or dangerous than another (e.g. diagnosing a disease where false negatives might be more critical than false positives).

To assess how well each binary classification model performs, several metrics are computed based on the confusion matrices obtained, as discussed in [20] and [21] .

$\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TN}+\text{FP}+\text{TP}+\text{FN}}$ (1)

$\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}$(2)

$\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}$ (3)

$\text{F1-Score}=\frac{2\times \text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}}$ (4)

$\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}$ (5)

$\text{Misclassification Rate}=\frac{\text{FP}+\text{FN}}{\text{TN}+\text{FP}+\text{TP}+\text{FN}}$ (6)

$K=\frac{\text{P agree}-\text{P chance}}{1-\text{P chance}}$ (7)

A Kappa value close to 1 indicates a strong agreement between predicted and actual values, adjusting for chance. A Kappa coefficient of zero in this case represents agreement equivalent to chance between the climatic and non-climatic variables.

One dependent variable and ten independent variables, inclusive of both climatic and non-climatic factors, were considered in the model fitting. Table 1 provides a summary of the variable types and levels where applicable. All categorical predictors including wealth index, malaria endemicity, education level, and mosquito net ownership were one-hot encoded. Binary indicator columns were created for each level of the categorical variables, dropping the reference level to avoid multicollinearity. Numeric variables such as altitude were normalized to transform into 0 - 1 continuous variables.

Performance metrics including accuracy, recall, specificity and precision were computed based on the confusion matrices and the results are discussed here for each classifier. Table 2 gives a summary of the performance measures under different models with the comparative abilities displayed in Figure 1 .

The logistic model predicts 616 individuals to have malaria, and this accounts for 6.04% correct prediction. (True Positive = 616, False Positive = 249, precision = 0.06038). Secondly the logistic regression models indicate that the rate of an individual to have malaria to be predicted to fall in the same category by our model is 45.714 percent. (Recall = 0.45714). Specificity of 0.71214, this implies that the model can correctly classify individuals without malaria 71.21% of the time. Accuracy should be as high as possible. It answers the question of how correct our classifier is generally. The logistic regression model is 70.22% correct (Accuracy = 0.7022) in malaria prediction.

The Support Vector Machine model had both precision and recall of 0, but correctly predicted malaria status of individuals without malaria (specificity = 1.000) with a 70.56% accuracy. The random forest model predicts 576 individuals to have malaria, which accounts for 64.67% correct prediction. (True Positive = 576, False Positive = 157, precision = 0.6467).

The decision tree predicts 589 individuals to have malaria, which accounts for 46.51% correct predictions. (True Positive = 589, False Positive = 225, precision = 0.46512). Secondly the decision tree classifier indicates that the likelihood of an individual suffering from malaria to be predicted as having malaria is 15.09% (Recall = 0.15094). Specificity of 0.9276 implies that our model can predict those without malaria correctly with 92.76% of the time. Accuracy should be as high as possible as it shows how correct the classifier is generally. The decision tree is 69.89% correct (Accuracy = 0.6989).

Further, the decision tree classifier indicates that the likelihood that an individual who has malaria is correctly predicted to have malaria by the model is 40.75% (recall = 0.4075). A specificity value of 0.9071 implies that the model correctly predicts absence of malaria in individuals who do not have the disease 90.71% of the time. Accuracy should be as high as possible. It answers the question of how correct our classifier is generally. The decision tree is 76% correct (Accuracy = 0.76).

The Naïve Bayes classifier predicts 597 individuals to have malaria, and this accounts for 8.679 percent correct prediction. (True Positive = 597, False Positive = 38, precision = 0.08679). Secondly, the decision tree classifier indicates that the rate of an individual having malaria is predicted to fall in the same category by our model at 37.71% (Recall = 0.37705). Specificity of 0.71156 implies that our model can predict those without malaria correctly, with a 71.16% prediction rate.

The five classifiers were evaluated with respect to how well they fit the malaria sample data based on the kappa coefficient and model accuracy. The higher the Kappa coefficient the more accurate the classification model. A kappa coefficient of 1 is desired, such that the predicted value equals the observed value. The model with the highest accuracy level and Kappa coefficient value is perceived to be the best fitted model. The goodness of fit measures are summarized in Table 3 and comparative performance of models displayed in Figure 2 . The Random Forest has the highest accuracy (0.76) with a 95% confidence interval (0.7307, 0.7876) and Kappa coefficient (0.3526) value and it is therefore better than Logistic model, Decision tree, Support vector model and naïve Bayes model. Though the Random Forest had the highest Kappa coefficient, its value is relatively low, indicating only a moderate compatibility between our classifier and the true class labels while controlling for accuracy.

McNemar’s test was used to compare predictive accuracy of two competing models. A null hypothesis of no difference in predictive ability was tested. The Logistic model, Naïve Bayes model, Support Vector model and Decision tree all had the same McNemar’s p-value of 2e−16, while Random forest had a McNemar’s p-value of 4.111e−16. Random forest had the highest accuracy and Kappa coefficient value (0.76, 0.3526 respectively). Support vector model came a close second to the Random Forest in terms of accuracy at 71%. Feature Importance and contribution analyses was performed to understand which predictor variables most influenced the Random Forest model. We extracted both Gini importance (computed as the mean decrease in impurity) for each feature, and permutation importance (computed as the average drop in test‐fold accuracy when a feature is randomly permuted). The top three predictors by Gini importance were: use of insecticide-treated nets, household wealth quintile and normalized altitude (height above sea level). Permutation importance rankings mirrored this ordering, confirming climate variables as primary drivers with socioeconomic factors also contributing substantially to incidence of malaria.