Performance of Different Machine Learning Algorithms for Credit Risk Classification ()
1. Introduction
Microfinance Institutions (MFIs) are financial establishments that offer financial services, basically loans and savings accounts, to individuals and Small and Medium Enterprises (SMEs) who more often fail to meet the traditional banking threshold due to lack of formal regular income or collateral to secure credit advance services. According to [1] “the financial performance of MFIs in Kenya continues to plummet and fluctuate unpredictably despite having implemented measures to limit loan defaults, as demonstrated over a five-year period spanning the years 2016 to 2020”. The study noted that the default rate among Microfinance Institutions (MFIs) was significantly higher than it was in the broader banking sector. Although the specific default rate varied, it was generally considered to be around 15% or higher, sometimes reaching 30.8%. This roughly doubled the acceptable industry average and was considerably higher than the non-performing loan rates in the commercial banking sector, which typically ranged from 13.8% to 14.47%.
2. Statement of the Problem
Despite the fact that Microfinance institutions (MFIs) in Machakos County play a crucial role in providing financial support to underserved populations, specifically low-income individuals with no formal regular income and small-scale entrepreneurs who may not access the traditional banking system. According to [2] and [3], most of their products and programs have collapsed due to repayment problems, threatening their financial sustainability and the ability to extend additional credit. This challenge could be attributed to poor credit risk assessment of loan applicants prior to advancing loans. The current available Machine Learning algorithms present a prime opportunity not only to improve credit risk assessment and reduce the current high nonperforming loan ratios but also to give the MFIs a competing edge. The adoption and application of machine learning in microfinance institutions in Kenya, particularly in Machakos County, remains underexplored. Therefore, this research sought to improve credit risk assessment by applying machine learning techniques among MFIs operating within Machakos, Kenya.
3. Literature Review
The research findings by [4] showed that poor credit risk assessment, had a significant impact on the nonperformance of loans advanced by MFIs in Kigali, Rwanda. The study by [5] recommended that an application of effective credit risk score management procedures which is sequentially developed and executed to specification, particularly through credit risk management information systems among all money lending institutions, would drop the default rates to the bare minimum in Kenya. Based on the study by [6], which utilized the machine learning methodology and techniques for credit scoring analysis by deploying logistic regression, random forest and XG boosting algorithms. XGBoost method approach was the best performer in all metrics evaluated. However, logistic regression remained the benchmark in the classification tasks mainly because of its interpretability compatible with the requirements of financial regulators. According to the study conducted by [7], “XGBoost has been optimized to increase GMB’s speed and prediction performance; it is advantageous because it is scalable and it can be integrated into a different platform”. Besides, “it is faster than other algorithms due to less resource usage”, the study noted. In their research [8], who implemented a machine learning model using the light gradient boosting machine (LightGBM) classification algorithm, and it was optimized using extreme gradient boosting (XGBoost) and identify feature importance to increase the accuracy of bankruptcy prediction. The research results obtained after going through the model testing process using different evaluation metrics that included the confusion matrix got the performance accuracy of “LightGBM + XGBoost feature importance of 99.227%”. The research by [9] on machine learning and credit risk scoring titled: “Empirical evidence from small- and mid-sized businesses” observed that one of the benefits of random forests is their capacity to handle unbalanced data and variables with missing values. Additionally, the algorithm mitigates the issue of arbitrary variable selection seen by certain alternative models. The integration of diverse data collecting, processing and analysis methods enhances precision and innovation in problem-solving. The interrelationships between the variables of the study are illustrated in the conceptual framework given in Figure 1.
![]()
Figure 1. The conceptual framework.
The loan requested could be categorized into different types depending on the criteria used. Individual loan applicant characteristics included the sex category, the number of years lived, marriage status, residence, and the highest formal training level attained. The details of SMEs business unit included, the business location, time in operation, previous loan payment behavior, and cash flow. The response variable was a credit risk status, categorical label of a loan applicant, categorized as either a default or non-default.
4. Methodology
The study adopted a mixed research design under supervised machine learning approach. The target population for this project were the loaned individuals and SMEs within the period 2020 to 2024 by the 21 registered MFIs operating in Machakos County, Kenya. The predictor variables included: the loan; amount, time dispersed, loan applicant’s income and the loan purpose. The applicants bio data captured included: age, gender, highest formal education level, location, previous loan repayment history, marital status and the number of dependants. Business related factors included: cash flow, area of operation, previous loan repayment history. The response variable was the individual’s and SMEs credit status as a categorical label (defaulter or non-defaulter).
Preprocessing
To ensure the data structure conformed to the machine learning formats, a preprocessing analysis was carried out. In order to ensure that there was no vital information unnecessarily got lost, mean imputation was performed for numeric features to address the missing data values while mode imputation was used for categorical features. Capping off the most extreme data values, categorical encoding, erasing all the duplicates was done before scaling the continuous variables into a zero to one range using normalization techniques so as to accommodate most of the ML algorithms.
5. Models
The study used: Logistic Regression algorithm, Extreme Gradient Boosting and Random forests ensemble method to analyze the data for prediction and classification of loan applicants as defaulters or non-defaulters. Rstudio, Excel and Python programming languages were deployed for data pre-processing, manipulation and classification.
5.1. Logistic Regression
Logistic regression method for classification was used Such that the dichotomous response variable of whether the applicant’s status is non-defaulter (0) or defaulter (1) such that; by letting the independent design matrix be X, then the response variable y with the binary values was:
Then the
such that:
(5.1)
where
is the likelihood of the credit risk score status of a loan applicant. The sigmoid function is most appropriate for performing logistic regression because it maps the linear combination of input features to a probability. The sigmoid function is given by Equation (5.2).
(5.2)
where
is the input variable such that; As
, the function approaches 1 and as
, the function approaches 0 resulting in the sigmoid shape illustrated in Figure 2. The sigmoid function takes any real input log-odds and gives output probability. Such that
. where x is a linear function of multiple explanatory variables such that
Figure 2. The Sigmoid function.
In the logistic model,
is interpreted as the probability of a binary response variable
whereby, the normal threshhold for the event being successful ranges between 0.5 to 1, otherwise it is deemed not successful. That is in this case, the probability of failing to pay will range between 0.5 to 1. Next, the log odds inverse is explained as
(5.3)
Then equivalently after exponentiation in both sides, we get the odds such that:
(5.4)
where
is the probability that the loan is not paid given some linear combination of the features.
is the intercept from linear regression when the numeric feature is equal to zero and the categorical feature is equal to the referenced feature. The study used both the regression coefficients and the odds ratio to establish the feature contribution and importance in the model.
5.2. Random Forest
Decision tree which is a simple model that divides data into different categories based on certain characteristics, is the foundation of the random forest. Decision trees mostly use entropy and information gain to decide where to split the data. Such that letting
.
where
is the probability of each class in the dataset and n represents the number of classes. Main idea is to select the feature that leads to the greatest reduction in entropy (information gain). According to [10], “A Random Forest model is a robust supervised ML algorithm composed of a tree-like structure that uses a set of decision trees to make predictions. Whereby each tree is trained on a random subset (with replacement) and a random subset of features”. This concept is well illustrated in Figure 3.
Figure 3. The random forest structure.
The final prediction for classification tasks is decided upon by a majority vote, while for the regression ones, it is the arithmetic mean of the predictions from all the trees.
5.3. Extreme Gradient Boosting
The Extreme Gradient Boosting, also referred to as XGBoost, is a ML model used for supervised learning challenges. It uses training data to predict a target variable. Its main strength is that it is a scalable and optimized algorithm that does improve the speed and prediction performance of Gradient Boosting Machines (GMB). It does either classification or regression based on the explanatory variables which could either be quantitative and, or qualitative. The XGBoost relies on the intuition that the best possible succeeding model, when combined with preceding models, minimizes the overall prediction error as illustrated in Figure 4. The errors for the classification tasks are minimized through the random forest process illustrated in Figure 4. Each succeeding model majors on misclassifications or errors of the preceding model and tries to reduce it in the succeeding iterations.
Figure 4. The Xgboost for classification.
5.4. Model Evaluation Metrics
The following classification metrics were used to evaluate the performance of different machine learning algorithms for this study. The selected metrics were preferred under the assumptions that, in a binary classification task, the instances of data are typically predicted to be either positive or negative such that, where the instance is present (loan default) is labeled positive while a negative implies that the instance does not differ from the baseline in this respect, no default. Consequently, each predicted binary label has four possible outcomes: True Positive (TP) which is a correctly predicted positive outcome, True Negative (TN) which is a correctly predicted negative outcome, False Positive (FP) which is a negative instance predicted to be positive, and a False Negative (FN) which is a positive instance predicted to be negative. Therefore, the metrics included; A confusion matrix, which is a 2 by 2 matrix containing the counts of TP, TN, FP, and FN observations. A confusion matrix shows how well the classification model is performing, and it was an output of Python and R codes used to analyze the data for this study. The confusion matrix is expressed as:
![]()
The confusion matrix counts were used to determine other metric evaluation measurements, as discussed below.
Accuracy was used to display the proportion of correctly classified instances, whether negative or positive and its value lies within 0 and 1.
The Precision score used to estimate the proportion of true positive predictions out of all positive predictions. Precision score is calculated as.
Sensitivity, which is usually referred to as recall, was also used to evaluate the performance of the models. It is calculated as:
Measures the proportion of true positive predictions out of all actual positives. The F1-Score which is the harmonic mean of precision and recall, was used to rank the performance of the models. It is calculated as;
It is always better to apply different evaluations metrics, especially in imbalanced datasets. Therefore, in addition to the above classification metrics, Area Under the Receiver Operating Characteristic (ROC) Curve and the decision tree classifier were also used. The AUC plots, True Positive Rate (Recall) vs. False Positive Rate at various thresholds. It ranges from 0 to 1, whereby a closer value to 1 means better performance. The Brier score (BS) which is a statistical measure used to evaluate the accuracy of probabilistic predictions—particularly in binary classification problems was also used.
(5.5)
where
1)
= total number of predictions.
2)
= predicted probability that default will occur.
3)
= actual outcome (1 if default occurred, 0 otherwise).
6. Results
6.1. Sample Size
The study used learning curve to choose a sample size appropriate for model training. Typically, the ideal sample was the saturation point where the curve no longer rises but plateau, indicating that there was no further significant performance improvement by adding more data. Learning models may stabilize at slightly different levels of sample size as shown in Figure 5 and the other models used in this study, plateau started at 3000 training sample size.
Figure 5. The learning curve.
Therefore, based on the learning curves from the three models, the ideal sample size for training data ranged between 3000 to 6000. The study used 6770 individual and small business loan accounts from the 21 registered microfinance institutions operating within Machakos County. Each microfinance was stratified to preserve representation, and then a unique identifier was assigned for each account. The accounts were randomly sampled within each stratum proportional to size without replacement.
6.2. Study Features and Label
The study features included demographic factors such as marital status, gender, age of the applicant, loan amount, credit history of the applicant, operating area and the date of disbursement between 2020 to 2024. Each loan was identified by the unique loan ID with labeled information on the status of the default payments. Table 1 gives the features and label summary.
Table 1. Study features and label.
|
Name |
Data type |
Units |
Coding of levels |
Features |
Loan ID |
Numeric |
|
|
|
Gender |
Categorical |
|
1 = female, 2 = male |
|
Marital status |
Categorical |
|
1 = single, 2 = married, 3 = others |
|
Dependents |
Categorical |
|
0 = None, 1 = One, 2 = Two, +3 = over three |
|
Education |
Categorical |
|
1 = Primary, 2 = Secondary, 3 = Post-Secondary |
|
Loan Purpose |
Categorical |
|
1 = Business, 2 = Personal, 3 = Others |
|
Applicant Income |
Numeric |
Kenya Shs |
|
|
Applicant Age |
Numeric |
Years |
|
|
Loan amount |
Numeric |
Kenya Shs |
|
|
Month Disbursed |
Categorical |
|
January to December |
|
Area of operation |
Categorical |
|
1 = Rural, 2 = Urban, 3 = Semi urban |
Label |
Default status |
Categorical |
|
0 = Not defaulted, 1 = Defaulted |
6.3. Data Exploratory
The study explored both the target and the features to get an overview of the target and features distribution and their general relationships. The findings are discussed in the following sections.
6.4. Target Distribution
The distribution of the response variable among the sampled loan accounts was performed. About a third of the loan applicants defaulted on payment, as shown in Figure 6. Based on the findings displayed in Figure 6, non-defaulters outnumbered
Figure 6. The loan default status.
the defaulters implying an imbalanced dataset. Since no resampling techniques such as SMOTE or under sampling were applied, the use of class weighting mitigated bias. The residual imbalance is acknowledged as a limitation. The dataset was partitioned into an 80/20 stratified train/test split. Further visualization was performed as displayed in the following sections.
6.5. Target and Features Distribution
This section related the target to the features of the study such as sex, marital status, credit history of the applicants and loan purpose as summarized in Figure 7.
Proportionately the higher default was by loan applicants with secondary school certificates, more default was by business loan applicants 18% compared to loan applicants for personal reasons 15.1%. The least default was by those without any credit history followed by those with more than three years’ experience. Male applicants were more likely to default compared to their female counterpart applicants.
Figure 7. The target feature exploratory.
7. Metric Evaluations
7.1. Logistic Regression Model Evaluation
Several model performance evaluations were carried out, including confusion matrix, recall, and F1-score. A confusion matrix also known as error matrix was used to visualize the breakdown of the predicted class of loan default status based on the historical loan default from loan applicant accounts in Machakos County. The confusion matrix summarized the true positives, false positives, true negatives and false negatives as shown in Table 2. Based on the Confusion matrix values, the model posted 935 true negatives but it also misclassified 45 accounts as positives. There were 237 false negatives alongside 137 true positives, the computed accuracy level was 0.791728212. The study also used ROC (Receiver Operating Characteristic) curve to plot the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings as shown in Figure 8. The Area Under the ROC Curve (AUC) was 0.785 implying a strong degree of separability between the binary classification of defaulters and non-defaulters. Other evaluation scores which included recall and F1-Score are summarized in Table 3.
Table 2. The confusion matrix.
Classification |
0 |
1 |
0 |
935 |
45 |
1 |
237 |
137 |
Figure 8. The ROC curve.
Table 3. The logistic regression evaluation results.
|
Classification |
Precision |
Recall |
F1-Score |
Support |
|
0 |
0.95 |
0.79 |
0.86 |
896 |
|
1 |
0.37 |
0.75 |
0.50 |
458 |
Accuracy |
|
|
|
0.79 |
1354 |
Macro avg |
|
0.66 |
0.77 |
0.68 |
1354 |
Weighted avg |
|
0.75 |
0.78 |
0.73 |
1354 |
7.2. Random Forest Model Evaluation
The necessary libraries were imported prior to data preprocessing and data analysis. Data were split into training and testing sets (80% train, 20% test) prior to training random forest model. The random forest algorithm performed better than the logistic classification algorithm in contrary to [11] whose logistic regression models through stepwise selection criteria outperformed the other models with an overall accuracy of 93.33% in his research on identifying the optimal model for financial distress prediction. However, the predictions in the test data by using the random forest model were in accordance with [9]. whose research on empirical evidence from small and medium businesses observed that one of the benefits of random forests over logistic regressions was their capacity to handle unbalanced data and variables with missing values. The predictions on the test data results of the random forest model are summarized in Table 4. The model performed better than logistic classification model in precision, F1-score and accuracy.
Table 4. The random forest evaluation results.
|
Classification |
Precision |
Recall |
F1-Score |
Support |
|
0 |
0.92 |
0.99 |
0.95 |
896 |
|
1 |
0.80 |
0.39 |
0.52 |
458 |
Accuracy |
|
|
|
0.81 |
1354 |
Macro avg |
|
0.86 |
0.69 |
0.74 |
1354 |
Weighted avg |
|
0.88 |
0.79 |
0.80 |
1354 |
7.3. XG Boost Model Evaluation
The XG Boost findings are summarized in Table 5. The model had a good accuracy level (83%) with a strong detection to non-defaulters which posted very high recall and high F1-score. Its detection to actual positives was comparatively low but good enough to detect two thirds. The weighted average was very good across different metric evaluation methods. The findings were in agreement with the study by [12], that machine learning has come out as a strong apparatus in overcoming the limitations of traditional business analysis statistics.
Table 5. The XG Boost evaluation results.
|
Classification |
Precision |
Recall |
F1-Score |
Support |
|
0 |
0.82 |
0.97 |
0.89 |
896 |
|
1 |
0.89 |
0.53 |
0.66 |
458 |
Accuracy |
|
|
|
0.83 |
1354 |
Macro avg |
|
0.85 |
0.75 |
0.78 |
1354 |
Weighted avg |
|
0.84 |
0.82 |
0.81 |
1354 |
7.4. Brier Score
For each model, the predicted probabilities of loan default were compared with the actual outcomes of the defaults in the test set. Calibration curves were constructed using 10 equally spaced probability bins, plotting the observed default rate within each bin against the mean predicted probability. The calibration curves generated to visualize the agreement between predicted probabilities and observed default rates is shown in Figure 9. Among the three models, XG Boost showed the best calibration (lowest Brier score), followed by Logistic Regression, while Random Forest exhibited comparatively weaker probability calibration. This meant that Logistic Regression and XG Boost predictions aligned more closely with the diagonal line of perfect calibration.
Figure 9. The Brier score.
8. Summary of the Findings
This study focused on performing different machine learning techniques in order to enhance credit risk assessment for microfinance institutions in Machakos County, Kenya. The study findings demonstrated that machine learning models, especially the XG Boost models which outperformed both the logistic regression models and random forest models by posting 83% prediction accuracy and 0.202 brier score was most appropriate in predicting loan applicants’ repayment outcome. The models proved more effective in classifying high- and low-risk loan applicants. The weighted average was very good across different metric evaluation methods. Alongside the classification evaluation metrics, Area Under the Receiver Operating Characteristic Curve (ROC) and the Confusion matrix were used to evaluate the logistic algorithm performance. The study findings were clear indication that by integrating machine learning into credit assessment risks can reduce loan default rates, improve on the institutional credit advancement sustainability, and expand financial inclusion by lending more low income applicants and SMEs. The study also confirmed the suitability and applicability of machine learning techniques, in predictive and classification performance within the microfinance institutions. The findings underscore the probable of data-driven decision-making in transforming the microfinance institutions’ lending practices in Kenya.
9. Recommendations
Microfinance institutions must invest in capacity building for handling big data, modeling, and output interpretation for evidence-based decision making. County and National governments should develop a legal framework to govern ethical and open use of machine learning assessment while observing the data protection Act. Explore more different machine learning techniques and compare their performance with the traditional credit score assessment performance.
Author Contributions
Conceptualization: MMK and JMK, methodology: MMK, JMK and AK, software: MMK and JMK, Data Analysis: MMK, MMK, AK and writing original draft preparation: MMK, JMK and AK, supervision: JMK and AK.