Performance of Different Machine Learning Algorithms for Credit Risk Classification

Abstract

Microfinance institutions in Kenya play a unique role in promoting financial inclusion, loans, and savings provision, especially to low-income individuals and small-scale entrepreneurs. However, despite their benefits, most of their products and programs in Machakos County have been reducing due to repayment challenges, threatening their financial ability to extend further credit. This could be attributed to ineffective credit scoring models which are not able to establish the nuanced non-linear repayment behavior and patterns of the loan applicants. The research objective was to enhance credit risk scoring for microfinance institutions in Machakos County using supervised machine learning algorithms. The study adopted a mixed research design under supervised machine learning approach. It randomly sampled 6771 loan application account records and repayment history. Rstudio and Python programming languages were deployed for data pre-processing and analysis. Logistic regression algorithm, XG Boosting and the random forest ensemble method were used. Metric evaluations used included the performance accuracy, Area under the Curve and F1-Score. Based on the study findings: XG Boosting was the best performer with 83.3% accuracy and 0.202 Brier score. Development of legal framework to govern ethical and open use of machine learning assessment was recommended. A similar research but using different machine learning algorithms, locations, and institutions, to ascertain the validity, reliability and the generalizability of the study findings was recommended for further research.

Share and Cite:

Kasina, M. M., Kihoro, J. M. and Kibet, A. (2025) Performance of Different Machine Learning Algorithms for Credit Risk Classification. Journal of Data Analysis and Information Processing, 13, 504-519. doi: 10.4236/jdaip.2025.134029.

1. Introduction

Microfinance Institutions (MFIs) are financial establishments that offer financial services, basically loans and savings accounts, to individuals and Small and Medium Enterprises (SMEs) who more often fail to meet the traditional banking threshold due to lack of formal regular income or collateral to secure credit advance services. According to [1] “the financial performance of MFIs in Kenya continues to plummet and fluctuate unpredictably despite having implemented measures to limit loan defaults, as demonstrated over a five-year period spanning the years 2016 to 2020”. The study noted that the default rate among Microfinance Institutions (MFIs) was significantly higher than it was in the broader banking sector. Although the specific default rate varied, it was generally considered to be around 15% or higher, sometimes reaching 30.8%. This roughly doubled the acceptable industry average and was considerably higher than the non-performing loan rates in the commercial banking sector, which typically ranged from 13.8% to 14.47%.

2. Statement of the Problem

Despite the fact that Microfinance institutions (MFIs) in Machakos County play a crucial role in providing financial support to underserved populations, specifically low-income individuals with no formal regular income and small-scale entrepreneurs who may not access the traditional banking system. According to [2] and [3], most of their products and programs have collapsed due to repayment problems, threatening their financial sustainability and the ability to extend additional credit. This challenge could be attributed to poor credit risk assessment of loan applicants prior to advancing loans. The current available Machine Learning algorithms present a prime opportunity not only to improve credit risk assessment and reduce the current high nonperforming loan ratios but also to give the MFIs a competing edge. The adoption and application of machine learning in microfinance institutions in Kenya, particularly in Machakos County, remains underexplored. Therefore, this research sought to improve credit risk assessment by applying machine learning techniques among MFIs operating within Machakos, Kenya.

3. Literature Review

The research findings by [4] showed that poor credit risk assessment, had a significant impact on the nonperformance of loans advanced by MFIs in Kigali, Rwanda. The study by [5] recommended that an application of effective credit risk score management procedures which is sequentially developed and executed to specification, particularly through credit risk management information systems among all money lending institutions, would drop the default rates to the bare minimum in Kenya. Based on the study by [6], which utilized the machine learning methodology and techniques for credit scoring analysis by deploying logistic regression, random forest and XG boosting algorithms. XGBoost method approach was the best performer in all metrics evaluated. However, logistic regression remained the benchmark in the classification tasks mainly because of its interpretability compatible with the requirements of financial regulators. According to the study conducted by [7], “XGBoost has been optimized to increase GMB’s speed and prediction performance; it is advantageous because it is scalable and it can be integrated into a different platform”. Besides, “it is faster than other algorithms due to less resource usage”, the study noted. In their research [8], who implemented a machine learning model using the light gradient boosting machine (LightGBM) classification algorithm, and it was optimized using extreme gradient boosting (XGBoost) and identify feature importance to increase the accuracy of bankruptcy prediction. The research results obtained after going through the model testing process using different evaluation metrics that included the confusion matrix got the performance accuracy of “LightGBM + XGBoost feature importance of 99.227%”. The research by [9] on machine learning and credit risk scoring titled: “Empirical evidence from small- and mid-sized businesses” observed that one of the benefits of random forests is their capacity to handle unbalanced data and variables with missing values. Additionally, the algorithm mitigates the issue of arbitrary variable selection seen by certain alternative models. The integration of diverse data collecting, processing and analysis methods enhances precision and innovation in problem-solving. The interrelationships between the variables of the study are illustrated in the conceptual framework given in Figure 1.

Figure 1. The conceptual framework.

The loan requested could be categorized into different types depending on the criteria used. Individual loan applicant characteristics included the sex category, the number of years lived, marriage status, residence, and the highest formal training level attained. The details of SMEs business unit included, the business location, time in operation, previous loan payment behavior, and cash flow. The response variable was a credit risk status, categorical label of a loan applicant, categorized as either a default or non-default.

4. Methodology

The study adopted a mixed research design under supervised machine learning approach. The target population for this project were the loaned individuals and SMEs within the period 2020 to 2024 by the 21 registered MFIs operating in Machakos County, Kenya. The predictor variables included: the loan; amount, time dispersed, loan applicant’s income and the loan purpose. The applicants bio data captured included: age, gender, highest formal education level, location, previous loan repayment history, marital status and the number of dependants. Business related factors included: cash flow, area of operation, previous loan repayment history. The response variable was the individual’s and SMEs credit status as a categorical label (defaulter or non-defaulter).

Preprocessing

To ensure the data structure conformed to the machine learning formats, a preprocessing analysis was carried out. In order to ensure that there was no vital information unnecessarily got lost, mean imputation was performed for numeric features to address the missing data values while mode imputation was used for categorical features. Capping off the most extreme data values, categorical encoding, erasing all the duplicates was done before scaling the continuous variables into a zero to one range using normalization techniques so as to accommodate most of the ML algorithms.

5. Models

The study used: Logistic Regression algorithm, Extreme Gradient Boosting and Random forests ensemble method to analyze the data for prediction and classification of loan applicants as defaulters or non-defaulters. Rstudio, Excel and Python programming languages were deployed for data pre-processing, manipulation and classification.

5.1. Logistic Regression

Logistic regression method for classification was used Such that the dichotomous response variable of whether the applicant’s status is non-defaulter (0) or defaulter (1) such that; by letting the independent design matrix be X, then the response variable y with the binary values was:

Y={ 0 if non-defaulter 1 if a defaulter

Then the log( Odds( defaulter ) )=logitπ such that:

log π 1π = β 0 + β 1 x 1 ++ β p x p (5.1)

where π is the likelihood of the credit risk score status of a loan applicant. The sigmoid function is most appropriate for performing logistic regression because it maps the linear combination of input features to a probability. The sigmoid function is given by Equation (5.2).

S( x )= 1 1+ e x (5.2)

where x is the input variable such that; As x , the function approaches 1 and as x , the function approaches 0 resulting in the sigmoid shape illustrated in Figure 2. The sigmoid function takes any real input log-odds and gives output probability. Such that σ:( 0,1 ) . where x is a linear function of multiple explanatory variables such that

S( x )=p( x )= 1 1+ e β 0 + β i x i

Figure 2. The Sigmoid function.

In the logistic model, p( x ) is interpreted as the probability of a binary response variable y whereby, the normal threshhold for the event being successful ranges between 0.5 to 1, otherwise it is deemed not successful. That is in this case, the probability of failing to pay will range between 0.5 to 1. Next, the log odds inverse is explained as

logitp( x )=ln p( x ) 1p( x ) = β 0 + β i x i (5.3)

Then equivalently after exponentiation in both sides, we get the odds such that:

p( x ) 1p( x ) = e β 0 + β i x i (5.4)

where p( x ) is the probability that the loan is not paid given some linear combination of the features. β 0 is the intercept from linear regression when the numeric feature is equal to zero and the categorical feature is equal to the referenced feature. The study used both the regression coefficients and the odds ratio to establish the feature contribution and importance in the model.

5.2. Random Forest

Decision tree which is a simple model that divides data into different categories based on certain characteristics, is the foundation of the random forest. Decision trees mostly use entropy and information gain to decide where to split the data. Such that letting

entropy=H,thenH( S )= i n p i log 2 p i .

where p i is the probability of each class in the dataset and n represents the number of classes. Main idea is to select the feature that leads to the greatest reduction in entropy (information gain). According to [10], “A Random Forest model is a robust supervised ML algorithm composed of a tree-like structure that uses a set of decision trees to make predictions. Whereby each tree is trained on a random subset (with replacement) and a random subset of features”. This concept is well illustrated in Figure 3.

Figure 3. The random forest structure.

The final prediction for classification tasks is decided upon by a majority vote, while for the regression ones, it is the arithmetic mean of the predictions from all the trees.

5.3. Extreme Gradient Boosting

The Extreme Gradient Boosting, also referred to as XGBoost, is a ML model used for supervised learning challenges. It uses training data to predict a target variable. Its main strength is that it is a scalable and optimized algorithm that does improve the speed and prediction performance of Gradient Boosting Machines (GMB). It does either classification or regression based on the explanatory variables which could either be quantitative and, or qualitative. The XGBoost relies on the intuition that the best possible succeeding model, when combined with preceding models, minimizes the overall prediction error as illustrated in Figure 4. The errors for the classification tasks are minimized through the random forest process illustrated in Figure 4. Each succeeding model majors on misclassifications or errors of the preceding model and tries to reduce it in the succeeding iterations.

Figure 4. The Xgboost for classification.

5.4. Model Evaluation Metrics

The following classification metrics were used to evaluate the performance of different machine learning algorithms for this study. The selected metrics were preferred under the assumptions that, in a binary classification task, the instances of data are typically predicted to be either positive or negative such that, where the instance is present (loan default) is labeled positive while a negative implies that the instance does not differ from the baseline in this respect, no default. Consequently, each predicted binary label has four possible outcomes: True Positive (TP) which is a correctly predicted positive outcome, True Negative (TN) which is a correctly predicted negative outcome, False Positive (FP) which is a negative instance predicted to be positive, and a False Negative (FN) which is a positive instance predicted to be negative. Therefore, the metrics included; A confusion matrix, which is a 2 by 2 matrix containing the counts of TP, TN, FP, and FN observations. A confusion matrix shows how well the classification model is performing, and it was an output of Python and R codes used to analyze the data for this study. The confusion matrix is expressed as:

The confusion matrix counts were used to determine other metric evaluation measurements, as discussed below.

Accuracy= TP+TN TP+TN+FP+FN [ 0,1 ]

Accuracy was used to display the proportion of correctly classified instances, whether negative or positive and its value lies within 0 and 1.

The Precision score used to estimate the proportion of true positive predictions out of all positive predictions. Precision score is calculated as.

Precision= TP TP+FP [ 0,1 ]

Sensitivity, which is usually referred to as recall, was also used to evaluate the performance of the models. It is calculated as:

Recall= TP TP+FN [ 0,1 ]

Measures the proportion of true positive predictions out of all actual positives. The F1-Score which is the harmonic mean of precision and recall, was used to rank the performance of the models. It is calculated as;

F1Score= 2PrecisonRecall Precision+Recall [ 0,1 ]

It is always better to apply different evaluations metrics, especially in imbalanced datasets. Therefore, in addition to the above classification metrics, Area Under the Receiver Operating Characteristic (ROC) Curve and the decision tree classifier were also used. The AUC plots, True Positive Rate (Recall) vs. False Positive Rate at various thresholds. It ranges from 0 to 1, whereby a closer value to 1 means better performance. The Brier score (BS) which is a statistical measure used to evaluate the accuracy of probabilistic predictions—particularly in binary classification problems was also used.

BS= 1 N i=1 N ( f i o i ) 2 (5.5)

where

1) N = total number of predictions.

2) f i = predicted probability that default will occur.

3) o i = actual outcome (1 if default occurred, 0 otherwise).

6. Results

6.1. Sample Size

The study used learning curve to choose a sample size appropriate for model training. Typically, the ideal sample was the saturation point where the curve no longer rises but plateau, indicating that there was no further significant performance improvement by adding more data. Learning models may stabilize at slightly different levels of sample size as shown in Figure 5 and the other models used in this study, plateau started at 3000 training sample size.

Figure 5. The learning curve.

Therefore, based on the learning curves from the three models, the ideal sample size for training data ranged between 3000 to 6000. The study used 6770 individual and small business loan accounts from the 21 registered microfinance institutions operating within Machakos County. Each microfinance was stratified to preserve representation, and then a unique identifier was assigned for each account. The accounts were randomly sampled within each stratum proportional to size without replacement.

6.2. Study Features and Label

The study features included demographic factors such as marital status, gender, age of the applicant, loan amount, credit history of the applicant, operating area and the date of disbursement between 2020 to 2024. Each loan was identified by the unique loan ID with labeled information on the status of the default payments. Table 1 gives the features and label summary.

Table 1. Study features and label.

Name

Data type

Units

Coding of levels

Features

Loan ID

Numeric

Gender

Categorical

1 = female, 2 = male

Marital status

Categorical

1 = single, 2 = married, 3 = others

Dependents

Categorical

0 = None, 1 = One, 2 = Two, +3 = over three

Education

Categorical

1 = Primary, 2 = Secondary, 3 = Post-Secondary

Loan Purpose

Categorical

1 = Business, 2 = Personal, 3 = Others

Applicant Income

Numeric

Kenya Shs

Applicant Age

Numeric

Years

Loan amount

Numeric

Kenya Shs

Month Disbursed

Categorical

January to December

Area of operation

Categorical

1 = Rural, 2 = Urban, 3 = Semi urban

Label

Default status

Categorical

0 = Not defaulted, 1 = Defaulted

6.3. Data Exploratory

The study explored both the target and the features to get an overview of the target and features distribution and their general relationships. The findings are discussed in the following sections.

6.4. Target Distribution

The distribution of the response variable among the sampled loan accounts was performed. About a third of the loan applicants defaulted on payment, as shown in Figure 6. Based on the findings displayed in Figure 6, non-defaulters outnumbered

Figure 6. The loan default status.

the defaulters implying an imbalanced dataset. Since no resampling techniques such as SMOTE or under sampling were applied, the use of class weighting mitigated bias. The residual imbalance is acknowledged as a limitation. The dataset was partitioned into an 80/20 stratified train/test split. Further visualization was performed as displayed in the following sections.

6.5. Target and Features Distribution

This section related the target to the features of the study such as sex, marital status, credit history of the applicants and loan purpose as summarized in Figure 7.

Proportionately the higher default was by loan applicants with secondary school certificates, more default was by business loan applicants 18% compared to loan applicants for personal reasons 15.1%. The least default was by those without any credit history followed by those with more than three years’ experience. Male applicants were more likely to default compared to their female counterpart applicants.

Figure 7. The target feature exploratory.

7. Metric Evaluations

7.1. Logistic Regression Model Evaluation

Several model performance evaluations were carried out, including confusion matrix, recall, and F1-score. A confusion matrix also known as error matrix was used to visualize the breakdown of the predicted class of loan default status based on the historical loan default from loan applicant accounts in Machakos County. The confusion matrix summarized the true positives, false positives, true negatives and false negatives as shown in Table 2. Based on the Confusion matrix values, the model posted 935 true negatives but it also misclassified 45 accounts as positives. There were 237 false negatives alongside 137 true positives, the computed accuracy level was 0.791728212. The study also used ROC (Receiver Operating Characteristic) curve to plot the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings as shown in Figure 8. The Area Under the ROC Curve (AUC) was 0.785 implying a strong degree of separability between the binary classification of defaulters and non-defaulters. Other evaluation scores which included recall and F1-Score are summarized in Table 3.

Table 2. The confusion matrix.

Classification

0

1

0

935

45

1

237

137

Figure 8. The ROC curve.

Table 3. The logistic regression evaluation results.

Classification

Precision

Recall

F1-Score

Support

0

0.95

0.79

0.86

896

1

0.37

0.75

0.50

458

Accuracy

0.79

1354

Macro avg

0.66

0.77

0.68

1354

Weighted avg

0.75

0.78

0.73

1354

7.2. Random Forest Model Evaluation

The necessary libraries were imported prior to data preprocessing and data analysis. Data were split into training and testing sets (80% train, 20% test) prior to training random forest model. The random forest algorithm performed better than the logistic classification algorithm in contrary to [11] whose logistic regression models through stepwise selection criteria outperformed the other models with an overall accuracy of 93.33% in his research on identifying the optimal model for financial distress prediction. However, the predictions in the test data by using the random forest model were in accordance with [9]. whose research on empirical evidence from small and medium businesses observed that one of the benefits of random forests over logistic regressions was their capacity to handle unbalanced data and variables with missing values. The predictions on the test data results of the random forest model are summarized in Table 4. The model performed better than logistic classification model in precision, F1-score and accuracy.

Table 4. The random forest evaluation results.

Classification

Precision

Recall

F1-Score

Support

0

0.92

0.99

0.95

896

1

0.80

0.39

0.52

458

Accuracy

0.81

1354

Macro avg

0.86

0.69

0.74

1354

Weighted avg

0.88

0.79

0.80

1354

7.3. XG Boost Model Evaluation

The XG Boost findings are summarized in Table 5. The model had a good accuracy level (83%) with a strong detection to non-defaulters which posted very high recall and high F1-score. Its detection to actual positives was comparatively low but good enough to detect two thirds. The weighted average was very good across different metric evaluation methods. The findings were in agreement with the study by [12], that machine learning has come out as a strong apparatus in overcoming the limitations of traditional business analysis statistics.

Table 5. The XG Boost evaluation results.

Classification

Precision

Recall

F1-Score

Support

0

0.82

0.97

0.89

896

1

0.89

0.53

0.66

458

Accuracy

0.83

1354

Macro avg

0.85

0.75

0.78

1354

Weighted avg

0.84

0.82

0.81

1354

7.4. Brier Score

For each model, the predicted probabilities of loan default were compared with the actual outcomes of the defaults in the test set. Calibration curves were constructed using 10 equally spaced probability bins, plotting the observed default rate within each bin against the mean predicted probability. The calibration curves generated to visualize the agreement between predicted probabilities and observed default rates is shown in Figure 9. Among the three models, XG Boost showed the best calibration (lowest Brier score), followed by Logistic Regression, while Random Forest exhibited comparatively weaker probability calibration. This meant that Logistic Regression and XG Boost predictions aligned more closely with the diagonal line of perfect calibration.

Figure 9. The Brier score.

8. Summary of the Findings

This study focused on performing different machine learning techniques in order to enhance credit risk assessment for microfinance institutions in Machakos County, Kenya. The study findings demonstrated that machine learning models, especially the XG Boost models which outperformed both the logistic regression models and random forest models by posting 83% prediction accuracy and 0.202 brier score was most appropriate in predicting loan applicants’ repayment outcome. The models proved more effective in classifying high- and low-risk loan applicants. The weighted average was very good across different metric evaluation methods. Alongside the classification evaluation metrics, Area Under the Receiver Operating Characteristic Curve (ROC) and the Confusion matrix were used to evaluate the logistic algorithm performance. The study findings were clear indication that by integrating machine learning into credit assessment risks can reduce loan default rates, improve on the institutional credit advancement sustainability, and expand financial inclusion by lending more low income applicants and SMEs. The study also confirmed the suitability and applicability of machine learning techniques, in predictive and classification performance within the microfinance institutions. The findings underscore the probable of data-driven decision-making in transforming the microfinance institutions’ lending practices in Kenya.

9. Recommendations

Microfinance institutions must invest in capacity building for handling big data, modeling, and output interpretation for evidence-based decision making. County and National governments should develop a legal framework to govern ethical and open use of machine learning assessment while observing the data protection Act. Explore more different machine learning techniques and compare their performance with the traditional credit score assessment performance.

Author Contributions

Conceptualization: MMK and JMK, methodology: MMK, JMK and AK, software: MMK and JMK, Data Analysis: MMK, MMK, AK and writing original draft preparation: MMK, JMK and AK, supervision: JMK and AK.

Conflicts of Interest

All authors have read and agreed to the published version of the manuscript.

References

[1] Nyambu, C.M. and Ogada, J.M. (2024) Non-Performing Loan Characteristics and Financial Performance of Microfinance Banks in Kenya. International Journal of Social Sciences Management and Entrepreneurship, 8, 926-940.
[2] Kimani, A., Arasa, R. and Karau, J. (2024) Resource Allocation and Performance of Microfinance Institutions in Machakos County, Kenya. International Journal of Finance and Accounting, 3, 28-35.[CrossRef]
[3] Muthini, J.M. and Ndede, F.W.S. (2022) Socio-Economic Factors and Table Banking Loans Default Levels among Women Groups in Machakos County, Kenya. International Journal of Current Aspects in Finance, Banking and Accounting, 4, 91-103.[CrossRef]
[4] Twesige, D., Uwamahoro, A., Ndikubwimana, P., Gasheja, F., Misago, I.K. and Hategikimana, U. (2021) Causes of Loan Defaults within Microfinance Institutions: Learning from Micro and Small Business Owners in Rwanda: A Case of MSEs in Kigali. Rwanda Journal of Social Sciences, Humanities and Business, 2, 27-49.[CrossRef]
[5] Karanja, S.G. and Simiyu, E.M. (2022) Credit Management Practices and Loan Performance of Microfinance Banks in Kenya. Journal of Finance and Accounting, 6, 108-139.[CrossRef]
[6] Susana, D.M.P. (2024) Optimizing Credit Scoring Models in Face of Global Economic Uncertainty: A Comprehensive Risk Analysis in Banking Loans. Master’s Thesis, Universidade NOVA de Lisboa (Portugal).
[7] Trizoglou, P., Liu, X.L. and Lin, Z. (2021) Fault Detection by an Ensemble Framework of Extreme Gradient Boosting (XGBoost) in the Operation of Offshore Wind Turbines. Renewable Energy, 179, 945-962.[CrossRef]
[8] Syafei, R.M. and Efrilianda, D.A. (2023) Machine Learning Model Using Extreme Gradient Boosting (XGBoost) Feature Importance and Light Gradient Boosting Machine (LightGBM) to Improve Accurate Prediction of Bankruptcy. Recursive Journal of Informatics, 1, 64-72.[CrossRef]
[9] Bitetto, A., Cerchiello, P., Filomeni, S., Tanda, A. and Tarantino, B. (2023) Machine Learning and Credit Risk: Empirical Evidence from Small-and Mid-Sized Businesses. Socio-Economic Planning Sciences, 90, Article 101746.[CrossRef]
[10] Salman, H.A., Kalakech, A. and Steiti, A. (2024) Random Forest Algorithm Overview. Babylonian Journal of Machine Learning, 2024, 69-79.[CrossRef]
[11] Zizi, Y., Jamali-Alaoui, A., El Goumi, B., Oudgou, M. and El Moudden, A. (2021) An Optimal Model of Financial Distress Prediction: A Comparative Study between Neural Networks and Logistic Regression. Risks, 9, Article 200.[CrossRef]
[12] Rahaman, M., Rani, S., Islam, R. and Bhuiyan, M.M.R. (2023) Machine Learning in Business Analytics: Advancing Statistical Methods for Data-Driven Innovation. Journal of Computer Science and Technology Studies, 5, 104-111.[CrossRef]

Copyright © 2026 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.