The segmentation model was implemented in four stages: data collection, data pre-processing, database design, and model training.

2.1.1. Data Collection

Equipment for data collection

The Canon Rebel T8i camera was used to obtain high-quality data. This camera can capture high-resolution videos. High-quality images with no noise are extracted from these videos. This camera offers the ability to adjust the image flow obtained from the videos and to have up to 60 frames per second with a resolution of 1920 × 1080.

The data come from videos taken on some asphalt-surfaced roads in Benin with low. The data were divided into data for model design and data for evaluating the degradation index calculation program. Table 1 presents the Structure of the collected data.

Table 1. Structure of the collected data.

Collection method

The collection carried out on the defined road axes initially consisted of a manual survey of the damage. In fact, for each degradation, parameters such as the following were measured in accordance with the Vizir method:

length and width for calculating the surface area of the tears, such as peeling. the width of the lips for the cracks.the depth of the deformations.the diameter of the potholes.

The severity of each type of damage encountered was assessed using the Vizir method. For damage that did not require measurement, such as crazing, edge spalling, repairs, and other issues, a visual assessment of severity was made based on the criteria defined by the VIZIR method. The photographer then filmed the roadway, holding the camera at a height of 1.20 m and at an angle of 45˚.

2.1.2. Data Pre-Processing

The collected videos underwent a five-step pre-processing process: image extraction, selective sorting, image resizing, pixel normalization, and image annotation.

Image extraction

Image extraction is the process of obtaining, according to a fixed step, these images that constitute the video. The extraction was possible using a script written in Python in the PyCharm environment.

Selective sorting

To perform effectively, the model requires clear and varied images so that it can extract as many characteristics as possible. To achieve this, selective sorting was carried out to remove redundant (duplicate) or blurred images from the databases.

Resizing images

Research has shown that an image that is too large requires more RAM than necessary, which slows down learning by neural networks.

The images in the databases were reduced by a quarter of their original size, from 1920 × 1080 pixels to 480 × 240 pixels. This resizing was done using a Python script called Resizing, written in PyCharm.

Pixel Normalization

Pixel normalization, like resizing, simplifies data to reduce computation time for neural networks. In this work, min-max normalization was used thanks to a Python script in PyCharm called pixel normalization to perform pixel normalization.

Image annotation

The model design through fine-tuning involves supervised learning. This requires prior identification of the images. Annotation was carried out based on the data collection summary table to indicate the damage present in each image and its severity. This step defines the model’s performance because if there is no consistency between the annotations made, the model will have difficulty extracting the right characteristics to recognize the damage on its own. The annotation was carried out on the Roboflow platform using the Roboflow annotated feature. The polygon tool was used to accurately indicate the damage present in the images to the models, as bounding boxes do not allow us to accurately and distinctly capture our damage with irregular contours. Once the outline was drawn, the damage class and severity level were used to label damage considered type A by the VIZIR method, and only the damage class for damage considered type B. This is because type B damage is not used to calculate the SDI. Thus, when labeling, it was sufficient to indicate the class of degradation encountered after outlining. Figure 1 shows an example of an annotated image using the polygon tool.

Figure 1. Example of an annotated image using the polygon tool.

2.1.3. Creation of Databases

The third step involved creating various databases. This involved creating a folder containing the annotated images, which were structured into training, validation, and test datasets. Before generating the databases, data augmentation was performed. Data augmentation involves artificially increasing the size of the dataset by creating modified versions of our annotated images. This process reduces overfitting and enhances the model’s generalization. Ultimately, the goal is to make the model more robust to potential variations and distortions that may be present in the input data. This includes changes in lighting, orientation, and scale.

After that, the databases were subdivided into three sets. One set, called the “training set”, contains the images used to train the model. A second set, called the “validation test”, contains the photos used to validate the model. The last set, called the “test set”, serves as test data.

Once the subdivision was completed, the designed databases were exported in COCO segmentation (Common Objects in Context) and YOLO V11 formats. Five databases are created, and their compositions are presented in Table 2. Table 3 presents the classes from dataset 5.

Converting databases to grayscale

The latest database was converted to grayscale using a Python script called Grayscale Conversion in the PyCharm IDE, which uses the CvtColor method with the BGR2 GRAY filter from the OpenCV library. This choice was made because conversion to grayscale allows for faster processing, which is crucial for real-time applications. It also simplifies the model’s input data, allowing it to focus on shapes and textures rather than color variations, which are not criteria for distinguishing between degradation classes. The conversion does not affect the outlines drawn around the damage or the annotations made.

Table 2. Composition of the databases created.

Table 3. Presentation of classes from the latest database.

2.1.4. Training of YOLOV11 Instance Segmentation and ROBOFLOW3.0 Instance Segmentation Models

Once the various databases had been designed, they were used to train the different transfer learning models: YOLOV11 instance segmentation and the Roboflow 3.0 Instance Segmentation model. YOLOV11 and Roboflow 3.0 models were selected because of their optimal balance between speed and accuracy, as well as their strong performance in transfer learning. These characteristics make them particularly well-suited for road environments, where scenes must be analyzed quickly and with a high level of reliability. We chose these models because they have a very large community, comprehensive tutorials, are easier to implement, and are available on the Roboflow platform, which we used for annotation and database creation.

The training took place on the Roboflow platform. At the end of each training session, the performance of the models was evaluated using the following metrics:

Precision.mAP (mean Average Precision).Recall.Confusion matrix.

A confidence threshold of 70% was set on the platform.

The weight of the model recovered after training the latest database was inserted into the code for object tracking using one of the videos from the data collection on the Zè-Zè-centre intersection. Thanks to the weight of the model, the object tracking algorithm will be able to determine the damage present in the videos.

To track damage, the ByteTrack algorithm, renowned for its accuracy and reliability, and the Python Supervision library, which provides connectors to models such as YOLO, were used. Thanks to this Python library, which offers a range of annotators and numerous highly customizable utilities, it is possible to view, load, split, merge, and save our datasets in different formats.

The Python script identifies all damage present on the filmed road surface and then draws a boundary polygon around the detected problems. It then assigns each defect an identifier called an “Id-tracker” and flags their presence when the camera passes over them. The code ultimately produces a video showing how the damage was tracked within the video.

2.3.1. Estimation of the Extent and Severity of Damage

The method used to estimate the extent consists, first, of retrieving the length, width, and coordinates of the center of the bounding boxes predicted by the model, as well as the identifier (id) of the predicted damage. This information is retrieved following the detections made by the model.

Then, based on the parameters set during field collection, namely:

a constant collection speed of 10 km/h;a camera viewing angle fixed at 45˚;a shooting height of 1.20 m, so that the filmed video covers the entire width of the road.

The number nif of image frames from which one leaves the visual field of an image could be estimated.

Field measurements have shown that the width of an image taken while respecting the previously mentioned parameters occupies = 5.5 m of the roadway length. This information, combined with the speed set at 20 km/h and the camera frame rate set at 60 frames per second, was used to find this number nif = 119 image frames.

This information was used to calculate the extent of images spaced 119 frames apart. This was done using a Python script called calcul_étendue, written in PyCharm, which extracts images with a step size of 119 frames and then uses the detection model to make predictions on these images. Following the predictions, for the damage detected, the code retrieves the coordinates of the bounding box to calculate the areas occupied by each of the areas of damage present in the image. It then calculates the ratio between this area and that of the image, which is (1920 × 1080). This ratio, thus calculated, is the extent of the degradation calculated by the formula:

with S T the total area of an image in p i x e l 2 (1080 × 1920 in our case).

2.3.2. Estimation of the Severity

During annotation, type A damage was annotated in the format “Damage name” + “Severity level”. This resulted in classes such as Ruts 3, indicating that the damage detected was a rut with a severity level of 3.

2.3.3. Evaluation of the Degradation Index

The degradation index, which characterizes the surface condition of our roads, is obtained by combining the extent and severity of the degradation. The method used until now has allowed the detection of the degradations with their level of severity and to estimate their extent. The SDI is obtained by combining the cracking index obtained from the extent and severity of cracks and alligator cracks, and the deformation index Id obtained by agglomerating the extent and severity of the deformations (Rut, Bead, Subsidence, Wane, etc.). The Vizir method presents tables and calculation principles for various indices (cracking index, deformation index, and surface degradation index).

The code then estimates the cracking index by combining the severity of longitudinal and transverse cracks and crazing with their extent, as indicated in the VIZIR method and presented in Table 4. The same was done for deformations. The maximum cracking index obtained on the 5.5 m sections is the cracking index for the 200 m section. The same applies to the deformation index. The values obtained for the cracking and deformation indices of the 200 m section are combined, as shown in Table 5, to calculate the deterioration index. If repairs were detected in the video, the SDI index was corrected in accordance with the VIZIR method specifications specified in Table 6.

Table 4. Methodology for evaluating crack and deformation indices using the Vizir method.

Table 5. Methodology for assessing the degradation index according to the VIZIR method.

Table 6. Evaluation of the degradation index considering repairs according to the Vizir method.

Once the code was complete, 200-meter video sequences from the data collection carried out on the Crossroad Zè-Zè Centre road and the Sèhouè-Massi section were used to evaluate its performance in PyCharm. Subsequently, thanks to the manual survey carried out during the collection, the index was calculated manually and compared with the results from the various videos to improve the program’s performance. After testing, the code was deployed within a mobile application.

Figure 2 and Figure 3 show the mAp50 score obtained from validation data for models designed before and after switching to 19 classes, respectively. The performance of the models designed has gradually improved. Initially, marks appearing on the road surface as cracks were annotated and assigned to the traces of track accident class. These marks were the main sources of background errors. In addition, using various confusion matrices, other sources of background errors related to blurred images and annotation errors were corrected. The preprocessing process, in particular pixel normalization, reduced the model’s sensitivity to variations in light intensity (sun, shade).

Figure 2. mAP50 scores obtained from validation data for models designed before switching to 19 classes.

Figure 3. mAP50 values obtained from validation data for models designed after switching to 19 classes.

Furthermore, the number of classes has been reduced in the latest databases because pathologies in the class of material up lift or displacement are not useful for calculating the degradation index. As a result, the classes created to indicate their severity levels have been merged. For example, the Potholes 1 and Potholes 2 classes have been merged into the Potholes class. It is noticed in these Figures that the combination of some classes increases the mAP50.

Figure 4 shows that the mAP50 precision increased for RIS3.4 and RIS3.5. The comparison of the results in Figure 4, shows that the grayscale conversion improved the model’s performance, resulting in a 2% improvement in accuracy, a 2.8% improvement in mAP, and a 0.2% improvement in Recall.

Figure 4. Evaluation of the impact of grayscale conversion on mAP50 for each degradation class.

Table 7 summarizes the Precision, the mAP50, and the recall of all tested models. Analysis of Table 7 shows that the best model obtained, named RIS 3.5, was achieved after training the Roboflow instance segmentation model on the fifth database converted to grayscale.

Table 7. Summary of the performance of the models designed.

The confusion matrix of the model RIS3.5 is presented in Figure 5. This confusion matrix indicates that there is low precision in predicting some types of degradation, such as raveling.

Figure 5. Confusion matrix of the best model RIS3.5.

Many authors have also tried other approaches to detecting pavement deterioration. Table 8 compares the developed model with those found in the literature. The number of pathology classes, methods, and evaluation metrics varies from one author to another. A comparison was made between the results obtained by the designed model and those found in the literature to assess its relevance. This comparison is presented in the following table.

The results presented in Table 8 show that the model RIS3.5 has good metrics compared to other models. It stands out for its good balance between accuracy, mAP, and recall. In addition, between the developed model is the only one that detects 19 classes of damage.

The surface degradation index calculation program was tested on the videos collected. A manual calculation of the index was also performed to verify the reliability of the program. The results obtained are presented in Table 9 and Table 10 for two sections of the road.

Table 8. Comparison of the model with those found in the literature.

Table 9. Degradation index found manually and with the model on the Carrefour Zè-Zè Centre axis.

Table 10. Degradation index found manually and with the model on the Sèhouè-Massi axis.

3.3.1. Section on the Carrefour Zè-Zè Centre Axis

Table 9 and Table 10 demonstrate that the model accurately identified the damage present on the filmed road surface. There is a similarity between the predictions made by the model and those obtained through the manual survey conducted in the field. The SDI obtained by manual calculation is equal to one, given that the damage recorded on this section of road is mostly rutting. The SDI obtained by aggregating the SDI determined per image (5.5 m) with this program matches that determined manually. The program, therefore, also performed well in this respect.

3.3.2. Second Portion of 200 m (Sèhouè-Massi Axis)

In this case, the model detected cracks, tears, and some deformations. The detections made and the severity levels found by the model correspond to those observed in the field. This demonstrates that the model is working properly. Next, using various Python codes, the extent of the multiple types of damage detected and the damage index per image were evaluated. After agglomeration, the SDI was evaluated at 200 m intervals. The program found a damage index of five before and after corrections, while the manual calculation gave an index of four before correction and five after correction. This discrepancy is because the model detects repairs and already incorporates the related corrections when calculating the SDI for each image. As a result, at the model level, the SDI before correction is identical to that after correction.

For the two sections of road studied, the model closely matching the detections made in the field, accurately detected the various pathologies. The program also accurately predicted the SDI. These positive results demonstrate the reliability of this approach.

3.3.3. Model Limitations and Application Constraints

The designed model has certain limitations that should be highlighted. The model does not allow all types of damage to be identified. Certain types of damage, such as road surface deflection or road subsidence, cannot be detected by the model because they were not recorded during the data collection process. The accuracy of detection remains sensitive to adverse weather conditions such as rain, humidity, or the presence of standing water on the road surface. Furthermore, changing the camera or shooting angle would affect the model’s performance.