Traditional applications of UAV target detection technology require the generation and collection of large volumes of video during routine inspections. Deep learning-based target detection allows for efficient and accurate identification of objects to be detected. While there are managers who process manually in data centres or deploy intelligent detection frameworks on cloud servers for automated processing, solutions via over-the-horizon communication are not suitable for the demanding tasks that arise from the proliferation of video volumes. With the performance and speed of intelligent computing devices constantly pushing new limits and more and more video processing tasks being placed on devices at the edge of the network, bringing them as close to the data source as possible, now is the beginning of the era of edge computing and edge devices. An intelligent autonomous sensing inspection system that incorporates cloud, edge computing, deep learning and big data technologies consists of the following three main components.

1) Data acquisition and analysis system

2) Real-time autonomous visual monitoring system for UAVs

3) High-performance single board devices for UAVs that can run real-time inspection depth models

The autonomous inspection system designed in this project is mainly based on the above 3 components. Based on the above analysis, a system model for real-time UAV detection is proposed in this paper, and the system architecture diagram is shown in Figure 1. This inspection process is divided into two phases: the first phase is the training phase, where data collection, dataset production and model training are to be completed to obtain training weights; the second phase is the inspection phase, where the inspection model and the modified business model (to achieve statistical, functional wearability) are deployed in advance to the edge-side single board device. This enables real-time monitoring of the aerial inspection process of drones, which require rapid and continuous analysis of incoming data to be able to parse the world around them and take action within millisecond response times.

This time constraint makes it impossible to rely entirely on the cloud to process the data stream when detecting, but rather the processing must be done locally. However, there is a disadvantage to processing locally: the edge device hardware does not have the computing power available on the cloud and cannot compromise on accuracy and speed depending on the actual requirements. The solution to this problem is either to use stronger and more efficient hardware, or not to use as complex a deep neural network. To get the best results, a balance has to be found between the two points.

Naeem et al. deployed the yolo series of algorithms using Raspberry Pi 4, Nvidia Jetson Nano, Nvidia Jetson TX2, and Nvidia AGX Xavier devices to evaluate the use of autonomous deep learning target detection algorithms on these airborne devices and use this for powerline inspection. But they did not experiment with the latest Nvidia Jetson NX [10] .

For graphical image processing UAVs, the core device for edge computing is the aircraft’s Single Board Computer (SBC), and the most advanced SBCs available are NVIDIA’s range of visual reasoning modules. NVIDIA’s leading single board computers are shown in Table 1.

On the NX, the NVIDIA Deep Learn Accelerator (NVDLA) engine and GPU run simultaneously at INT8 precision, while on the Jetson Nano and Jetson TX2, the GPU runs at FP16 precision. jetson Xavier NX delivers 10 times more performance compared to Jetson TX2 with the same power and 25% less footprint. The kit’s small size and light weight compared to the AGX led to the selection of the NX as the single board device to carry the UAV.

In this paper, we use the mean average precision (mAP) to measure the goodness

of the network model accuracy, mAP means the average of the APs of all categories within all images, which is calculated as

mAP = 1 N × ∑ i = 1 N AP i × % (1)

where: N—number of categories, which is 3 in this experiment; AP—Average accuracy per class, refers to the area under the P-R curve of a specific category within all images. AP is calculated as follows

AP = ∫ 0 1 P ( R ) d R (2)

P = TP TP + FP (3)

R = TP TP + FN (4)

where: TP indicates predicted with mining card, actual with mining card; FP indicates predicted with mining card, actual without mining card; FN indicates predicted without mining card, actual with mining card; TN indicates predicted without mining card, actual without mining card. The AP calculation of COCO will take multiple cross-merge ratios for calculation. In this experiment, the result of IOU = 0.5 is used to calculate AP.

In order to select a network model suitable for real-time target detection at NX, it is necessary to evaluate the performance of different target detection network models operating on NX, and therefore a quantitative criterion is proposed in this section. This criterion is based on the fact that a large amount of computational resources will be used by the UAV for real-time monitoring, and that the computational resource occupation will affect the operation of other functional modules of the overall detection system (e.g. distance detection, obstacle avoidance, etc.), and the performance of each model is analyzed comprehensively in this paper in relation to its occupation of hardware resources.

The NX single-board device on board the UAV turns on the maximum performance mode (MAXN mode) during the target detection of the mining card, closes all irrelevant terminal windows, and ensures that only one network model detection program and the jtop system resource monitoring program are opened for each test. This paper uses a scoring system to evaluate the overall performance of the network model, which is calculated using the formula

Grade = ∑ i = 1 3 w i g i (5)

where: Grade is total score; g₁, g₂, g₃ are mAP item score, average frame rate item score, hardware test score; w₁, w₂, w₃ are the weighting factor of the above 3 scores, which sums to 1.

When evaluating the drone detection resource score, GPU, CPU and RAM usage is considered and a corresponding score is assigned to each sub-category. As a rule of thumb, fluency is the most important factor for real-time drone detection, followed by object detection accuracy and hardware resource consumption respectively. For this reason, all evaluation items in real-time UAV monitoring are weighted and given different weighting factors according to their importance, with 1 being the highest weighting factor. This paper therefore assigns the following values to the weighting coefficients of all the test terms according to their importance: w₁ is 0.4, w₂ is 0.4 and w₃ is 0.2. A maximum standard score of 5 points was established for the scoring of the test items and the final score was calculated by decreasing the standard score by 1 point for each 20%. The item-specific evaluation criteria can be found in Table 2.

w₁ is the frame rate test weight, w₂ is the resource usage test weight and w₃ is the detection accuracy test weight.

The model training test platform is a H3C UniServer R4900 G3 dual-way rackmount server with Intel(R) Xeon(R) Bronze 3204 CPU, 16 GB RAM, NVIDIA A10 GPU and 24 GB video memory to build the DarkNet deep learning framework. On this basis, the training trials of the target detection model were completed.

The visual inference module on board the drone is NVIDIA’s NX single board device with a Volta architecture GPU containing 384 CUDA compute cores, each chip hitching 48 Tensor Cores, with over 59.7 GB/s of memory bandwidth and 8 GB of 128-bit LPDDR4x memory with a maximum frequency of 1600 MHz and an arithmetic power of 21 TOPS. As mentioned earlier, the YOLO family of target detection algorithms (YOLOv3, YOLOV4, YOLOv3-tiny and YOLOv4-tiny) will be used in this experiment.

Among them, the training parameters of YOLOv3, YOLOV4, YOLOv3-tiny and YOLOv4-tiny are the same: learning rate 0.001. A batch training method is used to divide the training set as well as the test set into four batches, both with a batch-size of 4, and the dataset is set to a uniform resolution of 640 × 640 for

T1 is CPU usage, core occupancy ≥ 95% is “core full”; T2 is GPU usage; T3 is RAM usage.

training and testing.

Data were collected using a DJI Royal 2 zoom version of the drone (Figure 2(a)) and real-time monitoring was carried out using a research drone p600 equipped with a Jetson Xavier NX (Figure 2(b)).

The dataset in this experiment was taken from the aerial video taken and produced by a UAV at the Dongsheng coalfield in Ordos, as shown in Figure 3. The video files were split with a frame rate of 10 images/s to eliminate redundant and repetitive images. In order to enrich the diversity of the dataset and increase the generalization ability of the model in practical use, a small part of the dataset was searched on the Internet or by means of screen shading, etc. A total of 1863 images were collected as the original images.

The data set was divided into a training set, a validation set and a test set according to 8:1:1, and the data set was expanded by data enhancement methods such as Mosaic and Mixup during training.

In this paper, four different deep learning models YOLOv3-DarkNet53, YOLOv3-tiny, YOLOv4-CSPdarknet53 and YOLOv4-tiny were trained to construct

the autonomous target detection algorithm, with 6000 (Figure 4) and 12,000 (Figure 5) iterations on the dataset respectively.

From Figure 4 it can be seen that YOLOv4-tiny after a relatively small number of iterations the model basically converges to 0.6, and YOLOv3 and YOLOv4 quickly reach a high training accuracy of 91.3% and 95.1% respectively.

Compared to YOLOv4 and YOLOv3, YOLOv4-tiny and YOLOv3-tiny have fewer convolution layers, which improves their suitability for real-time processing,

but theoretically reduces accuracy slightly. As can be seen from Figure 5, the training accuracy of YOLOv4 is higher (4.6% higher than YOLOv3), the convergence of the YOLOv3 and YOLOv4tiny algorithms is better, and the loss rate of YOLOv4tiny drops to 0.65 or less with little variation after a small number of iterations. Of all four models, YOLOv4-tiny gave the best training results with an accuracy of 80% and a loss of 0.62. YOLOv3 also had a low loss of 0.8 and an accuracy of 91.7%, but it used 53 convolutional layers, which made it significantly more computationally expensive compared to the 29 convolutional layers used in YOLOv4-tiny.

The results of real-time video detection on NX are showing on Table 3. The table lists the detection speed of the algorithms and the operation of hardware devices such as CPU, GPU and memory, based on the results below to obtain the grades in 5.3.

Based on the real-time detection results in Table 3 and the detection accuracy mAP values of the four network models (YOLOv3: 92.8, YOLOv3-tiny: 79.5, YOLOv4: 97.1, YOLOv4-tiny: 80.7), the scoring results of each model on NX were calculated using the scoring criteria in Section 3.4, as shown in Table 4.

YOLOv4-tiny and YOLOv3-tiny scored highest in resource usage with their lightweight size. Although YOLOv4 and YOLOv3 have higher detection accuracy, they do not score well in real-time target detection because of their slow detection speed and high resource usage. Other modules will also request access to GPU resources when the drone is performing real-time monitoring. Considering the high GPU usage of YOLOv3 and YOLOv4, using YOLOv4-tiny may be more beneficial to the overall performance of the system [10] .