The RT-DETR model has set a precedent for the first real-time end-to-end object detection model. This object detection model, based on the Transformer architecture, aims to surpass YOLO and achieve more efficient and rapid real-time object recognition. RT-DETR fully leverages the potential of Transformers in the field of computer vision, elevating the performance of real-time object detection to new heights. Like DETR, RT-DETR consists of a backbone network and a hybrid encoder.

The backbone is the initial stage where input images are processed to extract features. This typically involves a convolutional neural network that reduces the spatial dimensions of the image while increasing depth, generating feature maps at various stages (S3, S4, S5).

The hybrid encoder comprises two main parts: Attention-based Intra-scale Feature Interaction (AIFI) and Cross-scale Feature Fusion (CCFF).

AIFI processes only the S5 feature map from the backbone. Since S5 represents the deepest layer, it contains the richest semantic information about complete objects and their context in the image—making it highly suitable for the Transformer-based attention to capture meaningful relationships between objects. Although S3 and S4 contain useful low-level features such as edges and object parts, applying attention to these layers would be computationally expensive and less efficient because they do not yet represent complete objects that need to be related to each other.

CCFF, based on CNN layers, is responsible for combining the S3, S4, and AIFI (S5) feature maps. Its task is to fuse features from different scales, ensuring that the final feature map contains information from a variety of resolutions. The core of CCFF is a special block called RepBlock, which uses a structure named RepConv. RepConv allows the network to switch between different forms of convolution operations during training and inference, thus improving efficiency without sacrificing performance.

Following this, uncertainty queries are performed, and after selecting the most confident queries, RT-DETR uses a Transformer decoder with multi-scale deformable attention to predict object locations and categories. The decoder architecture is designed for efficient processing of multi-scale feature maps while maintaining high precision.

To facilitate the deployment of the model on embedded devices for on-site real-time forest fire detection while maintaining its detection performance, several improvements have been made. Firstly, to address the issues of large parameter size and computational complexity in RT-DETR, the backbone network has been replaced with a lightweight MobileNetV4 structure. To counteract the decrease in detection accuracy after lightweighting, the encoding method has been improved to the LPE (Learning Potential Encoding) method, which allows the model to better learn feature information. Additionally, the SDI (Scale-Discrepancy Integration) module has been employed to enhance the integration of fire features at different resolutions. The improved structure is shown in Figure 1 .

The latest generation of MobileNets, MobileNetV4 [8] , presents a general and efficient architecture design tailored for mobile devices. At its core, the research team has introduced a search block known as Universal Inverted Bottleneck (UIB). This structure is unified and flexible, ingeniously integrating the Inverted Bottleneck (IB) [9] , ConvNext [10] , Feed-Forward Network (FFN), and an innovative Extra Depthwise (ExtraDW) [11] variant. In addition to UIB, the research team has also proposed Mobile MQA, an attention block specifically designed for mobile accelerators that can significantly enhance acceleration performance by 39%. Furthermore, to further improve the search efficiency of MNv4, the team has developed an optimized Neural Architecture Search (NAS) recipe.

(1) UIB

As shown in Figure 2 , in the design of the inverted bottleneck, two optional Depthwise layers have been added. They are placed before the expansion layer and between the expansion layer and the projection layer. The inclusion of these layers is a key decision in the Neural Architecture Search (NAS) optimization process, leading to the creation of a new network architecture. Although this adjustment may seem simple, it ingeniously integrates multiple key network blocks, including the original Inverted Bottleneck (IB) block, the ConvNext block, and the Feed-Forward Network (FFN) block from ViT. Additionally, the UIB block introduces an innovative variant, the Extra Depthwise Inverted Bottleneck (ExtraDW) block.

The Inverted Bottleneck (IB) increases the model’s capacity at the cost of increased computational expense by performing spatial mixing on the expanded feature activations. ConvNext achieves a less costly spatial mixing by performing the spatial mixing operation before feature expansion and allows for the use of larger convolution kernel sizes. ExtraDW is a new variant proposed in this article that enhances the network’s depth and receptive field at a lower cost, combining the advantages of ConvNext and the Inverted Bottleneck (IB). The FFN consists of two stacked 1x1 pointwise convolutions (PW) layers, interspersed with activation layers and normalization layers. PW operations are among the most effective on accelerators, but their impact is most significant when used in conjunction with other network blocks.

(2) Mobile MQA

MQA significantly reduces the burden of memory access by implementing the sharing of keys and values across all attention heads. Building on this, a strategy of Spatial Reduction Attention (SRA) is further adopted, applying an asymmetric spatial downsampling technique to the keys and values to enhance computational efficiency. In the implementation details of MQA, a 3 × 3 depthwise separable convolution is used to replace the traditional average pooling step, effectively halving the spatial resolution of the keys and values. This dramatically reduces the inference time. The following is the mathematical expression for Mobile MQA:

$MobileMQA\left(x\right)=Concat\left(attentio{n}_1\cdots attentiuo{n}_n\right)W^O$ (1)

$attentio{n}_j=\left(\frac{\left(X{W}^{Q_j}\right){\left(SR\left(X\right)W^k\right)}^T}{\sqrt{d_k}}\right){\left(SR\left(X\right)W^k\right)}^V$ (2)

SR stands for spatial reduction, which involves asymmetric spatial downsampling of keys and values to further enhance efficiency.

In the attention mechanism of transformers, to address the issue of ambiguous word vector positions in parallel operations, positional encoding is used to imbue each word vector with positional semantic information. This allows the attention mechanism to operate in parallel more effectively. The original positional encoding in the Transformer model uses a combination of sine and cosine functions [12] , which is relatively fixed and may not be ideal for expressing complex positional information. Therefore, Localized Positional Encoding (LPE) is adopted.

LPE treats positional information as a parameter during the training process, continuously adjusting the encoding information in each training iteration. This allows for a more thorough representation of the positional relationships between each vector, thereby enhancing the effectiveness of the attention mechanism. Moreover, since LPE is trained on different datasets, it can adapt to various data distributions.

As shown in Figure 3 , each element in the token sequence finds its corresponding positional encoding through a lookup in the embedding table. The positional encoding is then added to the element to obtain a vector representation that includes positional information. Subsequently, during training, the loss function is computed, and during the backpropagation process, the weight information in the position embedding is optimized. This results in more suitable positional information for each element, leading to better training outcomes for the attention mechanism.

In the basic RT-DETR module, the feature fusion module aims to reduce computational complexity and the consumption of computational resources. It overlooks low-level detail information and prioritizes high-level semantic information, leading to insufficient fusion of high and low-level feature information. Therefore, the HLF module is proposed to enable more thorough fusion of high and low-level feature information. The schematic diagram of the module structure is shown in Figure 4 .

The illustration shows the HLF (Hierarchical Level Fusion) module. Initially, the feature layers of various scales from the encoder input are taken as inputs and processed through channel attention mechanisms [13] and spatial attention mechanisms [14] , respectively. The spatial attention mechanism focuses on identifying key regions within the feature maps, while the channel attention mechanism concentrates on selecting the primary channels within the feature maps. By utilizing these two attention mechanisms, the feature maps are able to fuse local spatial details with global channel information, resulting in a more expressive feature representation. This process is as shown in Equation 3:

$f_i^1={\phi }_i^c\left({\phi }_i^s\left(f_i^0\right)\right)$ (3)

Formula 3 represents the feature integration process on, where $f_i^0$ denotes the feature layer at the i-th scale that is input after passing through the encoder. ${\phi }_i^s$ represents the feature at the i-th layer after being processed by the spatial attention mechanism, and ${\phi }_i^c$ indicates the feature at the i-th layer after being processed by the channel attention mechanism.

Following this, feature scale adjustment is performed. If the size of the existing feature map is smaller than the target size, bilinear interpolation is used to upscale it to the target resolution. If the existing feature map is larger than the target size, adaptive average pooling is applied to downscale it to the target resolution. If the feature map’s size already matches the target resolution, it is used directly. This process is shown in formula 4:

$f_{ij}^3\{\begin{array}{l}D\left(f_i^2,\left(H_i,W_i\right)\right)j<i\\ I\left(f_i^2\right)j=i\\ U\left(f_i^2,\left(H_i,W_i\right)\right)j>i\end{array}$ (4)

Formula 4 represents the fusion of feature maps at different resolutions. D denotes downsampling operations when the image resolution is higher than the target image resolution. I indicates that the image is used directly when the image resolution is equal to the target image resolution. U represents upsampling operations using bilinear interpolation when the image resolution is lower than the target resolution, to restore it to the target resolution.

For each feature map that has been resized, a 3 × 3 convolution kernel is applied for smoothing. This 3 × 3 convolution effectively removes interference noise while preserving the image details, resulting in clearer feature maps.

This process is shown in Equation 5 as follows:

$f_{ij}^5={\theta }_{ij}\left(f_{ij}^3\right)$ (5)

where θij represents the parameters of the smooth convolution, and fij is the j-th smoothed feature map at the i-th level.

Finally, a Hadamard product [15] operation is applied to all the adjusted feature maps to generate the merged feature map. The Hadamard product is an effective method that combines two feature maps while retaining rich detail information.

In this experiment, the training parameters are set as follows: the number of epochs is set to 100, image size is set to 640, batch size is set to 16, and the learning rate (lr) is set to 0.1. The optimizer chosen is AdamW, with a weight decay of 0.0001 and a global gradient clipping range of 0.1. The network was not initialized with pre-trained weights. Data augmentation includes random color distortion, expansion, cropping, flipping, and resizing. The linear warm-up steps are set to 10. The Hardware and Software Configuration of the experimental environment is shown in Table 1 below.

As shown in Table 1 , The operating system used is Windows 11, with Python version 3.8.8. The PyTorch version is 2.20. For GPU, it’s equipped with an NVIDA GEForce RTX 3060, and the CPU is the 12th Gen Intel Core i5-12400F.

The dataset used in this experiment was created personally. The author provides the dataset at the following address: https://download.csdn.net/download/m0_64879847/88535127 . To evaluate the effectiveness of the improved RT-DETR detection algorithm in the field of small target detection using unmanned aerial vehicle (UAV) aerial photography. The data was collected using a drone, The drone model is the DJI M300 RTK, operating at an altitude of 300 - 500 meters, and is equipped with an H20T payload camera. The video resolution is 1920 × 1080 at 30 frames per second. Data is transmitted to the backend server through CAN/4G/WiFi and other communication protocols. After frame extraction and data cleansing, an aerial forest fire image dataset is produced. This dataset encompasses records of forest fire incidents within the central region of the country over the past five years, spanning from 2020 to 2025 and covering multiple forested areas. The dataset is categorized into two classes: the flame dataset and the non-flame dataset. To enhance the completeness of the sample dataset, images of both small and large fire spots, as well as various environmental backgrounds under different seasonal conditions and daylight/darkness scenarios, have been collected, encompassing a range of forest vegetation coverage states. For the object detection task, 3699 images were selected as the training set, 800 images as the test set, and 500 images as the validation set. The resolution is 512 × 512 pixels.

The evaluation metrics used in this experiment are precision (P), recall®, mean Average Precision with an IOU threshold of 0.5 (mAP50), number of parameters (Parameter), and GFLOPS.

Precision refers to the proportion of samples that we predict as positive that are actually positive. Recall refers to the proportion of samples that are actually positive that we correctly predict as positive. mAP50 is an indicator that refers to the calculation of Average Precision (AP) with an IoU threshold set to 0.5. This means that a prediction is considered accurate (i.e., judged as a true positive) only when the IoU score between the predicted bounding box and the ground truth bounding box is not less than 0.5. The setting of the IoU threshold directly affects the calculation of AP, and mAP50 is a widely adopted evaluation standard because it effectively balances the trade-off between the accuracy of the bounding box localization and the detection criteria. The formulas for P, R, and mAP are as follows:

$\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}$ (6)

$\text{Pecall}=\frac{\text{TP}}{\text{TP}+\text{FN}}$ (7)

$\text{mAP}=\frac{{\displaystyle \underset{i=1}{\overset{k}{\sum }}{\text{AP}}_i}}{k}$ (8)

TP stands for the number of positive samples that are predicted as positive, FP stands for the number of negative samples that are predicted as positive, and FN stands for the number of positive samples that are predicted as negative.

mAP represents the sum of the average precision values for all classes divided by the number of classes.

Here is the translation of your last sentence with the correct context:

mAP represents the sum of the average precision values for all classes, normalized by the number of classes.

The results in Table 2 show that, when compared with the classic algorithms of the YOLO series through the improved RT-DETR, the enhanced algorithm demonstrates improvements in terms of precision (P), recall®, and mAP50. This indicates that RT-DETR, as a detection algorithm, can perform well in the field of forest fire prevention.

As shown in Table 3 , The ablation study validates the performance improvement of the model with the addition of different modules. Firstly, by employing the lightweight module MobileNetv4 and through meticulous design of the network

architecture, such as the judicious selection of layer width (number of channels), stride, and layer stacking methods, it is possible to reduce the complexity of the model while maintaining accuracy. MobileNetV4 utilizes an improved inverted residual structure, which first increases the number of channels with pointwise convolutions, then applies depthwise convolutions, and finally uses pointwise convolutions to reduce the number of channels. This design aids in enhancing the network’s representational capacity while maintaining its lightweight nature. Through the detailed design of the network architecture, the model’s parameters and GFLOPS are reduced by approximately two-thirds. By employing Learnable Positional Encoding (LPE), positional encoding becomes a learnable form, allowing for specific adjustments of the parameters based on the dataset, thereby providing a more accurate description of the positional relationships between targets. This is beneficial for feature extraction, leading to an increase in both the R and mAP50 metrics by about 2 percentage points. Finally, the Hierarchical Local-Global Module (HLM) is added. This module further enhances the fusion and connection between high-level semantic features and low-level detail features, leading to a more comprehensive expression of target features. As a result, the model shows an almost one-percentage-point increase in both the P and mAP50 metrics. This confirms the superior performance of the improved model over the original model in the domain of forest fire detection.

It can be clearly observed from Figure 5 that the optimized RT-DETR model excels in extracting flame features and fusing flame features at different scales, thereby significantly improving the recall rate and enhancing the precision of the model’s detection. Particularly in the detection of small target flames, its detection capability has been significantly enhanced.