Early LPR systems relied on conventional object detectors such as Faster R-CNN and SSD, which assume axis-aligned bounding boxes. However, this assumption is often violated in real-world scenarios where license plates appear tilted, rotated, or partially occluded due to perspective distortion. To overcome this limitation, oriented bounding box (OBB) detection has been introduced to better capture the spatial orientation of plates.

HPR-Net [16] proposed a two-stage framework that combines YOLOv5-OBB for oriented detection with a Transformer-based recognizer for character sequence decoding. Similarly, recent YOLOv8-OBB variants [6] enhance angular accuracy through direct angle regression and hybrid loss functions, enabling precise localization of plates under complex viewpoints. These improvements significantly boost the performance of downstream recognition modules by providing more accurate region proposals.

Despite their effectiveness in handling geometric distortions, these methods are typically designed for specific regions and lack mechanisms for cross-domain generalization or multilingual support. This limitation motivates the development of more flexible and adaptive architectures, such as the one proposed in this work.

The second stage of LPR focuses on recognizing characters from detected plate regions, often under challenging conditions such as motion blur, low resolution, contrast variations, or partial occlusion. Early approaches were dominated by Convolutional Recurrent Neural Networks (CRNN) [17] , which combine CNN-based feature extractors with RNNs and Connectionist Temporal Classification (CTC) for sequence prediction.

More recent methods have incorporated attention mechanisms to improve recognition robustness. For instance, UR-LPR [18] integrates EfficientNet-B4 with the Convolutional Block Attention Module (CBAM) [9] to strengthen spatial feature representation, yielding improved performance under degraded visual conditions. Other works have explored Vision Transformers (ViT) [19] and Swin Transformers [20] , leveraging their global receptive fields and ability to model long-range dependencies in sequential character patterns.

Our approach advances this line of research by employing a Transformer-based recognition module enhanced with attention regularization and constrained decoding based on region-specific syntax rules, enabling higher accuracy and better generalization across diverse plate formats.

Most existing LPR models are trained and evaluated on datasets from specific geographic regions (e.g., CCPD [13] , AOLP, UFPR), assuming consistent environmental and design characteristics. However, real-world deployment requires generalization across varying conditions-including lighting, weather, camera angles, plate colors, fonts, and regional formatting rules-which introduces significant domain shifts.

To mitigate such discrepancies, domain adaptation techniques have been explored in related vision tasks. Methods such as Maximum Mean Discrepancy (MMD) [21] , adversarial domain adaptation (e.g., ADDA [22] ), and test-time adaptation (e.g., TENT [23] ) have shown effectiveness in aligning feature distributions across domains. However, their application in LPR remains limited.

While UR-LPR demonstrates implicit robustness through end-to-end training, it does not explicitly address domain shift. In contrast, our work introduces an explicit, lightweight domain adaptation module that combines MMD, adversarial training, and test-time fine-tuning to dynamically adapt to unseen environments, improving model robustness without requiring extensive retraining.

In global deployments, LPR systems must handle license plates written in multiple scripts-such as Arabic, Cyrillic, Latin, and Thai-as well as non-standard formats including vertical, double-line, or irregular layouts. However, the majority of current systems are restricted to Latin-character plates, limiting their applicability in multilingual regions.

Some efforts have aimed at expanding character vocabularies to support mixed-script recognition [24] , but comprehensive multilingual LPR remains an open challenge. Additionally, real-world images are often degraded by adverse weather, motion blur, low illumination, or compression artifacts, further complicating recognition.

To address these issues, synthetic data augmentation techniques-such as simulating rain, fog, JPEG compression, or noise injection-have been employed to improve model robustness [25] .

Our approach addresses both challenges by introducing a syntax-guided, vocabulary-aware decoding mechanism that adapts to country-specific plate structures and supports cross-lingual recognition. Combined with synthetic augmentation and attention-enhanced feature learning, our framework achieves reliable performance even under low-quality imaging conditions and diverse linguistic contexts.

In license plate recognition (LPR) systems, detection is a critical step often compromised by non-ideal visual conditions. In real-world environments, plates may appear tilted, partially occluded, distorted, or captured at oblique angles. Traditional detectors based on axis-aligned bounding boxes generally fail to accurately locate these distorted objects. To overcome these limitations, our method relies on an oriented extension of the YOLOv8 detector, where each plate instance is represented by an oriented bounding box (OBB) enhanced by an angle regression branch.

Each predicted box is parameterized by five elements:

$B=\left(x_c,y_c,w,h,\theta \right)$

where $\left(x_c,y_c\right)$ are the coordinates of the box center, $w$ and $h$ are its width and height, respectively, and $\theta$ is the orientation angle (in radians) defined in the interval $\left[-\pi ,\pi \right]$. This formalism allows for a more geometrically faithful representation of tilted or distorted plates, significantly improving subsequent recognition performance.

The base architecture relies on YOLOv8 for its execution speed and ability to extract efficient visual representations. We extend the standard prediction head to incorporate a dedicated channel for angle $\theta$, in addition to the usual outputs related to localization and classification. This channel is trained using an angular loss function tailored for periodic variables. Specifically, the loss

$L_{\theta }=\text{min}\left({\left(\hat{\theta }-\theta \right)}^2,{\left(2\pi -\left|\hat{\theta }-\theta \right|\right)}^2\right)$

avoids discontinuities during learning at the boundary −π/+π.

The overall training of the detector is guided by a hybrid cost function combining three main components:

$L_{\text{det}}={\lambda }_{\text{cls}}\cdot L_{\text{cls}}+{\lambda }_{\text{IoU}}\cdot L_{\text{IoU}}+{\lambda }_{\theta }\cdot L_{\theta }$

where $L_{\text{cls}}$ is the classification loss (binary cross-entropy), $L_{\text{IoU}}$ is the localization loss based on intersection over union (IoU), and $L_{\theta }$ is the angular regression loss. The coefficients ${\lambda }_{\text{cls}},{\lambda }_{\text{IoU}},{\lambda }_{\theta }$ allow for the weighting of these terms for balanced learning.

To enhance the robustness of the detector to geometric and photometric variations, we have integrated a targeted data augmentation strategy during training. This includes random affine transformations (rotations from 0˚ to 180˚), partial cutouts, Gaussian blur, lighting alterations, and the use of a synthetic rendering engine (Unreal Engine) to generate plates in rare or complex positions. This approach enables the model to better generalize to out-of-distribution scenarios.

During inference, the model outputs are projected as oriented polygons, calculated from $\left(x_c,y_c,w,h,\theta \right)$. A geometric transformation module then extracts a rectified region (aligned crop) through rotation, which is passed to the recognition module. This explicit separation between oriented detection and recognition allows for better modularity while optimizing the overall pipeline accuracy.

This initial step ensures robust localization of plates under varied conditions while preparing a clean encoding for the recognition stage. The choice of YOLOv8-OBB, combined with direct angle regression and multi-component supervision, constitutes an effective foundation for complex visual contexts.

Once the plate is detected and cropped (rectified) by the oriented detection module, the next task is to recognize the sequence of characters it contains. This step is complicated by many factors: motion blur, low resolution, uneven lighting, partial occlusion, and especially linguistic variability (different alphabets, national formats, etc.). To address this, our architecture relies on a two-stage recognition module combining a visual Transformer encoder, enhanced by adaptive spatial attention (CBAM), and sequential decoding under syntactic constraints based on the plate formats specific to each country or region. Figure 1 provides an overview of the complete pipeline. The overall architecture of the Transformer-based recognition module is illustrated in Figure 2 .

The visual encoder uses a Vision Transformer (ViT) architecture. Unlike traditional CNNs or CRNNs, Transformers can model long-range dependencies between different parts of the image, which is crucial for correctly interpreting distorted or partially visible characters.

The rectified plate image, of size $I\in {ℝ}^{H\times W\times 3}$, is first divided into small non-overlapping patches (e.g., 16 × 16 pixels), each projected linearly into a vector space of dimension $d$ by an embedding layer E. To this sequence of vectors, we add a positional encoding $E_{\text{pos}}$ to preserve the spatial structure of the image:

$Z_0=\left[x_{\text{cls}};x_{p_1}E;x_{p_2}E;\cdots ;x_{p_N}E\right]+E_{\text{pos}}$

where $x_{\text{cls}}$ is a classification token (optional), $x_{p_i}$ represents the $i$-th patch, and $N=\frac{HW}{P^2}$ is the total number of patches.

To increase the encoder’s robustness against perturbations (blur, noise, low contrast), we integrate adaptive spatial attention through CBAM (Convolutional Block Attention Module). This module dynamically refines feature responses by focusing on the most informative regions and channels. In practice, CBAM is inserted after the patch embedding step or between Transformer blocks to guide attention towards relevant characters.

The goal of the decoder is to produce an ordered sequence of characters $\left(y_1,y_2,\cdots ,y_T\right)$ corresponding to the content of the plate. Instead of using naive decoding, our method leverages a constrained dictionary $D_c$ specifying syntactically valid character combinations based on the country or region.

Decoding is performed using an autoregressive Transformer decoder, which generates one character at a time, conditioning each prediction on the previous output and the representations extracted by the encoder. At each step $t$, the prediction is defined as:

$y_t=\arg \underset{c\in A_c\left(t\right)}{\max }P\left(y_t=c|y_{<t},Z\right)$

where $A_c\left(t\right)\subset D_c$ is the restricted set of valid characters at step $t$, and $Z$ is the contextual representation of the image from the encoder. This constraint injects prior knowledge about formats (e.g., 2 letters followed by 4 digits), reducing common semantic or syntactic errors in CTC-type methods.

The total vocabulary used consists of the union of several alphabets: Latin, Cyrillic, Arabic, Thai, etc., as well as common digits and symbols. A dynamic masking is applied to the vocabulary at each decoding step to respect the rules defined in $D_c$.

The syntactic constraints for license plate formats are explicitly encoded as deterministic rules rather than learned patterns. For each supported country/region, we have manually curated a formal specification of valid plate formats based on official government documentation. These specifications define the exact sequence of character types (letters, digits, special symbols), regional codes, and positional constraints. During inference, the function GetCountrySyntax (Pi) (Algorithm 0.0.6, line 7) dynamically selects the appropriate constraint dictionary based on either: (a) explicit country identification from the input image, or (b) the domain adaptation module’s prediction of the most probable region. The constrained decoding mechanism then applies these rules through a dynamic vocabulary masking approach, where at each decoding step t, only characters compliant with the current position’s syntax rules remain unmasked in the prediction space. This implementation approach was chosen deliberately over a learned syntax model for three key reasons: (1) license plate formats are strictly defined by governmental authorities with zero tolerance for variation; (2) manual encoding ensures 100% compliance with official standards; and (3) it eliminates the need for extensive region-specific training data that would be required for a learned approach. The constrained dictionary Dc is implemented as a finite-state machine where each state represents a valid position in the plate sequence, and transitions are permitted only to characters that satisfy the regional syntax rules.

The encoder-decoder pair is trained end-to-end, with a cross-entropy loss between the predicted sequence and the target sequence:

$L_{\text{rec}}=-\underset{t=1}{\overset{T}{{\displaystyle \sum }}}\log P\left(y_t^{\text{true}}|y_{<t}^{\text{true}},Z\right)$

This loss is combined with an attention regularization (if activated), which encourages the model to focus its attention weights on the expected textual areas.

Teacher forcing, character masking, and attention dropout strategies are also employed to stabilize learning and prevent overfitting.

License plate recognition (LPR) systems typically suffer from a strong domain bias: when trained on a specific dataset (e.g., CCPD, UFPR), their performance drops drastically on data from other geographical or environmental contexts. To address this out-of-distribution fragility, we propose a lightweight yet effective domain adaptation module based on three strategies: (i) distribution alignment via Maximum Mean Discrepancy (MMD), (ii) adaptation through adversarial learning, and (iii) self-supervised fine-tuning during testing.

MMD allows reducing the statistical divergence between the source domain $\mathcal{S}$ and the target domain $\mathcal{T}$. Let $\varphi \left(x\right)$ be a projection function into a Hilbert space (RKHS), the loss is defined as:

${ℒ}_{\text{MMD}}={\Vert \frac{1}{n_s}{\displaystyle \underset{i=1}{\overset{n_s}{\sum }}\varphi \left(x_i^s\right)}-\frac{1}{n_t}{\displaystyle \underset{j=1}{\overset{n_t}{\sum }}\varphi \left(x_j^t\right)}\Vert }^2$ (1)

This function is added to the training phase of the backbone to encourage convergence towards visually and statistically coherent representations.

We also employ an adversarial strategy to learn domain-invariant representations. A discriminator $D$ is trained to distinguish between the source and target domains while the encoder $G$ learns to make them indistinguishable. This leads to the following loss game:

${ℒ}_D=-{\mathbb{E}}_{x\in \mathcal{S}}\log D\left(G\left(x\right)\right)-{\mathbb{E}}_{x\in \mathcal{T}}\log \left(1-D\left(G\left(x\right)\right)\right)$ (2)

${ℒ}_{\text{adv}}=-{ℒ}_D$ (3)

A gradient reversal mechanism is used to train the encoder jointly with the discriminator.

To adapt to gradual distribution drifts, we propose a test-time adaptation strategy without explicit supervision. It combines entropy minimization:

${ℒ}_{\text{ent}}={\displaystyle \underset{i=1}{\overset{N}{\sum }}H\left(P\left(y_i|x_i\right)\right)}=-{\displaystyle \underset{i=1}{\overset{N}{\sum }}{\displaystyle \underset{c}{\sum }P\left(y_i=c|x_i\right)\log P\left(y_i=c|x_i\right)}}$ (4)

and temporal coherence regularization:

${ℒ}_{\text{cons}}=\text{KL}\left(P^{\left(t\right)}\parallel P^{\left(t-1\right)}\right)$ (5)

These losses are utilized during testing to refine representations without relying on real labels. The total loss used during training is defined as follows:

${ℒ}_{\text{total}}={ℒ}_{\text{det}}+{ℒ}_{\text{rec}}+{\lambda }_{\text{mmd}}\cdot {ℒ}_{\text{MMD}}+{\lambda }_{\text{adv}}\cdot {ℒ}_{\text{adv}}+{\lambda }_{\text{tt}}\cdot \left({ℒ}_{\text{ent}}+{ℒ}_{\text{cons}}\right)$ (6)

where the terms ${\lambda }_i$ allow for dynamically weighting the importance of each component according to the learning scenario.

To better illustrate the sequential functioning of our architecture, Algorithm 0.0.6 presents the entire inference pipeline proposed for robust multilingual license plate recognition in multi-domain environments. The process is organized into three main stages: (i) oriented plate detection using a YOLOv8 detector modified to produce bounding boxes with explicit orientation, (ii) extraction and recognition of characters using a visual Transformer encoder enriched by spatial attention (CBAM), coupled with a syntactically constrained decoder, and (iii) a domain adaptation module, activated during training or online, which dynamically adjusts representations to compensate for inter-domain divergences. This pseudocode highlights the various functions involved, as well as the mechanisms for conditional adaptive regularization.

Model performance is assessed using multiple relevant metrics. First, Accuracy (ACC) measures the exact match rate of the entire license plate string. Second, Character Accuracy (CA) computes the per-character correct recognition rate.

The Robustness Score (RS) is calculated as a weighted average of character accuracy (CA) across five critical challenging conditions commonly encountered in real-world license plate recognition:

$RS={\displaystyle \underset{i=1}{\overset{5}{\sum }}{w}_i\times C{A}_i}$ (7)

where:

The specific conditions and their corresponding weights are:

The weights were determined through field studies of real-world traffic camera footage across 12 metropolitan areas, reflecting the relative frequency and impact of each condition on recognition performance ( Table 2 ). During evaluation, each test image is processed through all five degradation scenarios, and the character accuracy is computed separately for each condition. The final Robustness Score provides a single metric that holistically represents the system’s resilience across the most common challenging scenarios encountered in practical deployments. This metric complements traditional accuracy metrics by specifically measuring performance under adverse conditions that significantly impact real-world LPR system reliability.

The Adaptation Gain (%) quantifies the relative improvement in recognition performance specifically attributable to the domain adaptation module. It is calculated as follows:

$\text{Adaptation Gain}\left(\text{\%}\right)=\left(\frac{{\text{ACC}}_{\text{adapted}}-{\text{ACC}}_{\text{baseline}}}{{\text{ACC}}_{\text{baseline}}}\right)\times 100$

where:

As shown in our ablation study ( Table 4 ), when comparing our full model against the configuration “No Domain Adaptation,” the calculation would be:

$\text{Adaptation Gain}\left(\text{\%}\right)=\left(\frac{91.5-84.9}{84.9}\right)\times 100=7.8\text{\%}$

The slight discrepancy with the 7.1% reported in Table 3 arises because Table 3 presents results averaged across multiple cross-domain scenarios, while the ablation study shows performance on a single representative scenario.

This definition ensures that the metric reflects only the improvement due to adaptation mechanisms (MMD, adversarial training, and test-time fine-tuning) rather than the inherent capabilities of the detection or recognition components.

Regarding training specifications, we employ a YOLOv8-OBB-based detector, initialized on COCO and fine-tuned on the CCPD and AOLP datasets. For the encoder, we adopt a ViT-B/16 architecture with a CBAM module integrated into each Transformer block.

The decoder is an autoregressive Transformer that incorporates dynamic syntactic constraints. Optimization is performed using AdamW with an initial learning rate of 10⁻⁴ and a cosine annealing learning rate schedule.

We set the batch size to 32 and train for 50 epochs. Furthermore, we incorporate domain adaptation during training using Maximum Mean Discrepancy (MMD) and adversarial learning techniques. Test-time fine-tuning is enabled for cross-domain scenarios to enhance adaptation performance.

We compare our method with several representative LPR systems:

Table 3 reports the overall plate-level accuracy (ACC), character accuracy (CA), robustness score, and adaptation gain under a representative cross-domain scenario. As illustrated in Figure 3 , our approach consistently outperforms existing baselines across all metrics, achieving a substantial boost of +7.1% in adaptation performance compared to UR-LPR Net.

To evaluate the relative importance of each architectural component, we conducted an ablation study by selectively removing one module at a time from our full model. The configurations tested include:

As shown in Table 4 and visualized in Figure 4 , every component contributes meaningfully to performance. Notably, disabling the adaptation module leads to a drop of 6.6% in plate accuracy, and removing the syntax-constrained decoder results in more than 3.7% loss in CA.

We further assess the ability of our system to adapt across domains, particularly when training and testing data differ in style, region, and language. Figure 5 highlights the adaptation gain observed in three transfer scenarios. On average, our adaptation module improves performance by 5.7% across settings, demonstrating its effectiveness in minimizing distribution shifts.

To evaluate the practical feasibility of our framework for real-time deployment on edge devices, we conducted a comprehensive computational efficiency analysis. Table 4 presents the inference speed and resource requirements of our complete system compared to state-of-the-art methods under identical conditions (NVIDIA RTX 3090 GPU, Intel Xeon W-2245 CPU @ 3.90 GHz, 32 GB RAM).

Our complete framework processes license plates at 32.6 FPS on a high-end GPU, which meets the minimum requirement for real-time operation (typically 25-30 FPS for traffic monitoring applications). The slight decrease in speed compared to YOLOv8+CRNN is justified by the significant gains in cross-domain accuracy (91.5% vs 78.4%) and robustness. Notably, our optimized version (with pruned Transformer layers and simplified domain adaptation) achieves 42.3 FPS while maintaining 89.7% accuracy, making it suitable for edge deployment ( Table 5 ).

The computational breakdown by module reveals:

When deployed on embedded systems, our optimized model achieves 9.8 FPS on a Raspberry Pi 4 with Coral TPU acceleration, sufficient for stationary traffic monitoring applications where vehicles move at moderate speeds. For resource-constrained environments, we recommend:

1. Disabling test-time adaptation during inference (reduces latency by 17%)

2. Using quantized models (INT8) for the recognition module (reduces model size by 75%)

3. Implementing selective domain adaptation (only when confidence scores fall below threshold)

These optimizations allow our system to maintain 87.3% accuracy while achieving 15.2 FPS on Jetson Nano platforms, demonstrating its viability for practical deployment in intelligent transportation systems. The modular architecture also enables selective component activation based on available resources, providing flexibility for various deployment scenarios from cloud servers to edge devices.

Our results clearly show that the proposed architecture significantly outperforms competitive baselines, particularly in complex and multilingual cross-domain conditions. Each module contributes to this success: the YOLOv8-OBB detector provides precise alignment even under oblique perspectives, the Transformer decoder combined with constrained syntax effectively reduces invalid predictions, and the domain adaptation module ensures generalization to new regions without requiring manual re-training.

The ablation study confirms that no single module is sufficient on its own: removing any component leads to noticeable performance degradation. In addition, the adaptation module proves critical when facing distributional drift, a common phenomenon in real-world surveillance applications. By integrating these components into a unified and modular framework, we enable robust, multilingual, and adaptive license plate recognition suitable for deployment at global scale.