The proposed study presents an HGR model on Vision Transformer (ViT) architecture with a transfer learning modality. The general workflow involves gait frame detection on video sequences, image processing, dataset division, and transformer-based feature detection and classification. The complete architecture of the proposed system is illustrated in Figure 1, which outlines the sequential processing pipeline from data acquisition to final recognition.

CASIA-B gait dataset, which is publicly available, was used to test the proposed framework. The data set consists of gait sequences recorded on 124 participants in controlled conditions in an indoor environment at 11 viewing angles (0˚ to 180˚ with 18˚ intervals). All subjects completed ten walking sequences of which six of them were normal walking sequences (NM), two walking sequences with a carried bag (BG), and two walking sequences with a coat (CL), which were all common covariate conditions that influenced gait recognition. Video-to-image extraction process was used to convert video sequences into image frames. Six frames of every video sequence were uniformly sampled to ensure balance in representation and computational efficiency. The extracted images were all resized at 256 × 256 pixels in three colour channels. The resulting processed data was then further subdivided into training and testing groups so that the performance can be assessed impartially.

Figure 1.The architecture of the proposed HGR method.

The suggested framework employs Vision Transformer models to use self-attention mechanisms to learn the global representation of features. In contrast to traditional convolutional neural networks, where local spatial patterns are addressed, ViTs subdivide images into fixed patch sizes and operate on the sequences of the tokens, which allows to effectively model long-range effects. This study adopted two variants of transformers that were pre-trained.

ViT-B/16 and ViT-L/32, where the numbers indicate patch sizes used for image segmentation. Let the input image be represented as

where L , C , and O denote image height, width, and number of channels, respectively. The image is partitioned into non-overlapping patches, which are flattened to form patch vectors

where p represents the patch resolution and N denotes the total number of patches, computed as:

This patch-based representation significantly reduces computational complexity while preserving spatial information.

Each flattened patch was linearly projected into an embedding space using a learnable projection matrix. To retain spatial relationships between patches, positional embeddings were added to the patch embeddings. A learnable classification token was appended to the sequence to enable global representation learning for classification.

The embedded token sequence was formulated as:

where E denotes the embedding matrix and x c l a s s represents the class token. This combined sequence served as input to the transformer encoder.

The transformer encoder was composed of several layers, each with a multi-head self-attention (MSA) module then a multi-layer perceptron (MLP) as in Figure 2. Residual connections and layer normalisation were added in order to enhance the stability of training and prevent the problem of vanishing gradients.

Figure 2. The architecture of the transformer encoder.

For the l t h encoder layer, the computations were expressed as:

where LN denotes layer normalisation.

The MSA module performs parallel self-attention operations across multiple heads to capture diverse feature relationships. For each head, the input embeddings were projected to query ( q ), key ( k ), and value ( v ) matrices:

where U q , U k , and U v are learnable weight matrices.

The attention weights were computed using the scaled dot-product mechanism:

The self-attention output was then obtained as:

Outputs from all heads were concatenated and linearly transformed:

as depicted in Figure 3.

Figure 3. The architecture of MSA.

The MLP block comprised two fully connected layers with Gaussian Error Linear Unit (GeLU) activation, as shown in Figure 4. GeLU assigns weights based on both magnitude and sign, enabling improved nonlinear representation learning compared to ReLU.

The MLP output was expressed as:

where z n denotes the embedding of the n t h patch.

The classifier head was replaced with a single linear layer with classes outputs followed by softmax with Cross-Entropy loss, ensuring compatibility with single-label multi-class recognition.

Model optimisation was performed using the Adam optimizer, and categorical cross-entropy loss was employed:

where y t represents the ground truth label, y ^ t denotes the predicted probability, and T is the number of samples.

Both ViT-B/16 and ViT-L/32 models were fine-tuned using learning rates of 0.001 and 0.0001, a batch size of 16, and 200 training epochs.

Experiments were conducted using four selected viewing angles of the CASIA-B dataset (0˚, 18˚, 36˚, and 54˚) to assess performance. Model performance was evaluated using training accuracy, testing accuracy, training loss, and testing loss.

The models were evaluated under three experimental setups: a 70:30 data split, a frontal view analysis, and a cross-view analysis. In the 70:30 configuration, the NM, BG, and CL classes of the CASIA-B dataset were included. Each class contained sequences from all subjects. This experiment was designed to examine how well the models handled variations in appearance.

A second set of experiments focused on subject recognition under different appearance conditions. Three evaluations were carried out: gallery versus NM, gallery versus CL, and gallery versus BG. The gallery set consisted of sequences NM-01 to NM-04, which were used for training. The corresponding test sets were NM-05 to NM-06, CL-01 to CL-02, and BG-01 to BG-02. These sequences allowed the system to be analyzed under normal walking, clothing variations, and carrying conditions.

The third experimental setup was intended to measure cross-view performance. The model was trained using sequences captured at the 000˚ view. It was then tested using sequences captured at the 180˚ view. This configuration introduced a significant change in viewpoint, providing insight into how effectively the model generalized to unseen angles. All experiments were implemented in a Jupyter Notebook environment on a system equipped with an Intel Core i5 (9th generation) processor, 32 GB RAM, and an NVIDIA RTX 2060 GPU.

The proposed Vision Transformer-based HGR framework was evaluated using four viewing angles of the CASIA-B dataset, namely 0˚, 18˚, 36˚, and 54˚. All experiments were implemented in a Jupyter Notebook environment using an Intel Core i5 (9th generation) processor, 32 GB RAM, 1.5 TB SSD storage, and an NVIDIA RTX 2060 GPU with 8 GB memory. Two pre-trained transformer models, ViT-B/16 and ViT-L/32, were fine-tuned using a batch size of 16 for 200 epochs. Learning rates of 0.001 and 0.0001 were examined to assess model sensitivity to optimisation settings.

Model performance was evaluated using training accuracy (Trn-ACR), testing accuracy (Tst-ACR), training loss (Trn-LS), and testing loss (Tst-LS).

The ViT-B/16 model was first evaluated using a learning rate of 0.001 across the selected CASIA-B viewing angles. At 0˚, the model achieved a training accuracy of 93.40% and a testing accuracy of 91.47%, with corresponding training and testing losses of 17.59% and 22.01%, respectively. At 18˚, training and testing accuracies decreased slightly to 89.91% and 86.80%, with losses of 26.82% and 34.27%. For the 36˚ angle, the model obtained a training accuracy of 89.22% and a testing accuracy of 86.28%, while training and testing losses were recorded as 28.15% and 34.56%. At 54˚, the performance further declined, with training and testing accuracies of 88.95% and 83.43%, and losses of 28.44% and 40.50%.

The detailed numerical results are presented in Table 1, while the corresponding training and validation curves are illustrated in Figure 4.

Table 1.Results for the ViT-B/16 model using CASIA-B at a learning rate of 0.001.

When the learning rate was reduced to 0.0001, the ViT-B/16 model exhibited a noticeable decrease in recognition performance. At 0˚, the training accuracy was 89.98% and the testing accuracy was 87.86%, with losses of 29.34% and 32.10%, respectively.

Figure 4.Training curves for the ViT-B/16 model using CASIA-B at a learning rate of 0.001.

At 18˚, performance dropped to 82.93% training accuracy and 81.15% testing accuracy. For 36˚, the model attained 81.20% training accuracy and 79.33% testing accuracy, with increased losses exceeding 50%. At 54˚, training and testing accuracies were recorded as 82.60% and 78.84%, respectively.

The complete results are summarised in Table 2, with training behaviour illustrated in Figure 5.

Table 2.Results for the ViT-B/16 model using CASIA-B at a learning rate of 0.0001.

For the frontal-view analysis using the 000˚ angle of the CASIA-B dataset, four sets were used in the experiments: the gallery view, NM, BG, and CL. The gallery view consisted of four sequences (NM-01 to NM-04), while NM-05 to NM-06, BG-01 to BG-02, and CL-01 to CL-02 were used as the evaluation sets. Each set contained gait sequences from all 124 subjects. The objective of this experiment was to investigate the model’s ability to handle variations in appearance.

Figure 5.Training curves for the ViT-B/16 model using CASIA-B at a learning rate of 0.0001.

Using the gallery versus NM configuration, ViT-B/16 achieved a training accuracy of 100%, a test accuracy of 98.51%, a training loss of 0.00, and a test loss of 1.10. Under the gallery vs. CL setting, the model obtained training and test accuracies of 100% and 33.20%, respectively, with corresponding losses of 0.00 and 7.02. In the gallery versus BG evaluation, the training accuracy, test accuracy, training loss, and test loss were 100%, 98.43%, 0.00, and 2.10, respectively.

These results indicate that ViT-B/16 performs strongly on the NM and BG sets but exhibits a notable decline in performance on the CL set. The detailed results are provided in Table 3.

Table 3.Performance of ViT-B/16 for frontal view analysis using 00˚ angle of CASIA-B.

The training curves for all experiments using frontal view on ViT-B/16 are illustrated in Figure 6.

For the cross-view analysis of ViT-B/16, 70% of the gait sequences captured at the 000˚ view of the CASIA-B dataset were used for training, whereas 30% of the sequences captured at the 180˚ view were used for testing. The evaluation employed four metrics: training accuracy, test accuracy, training loss, and test loss. The model achieved a training accuracy of 100%, a test accuracy of 38.95%, a training loss of 0.00, and a test loss of 11.1123. The corresponding training curve for the cross-view evaluation using ViT-B/16 is presented in Figure 7.

Figure 6.Training curves of ViT-B/16 for frontal view analysis using 00˚ angle of CASIA-B.

Figure 7.Training curves of ViT-B/16 for cross view analysis.

The ViT-L/32 model demonstrated improved recognition performance compared with ViT-B/16 when trained using a learning rate of 0.001. At 0˚, the model achieved a training accuracy of 96.33% and a testing accuracy of 94.02%, with reduced losses of 10.98% and 16.29%, respectively.

At 18˚, training and testing accuracies were 90.96% and 87.64%. At 36˚, the model obtained 89.94% training accuracy and 86.69% testing accuracy. At 54˚, performance decreased to 89.38% training accuracy and 83.12% testing accuracy.

The quantitative outcomes are shown in Table 4, and the training trends are illustrated in Figure 8.

Table 4.Results for the ViT-L/32 model using CASIA-B at a learning rate of 0.001.

At the lower learning rate of 0.0001, ViT-L/32 achieved slightly reduced performance relative to its higher learning rate configuration. At 0˚, training and testing accuracies were 92.25% and 90.75%, respectively. Similar performance was observed at 18˚, with a testing accuracy of 90.69%.

Figure 8.Training curves for the ViT-L/32 model using CASIA-B at a learning rate of 0.001.

However, at higher viewing angles, performance decreased to 83.52% at 36˚ and 80.23% at 54˚. The complete results are provided in Table 5, while the training curves are depicted in Figure 9.

Table 5.Results for the ViT-L/32 model using CASIA-B at a learning rate of 0.0001.

Figure 9.Training curves for the ViT-L/32 model using CASIA-B at a learning rate of 0.0001.

The frontal-view analysis using the 000˚ angle of the CASIA-B dataset followed the same experimental protocol described previously for ViT-L/32. Under the gallery vs. NM evaluation setting, ViT-L/32 achieved a training accuracy of 100%, a test accuracy of 96.27%, a training loss of 0.00, and a test loss of 2.15. When evaluated under the gallery vs. CL setting, the model obtained training and test accuracies of 100% and 42.09%, respectively, with corresponding training and test losses of 0.00 and 6.23. For the gallery vs. BG setting, the training accuracy, test accuracy, training loss, and test loss were 100%, 89.98%, 0.00, and 8.21, respectively.

Based on these results, it is evident that ViT-L/32 performs strongly on the NM and BG sets, whereas its performance declines considerably on the CL set. The detailed results are presented in Table 6.

Table 6.Results of ViT-L/32 for frontal view analysis using 00˚ angle of CASIA-B.

The training curves for the frontal view experiments using ViT-L/32 are depicted in Figure 10.

For the crossview analysis of ViTL/32, 70% of the gait sequences captured at the 000˚ view of the CASIAB dataset were used for training, while 30% of the sequences captured at the 180˚ view were used for testing. The evaluation considered four metrics: training accuracy, test accuracy, training loss, and test loss. The model achieved a training accuracy of 100%, a test accuracy of 38.84%, a training loss of 0.00, and a test loss of 12.70. The training curves corresponding to this crossview evaluation using ViT-L/32 are presented in Figure 11.

Table 7 is a comparative analysis of the suggested transformer-based framework and the latest HGR methods. The GaitSTAR approach recorded a 76.50% average recognition accuracy at selected angles, and spatial-temporal feature optimisation designs reached around 81.68%. View alignment methods based on GEI achieved average accuracy of less than 80%. The proposed ViT-L/32 model with an average testing accuracy of 87.87 and ViT-B/16 with 86.99 model showed better performance in all tested angles as well.

Figure 10.Training curves of ViT-L/32 for frontal view analysis using 00˚ angle of CASIA-B.

Figure 11.Training curves of ViT-L/32 for cross view analysis.

Table 7.Comparison of the proposed method with recent HGR techniques.

The computational complexity of both models, ViTB/16 and ViTL/32, was also evaluated, as summarized in Table 8. The two architectures exhibit distinct parameter counts and FLOP requirements, with ViTB/16 requiring approximately 44 GFLOPs and ViTL/32 requiring approximately 55 - 60 GFLOPs. This substantial disparity in computational cost arises primarily from differences in patch size, the number of transformer layers, and the dimensionality of the hidden representations. In comparison, ViTB/16 demonstrated greater efficiency, owing to its lower FLOP count and reduced inference latency compared to ViTL/32.

Table 8.Computational complexity of ViT-B/16 and ViT-L/32.

Models trained at a learning rate of 0.001 performed better than models trained at 0.0001 across all viewing angles, suggesting moderately higher learning rate enabled improved convergence and adaptation to features during fine-tuning. As illustrated in Table 1 and Figure 1, the ViT-B/16 model recorded its best test accuracy (Tst-ACR) at 0˚ of 91.47, where the poor performance reached 83.43 at 54˚. This apparent negative relationship with the horizontal angle reflects the expected difficulty in recognizing cross-view gait since the silhouettes vary dramatically in appearance and motion projection angles.

As presented in Table 2 and illustrated again in Figure 2, the ViTB/16 model trained with a 0.0001 learning rate showed a noticeable performance drop. This decline was most evident at the 36˚ and 54˚ viewpoints, where the training accuracy fell below 80%. This underperformance emphasises the importance of selecting appropriate hyperparameters in the application of transformer-based models, and they are also subject to optimisation schedules and data distribution.

Conversely, the ViT-L/32 model exhibited better overall feature discrimination with the highest Tst-ACR of 94.02% at 0˚ with 0.001-learning rate (Table 3, Figure 3). ViT-L/32 achieved accuracies of over 86% at even more challenging angles (18˚ and 36˚), much higher than the calculated analogs of ViT-B/16. This trend demonstrates the benefit of increased patch sizes and increased transformer layers to capture global gait dynamics and supports related literature indicating the usefulness of transformer attention processes in learning long-range spatial dependencies in vision tasks in intuition [18][19]. At the reduced learning rate of 0.0001 (Table 4, Figure 4), ViT-L/32 continued to achieve fairly high performance with Tst-ACRs exceeding 90%.at 0˚ and 18˚.

However, recognition at higher angles (36˚ and 54˚) again declined, illustrating that viewpoint variation remains a fundamental challenge for gait recognition systems even when using advanced architectures. The models were further assessed under frontalview appearance conditions, and the findings demonstrated that variations in clothing exerted a substantial influence on performance. Moreover, when evaluating view variation, a pronounced degradation in accuracy was observed, indicating a significant sensitivity of the models to changes in viewpoint.

This trend echoes the broader gait recognition literature, where cross-view variation is acknowledged as one of the most persistent sources of error in both appearance-based and model-based frameworks [20].

When compared with recent HGR methods (Table 7), the proposed transformer models achieve superior recognition accuracy across all evaluated angles. The ViT-L/32 model yielded an average accuracy of 87.87%, surpassing techniques such as spatiotemporal attention networks (e.g., GaitSTAR) and hybrid feature optimisation approaches, which have reported average accuracies in the mid-70s to low-80s for the same evaluation protocol. For example, the GaitSTAR method obtained approximately 76.5% average accuracy across the first four angles of CASIA-B, while conventional optimisation and deep learning frameworks ranged between ~71.6% and ~81.7% in recent literature [21]. These results demonstrate that transformer-based global attention aids in extracting discriminative gait features that are less sensitive to covariate variations compared to traditional CNN-based methods and handcrafted feature approaches.

In particular, methods such as Gait-ViT, which apply Vision Transformers to averaged silhouette templates (GEIs), have reported near-perfect CASIA-B performance in controlled settings (e.g., >99% accuracy) [19][21]. However, such results often stem from tailored preprocessing (e.g., GEI extraction) and evaluation conditions that may not reflect raw sequence inputs. The present study’s direct application of ViT models to resized frames (256 × 256) without silhouette averaging demonstrates competitive performance under raw input conditions, highlighting the flexibility and generalisability of transformer architectures for gait representation learning.

The training curves in all the experimental settings are visualised in Figures 1-4, which indicate that there are similar patterns of convergence during training and validation stages. It is also worth noting that models educated with the learning rate of 0.001 exhibited a smoother and quicker convergence, and the difference between training and testing losses were less, which suggests improved generalisation. Conversely, the 0.0001 learning rate curves exhibited larger loss gaps and slower rates of convergence, especially ViT-B/16, which reflect the underfitting behavior of more complex visual patterns. Such findings are consistent with the general knowledge on transformer optimisation in vision tasks, where the correct learning rate schedules and patch representations have a strong impact on representation consistency and prediction. Recent studies on gait transformer also propose that patch designs and positional encoding algorithms may have significant impact on recognition performance especially where conditions of cross-view and temporal variation are involved variability [13][22].

Although the performance is high, the findings validate that viewpoint variation is a core concern of gait recognition, despite the advanced attention-based models. Although ViT-L/32 alleviates these effects to some extent, recognition accuracy declined by approximately 10% across frontal and oblique views, which suggests that alternative strategies, including multi-view fusion, temporal augmentation, or hierarchical attention, can further strengthen them. Future directions could involve studies on self-supervised pretraining with large uncontrolled gait datasets (e.g., GREW) to improve functional generalisation and test performance in unconstrained real-world scenarios.