Communication is a fundamental process in human interaction, enabling the exchange of ideas, information and emotions [2]. For the deaf community, often deprived of hearing or speech, non-verbal communication, and more specifically sign language, becomes the main vector of interaction.

In Niger, disabled people represent 4.2% of the population, approximately 715,497 individuals, among whom there are nearly 24,000 deaf people [3]. The latter use Niger Sign Language (NiSL) to communicate. Sign language is a natural language in its own right, which is based on a visual-gestural modality involving the hands, facial expressions and body movements [4]. However, in an environment where the spoken language predominates, this community faces social isolation which hinders its integration.

Technological advances, particularly in artificial intelligence (AI), offer promising solutions to overcome this isolation. Systems based on digital gloves (data gloves) [5] or computer vision [6] have been developed for other sign languages. However, NiSL remains underrepresented in academic work and technological applications, suffering from a critical lack of digital resources.

This work aims to address this challenge by developing a system capable of recognizing NiSL gestures at word level in real time and translating them into text, using a simple webcam.

To design this translation system, we adopted a multi-step methodology, ranging from the linguistic study of the NiSL to the validation of the AI model.

The heart of this system is a Deep Learning model capable of processing sequential and multimodal data.

For the evaluation, we didn’t use part of these 270 sequences. We did a new, independent real-time recording: each sign was performed 10 times, making 90 test trials, with no overlap with the training data. So, the reported performance is calculated exclusively on these 90 independent trials. We trained and tested the model in four different configurations to assess the relative importance of each modality (hands, face, pose). The model was tested in real time via a webcam, performing 10 attempts for each of the 9 signs.

The average recognition rates obtained are as follows: (see Table 1, Figure 6 and Figure 7)

Hands only: 62.22%Hands + Face: 27.77%Hands + Pose: 35.55%Hands + Face + Pose: 37.77%

Table 1. Overall recognition rate based on features used.

Figure 6. Comparison of the overall accuracy rates of the four configurations.

Figure 7. Accuracy by sign according to feature configuration.

These results suggest that using only hand features achieves the best recognition rate. This score reflects the central role of manual gestures, which convey the majority of meaning in sign language. The lower complexity of this configuration (fewer key points to process) allows the model to efficiently focus on the most relevant information.

On the other hand, adding facial features significantly reduces accuracy. This can be explained by the high variability of facial expressions and the large number of extracted key points (468), which may introduce noise and distract the model from more discriminative hand features.

Including pose features slightly improves accuracy compared to using facial features alone, likely because body posture provides useful additional spatial context. However, its contribution remains secondary compared to hand gestures.

Finally, combining all features only marginally improves performance over face and pose alone, while remaining well below the hand-only configuration. The increased data dimensionality and the absence of a mechanism to weight the relative importance of each modality appear to dilute key signals, reducing the overall effectiveness of the model.

Our best result (62.22%) is encouraging and comparable to those reported in the literature for more extensively studied sign languages. For instance, studies conducted on the WLASL dataset for American Sign Language (ASL) reported accuracies of 62.63% [12], although it should be noted that experimental conditions differ significantly in terms of vocabulary size and dataset scale.

It should also be noted that, due to the binary nature of the collected data (correct/incorrect prediction), only accuracy and recall could be computed in this study. Metrics such as precision, F1-score, and confusion matrix require per-prediction class labels, which constitute a limitation of the current evaluation and a direction for future work.

This work has enabled the development of a functional prototype of a real-time translation system from Niger Sign Language to text, using Deep Learning techniques. The main contribution lies in the creation of a first video corpus for NiSL and in the demonstration that an LSTM model, trained on hand movements alone, can achieve a promising performance of 62.22%.

Our results emphasize that hands convey the essence of the message and that the addition of other modalities can, paradoxically, degrade performance due to the complexity and noise added.

To improve this system, future perspectives include the expansion of the corpus. The integration of attention mechanisms, such as those proposed in Transformers models [13], could also help to better prioritize information from different modalities. Ultimately, such a system could greatly promote the social and professional inclusion of deaf people in Niger.

Conceptualization, Moussa Idi Bachir and Bouwey Alley Ibrahim; methodology, Moussa Idi Bachir; software, Bouwey Alley Ibrahim; validation, Naroua Harouna, Kadri Chaibou, and Moussa Idi Bachir; resources, Bouwey Alley Ibrahim; data curation, Bouwey Alley Ibrahim; writing—original draft preparation, Moussa Idi Bachir and Kadri Chaibou; writing—review and editing, Kadri Chaibou and Morou Ganda Yahaya; supervision, Naroua Harouna.