Toward Real-Time Sentence-Level Sign Language Translation

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

BLEU-4 is the standard metric for evaluating sign language translation (SLT), but spoken-language metrics may not adequately reflect sign language proficiency. The multimodal, low-resource context of SLT allows models to exploit spurious correlations and spoken-language priors, rather than learning stronger sign representations. In this paper, we evaluate the relationship between spatio-temporal understanding and BLEU-4 across six SLT models on Phoenix-2014T and CSL-Daily, showing that gains in BLEU-4 are not on their own evidence of better sign language understanding. This work introduces an alternative inspired by language-learning assessment, using an open-weight-LLM QA protocol that measures salient content preservation. It aligns more closely with human rankings and is six to seven times more paraphrase-invariant than BLEU-4. Applied to SLT, this protocol targets content transfer, is more robust to train-test overlap, and gives a different picture of the field: the five gloss-free systems are largely within noise of one another on Phoenix-2014T, while the gloss-supervised system stands 9.3 points higher, a gap invisible to BLEU-4.

A Gloss-driven Indian Sign Language Production System Using Learned Pose Representations

Automated Multilingual Translator Using Neural Translation

The results indicate that a moderately sized, shared self-attention architecture can deliver production-quality multilin-gual translation within the resource constraints of an academic de-ployment, while surfacing clear directions – low-resource language coverage, domain adaptation, and speech-based extension – for con-tinued development.

TransSLR: A Lightweight Transformer for Sign Language Recognition

Automated Sign Language Recognition for under-represented languages remains a largely unsolved problem. Central African Sign Language (CASL) exemplifies this gap: the only available bench-mark, CASL-W60, has a best reported accuracy of 69.93%, and we show that the common heuristic of fine-tuning high-resource models fails to close it. This failure stems from two compounding factors: the limited scale of available CASL data and the significant lexical and visual domain gap between CASL and large-scale corpora such as WLASL, which renders pre-trained representations largely uninformative. To address this, we propose TransSLR, a lightweight Temporal Transformer Encoder trained from scratch on 64-frame normalized pose sequences, with average pooling and a classification head. By operating on geometric keypoint representations rather than raw RGB, TransSLR achieves signer-independent generalization without relying on visual appearance. On the CASL-W60 benchmark, TransSLR establishes a new state-of-the-art accuracy of 80.39%, surpassing the prior best by +10.46%. Beyond accuracy, our encoder-only design significantly reduces computational overhead, making deployment feasible in resource-constrained environments. We conduct extensive experiments on the CASL-W60 benchmark, comparing against RGB-based and multimodal baselines, and demonstrate that TransSLR achieves state-of-the-art performance.

End-to-end sentence-level Indian sign language translation with ISH-NEWS dataset and transformer model.

People with hearing impairments use Sign Languages (SLs) to communicate. They find it difficult to communicate with spoken-language users because spoken-language users do not understand SLs. We must encourage tools that allow sign language and spoken language users to communicate with one another. Sign language translation (SLT) attempts to translate sign-language videos into spoken language or vice versa. In India, the development of datasets for the Indian Sign Language (ISL) in India is progressing slowly due to researchers' discrete efforts. Currently, there is no publicly available dataset on ISL to evaluate sentence-level Continuous Sign Language Translation (CSLT) approaches that can be used by a transformer-based model. In the proposed work, we present the first ISL Dataset for CSLT, ISH-NEWS, that contains 4,222 sentence videos of over 6.5K words. We use the transformer-based translation model to evaluate its performance against the ISH-NEWS dataset and establish a baseline for model performance. Using data augmentation techniques, we increased the proposed model's BLEU-4 score by 8.46.

Sign Voice: Real-Time Sign Language Recognition and Speech Translation System

Communication between deaf or hearing-impaired people and the rest of society is still quite a social and technological challenge. The main reason for this is the fact that a very small number of people understand sign language. Although sign language is a very good and expressive way of communicating, the lack of sign language translation tools in real-time is one of the main barriers for inclusive communication in everyday environments. To solve this problem, we have developed SignVoice, a system that recognizes sign language in real-time and converts hands movements into spoken words automatically through computer vision and machine learning. The system we propose exploits MediaPipe to attain the most precise, efficient, and robust hand landmark detection under varied lighting conditions and different backgrounds. A hybrid classification schema has been implemented which allows the system to easily recognize both static and dynamic gestures. Static hand gestures are detected using a K Nearest Neighbors (KNN) classifier, dynamic gestures are analyzed by Long Short-Term Memory (LSTM) networks to understand sequential hand movements based on the change of time. This two-model system increases the recognition accuracy without requiring high computational power. Experimental results demonstrate that the proposed method performs extremely well in terms of recognition accuracy and at the same time demands very little time, thereby establishing the method as highly suitable for assistive communication applications. In summary, SignVoice is a step forward in addressing accessibility issues and promoting social inclusion by enabling communication between hearing-impaired individuals and the rest of the community.