PSL-128: A Multi-Signer Pakistan Sign Language Video Dataset (224x224 preprocessed release)

Abstract

PSL-128 is a multi-signer video dataset for word-level isolated sign recognition in Pakistan Sign Language (PSL), comprising 9,068 clips across 128 sign classes contributed by 68 signers. It is, to the authors' knowledge, the first large-scale, multi-signer, publicly released PSL video dataset for word-level recognition. Every prior PSL resource is image-based, alphabet-only, collected from a single signer, or privately held. This record contains the 224 x 224 preprocessed release. The full-resolution 1920 x 1080 source recordings (approximately 54.7 GB), which support re-extraction of pose landmarks at higher resolution, are released separately on Kaggle at https://www.kaggle.com/datasets/emusman371/psl-128-fhd, organised by signer, so that a single DOI resolves to both. The dataset is described in full in an accompanying data descriptor, and the reference implementation of its baseline is at https://github.com/muhammadusman371/psl128-i3d-r50. A separate benchmark study evaluating a range of architectures on this dataset is reported elsewhere. Access Files in this record are currently restricted while the accompanying papers are under review. The dataset can be obtained on reasonable request, either through the access-request form on this page or from the corresponding author at msaif23m012@pucit.edu.pk. Files will be opened publicly upon acceptance, under the same DOI. Contents Two ZIP archives contain the identical set of 9,068 clips in two directory layouts: by_class.zip groups them into 128 folders named after the sign class, and by_signer.zip regroups the same clips into 68 folders S01 to S68, one per signer. The two layouts are provided so that either evaluation protocol below can be used without regrouping; extract one, not both, into a training set. Alongside these are classes.txt (the 128 class names, sorted, with line order defining class_id), metadata.csv (one row per clip: filename, class, class_id, signer_id, session, resolution), md5sums.txt (checksums for every video in both trees, plus classes.txt and metadata.csv), the two split files video_level.csv and signer_independent.csv, and README.md, which documents the filename convention, parsing caveats, verification procedure and terms of use in full. The trained weights of the I3D ResNet-50 baseline reported in the data descriptor are included as i3d_r50_psl128_video_level_best.pth. A third archive, landmarks_psl.zip, holds pre-extracted MediaPipe landmark sequences, together with landmarks_psl.md5. It mirrors the by_class layout with one compressed NumPy archive per clip, each holding a single float32 array of shape (32, 258) under the key "landmarks": 33 pose landmarks as x, y, z, visibility, then 21 left-hand and 21 right-hand landmarks as x, y, z, with undetected blocks written as zeros and every sequence resampled to a fixed 32 frames. Coordinates are stored as MediaPipe returns them; all normalisation belongs in the training loader. Extraction was run on the 224 x 224 clips rather than the full-resolution source, so appearance-based and skeleton-based users start from the same material. The extraction script, extract_landmarks.py, is included so the step can be re-run at full resolution where better landmarks are needed. Clips are named S _T 224p .mp4, for example S1_T1_224p_listen.mp4. Note that the signer token is not zero-padded in filenames while the by_signer directories are, so prefer signer_id in metadata.csv to parsing filenames. One class name, ramzan(fast), contains parentheses; quote paths when scripting over the class folders in a shell. Dataset properties Clips were recorded on a Google Pixel 5 at 1920 x 1080 and 30 fps, each containing one isolated sign lasting 2 to 4 seconds, and downsampled to 224 x 224 with FFmpeg for this release. The dataset averages 70.8 clips per class, with a minimum of 51 and a maximum of 83, so it is close to balanced. Recording conditions were deliberately varied — indoor plain-wall settings, classrooms and outdoor parks; indoor LED lighting, natural daylight and partial shadow; signers aged roughly 14 to 25 with varied skin tone, clothing and build; both landscape and portrait orientation at varying camera distances — producing high intra-class variability by design. The vocabulary covers everyday objects, transport, professions, animals, emergency terms, weather and calendar vocabulary, Islamic and cultural vocabulary specific to PSL, and a small number of phrase-level items. Collection Collection took place on 20 recording days between September and December 2025 at Government Degree College of Special Education, Johar Town, Lahore, in three phases. In the first, 16 participants recorded 1,670 videos across roughly 100 classes, which a sign language expert then reviewed, retaining only those already consistent with accepted PSL conventions. In the second, 11 returning and 4 new participants were guided by the expert to perform standardised versions of each sign, with the vocabulary expanded to 128 classes and recordings verified against the National Institute of Special Education (NISE) PSL dictionary. In the third, 50 further students each contributed one video per class after viewing a gold-standard reference recording. Seventy participants were recorded in total; two took part only in Phase 1, performed only a small number of signs and did not complete a session, and because Phase 1 predates the standardisation carried out in Phase 2 those signs were not in the standardised form later verified against the NISE dictionary and did not survive expert review. Their recordings were not retained, on coverage and sign-correctness grounds and before any model was trained. All clips were annotated by the primary researcher and verified by the research supervisor. Evaluation protocols Two splits are provided so that results on PSL-128 are comparable across studies; please report which was used. The video-level split (video_level.csv) is a 70/15/15 partition over videos, yielding 6,347 / 1,360 / 1,361 clips; because it is drawn over videos, a signer's recordings may appear in more than one partition, so it measures generalisation across recordings and conditions rather than across people. The signer-independent split (signer_independent.csv) is drawn over signers instead, yielding 49 training, 9 validation and 10 test signers (6,304 / 1,382 / 1,382 clips); the validation and test signers contribute no training data at all. Signers were eligible for validation or test only if their recordings covered at least 90% of the 128 classes, and where a signer recorded across more than one sitting, all of their sessions stay together in the same partition. The session token in each filename (T1 to T5) marks a separate recording sitting rather than a collection phase: a signer needed a further sitting when one ran out of time before their class list was covered, resumed on a later day, so sittings are separated by at least a day and clothing and surroundings commonly differ between them. Ethics and permitted use Ethical approval was obtained from the Directorate General of Special Education, Punjab. Informed consent was taken from each participating student and, for minors, from their legal guardians. No participant's name or personal information was recorded, and no personally identifying metadata is included. The research was conducted in accordance with the Declaration of Helsinki and Higher Education Commission of Pakistan research ethics guidelines. The clips show identifiable individuals, many of whom were minors at the time of recording. Use is non-commercial and research-only under CC BY-NC 4.0. Users must not attempt to identify, contact or trace any participant; must not redistribute the video files, citing this DOI instead; must not use the clips for biometric identification, face recognition, surveillance, or any application whose purpose is recognising people rather than signs; and must not publish frames showing participants' faces outside the context of academic reporting on sign language recognition. Known limitations All 68 signers are male, so the dataset does not sample variation associated with female signers and reported accuracies should not be assumed to transfer unchanged to them; extending collection to female signers is a priority for future releases. The dataset covers isolated signs only, so continuous signing is out of scope. Non-manual grammatical markers — facial expression and mouthing — are present in the video but not annotated. At 224 x 224, extracted hand keypoints are the least reliable part of a pose estimate; the full-resolution Kaggle release should be used where higher-quality landmarks are needed. The 128 classes were recorded in Lahore and do not sample regional sign variation across Pakistan. The signer distribution is uneven by construction, ranging from 5 to 277 clips per signer, and five signers cover too few classes to be eligible for held-out partitions. Citation Usman, M., & Farooq, M. (2026). PSL-128: A Multi-Signer Pakistan Sign Language Video Dataset (224x224 preprocessed release) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.21946282 Please cite the accompanying data descriptor as well once it is published. Contact Muhammad Usman, M.Phil. candidate, Department of Computer Science, Punjab University College of Information Technology (PUCIT), University of the Punjab, Lahore, Pakistan, supervised by Muhammad Farooq. Email msaif23m012@pucit.edu.pk, LinkedIn https://www.linkedin.com/in/em-usman.