Jul 2026· ICCK Transactions on Advanced Computing and Systems· 0 citations· 89 references
TL;DR
A compact and deployable Convolutional Neural Network-Long Short Term Memory (CNN-LSTM) framework that combines a 2D convolutional backbone (AlexNet) for frame-level descriptors with an LSTM head for sequence modeling is proposed, indicating a robust, real-time-capable solution for video understanding in both offline analytics and online deployment.
Abstract
Human Action Recognition (HAR) in unconstrained video remains difficult due to cluttered backgrounds, camera motion, and long-range temporal dependencies. The recognition of human action is the most complex study in the area of Artificial Intelligence (AI) and Computer Vision (CV). Machine vision for online and offline video processing is typically employed in the development of human behavior recognition systems. In video broadcasting and analysis, identifying the type and content of human actions present in the footage is a fundamental requirement. In this article, we propose a compact and deployable Convolutional Neural Network-Long Short Term Memory (CNN-LSTM) framework that combines a 2D convolutional backbone (AlexNet) for frame-level descriptors with an LSTM head for sequence modeling. The approach is evaluated on three representative benchmarks KTH (6 classes), Hollywood-2 (12 actions), and UCF-50 (50 actions). The system attains strong aggregate performance, reaching 98.5% accuracy on KTH, 93.0% accuracy on UCF-50, and 96.0% mAP on Hollywood-2, with improvements distributed broadly across classes. Error–length analyses show that recognition quality rises steadily with longer clips and that the recurrent module extracts the largest gains, while precision–recall and ROC curves with superior Area Under the Curve (AUC) and reliability demonstrate improved ranking and probability calibration across operating points. Despite a larger parameter count than several baselines, the design achieves the most favorable efficiency profile measured at 18ms per frame latency, 135 mJ per frame energy, and the highest throughput placing it on the accuracy–latency–energy Pareto frontier. An ablation study confirms the centrality of temporal modeling (6-12 percentage-point drops without the LSTM), the benefit of longer temporal windows, and the usefulness of stepped learning-rate schedules for stable optimization. The results indicate a robust, real-time-capable solution for video understanding in both offline analytics and online deployment.
This paper systematically reviews the evolution of computer vision-based human motion recognition technology, from traditional handcrafted feature methods to convolutional neural networks and recurrent neural networks in the deep learning era, and then to the recently emerging Transformer architectures and vision-language models.
Xin Sun· International Conference on...· 0 citations
The article compares the performance of traditional machine learning techniques with recent deep learning architectures such as CNNs, RNNs, TCNs, and Transformers, based on accuracy, computational cost, and suitability for real-world disorderly plotting.
Disha Deotale, Madhushi Verma, P. Suresh et al.· Discover Artificial Intellig...· 0 citations
Aiming at the key problems such as the separation of action recognition and quality assessment tasks, coarse feedback granularity and high labeling cost in the automatic evaluation of rehabilitation training, this paper proposes PDDS‑Net. The framework takes human skeleton sequence as input and realizes action classification and location-level deviation location synchronously through the collaborative architecture of a global action recognition stream and a local part evaluation stream. In the local flow, the joint nodes are divided into three functional part groups: upper limb, trunk and lower limb. independent graph convolutional subnetworks are used for decoupled coding, and a contrastive learning strategy is used to drive the attention map to focus on abnormal joints without frame‑level annotations. This is mainly achieved through the adaptive fusion mechanism to dynamically integrate the confidence of the deep network and the matching score of dynamic time warping template to maintain decision stability under conditions of pose degradation. Experiments on the NTU RGB+D and PKU‑MMD public datasets show that the action recognition accuracy of PDDS‑Net reaches 93.7%, the average precision of position‑level feedback is 0.656, 82.3% of the original AUC is still maintained at a 40% joint dropout rate, and the single‑frame inference latency is 41.8 ms, which meets the requirements of real-time interaction. This study provides a replicable technical path for the validation of rehabilitation evaluation algorithms without clinical data collection.
Mingxiang Yang· Journal of Discovery Core· 0 citations
Human Activity Recognition (HAR) has gained significant attention in computer vision due to its wide range of applications in surveillance, social behaviour analysis, and human–computer interaction. Among various human-to-human interactions, handshake recognition is particularly important as it represents social intention and cooperative behaviour. This study presents an efficient and interpretable deep learning framework for automatic handshake recognition from video sequences. The proposed approach employs a pretrained 3D Convolutional Neural Network (3D CNN) to directly learn spatiotemporal features, enabling effective modelling of both motion dynamics and spatial relationships between interacting individuals. The experiments are conducted using two dataset namely UT-Interaction Human Interaction Dataset and SBU Kinect Interaction dataset, focusing exclusively on the handshake interaction as the target class. The dataset provides accurate ground-truth annotations, including temporal intervals and bounding boxes, which support precise localization and reliable recognition of handshake actions. Each dataset is split into 80% for training, 10% for validation, and 10% for testing to ensure robust performance evaluation. The experimental results demonstrated that the proposed 3D CNN-based framework achieved a handshake recognition accuracy of 98.92% on the UT-Interaction dataset, representing performance improvements of 9.72%, 6.32%, and 7.12% compared to CNN, BiLSTM, and RNN models, respectively.
S. Kumaravel, S. Veni· International Journal of Lat...· 0 citations
Action prediction from frames and videos is a well-studied problem. Models trained with a single modality, mostly vision, will fail in low-light conditions. Recent works have attempted to predict action categories using vision-language and audio-visual models. A challenge, however, is that some dataset annotations lack temporal ground truth and include only the vision modality. Relying on transformers and an intelligent Vision Language Model (VLM) is a viable solution, but deploying them on edge devices could lead to reduced performance and hallucinations. This work presents an Audio-Visual-Text (AVT)-based multi-step Pipeline for Action Prediction and Captioning (AVT-PAC) to address this problem. First, for an input video, we identify the area to focus on using the Region-of-Interest (ROI) Extraction module. CLIP and CLAP encoders are used for ROI prediction. However, the ROI extracted region may vary in duration, resulting in a large number of frames to be processed. To avoid learning from redundant frames, we uniformly sample key frames within the ROI extracted region using a keyframe extraction module. These keyframes are then used to train an Audio-Visual Action and Text-Aware Representation (AVATAR) model to predict actions and captions. Through systematic experiments, we demonstrated that the proposed AVATAR-TCN model beats the present state-of-the-art (SOTA) baselines on the AVE dataset. Code is available in https://github.com/Ifovia/AVT-PAC
A. R, Ambarish Parthasarathy, Sucharitha Devarakonda et al.· International Conference on...· 0 citations
Lip reading, the task of inferring spoken words purely from visual observation of lip movements, remains a
challenging problem even for trained human lip readers, who are typically able to correctly identify only about every second
word. This paper presents an automated lip reading system built using Convolutional Neural Networks (CNNs) to classify
spoken words from video sequences of a speaker's mouth region. The system uses a Haar Cascade classifier to localize the face
and mouth region in each video frame, followed by a dlib-based facial landmark detector to extract a precise lip Region of
Interest (ROI). The cropped and normalized lip images are then passed to a trained CNN, which was benchmarked against a
Long Short-Term Memory (LSTM) network and a Temporal Convolutional Network (TCN) to model the sequential nature of lip
movement, with the CNN architecture selected for deployment based on superior classification accuracy on the Lip Reading in
the Wild (LRW) dataset. The trained model was integrated into a real-time application capable of capturing live webcam video,
isolating the speaker's lip region frame-by-frame, and predicting the spoken word with an associated confidence score.
Experimental evaluation on a ten-class subset of the LRW dataset shows that the proposed CNN model is able to reliably identify
isolated words such as “Hello”, “Start”, “Stop”, and “Previous” in real time. The system demonstrates the practical feasibility of
vision-only speech recognition and its potential applications in assistive hearing technology, security and surveillance, and
human–computer interaction in audio-degraded environments
B. Arjun, M. Saad, Aaron Biju et al.· International Journal for Re...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
