A human-centered cognitive support framework for manual industrial assembly: integrating perception, agentic reasoning, and augmented reality guidance

AR and Intelligent Assistance for Technicians: Design, Delivery, and Real-World Impact

AR and Intelligent Assistance for Technicians: Design, Delivery, and Real-World Impact Table of Contents CHAPTER 1 IntroductionCHAPTER 2 The Evolution of Technical Support and AssistanceCHAPTER 3 Foundational Principles of Augmented RealityCHAPTER 4 Cognitive Ergonomics and Human-Computer InteractionCHAPTER 5 Hardware Systems for Industrial Augmented RealityCHAPTER 6 Software Architectures and Spatial ComputingCHAPTER 7 Designing User Interfaces for Technical TasksCHAPTER 8 Integrating Artificial Intelligence with Augmented RealityCHAPTER 9 Computer Vision for Object and State RecognitionCHAPTER 10 Authoring and Managing Augmented Reality ContentCHAPTER 11 Real-Time Data Integration and the Internet of ThingsCHAPTER 12 Remote Expert Collaboration and TelepresenceCHAPTER 13 Pedagogical Frameworks for AR-Based TrainingCHAPTER 14 Measuring Performance and Cognitive LoadCHAPTER 15 Safety, Ergonomics, and Occupational HazardsCHAPTER 16 Deployment Strategies and Organizational ChangeCHAPTER 17 Security, Privacy, and Data GovernanceCHAPTER 18 Case Studies in Manufacturing and MaintenanceCHAPTER 19 Emerging Trends and the Future of Intelligent AssistanceCHAPTER 20 Conclusion

Toward an Integrated Cognitive-Ergonomic Architecture for Human-Machine Interaction: Combining Cognitive Models with Human Factors Ergonomics

This paper presents an integrated approach to modeling human competencies by combining the theoretical foundations of cognitive architectures with principles from Human Factors Ergonomics (HFE). Through a comparative analysis of established cognitive models-SOAR, ACT-R, LIDA, and COCOM-we synthesize a tailored architecture designed to address the complexities of human-machine interaction (HMI) in dynamic environments. By contextualizing this model within ergonomic frameworks, we elucidate the mechanisms underlying decision-making, skill acquisition, and adaptive behavior, bridging the gap between cognitive theory and applied system design. Our framework is empirically grounded in industrial robotics applications, where operator expertise, normative knowledge, and real-time feedback loops are critical. The proposed architecture not only enhances the cognitive alignment of HMI systems but also provides a scalable methodology for designing intelligent, human-centered interfaces in high-stakes environments. This work advances both the theoretical understanding of human competencies and the practical implementation of adaptive, ergonomically optimized systems.

A multimodal foundation model-enabled agent for human–robot collaboration in construction

Fostering Computational Thinking for an Adaptive Construction Workforce: Experimental Study of VR-Based Human–Robot Interaction

Construction work is becoming increasingly technology-intensive, requiring transferable skills that enable effective human–robot interaction (HRI). We designed and evaluated a virtual-reality (VR) training environment that pedagogically integrates computational thinking (CT) elements (i.e., decomposition, pattern recognition, abstraction, and algorithm design) within HRI tasks relevant to construction. In this study, CT is conceptualized as a structured problem-solving competency that supports task formulation, decision-making, and interaction with automated systems in construction environments. Participants completed a progression of pick-and-place missions using a six-axis robotic arm on a Mecanum mobility platform across three settings: a warehouse (training), along with realistic construction and unfamiliar extraterrestrial construction (testing) environments. In a proof-of-concept, exploratory study with 14 undergraduate construction students, the CT-integrated training environment was compared with a baseline configuration featuring similar HRI tasks without explicit pedagogical CT integration. Outcomes included knowledge–skill–attitude measures and task performance indexed by accuracy–speed–energy–safety metrics. Relative to the control group, the experimental group (trained in the CT-integrated environment) demonstrated larger gains in CT knowledge and attitudes and achieved superior post-training performance in the realistic construction site scenario, with significant multivariate group differences favoring CT integration. Group differences were insignificant in the extraterrestrial construction site scenario, suggesting that near-term transfer is constrained by contextual factors under unfamiliar conditions. Across groups, pattern recognition and algorithm design were the most consistent predictors of efficient and safe task execution. These findings provide preliminary evidence that integrating CT within immersive VR-based HRI training can strengthen computational competencies and structured task performance in construction-relevant environments, while highlighting the need for expanded instructional designs to better support transfer to unfamiliar, high-complexity contexts.

CARTA: Context-Aware Dual Retrieval and Chain-of-Thought Task Allocation for Environment-Grounded Elderly Care

Enabling elderly individuals to age independently at home requires intelligent assistive systems that can understand complex care needs and coordinate appropriate responses. While large language models (LLMs) show promise for adaptive assistance, current eldercare systems suffer from critical limitations: they generate physically infeasible actions due to weak environmental grounding, cannot coordinate multi-faceted care needs through single-agent processing, and lack safety mechanisms to prevent hallucination of non-existent resources. We present CARTA (Context-Aware Dual Retrieval and Chain-of-Thought Task Allocation), a framework that addresses these gaps through two innovations. First, our dual-path retrieval architecture combines semantic knowledge retrieval with graph-constrained retrieval-augmented reasoning, constructing an explicit environment graph encoding object availability, functional affordances, and spatial relationships. This grounds LLM reasoning in physical reality, improving the environmental executability of generated plans by constraining them to objects and affordances actually present in the scene. Second, our Chain-of-Thought multi-agent coordination enables specialized agents to deliberatively decompose ambiguous requests, negotiate responsibilities based on capabilities and constraints, and execute interventions concurrently. Experimental evaluation demonstrates substantial improvements: CARTA achieves 81.8% environment constraint satisfaction versus 65.5% for BM25 and 76.4% for dense retrieval, preventing resource hallucination. On multi-agent task execution, CARTA attains 78.1% success rate on complex tasks and 75.0% on vague commands-a 7.9% improvement over state-of-the-art planners on ambiguous scenarios requiring adaptive coordination. These results establish that explicit environmental grounding and deliberative multi-agent reasoning are essential for safe, contextually appropriate LLM-based elderly care.

From Code to Collaboration: A Cognitive Agent Framework for Large Language Model (LLM)-Based Human-Vehicle Teaming

This study develops a human-centered cognitive-agent framework for understanding how large language models (LLMs) can support human-vehicle teaming in automated driving. Following PRISMA guidelines, we reviewed 1,126 records published between 2021 and 2025 and included 52 studies after screening and full-text assessment. The synthesis identified four recurring capability clusters: perception and awareness, reasoning and decision-making, action and control, and interaction and communication. Across these functions, LLMs show promise for improving semantic scene understanding, explainable decision-making, high-level planning, and bidirectional communication with drivers. However, hallucinations, incomplete physical grounding, non-deterministic reasoning, and latency remain important limitations in safety-critical settings. The findings suggest that LLMs are most effective as high-level cognitive partners integrated with verified task-specific modules rather than as standalone controllers. The proposed framework offers design guidance for safer, more transparent, and collaborative human-vehicle systems.