From Raw Signals to Understood States: A Perspective on Neuro-Symbolic and LLM-Driven Intelligent Sensing of Human Affective and Cognitive States

Abstract

Intelligent sensor technologies now provide unprecedented multimodal access to human affective and cognitive processes, spanning physiological, ocular, facial, kinematic, ambient, and behavioral streams. Yet, despite dramatic advances in acquisition and deep representation learning, the pipeline from raw signals to psychologically meaningful, actionable understanding remains fragile. Deep-only architectures excel at pattern extraction but struggle with contextual reasoning, uncertainty communication, and human-facing explanation; symbolic-only frameworks resist noisy, high-dimensional streams. This perspective argues that the field’s next inflection point lies not in richer sensors alone but in the interpretive layer that turns signals into states. We propose an integrative view in which neuro-symbolic fusion couples continuous sensor evidence to psychologically grounded symbolic primitives, LLMs act as auditable semantic reasoners rather than opaque classifiers, and explainability is treated as a design constraint rather than a post hoc addition. Synthesizing a decade of work on affective computing, fuzzy learner modeling, neuro-adaptive multimodal interaction, and explainable human–AI collaboration, we motivate a research agenda organized around grounded representations, uncertainty-calibrated LLM reasoning, and reflexive explanation. The aim is a class of sensing systems whose intelligence is measured not by accuracy alone but by the quality of the states they help humans understand.