An end-to-end framework for apparent personality inference leveraging vision-language models to introduce semantic priors and a semantic-enhanced facial representation module to capture subtle personality-related cues, together with an uncertainty-aware learning strategy to handle noisy and subjective annotations is proposed.
Abstract
Inferring apparent personality from facial images is important in social scenarios for embodied agents in human-robot interaction. Unlike inferring intrinsic personality traits via conversation, this task models first-impression personality perception based solely on facial appearance before interaction begins. Existing studies mainly focus on the Big Five personality model and often rely on language or multimodal inputs. As a result, it remains unclear whether facial cues alone can support meaningful associations with perceived personality traits. This question is particularly relevant for MBTI types, which are widely used in practice and more readily interpretable by large language models. To this end, we propose \textbf{GlanceFace}, an end-to-end framework for apparent personality inference leveraging vision-language models to introduce semantic priors and a semantic-enhanced facial representation module to capture subtle personality-related cues, together with an uncertainty-aware learning strategy to handle noisy and subjective annotations. Extensive experiments demonstrate strong performance on MBTI-based apparent personality benchmarks and reveal relationships between facial characteristics and perceived personality traits, highlighting its potential to support adaptive initial interaction strategies for embodied agents. The code and dataset are available at https://github.com/MrHuan3/GlanceFace.
SPIN is introduced, a cognitive-affective personality system-inspired inference pipeline for simulating human social-psychological behavior that suggests that structured personality-state inference can improve benchmark-level behavioral alignment beyond richer persona descriptions or generic multi-step reasoning.
Zhi-Bo Deng, Dong-Yuan Li, Shu-Wen Ge et al.· 0 citations
Robots operating in human-centered environments frequently encounter interaction errors due to limitations of underlying machine learning models, potentially negatively affecting user trust and adoption. Therefore, detecting when the models fail is critical for robust HRI systems. Existing approaches mainly rely on mod...
Pradip Pramanick, Alessandra Rossi, Onofrio La Scala et al.· IEEE International Symposium...· 0 citations
A century of psychology has found that the trait words people use to describe one another vary, but the relational structure among those traits, which ones go together and which oppose, is strikingly consistent across raters and cultures. We test whether the LLM (Qwen 2.5-7B-Instruct) reproduces this structure in its i...
Yi-Lin Geng, Omri Abend, Eduard H. Hovy et al.· Conference on Empirical Meth...· 0 citations
GesBench is introduced: a human-grounded benchmark for cue-level social perception in multimodal LLMs and shifts social evaluation from whether models eventually recognize meaning to when and where that meaning is formed.
Although the thoughts, feelings, and intentions of others are not directly accessible to perception, they are often inferred from observable behavior. Human movement may be especially important in this process because it provides continuously unfolding spatiotemporal structure through which socially meaningful interpre...
Mark Jaime, Elizabeth B. daSilva· PLoS ONE· 0 citations