Dynamic Facial Expression Recognition (DFER) has recently attracted significant interest due to its vital role in enabling empathetic and human-compatible technologies. Developing models that remain robust under in-the-wild variability is a key motivation for DFER research and its practical applications. Improving models through multimodal learning, which leverages audio and video data for richer, complementary representations, is one promising direction. However, existing methods still rely heavily on modality-specific encoders and coarse-grained content-level alignment, which hinders their ability to capture fine-grained emotional semantics and dynamic cross-modal interactions. To address this, we adopt parameter-efficient fine-tuning (PEFT) to facilitate audio-visual interaction. This strategy offers key advantages: (1) freezing parameters preserves upstream pretrained knowledge, ensuring that the model focuses solely on learning modules for audio-visual interaction and modal fusion; (2) a Mamba Fusion Adapter (MFAdapter) is inserted at each encoder layer to perform causal, audio-conditioned fusion over a frame-aligned token sequence, enabling efficient multi-level cross-modal injection with linear complexity; and (3) a Frame-level Feature Arrangement (FFA) strategy is introduced as a deterministic index-based arrangement scheme that arranges audio tokens at the video-frame rate, providing a frame-indexed temporal prior that supports the causal scan in MFAdapter; FFA reduces, but does not eliminate, coarse segment-level mismatch and is not claimed as verified frame-level synchronization. Notably, our method achieves competitive performance on DFEW and MAFW while updating 2.7% (4.7 million) of the model parameters, with the best WAR of 58.70% on MAFW among all compared methods and 76.62%/65.25% (WAR/UAR) on DFEW under the official five-fold cross-validation protocols.
Kang-Bo Ning, Shanshan Gao, Zhaoqiang Xia et al.· Italian National Conference...· 0 citations
Sequential recommendation aims to predict users’ next items of interest based on their historical interactions. Recently, Large Language Models (LLMs) have shown strong potential in this field due to their powerful semantic understanding. However, existing methods face two core challenges: First, LLMs may generate non-existent recommendations due to hallucinations; Second, when utilizing LLM-generated embeddings for transfer to downstream recommendation tasks in different domains, the dimensional and structural differences across spaces exacerbate space misalignment. To address these challenges, we propose UniRec, a unified space learning framework that achieves deep alignment through a hierarchical space transformation path: language space → collaborative space → real space. This framework employs a two-stage learning mechanism: The first stage guides LLMs to align semantic and collaborative spaces through collaborative instruction fine-tuning, introduces a real-space constrained generation mechanism to reduce hallucination problems, and is optimized for large-scale applications; The second stage extracts core semantic principal components through singular value decomposition and freezes the representations, combines with gating networks to adaptively fuse semantic and domain-specific collaborative signals, achieving efficient cross-domain transfer. Experiments show that UniRec achieves over 27% average improvement on in-domain datasets and 24% on out-of-domain datasets, while maintaining low memory overhead.
Jiarui Yang, Junmei Feng, Yihan Zhang et al.· ACM Transactions on Informat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.