Recent advancements in Large Language Model (LLM) agents have largely focused on extending context windows or implementing massive Retrieval-Augmented Generation (RAG) systems to retain long-term history. However, this store-everything approach causes high computational costs and digital hoarding, paradoxically leading to digital amnesia where key emotional contexts are buried under trivial data. To challenge this paradigm, we introduce the Affective Memory Architecture, drawing from the amygdala's role in memory modulation to equip AI agents with the essential capacity to actively forget. Unlike static summarization, our framework structures multimodal inputs into an Affective Scene Graph (ASG) and dynamically adjusts the memory resolution based on emotional salience. High-arousal core memories are preserved in rich, high-resolution episodic detail; low-salience routines are aggressively downsampled using novel Optical Context Compression to minimal vision tokens; and frequently reactivated patterns are consolidated into crystallized semantic insights. Through quantitative proof-of-concept modeling, we demonstrate that systematically managing the trivial not only resolves the digital hoarding problem but actively reduces proactive interference, enhancing overall recall clarity. Ultimately, this work offers a scalable, privacy-friendly blueprint for resource-efficient AI capable of evolving with users over time, fundamentally shifting the goal of AI memory from total recall to meaningful retention.
SoYeop Yoo, Sunghoon Im· Proceedings of the 32nd ACM...· 0 citations
Subject-driven image personalization---generating new images that preserve the identity of one or several reference subjects in novel scenes---is a foundational capability for modern visual content creation. It is currently dominated by generalized methods that fine-tune a pretrained multimodal diffusion transformer (MMDiT) on hundreds of thousands to millions of paired \emph{(reference, composed-target)} examples, where each composed target is a synthesized image of the subject in a novel scene. Producing such targets demands a costly multi-stage curation pipeline---LLM-based prompt generation, T2I-based composed-target synthesis, reference-subject extraction, VLM-based quality filtering, and correspondence labeling---and tightly couples each method to a particular target synthesizer and curation choice. We introduce \emph{CRAFT} (Constrained Reward via Attention Fine-Tuning), a single-step ReFL framework that fine-tunes a pre-trained \emph{reference-aware} MMDiT via LoRA adapters using a compact reference-only data construction---$10$K reference images and subject masks, with no composed-target supervision. CRAFT realizes a \emph{Where to look} principle: attention-level rewards align noise- and phrase-token attention with the correct reference subject, and the resulting per-subject attention masks gate a pixel-level identity reward to keep image-space supervision consistent with the learned attention routing. Applied to FLUX.2-klein-9B, CRAFT achieves state-of-the-art performance on XVerseBench \rev{while using no composed-target supervision---only $10$K reference-only samples, whereas prior generalized methods require $150$K to over $2$M composed-target pairs}. The same recipe transfers to other reference-aware backbones, consistently improving performance. Project page: https://jihun999.github.io/projects/CRAFT/.
Jihun Park, Kyoungmin Lee, Jongmin Gim et al.· 0 citations
In video-text retrieval, addressing the noisy correspondence problem is crucial for achieving accurate retrieval performance. Recent methods address this challenge by either suppressing the impact of noisy data or predictions distorted by noise. However, existing approaches often overlook two critical aspects: distinguishing hard noise from semantically ambiguous cases, and preserving latent associations within unmatched negative pairs. To address this limitation, we propose Noise-mined Adaptive Self-Labeling (NASL), which effectively manages noisy data during training. NASL consists of two loss functions: 1) Noise-mined Matching Loss (NML), which identifies and penalizes noisy data based on a two-stage suppression strategy, and 2) Adaptive Self-labeling Loss (ASL), which employs optimal transport to recover latent associations among false negatives in noisy conditions and provides soft supervision to prevent excessive penalization of semantically plausible pairs. Extensive experiments demonstrate that NASL improves the separation between clean and noisy data while effectively mitigating noise, leading to significant performance improvements on the MSR-VTT, DiDeMo, MSVD, LSMDC, and ActivityNet datasets.
Jeonghoon Kim, Hyeon Kang, Jihun Park et al.· ACM Transactions on Multimed...· 0 citations
Recent advancements in Large Language Model (LLM) agents have largely focused on extending context windows or implementing massive Retrieval-Augmented Generation (RAG) systems to retain long-term history. However, this store-everything approach causes high computational costs and digital hoarding, paradoxically leading to digital amnesia where key emotional contexts are buried under trivial data. To challenge this paradigm, we introduce the Affective Memory Architecture, drawing from the amygdala's role in memory modulation to equip AI agents with the essential capacity to actively forget. Unlike static summarization, our framework structures multimodal inputs into an Affective Scene Graph (ASG) and dynamically adjusts the memory resolution based on emotional salience. High-arousal core memories are preserved in rich, high-resolution episodic detail; low-salience routines are aggressively downsampled using novel Optical Context Compression to minimal vision tokens; and frequently reactivated patterns are consolidated into crystallized semantic insights. Through quantitative proof-of-concept modeling, we demonstrate that systematically managing the trivial not only resolves the digital hoarding problem but actively reduces proactive interference, enhancing overall recall clarity. Ultimately, this work offers a scalable, privacy-friendly blueprint for resource-efficient AI capable of evolving with users over time, fundamentally shifting the goal of AI memory from total recall to meaningful retention.
SoYeop Yoo, Sunghoon Im· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
