2026· Poster Volume 0008 The 2026 Twenty-Second International Conference on Intelligent Computing July 23-26, 2026 Toronto, Canada· 0 citations
TL;DR
This work proposes RIG (Regional Image-prompt Generation), a novel training-free framework for multi-reference personalized generation that significantly outperforms state-of-the-art adapter methods in terms of both personalization fidelity and text-layout alignment.
Abstract
Though zero-shot adapters excel in image personalization, they often encounter significant challenges in multi-reference personalized generation, specifically failing to precisely adhere to the spatial layouts described in text prompts and suffering from feature leakage between reference images. To address these two challenges, we propose RIG (Regional Image-prompt Generation), a novel training-free framework. For the first challenge, leveraging Multimodal Large Language Models (MLLMs), we introduce a layout planning binder. Leveraging Chain-of-Thought (CoT) reasoning, this module infers and generates precise global layouts from text prompts, while simultaneously binding reference images to their corresponding regions. For the second, we introduce a satially decoupled diffusion mechanism that isolates feature streams during attention computation. By injecting reference features exclusively into designated regions, this mechanism effectively prevents feature interference between reference images. Extensive experiments demonstrate that RIG significantly outperforms state-of-the-art adapter methods in terms of both personalization fidelity and text-layout alignment.
Subject-driven image personalization---generating new images that preserve the identity of one or several reference subjects in novel scenes---is a foundational capability for modern visual content creation. It is currently dominated by generalized methods that fine-tune a pretrained multimodal diffusion transformer (MMDiT) on hundreds of thousands to millions of paired \emph{(reference, composed-target)} examples, where each composed target is a synthesized image of the subject in a novel scene. Producing such targets demands a costly multi-stage curation pipeline---LLM-based prompt generation, T2I-based composed-target synthesis, reference-subject extraction, VLM-based quality filtering, and correspondence labeling---and tightly couples each method to a particular target synthesizer and curation choice. We introduce \emph{CRAFT} (Constrained Reward via Attention Fine-Tuning), a single-step ReFL framework that fine-tunes a pre-trained \emph{reference-aware} MMDiT via LoRA adapters using a compact reference-only data construction---$10$K reference images and subject masks, with no composed-target supervision. CRAFT realizes a \emph{Where to look} principle: attention-level rewards align noise- and phrase-token attention with the correct reference subject, and the resulting per-subject attention masks gate a pixel-level identity reward to keep image-space supervision consistent with the learned attention routing. Applied to FLUX.2-klein-9B, CRAFT achieves state-of-the-art performance on XVerseBench \rev{while using no composed-target supervision---only $10$K reference-only samples, whereas prior generalized methods require $150$K to over $2$M composed-target pairs}. The same recipe transfers to other reference-aware backbones, consistently improving performance. Project page: https://jihun999.github.io/projects/CRAFT/.
Jihun Park, Kyoungmin Lee, Jongmin Gim et al.· 0 citations
Layout-guided multi-instance generation is essential for controllable image synthesis in Multi-Modal Diffusion Transformers (MM-DiTs). However, integrating this capability into unified architectures remains challenging. Prior frameworks rely on redundant full-resolution canvas padding and Shifted-RoPE to manage multiple reference images. This mechanism drastically inflates computational overhead for sparse layouts and disrupts critical low-frequency RoPE features, creating a severe spatial-frequency compromise that blurs absolute spatial correspondence. To overcome these limitations, we propose ControlRef, a highly efficient and precise multi-instance synthesis framework. ControlRef utilizes a Unified Instance-Layout Control (UILC) attention mask to strictly decouple inter-instance semantic interactions and enforce precise regional binding. To further promote region-level spatial alignment, we introduce Anchored 4D-RoPE, a novel positional encoding mechanism that directly anchors tokens to their absolute geometric centers. By pre-aligning reference images to their corresponding bounding box resolutions, physically anchoring both layout and reference tokens to their absolute geometric centers, and stacking the references along the z-axis, Anchored 4D-RoPE natively preserves spatial priors and mitigates the spatial-frequency compromise without lossy shifting. Extensive experiments demonstrate that ControlRef achieves state-of-the-art visual fidelity and localization accuracy, while concurrently slashing inference latency by over 80% in sparse layouts and reducing memory overhead by 50% in dense scenarios.
Yunkai Yang, Yudong Zhang, Xinying Chen et al.· 0 citations
The AP-LCA approach introduces a novel local cross-modal alignment strategy that utilizes image cropping and similarity-based semantic contribution assessment to precisely map fine-grained descriptions to relevant local image regions.
Si-Ying Wu, Song Wu· International Conference on...· 0 citations
This work presents Gradient-free Analytical Trajectory Optimization Video Generation (GATO-Vid), a novel training-free and gradient-free approach for precise spatial guidance that significantly outperforms existing baselines in localization accuracy while introducing minimal computational overhead.
Guillaume Jeanneret, Mathis Koroglu, Hugo Caselles-Dupré et al.· 0 citations
PoseAdapter, a lightweight framework for high-fidelity 2.5D controllable image generation, and a Context-Aware Dual-Stream Representation, to resolve the generative trade-off between strict instance isolation and global coherence.
Yufeng Chi, Hui-Min Ma, Fan Gao et al.· 0 citations
Swift-Image achieves leading aggregate performance among evaluated open-source models with only 6B parameters and 243K GPU training hours; the compressed 3B model incurs nearly no loss, while few-step distillation further improves aggregate editing performance with substantially fewer sampling steps.
Taihang Hu, Zhaowen Wang, Zuan Gao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
