Recent advances in text-to-image (T2I) generation have enabled controllable image synthesis by incorporating conditions beyond text. However, most existing diffusion-based methods are limited to a single type of control condition (e.g., bounding boxes or keypoints), which restricts their flexibility. To address this limitation, we propose MixDiffusion, a training-free diffusion framework for multi-condition T2I generation. MixDiffusion theoretically supports an arbitrary number of control conditions, including bounding boxes, keypoints, sketches, depth maps, reference images, and text, by collaboratively integrating multiple pre-trained uni-condition diffusion models. The key insight of the proposed approach is to derive the predicted noise distribution in each denoising step of the diffusion-based multi-condition image generation model from the predicted noise distributions of multiple diffusion-based uni-condition models with a derived integration formula, which is supported by rigorous theory proof. Owing to its training-free nature, MixDiffusion is easy to deploy and readily extensible to new control modalities.
In recent years, the progress of diffusion models in text-to-image generation has been obvious to all. Text-based descriptions alone, however, are often insufficient for precisely controlling screen content. In order to solve this problem, researchers began to try to introduce additional conditions or visual references...
Experiments show that PixelControl improves structural fidelity and visual quality over existing controllable generation methods, with especially strong gains on boundaries and medium/small conditioned regions.
Xin Lin, Haodong Li, Zhi-Fei Zhang et al.· 1 citation
This study proposes DNF-SR, a dual-input and Negative-aware Feature fine-tuning method for Real-ISR that uses a dual-input strategy that concatenates the original LR image with the noisy LR input and feeds them into a diffusion-based image editing model, ensuring both high-fidelity one-step super-resolution and improve...
Shu-Hao Han, Wen-Jie Liao, Hayden Vance et al.· 1 citation
. As one of the core cutting-edge fields of AI-generated content, text-to-image generation has brought about dramatic changes to traditional content creation fields. This paper divides the development of text-to-image generation into three stages: Generative Adversarial Networks (GAN), Diffusion Models and the explorat...
Ran Tao· Proceedings of the 4th Inter...· 0 citations
TOLA is proposed, a Text-aware One-step Latent Adaptation framework without iterative image-text diffusion that consistently surpasses existing diffusion-based TSR methods by at least 2.72 dB in PSNR on CTR-TSR-Test.
Adaptive Frequency-Aware Diffusion (AFID), an adaptive framework for high-quality text-guided image inpainting, is proposed and comprehensive comparisons against prevailing state-of-the-art inpainting approaches solidly confirm its superiority in visual fidelity, intact-area preservation and text-prompt consistency.
