CloDe: A Multimodal Tuning-Free Clothing Designer

Abstract

Recent diffusion-based image generation models have enabled new directions in virtual try-on and garment synthesis. However, existing approaches often trade off between controllability and accessibility: tuning-based methods typically require additional training or specialized inputs, while tuning-free methods may lack structural control and fine-grained appearance guidance. In this work, we propose CloDe, a multimodal, tuning-free garment synthesis framework built on a pretrained diffusion inpainting model. Our method enhances controllability without finetuning the backbone by introducing three components: (i) a category-aware masking strategy that improves edit locality and reduces garment leakage, (ii) pose conditioning via ControlNet to better preserve human structure, and (iii) optional fabric conditioning using IP-Adapter to incorporate fine-grained appearance cues. Built on DiCTI, our approach maintains a fully zero-shot pipeline while improving synthesis quality across realism, pose consistency, and texture fidelity. We evaluate CloDe on VITON-HD using quantitative metrics and a user study, showing consistent improvements over DiCTI in both objective measures and perceptual quality.