Skip to content
Open access

Controllable Training-Free Diffusion Style Transfer via Latent Initialization and Attention Statistical Alignment

Sep 2026 · Applied Sciences · 0 citations · 10 references

TL;DR

A unified, training-free diffusion framework for image-guided, text-guided, and localized style transfer that incorporates z-domain initialization and attention statistical alignment provide an effective basis for flexible style transfer without additional training.

Abstract

Recent advances in diffusion models have demonstrated remarkable generative capabilities, but their application to style transfer remains limited by inference instability and poor adaptation to diverse styles. Many methods either rely on costly fine-tuning or sacrifice flexibility and controllability at inference time. We present a unified, training-free diffusion framework for image-guided, text-guided, and localized style transfer. Built on Denoising Diffusion Implicit Model (DDIM) inversion, the framework incorporates z-domain initialization, which aligns the initial content latent with style statistics; and key–value statistical alignment (KV-StatAlign), which progressively mixes attention keys and values and calibrates their head-wise logit energy during sampling. It further incorporates a Prompt-to-Prompt branch for reference-free text-guided stylization and mask-based composition for regional editing. Experiments on MS COCO and WikiArt show that the proposed method achieves the best FID score of 18.039 and ranks second in ArtFID, LPIPS, and CFSD among the evaluated methods. Our experiments indicate that z-domain initialization and KV-StatAlign improve stylization stability and balance style fidelity with structural preservation while supporting continuous control over style strength. These results demonstrate that latent initialization and attention statistical alignment provide an effective basis for flexible style transfer without additional training.

Read PDF

Similar papers

Preprint Aug 2026

Scale-Separated Conditioning for Style-Encoder-Free Diffusion Stylization

SEFS (Style-Encoder-Free Stylization), a style-encoder-free conditioning framework for diffusion transformers, improves content consistency and leakage diagnostics while retaining reference-style affinity, and ablations support the crop-resolution, re-normalization, and skip-fusion choices.

Jing-Tao Zhang, Haorui Gao, Youqin Liang et al. · 2 citations
Open access Oct 2026

DeStyle: A Unified Decoupled Framework for Style Transfer via Singular Value Purification and Attention-Guided Distillation

A major challenge in training-free image style transfer is to reproduce the artistic characteristics of a reference image while preserving the semantic identity and spatial structure of the content image. Existing methods often struggle to balance content preservation and stylistic expressiveness: aggressive feature in...

Xin-Ying Liu, Tian Xian, Xue-Lei Geng et al. · 0 citations
Preprint Aug 2026

Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models

This study studies a family of training-free techniques conceptually rooted in Classifier-Free Guidance, most of which were originally proposed on older U-Net diffusion models and validated using metrics that assess image quality in isolation, without accounting for compositional alignment or semantic correspondence.

A. Sergievskii, Artyom Turevich, Sergey Kastryulin · 0 citations
Preprint Aug 2026

NoisEasier: Test-Time Noise Optimization for Text-to-Video Generation

NoisEasier serves as both a flexible alternative and a complementary enhancement to reward-based fine-tuning, establishing test-time scaling as an effective paradigm for controllable text-to-video generation.

Yu-Jiang Pu, Yu Kong · 1 citation
Preprint Oct 2026

TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer

Reference-guided style transfer aims to preserve the semantic structure of a content image while transferring the visual appearance of a style reference. Recent diffusion-based methods achieve impressive stylization quality by exploiting strong pretrained generative priors. However, training-free approaches still face...

Duc Khoan Le, Kim Ngoc Tran, Minh Đức Nhật Lê et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.