RSLayout-DiT: Parameter-Efficient Layout-Controlled Remote Sensing Image Generation With Diffusion Transformers

Abstract

Layout-controlled remote sensing image generation is valuable for data augmentation, scene simulation, and object-level geospatial understanding. However, it remains challenging due to complex backgrounds, dense small objects, rotated objects, and complex object-context relationships. To address the limitations of existing methods in global structure modeling, object orientation preservation, and small-object generation, we propose RSLayout-DiT, a parameter-efficient Diffusion Transformer-based method for layout-controlled remote sensing image generation. The proposed method adopts the pretrained remote sensing-adapted flux (RS-FLUX) model as the generative backbone, uniformly renders object categories, horizontal bounding boxes, and oriented bounding boxes into layout condition images, and maps them into the latent space via a frozen variational autoencoder (VAE). The layout latents are concatenated with noisy image latents along the token dimension and jointly modeled, enabling the denoising network to simultaneously perceive textual semantics, image states, and spatial structural constraints. To reduce training costs while preserving the pretrained generative priors, the backbone network is frozen, and low-rank adaptation modules are introduced only into the attention layers. We further design an object-aware region-enhanced training strategy to impose adaptive weighted supervision on object regions, small-scale objects, low-frequency classes, and boundary regions. Experiments on the detection in optical remote sensing–remote sensing visual grounding (DIOR-RSVG) and dataset for object detection in aerial images (DOTA) datasets validate its effectiveness in object-level layout-controlled remote sensing image generation.