Bottlenecks in Small-Scale Diffusion Models across Noise Scheduling Inference and Data Scarcity

Abstract

The diffusion model is currently the most common method for generating high-fidelity images, but it requires a large amount of data for training and has a high computational cost. Even though ongoing improvements to architecture have made generation more efficient, it is still important to look into the operational bottlenecks and performance limits of generation systems that are cheap and have few resources. This research examines the performance of diffusion models on small-scale datasets, including MNIST and CIFAR-10, utilising a lightweight U-Net architecture. The experiments seek to determine the configuration that optimises visual fidelity within a constrained computational budget by systematically assessing noise scheduling designs, DDIM-based acceleration strategies, and scenarios characterised by extreme data scarcity. Experimental results show that the combination of cosine scheduling and EMA does not improve the quality of generated results. However, DDIM enables a controllable trade-off between sampling speed and image quality. Under severe data scarcity, training loss remains relatively low despite substantial degradation in FID and generated image quality.