Conditional Dynamical Systems for Image Generation

Abstract

Image generation is dominated by deep generative models on GPUs, whose computational and energy costs raise sustainability concerns. Emerging non-von Neumann substrates, including quantum, compute-in-memory, photonic, and thermodynamic platforms, promise greater efficiency, yet most work ports conventional neural network architectures onto them and mainly accelerates operations such as matrix multiplication. This does not fully exploit a native capability of many hardware substrates: relaxation toward low-energy states can itself compute at little cost. Building on this primitive, we develop dynamical systems for image generation based on Ising spins and Kuramoto oscillators. The internal states of both models evolve under dynamics with an explicit Lyapunov energy, and a compact decoder renders the final states as images. For conditional generation, we introduce local conditioning: programmed interactions remain fixed, and one class-dependent bias per node reshapes the landscape, avoiding additional reprogramming and calibration costs. On CIFAR-10, the Ising and Kuramoto models reach clean-FID scores of $6.40$ and $5.37$, respectively, with $4096$ nodes. With a CMOS-compatible hardware realization, the dynamical-system core is projected to cost ${\sim}110.6$~nJ per image, and the full computation, even with the decoder on a GPU, outperforms all GPU baselines in latency and energy. These results suggest that energy-descending dynamics on non-von Neumann substrates can serve as generative computation, offering an efficient path beyond GPUs.