PaintCopilot: Modeling Painting as Autonomous Artistic Continuation
TL;DR
PaintCopilot lets AI autonomously act on an artwork whose direction is still open, through interruptible, revisable delegation mechanisms spanning individual strokes, extended sequences, bounded regions, and prior stroke history.
Abstract
Existing neural painting methods are target-driven: given a reference image, strokes are optimized to reconstruct it, fixing the outcome before painting begins. We instead ask whether a model can predict plausible painting actions when the final image is unknown, a problem we call autonomous artistic continuation. PaintCopilot realizes this through three lightweight, task-specific models: a Target Predictor that infers a provisional visual target directly from the evolving canvas, a Stroke Predictor that formulates brushstroke generation as flow matching to capture multiple plausible continuations, and a Region Sampler that generates stroke sequences within artist-specified regions via a conditional VAE, trained on a constructed dataset of 3000 portraits with stroke-level supervision. This lets AI autonomously act on an artwork whose direction is still open, through interruptible, revisable delegation mechanisms spanning individual strokes, extended sequences, bounded regions, and prior stroke history. A two-week deployment with ten artists suggests that creative agency was shaped not simply by how much work artists delegated, but by their ability to continually negotiate the direction, scope, and persistence of AI participation.
