When Direct Manipulation Becomes a Guess: Productive Friction in AI-Mediated Multisensory Visualization
TL;DR
Reading the malleable interfaces of Iron Man 2 against systems from Blade Runner 2049, this work proposes productive friction, including cross-sensory renderings whose detail reflects model uncertainty, a deterministic-affordance mismatch.
Abstract
Generative visualization increasingly embeds large language models within direct manipulation and multisensory interaction. Speech, gaze, touch, gesture, sound, and haptics can make probabilistic inference feel like familiar, deterministic tool use. I call this gap a deterministic-affordance mismatch: deterministic interaction cues persist while AI weakens predictability, locality, reversibility, or provenance. Reading the malleable interfaces of Iron Man 2 against systems from Blade Runner 2049, I propose productive friction, including cross-sensory renderings whose detail reflects model uncertainty. Four frictions expose the inference boundary, match action to effect, make stochastic branches tangible, and attribute sensory agreement.
