Reliability-guided meta-control in intuitive physics

Abstract

Intuitive physical reasoning is an important part of daily life, but the computations underlying it remain debated. Some prominent accounts propose intuitive physics relies on mental simulation, with people evolving mental scenes forward through step-by-step. However, representing every intermediate mental state in detail can be computationally costly. We suggest that intuitive physical reasoning combines mental simulation with learned predictive abstractions, and that the trade-off between these two is determined by the prospective reliability of an abstraction. We implemented the abstraction part of our proposal as a recurrent neural network model that learns a predictive abstraction by directly predicting the object’s future position from a short visual history. This network bypasses intermediate states, while also estimating its own prediction error. Our network produced simplified linear trajectories that shortcut ground-truth physical trajectories (in line with human behavior), and its error estimates were associated with human pupil size during a physical prediction task. We further developed a meta-control model that used the network’s error estimate to arbitrate between abstraction (direct prediction by the network) and simulation (running a mental simulation). The meta-control model captures key patterns in human response time and accuracy, and its predicted abstraction and simulation phases corresponded to distinct eye-movement signatures, over and above an earlier blended model that uses a hand-designed abstraction. Our results suggest that efficient physical reasoning can emerge from an interaction between learned abstractions, estimates of how reliable these abstractions are, and selective engagement of detailed simulation based on reliability estimates. Author summary From balancing dishes to knowing how an ice-cream scoop will fall and splatter, people have an intuitive sense of physics that they use in daily life. It is possible that people do this by “mental simulation”, thinking through how a scene will unfold step-by-step. But this might be slow and costly. We suggest people might combine mental simulation with shortcuts learned from experience. We built a neural network that learned to (1) predict where an object would be in the future without creating every intermediate state, and (2) estimate when its own predictions were unreliable. A meta-model can use the reliability estimate to know when to use the shortcut, and when to simulate. We compared our model to people’s behavior when they performed a physical reasoning task. Our model captured key patterns in people’s response time, accuracy, and eye movements. Our results suggest that efficient physical reasoning may combine simulation, shortcuts, and mechanisms for trading those off.