Latent Reorganization by Fixed-Point Regularization in Recurrent Frame Prediction
TL;DR
Results suggest that geometric regularization of a recurrent bottleneck can act as a useful training-time prior without adding any inference overhead to the inference-time architecture.
Abstract
Regularization can improve generalization by constraining how a model uses its internal representation. In this paper, we study whether algebraic fixed-point constraints applied to the final LSTM hidden state during training can reorganize the recurrent latent space and improve held-out frame prediction. Rather than modifying the inference-time architecture, we introduce four training-time operators, GlobalHouseholder (reflection), GlobalGivens (rotation), Composition, and Lie Algebra, that bias the hidden state toward geometrically structured regions without changing the decoder pathway. Experiments across four datasets (indoor robot sequences, KITTI driving, Flying Shapes 2D, and Moving 3D Shapes) show that lightweight constraints consistently improve prediction on structured scenes, with GlobalGivens achieving up to +1.04 dB PSNR and −11.3% MAE over the unconstrained baseline on held-out Indoor sequences. The latent analysis reveals that the operators that generalize best are not those that compress the representation most aggressively but those that redistribute latent energy while preserving broad dimensional participation. Lie Algebra, despite collapsing activation variance by 81–96%, degrades under latent perturbation and does not match the lighter operators on structured datasets identifying over-constraint as a clear failure mode. These results suggest that geometric regularization of a recurrent bottleneck can act as a useful training-time prior without adding any inference overhead.