Skip to content

Learning a Flow to Self-Supervised Representations

Sep 2026 · arXiv.org · Vol abs/2609.29350 · 0 citations · 44 references
Computer Science Mathematics

TL;DR

Flow-Based Distribution Matching is introduced, a non-adversarial framework that learns this reference-directed geometry through spherical conditional velocity regression that achieves performance nearly on par with DM and remains competitive with existing SSL methods.

Abstract

We introduce Flow-Based Distribution Matching (FBDM), which learns self-supervised representations through non-adversarial velocity regression. The encoder and auxiliary field are trained jointly by regressing analytic velocities along spherical paths from learned representations to reference targets. Both augmented views share a reference target, with a capacity limit on each reference center and an explicit view-alignment loss. A structured reference supports more components than the auxiliary flow dimension while retaining geometric separation. The auxiliary field is discarded after pretraining; only the backbone is retained downstream. On three ResNet-18 benchmarks with matched auxiliary parameter counts, FBDM provides 2.17 to 2.36 times higher per-epoch training throughput than critic-based Distribution Matching (DM), with about 0.21 GiB additional peak allocated GPU memory. Experiments show that FBDM is competitive with existing SSL methods from CIFAR to ImageNet and performs comparably to DM on their shared benchmarks. Under stated conditions, our population analysis bounds nearest-centroid classification error on the initial normalized pretraining representation by the FBDM loss.

View source

Similar papers

Open access Sep 2026

Mixture of Lie-Group Kernels for Equivariance-Inspired Feature Learning.

MoLiegroup is introduced, a framework that embeds multiple Lie-group priors as specialized kernel experts and adaptively fuses them via a geometry-aware gating mechanism to learn equivariance-inspired geometric features and supplies a practical, architecture-agnostic geometric inductive bias that improves expressivity...

Yao-Xian Yang, Guipeng Lan, Shuai Xiao et al. · 0 citations
Open access Aug 2026

Enhancing self-supervised representation learning through lightweight learnable data augmentation

A lightweight learnable augmentation framework based on Extreme Learning Machines for self-supervised visual representation learning that improves linear evaluation performance relative to reproduced baselines across most settings and demonstrates that lightweight learnable augmentation can effectively enhance self-sup...

Mubarakah Alotaibi · 0 citations
#machine learning Preprint Sep 2026

Unifying Distributional Training for One-Step Visual Generation

A unified theoretical framework is introduced that separates distribution modeling from matching discrepancy and connects global objectives to pointwise feature updates through Wasserstein gradient flow and motivates MGFlow, which models feature distributions with Gaussian mixtures at an adjustable granularity between...

Chi Zhang, Hao-Yan Shi, Yue-Yi Liu et al. · 2 citations
Conference Aug 2026

Monocular distance estimation: from geometric foundations and deep learning innovations to industrial deployment challenges

It is concluded that future industrial deployment on edge-computing platforms will rely on a synergy between lightweight network architectures and multi-sensor fusion and self-supervised frameworks.

Zi-Kang Fan, Zi-Hao Xiang, Jiang-Sheng Liu · 0 citations
Preprint Sep 2026

Positive Pair Geometry Matters: Optimal Transport for Contrastive Learning of Visual Representations

This work proposes OTCLR, an optimal transport-aware framework for contrastive learning representations that generates geometry-consistent positive samples that serve as positive views that better preserve image structure while explicitly modeling spatial distributional geometry.

Akshit Nanda, Shahzad Ahmad, R. Padhy · 0 citations
Sep 2026

Adversarial Training for Pixel Diffusion

This work identifies direct output access to the image statistics being corrected as a key factor governing when adversarial post-training succeeds, the first systematic study of adversarial post-training for pixel diffusion.

Xin Lin, Zhi-Fei Zhang, Yu-Qian Zhou et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.