Skip to content

An Introduction to Compression-Based Machine Learning

Sep 2026 · arXiv.org · Vol abs/2609.21309 · 0 citations · 65 references
Computer Science

TL;DR

This work introduces and empirically validate a design framework for compression-based ML, finding compression-based methods competitive with conventional baselines and decisively stronger on malware.

Abstract

Any lossless compression algorithm (like gzip) may be converted into a machine learning method, via either Normalized Compression Distance or the Minimum Description Length principle. Any auto-regressive model may be converted into a lossless compression method via entropy coding. This seemingly circular dependence has unrealized potential in modern artificial intelligence and machine learning, and we survey and formalize the various strategies that have been used to leverage compression for machine learning. We introduce and empirically validate a design framework for compression-based ML, finding compression-based methods competitive with conventional baselines and decisively stronger on malware. We find that varying these design choices yields accuracy gains of up to 0.62.

View source

Similar papers

Conference Open access Aug 2026

Mixture-of-Experts-based Entropy Model for Learned Image Compression

This paper proposes a MoE-based Entropy model (MoEE) for learned image compression, allowing the model to selectively activate only the subset of parameters required for the input image.

Jonas Brenig, R. Timofte · 1 citation
#artificial intelligence Preprint Sep 2026

Entropy-Punctured Bloom Filters for Memory-Efficient Machine Learning

Memory-efficient feature representations are increasingly important in machine learning settings where storage, transmission cost, bandwidth, or privacy constraints limit access to raw data. Bloom Filter (BF) encodings provide compact probabilistic representations of engineered features, but their behavior under struct...

J. Cartmell, M. Cardei, Ionut Cardei · 0 citations
Open access Oct 2026

LogNexus: Effective Log Compression via Unified Redundancy Encoding

State-of-the-art log compressors typically rely on a decoupled “parse-then-compress” workflow, where parsing is optimized for semantic accuracy (i.e., event identification) rather than storage efficiency. Through a comprehensive empirical study, we reveal that this architectural decoupling prevents the exploitation of...

Yang Liu, Kai-Ming Zhang, Zhuang-Bin Chen et al. · 0 citations
#natural language process... Preprint Sep 2026

FlexComp: One Model for Every Ratio in Context Compression

FlexComp is proposed, a method-agnostic framework that decouples the ratio from both training and deployment and matches separately trained fixed-ratio specialists with minimal degradation across ICAE, 500xCompressor, and SAC on MRQA.

Kai-Yan Zhao, Zhong-Tao Miao, Akiko Aizawa et al. · 1 citation
Preprint Aug 2026

Lossy Compression via Sparse Regression Codes: Generalized Construction and Finite-length Bounds

This work generalizes the SPARC construction, and considers the class of \emph{additive orthogonal} regression codes, of which standard SPARCs are a special case, and derives nonasymptotic bounds on the squared-error distortion by tracking the evolution of the encoding residual across stages.

G. Reeves, R. Venkataramanan · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.