Salman Ahmadi-Asl

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

This survey provides a structured entry point to tensorized language models and clarifies when parameter savings can plausibly translate into memory efficiency, computational efficiency, or interpretability, and introduces a metric for the compression-realization gap between theoretical memory reduction and measured system-level speedup.