Author

Druv Pai

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Oct 2026

Information-Theoretic Scaling Laws for Generative Diffusion Training

Diffusion models trained on increasingly large datasets sampled from the same data distribution will converge to the same learned distribution. However, in practice, a model trained on finite data depends on the particular training dataset; this dependence is reflected in properties of the model's outputs, such as memo...

Druv Pai, Dejan Stancevic, Y. Ma et al. · 0 citations

#machine learning Preprint Oct 2026

Nested Sparsity for Scalable Pretraining and Continual Learning

Mixture-of-experts (MoE) transformers, the standard backbone for modern large-scale deep learning, employ conditional computation: only a sparse subset of model weights is used in each forward pass. In MoE transformers, this sparsity arises in the MoE operator, which generalizes the multi-layer perceptron in the classi...

Tian-Zhe Chu, Druv Pai, Y. Ma · 0 citations