Deploying LLMs on consumer-grade GPUs with insufficient memory to hold their weights can result in prohibitively slow inference, because decoding repeatedly transfers offloaded weights from system RAM or flash storage into GPU at much lower bandwidth than local GPU-memory access. Activation sparsity reduces these trans...
Junehyung Kim, Sankeerth Durvasula, N. Vijaykumar· 0 citations
Vision-Language-Action (VLA) models have emerged as foundational models for next-generation robotics. High VLA inference throughput is critical for meeting the control-rate requirements of robots. VLA models comprise two phases, a vision-language model (VLM) and an action head, that can be decoupled and executed asynch...
A. Li, Christina Giannoula, N. Vijaykumar· 0 citations
Squeeze3D is a novel framework that leverages implicit prior knowledge learnt by existing pre-trained encoders and decoders to compress 3D data at extremely high compression ratios and can flexibly support different formats, including meshes, point clouds, and radiance fields.
Rishit Dagli, Yu-Shi Guan, Sankeerth Durvasula et al.· 0 citations
Zero-knowledge machine learning (zkML) enables a server to perform verifiable inference while keeping model parameters private from the client. However, existing zkML systems incur prohibitive proof-generation costs. We observe that proof generation exhibits limited parallelism; that is, prover time does not decrease s...
Pawan Kumar Sanjaya, Christina Giannoula, Valdy Oktavian et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
