The growing computational power of modern GPU data centers has let a few industrial giants train and deploy large language models (LLMs) on massive amounts of data. Yet the sheer scale of these models and their training data makes centralized development increasingly hard to sustain, and at the same time shuts out smal...
Zi-Yue Luo, Jiaxuan Cai, C. Le Denmat et al.· 0 citations
Multi-agent reasoning with large language models (LLMs) raises a central question for collective decision-making: when agents disagree, whose decision should be trusted? Resolving such disagreements is difficult because agents'decisions are susceptible to non-semantic biases such as majority pressure and response-style...
Sungjae Lee, Soheil Abbasloo, Jabin Koo et al.· 0 citations
Across diverse tasks and roles, model scales, and MoE architectures, DEP achieves better overall accuracy than static pruning and merging baselines, and generalizes to workflows unseen in training without retraining.
Jabin Koo, Soheil Abbasloo, Sungjae Lee et al.· 0 citations
Confidence-Gated Transductive Test Generation (CoTT), which first uses an efficient inductive procedure and invokes transductive generation only when inductive confidence is low, improves output reliability while allocating extra computation only when needed.
Sungjae Lee, Youngsik Yoon, Seockbean Song et al.· 0 citations
This work presents HAT-MPI, an ML-based framework that identifies the UCX Eager-Rendezvous protocol threshold and the optimal collective algorithm on InfiniBand HPC clusters and introduces two complementary techniques that improve the conventional data collection and evaluation workflow, namely LLM-assisted HPC hardwar...
Sungjae Lee, Sam Tilford, Dhabaleswar K. Panda· IEEE International Symposium...· 0 citations
Preliminary results from a geo-distributed LLM training prototype that treats networking constraints as first-order design concerns are presented, motivating adaptive networking support for synchronization, compression, placement, telemetry, and recovery in geo-distributed LLM training.
Zi-Yue Luo, Jiaxuan Cai, Cedric Le Denmat et al.· Conference on Applications,...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.