Improving the safety of large language models (LLMs) often comes at the expense of utility, as globally applied safety tuning may affect model responses to both harmful and benign inputs. We propose \textbf{C}ontinuous \textbf{L}at\textbf{E}nt \textbf{A}dapter \textbf{R}outing (CLEAR), a conditional safety adaptation f...
Chengxiao Wang, Enyi Jiang, Xiaojing Liao et al.· 0 citations
Agent evaluations are increasingly used to compare LLMs and inform deployment decisions, yet ranks can reflect not only the model but also the effects of the evaluation conditions such as the scaffolds or tasks. This makes reliability claim-dependent: an evaluation that reliably ranks deployed systems may not reliably...
Michael Hardy, Ruhana Azam, Anka Reuel et al.· 0 citations
The paper concludes that inference-time interventions are inadequate for combating the Artificial Hivemind, and shows that more generalizable solutions are needed at the model training level, and that an inference-time intervention (prompting) reliably raises measured response diversity.
Rylan Schaeffer, Brando Miranda, Joshua Kazdan et al.· 0 citations
This work introduces HeadEntropy, a method that predicts answer correctness by measuring how susceptible each attention head's pattern is to modification under further gradient updates, and evaluates broadly across 5 instruction-tuned LLMs and 5 datasets spanning general knowledge, multi-hop reasoning, mathematical rea...
S. Ostmeier, Brian Axelrod, M. Varma et al.· 5 citations
This work uses text optimization to detect subliminal learning effects and describes them as legible prompts and presents SALVE (Search-Aided Latent Verbalization) as a method to proactively detect subliminal learning effects.
Nathan Hu, Sanmi Koyejo, Christopher Potts· 1 citation
This work adapt convergent and discriminant validity from the social sciences into an approach for interrogating AI benchmarks, applying it to 56 capability and safety benchmarks across 53 models and asking analogous questions at the item level using item response theory (IRT) models.
Meera Desai, Sang T. Truong, Hanna Wallach et al.· 0 citations
This work audits ChatGPT, Claude, and Gemini across seven systems and nine benchmarks spanning general capability, social bias, and sycophancy, finding systematic API--interface differences in both accuracy and consistency.
Jennifer Wang, Joachim Baumann, Daniel E. Ho et al.· 0 citations
Vision-Language Models (VLMs) for radiology report generation are typically trained on retrospective clinical reports, which suffer from omission noise: clinically present findings are left unreported due to the omission of subtle findings. For example, prior studies show that cardiomegaly may be omitted from ICU chest...
Y. Kobayashi, P. Ramesh, Muhammad Ahmed Chaudhry et al.· 0 citations
It is found that the model's coding proficiency is inversely related to predictive performance in this setting, with the LLM better understood as a generative solver conditioned on context rather than a faithful predictor of solver behavior.
Fagun Patel, Sang T. Truong, Duc Q. Nguyen et al.· 0 citations
This report argues that affirmative AI coverage with limits in the billions is achievable by 2030, but only with industry-wide coordination, and lays out an eight-component AI insurance stack spanning incident data collection, catastrophe modeling, standards, contract design, risk selection, pricing, monitoring, and cl...
Cristian Trout, Sanmi Koyejo, S. Romanosky et al.· arXiv.org· 0 citations
This work argues for a hybrid public-private approach: keeping the benchmark content private to maintain integrity while making the benchmark design public to build trust, which offers a path toward trustworthy measurement of ever-growing data-intensive AI systems.
Sang T. Truong, S. Wang, Angelina Wang et al.· 0 citations
Coding agents increase development velocity but also technical debt. Prior work reports only average effects across adopters, hiding wide differences between teams. We introduce RAMP (Repository AI Maturity Profile), a four-level cumulative maturity model grounded in version-controlled artifacts that teams commit to co...
Yegor Denisov-Blanch, Shyam Agarwal, Pavel Azaletskiy et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
