Large language model (LLM) agents are increasingly evaluated on long-horizon tasks, including autonomously running a business. Existing benchmarks for this setting are almost exclusively English-language, Western in retail context, and single-agent, with the agent observing the business directly. We study how these res...
Bhuvan Arora, Ankita Karmakar, Chaitanya Jain et al.· 0 citations
Testing four instruction-tuned models from three families in five Indic languages alongside English suggests that safety evaluations in Indic languages may not reflect how models treat their everyday users.
Trizal Garg, S. Mitra, Aryan Karanjkar et al.· 0 citations
A previously undocumented confound in how unlearning is evaluated on BatchNorm-based architectures is identified: a single forward pass over retain data, an operation that modifies no weight, can deterministically rewrite the model's normalization state and reverse the apparent surface-metric forgetting.
Aaryaman Kalani, Murari Mandal, Dhruv Kumar et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.