This work introduces What If TSF (WIT), a benchmark for evaluating whether models effectively respond to future scenarios, and shows that factual accuracy gains do not consistently translate into appropriate responses to alternative scenarios, demonstrating the need to evaluate multimodal forecasting beyond predictive...
Jin-Kwan Jang, Hyunbin Jin, Hyungjin Park et al.· 0 citations
A comprehensive evaluation across a diverse set of open-source LLMs of varying sizes that support Korean finds that even the strongest model generates correct answer only 31.6% of the time, underscoring the challenging nature of KoSimpleQA.
Donghyeon Ko, Ye Jin, Kyubyung Chae et al.· arXiv.org· 1 citation
MoNe is a lightweight modular neural memory that attaches to any frozen pretrained Transformer to enable long-context inference without retraining, achieving strong performance on needle-in-a-haystack and word extraction benchmarks from RULER, where ICL degrades sharply.
Won-Yong Cho, Kyubyung Chae, Tribhuvanesh Orekondy et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.