Large language models (LLMs), when acting as agents, are expected to take observed data in context, infer the latent state space underlying the world, and leverage it for downstream prediction. However, prior work demonstrated that LLMs struggle to use representations learned in context on a graph tracking task, where...
Kohsei Matsutani, Gouki Minegishi, C. Park et al.· 0 citations
Robot demonstration datasets used to train vision-language-action policies can contain a subtle but harmful failure mode: trajectories that are behaviorally correct but paired with the wrong language instruction. We study post-hoc auditing of these Instruction–Trajectory Mismatches (ITMs). Unlike failed rollouts, ITMs...
Simon Holk, Ryosuke Takanami, T. Matsushima et al.· IEEE Robotics and Automation...· 0 citations
Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters. Existing studies, however, focus almost exclusively on Transformer backbones, leaving open whether this principle also applies to state-space language...
Zhen-Xuan Yu, Takeshi Kojima, Yutaka Matsuo et al.· 0 citations
A training-free adaptive pruning method designed specifically for batched inference in LRMs, built on two components: periodic top-k selection over the aggregated importance scores, unaffected by the shift that aggregation induces in the activation distribution, and based on the observation that important neurons re-fi...
Yongmin Kim, Shota Takashiro, Yusuke Iwasawa et al.· 0 citations
A taxonomy of CoT is proposed consisting of Explicit CoT, which outputs all operations without aggregation, Composed CoT, which combines multiple operations into a single step, and Implicit CoT, which omits intermediate operations.
Kohsei Matsutani, Gouki Minegishi, Takeshi Kojima et al.· arXiv.org· 1 citation
This study establishes a reliable approach of data generation, training, and benchmarking, paving the way toward further bootstrapping the quality of many-to-many translation for programming languages.
Kouki Yuki, Jie Zeng, Kyoko Ogawa et al.· 0 citations
Readout Feedback (RoFB), a test-time intervention that converts intermediate predictions into token-wise pairwise coupling forces injected into the latent dynamics, is introduced, suggesting that closed-loop steering of latent dynamics can serve as a complementary inference-time control mechanism for recurrent reasonin...
Shunsuke Kamiya, Masanori Koyama, Seongcheol Jeong et al.· 0 citations
Looped Mamba and Looped Hybrid Mamba-Transformer architectures, which repeatedly apply a shared Mamba block to introduce explicit finite-depth recurrent computation, are investigated and adaptive exit-state selection improves downstream performance at intermediate depths, while actual inference-time savings require add...
Zhenxuan Yu, Takeshi Kojima, Yutaka Matsuo et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.