How many tokens from its context does a language model actually use, and what determines that number? We study this question through self-attention. Without retraining, we retain only the tokens with the highest attention weights at each head, layer, and query, keeping their original weights unchanged. By varying the s...
Timur Mudarisov, M. Burtsev, Radu State· 0 citations
Transformer language models build predictions through successive residual updates, but how their representations become specific to an eventual outcome remains unclear. We study this process by comparing intermediate residual states with their own final states and an empirical bank of final states from other contexts....
Timur Mudarisov, M. Burtsev, Radu State· 0 citations
Approaches to incorporating human awareness into mobile robot decision-making mainly focus on collision avoidance in low-level motion planning, often overlooking the challenges posed by human presence and high-level behavior. To address this vacancy, we present HINT-Plan, a novel approach to integrate human intention p...
Yu-Chen Liu, Luigi Palmieri, Lujun Li et al.· 0 citations
Detecting whether a jailbreak attack against a large language model (LLM) has succeeded is a challenging problem in AI security. Existing approaches rely on brittle string matching, expensive LLM-based judges, or task-specific classifiers that often fail to generalize. This paper investigates whether semantic coherence...
Costabile Di Gregorio, Lama Sleem, Jérôme François et al.· Annual International Compute...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.