Towards Effective Theory of LLMs: A Representation Learning Approach
The results suggest that LLM computation admits useful effective descriptions via RET: high-level, dynamically meaningful variables for interpretation, prediction, and control.
2 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
The results suggest that LLM computation admits useful effective descriptions via RET: high-level, dynamically meaningful variables for interpretation, prediction, and control.
TRAP, a one-sided penalty on tokenwise TRA that acts only where the target model pulls ahead of its reference, brings memorization near the level of an untrained model at little utility cost, where generic regularizers barely move and differential privacy gives up most of what fine-tuning bought.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.