PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents
Pseudo Self-Distillation is presented, a framework that enables small language models to construct hierarchical memory representations by distilling behavior from a strong black-box oracle through a multi-stage training pipeline, with off-policy PSD achieving the strongest results across most conditions.
Pirzada Suhail, Meng-Lin Xia, Xu-Chao Zhang et al.
· 0 citations