Weikang Xiang

Symbolic Preference Distillation: Advancing Small Language Models for Mental Health Analysis.

This paper proposes symbolic preference distillation (SyPD), a reasoning optimization strategy that leverages specialized domain knowledge to perform SLMs' error analysis and generate symbolic knowledge via a teacher, and proposes a preference distillation method that guides an SLM to align with high-quality and clinically relevant reasoning derived from the teacher LLM through preference signals and symbolic knowledge, without requiring access to the teacher's output probabilities.