Soft Token Alignment for Cross-Lingual Reasoning
SOLAR is proposed, an auxiliary objective for supervised fine-tuning that aligns soft-token representations across languages, using English as a pivot and strengthens final-layer cross-lingual similarity and substantially reduces language-cluster separability, suggesting that aligning soft-token representations helps p...