Kongzi: A Historical Large Language Model with Fact Enhancement
Kongzi is proposed, a language model for historical understanding and reasoning that combines continued pre-training on historical corpora, supervised fine-tuning, and fact-enhanced reinforcement learning that improves the mean answer scores of both model sizes and their CMMLU-Chinese-History accuracy.