GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation
GoiBERT, a family of two compact Vietnamese encoders with 26M and 49M parameters that operate directly on raw text, shows that compression is nearly lossless for sentiment and classification tasks but remains costly for inference and entity recognition, which points to clear directions for future work.
Quang Anh Kiet Tran, Nam Anh Tran, Manh-Hung Trinh et al.
· 0 citations