Across seven Southeast Asian languages, broad capability gains are observed with the 120B-A12B model showing broader and more consistent improvements across tasks, with the 30B-A3B model showing broader and more consistent improvements across tasks.
Abstract
We introduce Nemotron-SEA-LION-v4.8, a family of Southeast Asian Languages In One Network (SEA-LION) models built upon NVIDIA Nemotron 3. The family includes 30B-A3B and 120B-A12B models, with both continued-pretrained base checkpoints and post-trained variants. We adapt the models using Southeast Asian, reasoning, code, and multilingual parallel data, followed by post-training with supervised fine-tuning and online on-policy distillation. On SEA-HELM, the 30B-A3B model improves the overall SEA score from 46.06 to 51.57, while the 120B-A12B model improves from 49.30 to 63.44. Across seven Southeast Asian languages, we observe broad capability gains with the 120B-A12B model showing broader and more consistent improvements across tasks.
To support long contexts efficiently, Sparse Gated Attention (SGA), which combines sparse attention with gated attention, and adopt Gated Norm (GN) to stabilize large-scale training is introduced, which keeps 4-bit NVFP4 serving within one point of FP8 accuracy.
Cheolseung Baek, Dhammiko Arya, Eunki Kim et al.· 0 citations
Manac-a-1B is the strongest model below the 7B scale, exceeding both Tucano-1b1 and Tucano-2b4 on LAMBADA-PT with large paired margins and is competitive on commonsense completion and near chance on multiple-choice reasoning, as are all small base models.
Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fábio Porto· 0 citations
This paper describes the system for Task~2 of the second Multilingual Conversational Speech Language Model (MLC-SLM) Challenge, which adapt Qwen3-Omni-30B-A3B-Instruct with a segment-evidence-aware data and post-training pipeline and obtains 90.92% accuracy on the final official evaluation set.
H. Le, L. Nguyen, Minh Tri Dao· 2nd Workshop on Multilingual...· 1 citation
Instella-MoE is introduced, a fully open Mixture-of-Experts (MoE) language model with 16 billion total parameters and 2.8 billion active parameters per token trained entirely from scratch on AMD Instinct MI300X and MI325X GPUs, establishing a strong, fully open foundation for efficient, high-performing MoE models and r...
Jiang Liu, Sudhanshu Ranjan, Prakamya Mishra et al.· 0 citations
Evaluation across the training stages shows that chat-vector merging transfers instruction following but preserves the donor model's reasoning language, while SFT with limited Cantonese reasoning data substantially shortens or removes reasoning traces and reduces benchmark performance.
Tsz-Chung Cheng, Chung-Shing Cheng, Chaak-ming Lau et al.· 0 citations
Ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish language model pretrained from scratch on 13.5B tokens of openly licensed text and instruction-tuned for chat, is described and three findings are believed to transfer to other small-model efforts.