Caduceus: MoE Foundation Models for Unifying Biological and Natural Language

Inventive Meta-Learning Strategies for Cross-Domain Natural Language Understanding in Distributed Knowledge Bases

Natural Language Understanding (NLU) addresses cross-domain challenges in understanding language due to inconsistent data distributions, uncertain concept semantics, and inconsistent language use specific to individual domains across different geographical locations. The majority of classical deep learning systems perform poorly because it cannot generalize, leading to cross-domain underperformance in practical settings. Therefore, this research develops a new class of meta-learning that provides a systematic means to advance cross-domain adaptability for NLU in the context of distributed knowledge systems. More specifically, this approach combines federated learning methods, model-agnostic meta-learning (MAML), and domain-adaptive transformers for the first time to enable rapid extraction of domain-invariant representations while concurrently maintaining local contextual semantics across reasonably distributed datasets. The author conducted benchmark assessments on the most recent multi-domain evaluations and cross-lingual and domain-shifted data to simulate distributed knowledge systems. The custom model outperformed all baselines, transformers, and fine-tuning models by an average of 12.8% cross-domain accuracy, 15.3% lower generalization error, and 18.6% faster convergence rate. In the scenario, the new model decreased to less than 90% and maintained over 90% of its performance after a 40% reduction in training samples. Meta-learning enables superior transfer efficiency and model adaptation to new, previously unseen domains with minimal training. The new meta-learning frameworks successfully addressed the required performance and adaptability for advanced intelligent information systems for cross-domain NLU in distributed knowledge bases.

Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench

PredicateLongBench is proposed, a benchmark that stress-tests long-context reasoning by asking models to identify the longest contiguous subsequence of words in a long input that satisfies given predicates/constraints drawn from a broader predicate class.

Domain-Specific Data Synthesis for LLMs through Minimal Sufficient Representation Learning

Large Language Models have demonstrated remarkable progress in general-purpose capabilities and can achieve strong performance in specific domains through fine-tuning on domain-specific data. However, acquiring high-quality data for target domains remains a significant challenge. Existing data synthesis approaches follow a deductive paradigm, heavily relying on explicit domain descriptions expressed in natural language and careful prompt engineering, limiting their applicability in real-world scenarios where domains are difficult to describe or formally articulate. In this work, we tackle the underexplored problem of domain-specific data synthesis through an inductive paradigm, where the target domain is defined only through a set of reference examples, particularly when domain characteristics are difficult to articulate in natural language. We propose a novel framework, DOMINO, that learns a minimal sufficient domain representation from reference samples and leverages it to guide the generation of domain-aligned synthetic data. DOMINO integrates prompt tuning with a contrastive disentanglement objective to separate domain-level patterns from sample-specific noise, mitigating overfitting while preserving core domain characteristics. Theoretically, we prove that DOMINO expands the support of the synthetic data distribution, ensuring greater diversity. Empirically, on challenging coding benchmarks where domain definitions are implicit, fine-tuning on data synthesized by DOMINO improves Pass@1 accuracy by up to 4.63% over strong, instruction-tuned backbones, demonstrating its effectiveness and robustness. This work establishes a new paradigm for domain-specific data synthesis, enabling practical and scalable domain adaptation without manual prompt design or natural language domain specifications. The full version is available at https://arxiv.org/abs/2605.30039.

NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models

Large language models (LLMs) have demonstrated strong performance across a wide range of tasks, but ensuring their reliability in highly technical domains remains a significant challenge. In nuclear engineering, problem solving often requires not only factual knowledge but also quantitative reasoning and conceptual understanding. To address the need for systematic evaluation in this domain, we introduce NuclearQAv2, a benchmark for assessing LLMs on nuclear engineering knowledge. The benchmark comprises approximately 1,240 question-answer pairs spanning three categories: boolean, numeric, and verbal. NuclearQAv2 is constructed using a hybrid pipeline that combines expert-authored questions, existing datasets, and LLM-assisted generation from domain-specific technical corpora. By leveraging structured prompting for both automated question generation and response evaluation, the proposed framework enables scalable benchmark construction and evaluation. We evaluate a diverse set of LLMs using NuclearQAv2 and observe substantial performance differences across task types. While the models generally perform well on factual questions, quantitative reasoning and conceptual understanding remain considerably more challenging. These results highlight the importance of multi-faceted evaluation frameworks and establish NuclearQAv2 as a scalable benchmark for assessing LLM capabilities in technical domains.

MMCP-GEN: A Modality-Extensible Diffusion Language Model for Conditional Protein Sequence Generation

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages

Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented languages spanning 6 language families -- ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales -- small, mid-size, large, and closed-source ensembles -- probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability. We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.