Paving the way for greener ammonia production
New MIT research could lead to better materials for a fossil-fuel-free process for making the chemical that's essential to fertilizer and other products.
More from the blog
A better way to turn 2D designs into 3D models for rapid prototyping
Researchers developed an automated framework that helps AI models generate CAD programs more accurately and efficiently.
A better way to model the behavior of metal alloys
MIT researchers’ approach captures subtle atomic patterns, improving predictions of material properties.
Four ways Google Research scientists have been using Empirical Research Assistance
Data Mining & Modeling
Related papers
Convergent Evolution: How Different Language Models Learn Similar Number Representations
This paper identifies two different routes through which models can acquire geometrically separable features: they can learn them from complementary co-occurrence signals in general language data, including text-number co-occurrence and cross-number interaction, or from multi-token addition problems.
SimulRAG: Simulator-based RAG for Grounding LLMs in Long-form Scientific QA
SimulRAG, a simulator-based RAG framework with a generalized retrieval interface that translates between text and simulator parameters/outputs, is proposed, which improves informativeness and factuality over the strongest adapted RAG baselines, while UE+SBA enhances claim-level efficiency and quality.
The Null Token Knows: Reducing Message-Free Hallucination in ASR and NMT
The null token is turned into a diagnostic lens on hallucination and motivate evaluating abstention methods by both suppression and deletion costs, rather than by hallucination reduction alone.
Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu
It is challenging to detect hate speech in Low Resource Languages (LRLs) because of the absence of annotated data, the informality of its language structure, and the lack of standardized grammar. A good example of such a challenge is Roman Urdu which is broadly used by South Asians on social media and has a high variation while lacking contextually consistent spellings. The objective of this paper is to conduct a comprehensive assessment of Large Language Models (LLMs) for Hate Speech Detection (HSD) in Roman Urdu script and fine-tune these models using the Parameter-Efficient Fine-Tuning (PEFT) method called Low-Rank Adaptation (LoRA). To evaluate zero-shot inference, we benchmarked it against PEFT on different transformer models, including Mistral, LLaMA, Falcon, and multilingual BERT. Experiments are conducted on the PURUTT (Parallel Urdu and Roman Urdu Corpus for Toxic Comments and Transliteration) dataset with over 72,000 annotated comments. The results suggest that zero shot models perform moderately (F1 = 0.56), but updating a small fraction of the model trainable parameters improves the classification performance significantly (F1 > 0.93). Our results have shown that PEFT delivers outstanding performance alongside excellent computational efficiency, making it highly suitable for low-resource language processing tasks.