This work presents the most comprehensive evaluation of LLM safety capabilities to date, systematically testing models across datasets that are organized into four distinct categories, and uncovers critical blind spots.
Abstract
Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades detection, the range of risks these models pose continues to grow. While both specialized content moderators and general-purpose LLMs are being used as safety layers, the question of which model is best suited for which type of harmful content remains unanswered. We present the most comprehensive evaluation of LLM safety capabilities to date, systematically testing \textbf{53} models across \textbf{11} datasets that we organize into four distinct categories. Our evaluation under both prompt-only and prompt-response settings uncovers critical blind spots: large frontier models that lead on one category fall significantly behind smaller, specialized alternatives on others, and real-world conversational safety remains largely unsolved across all model families. These findings challenge the assumption that scale alone ensures safety, and provide the community with a structured framework for informed model selection.
LifeSciBench is introduced, a benchmark of 750 expert-authored tasks designed to evaluate whether language models can handle realistic life science research work, with each constituent task paired with a human expert-written rubric.
Amelia Liu, Andrew Ho, Anne Marie Droste et al.· bioRxiv· 2 citations
A rapidly advancing precision-therapy pipeline-including antisense oligonucleotides to upregulate the intact allele, AAV-based gene replacement, CRISPR-mediated transcriptional activation, epigenetic modulators, and rational pathway-targeted small molecules-offers realistic prospects for disease modification.
Autistic children experience disproportionately high rates of anxiety, yet common interventions such as cognitive behavioural therapy and traditional mindfulness practices may be less effective due to their cognitive and abstract demands. Yoga nidra, a form of guided meditation using concrete visualisations and breath awareness, may offer a more accessible alternative. This mixed methods study evaluated the feasibility and pilot efficacy of a co-designed six-week online yoga nidra intervention targeting anxiety in autistic children aged 8-14 years. Neurophysiological and psychological data were collected from 13 participants using parent- and self-reported measures of anxiety (ASC-ASD), intolerance of uncertainty (IUSC), and emotion dysregulation (EDI), alongside heart rate variability (HRV). Notably, the cohort included a minimally speaking child, which demonstrates the potential for the intervention to extend to autistic children with language support needs, a group who are often underrepresented in similar research. Results indicated no statistically significant changes in anxiety over time; however, medium effect sizes were observed in self- (Hedge's g = 0.55) and parent-reported anxiety (Hedge's g = 0.45), with three participants moving from clinically significant to non-significant anxiety levels post-intervention. Additionally, intolerance of uncertainty and emotion dysregulation demonstrated small to medium effect size reductions (IUSC, Hedge's g = 0.50; EDI-Reactivity, Hedge's g = 0.55; EDI-Dysphoria, Hedge's g = 0.25), suggesting potential benefits of yoga nidra in these areas. Greater attendance was significantly associated with self-reported reductions in anxiety (ASC-ASD-SR, r = .757, p < .05), but not the other outcome measures. Unexpectedly, HRV outcomes indicated reduced autonomic functioning post-intervention. Additionally, qualitative data from semi-structured interviews with three child participants and their mothers from the study were analysed using reflexive thematic analysis, revealing five key themes of their experience: Thinking About the Body is Hard; We Want More; Homework Sucks; Making Mindfulness Concrete; and On-Screen and At Home is Convenient. While the online format was appreciated for its accessibility, challenges with adherence and scheduling highlighted the need for more flexible delivery models. This study advances upon prior research by being the first to develop a co-designed yoga nidra intervention specifically for autistic children. Findings support the feasibility of yoga nidra as a complementary intervention for autistic children and suggest directions for future research, including larger trials and further co-design with the autistic community.
Tundi Loftus, Shu H Yau, Sophia Soares et al.· Research in Developmental Di...· 1 citation
TestifAI, a deep learning testing framework for efficient and accurate estimation of robustness against combinations of perturbations, is proposed and partial model tomography is introduced, a novel approach to reconstructing model behaviour in a multi-perturbation space from tests that apply only a small number of perturbations.
Arooj Arif, T. Hartung, E. Botoeva et al.· 1 citation
A reproducible, license-aware knowledge-distillation recipe addressing the constraint of deploying a safety layer for large language models on commodity hardware by partitioning the corpus into seven safety categories aligned to a public hazard taxonomy.
Edson Rodrigues da Cruz Filho, Paulo Ricardo Ferreira Neves, P. H. Falsetti et al.· 0 citations
Large Language Models show potential in their diagnostic accuracy and consequent ability to reduce clinician burden, and may provide the greatest benefit when used to optimise referral quality at source, improving both clinician and potentially LLM triage downstream.
K. Surendran, I. Aziz, Glyndwr Jenkins· Current Surgery Reports· 0 citations
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
