This study applies mechanistic interpretability techniques to uncover and analyze computational circuits within LLMs during biomedical question answering, and employs Automatic Circuit Discovery in combination with Edge Attribution Patching to enable scalable circuit extraction beyond toy settings.
This work presents a scalable, reproducible framework for evaluating, optimizing, and interpreting LLMs for biomedical knowledge extraction, with a focus on gene–gene regulatory relation prediction, pathway component recognition, multimodal pathway figure understanding, and automated prompt optimization.
THGAgents utilizes collaborative and dynamically updating agents to build a Traceable Causal Knowledge Graph, which serves as the foundation for the evidence-based knowledge structure and employs an LLM-driven heuristic search algorithm to traverse the complex network, balancing both novelty and rigor to deduce strict, evidence-based mechanistic causal chains.
Mingjia Yang, Kunhua Dong, K. Lim et al.· 0 citations
A mechanistic analysis of the jailbreaking behavior in a large-scale, safety-aligned LLM, focusing on LLaMA-2-7B-chat-hf identifies computational circuits responsible for generating affirmative responses to jailbreak prompts and uncovers key attention heads and MLP pathways that mediate adversarial prompt exploitation.
Paria Mehrbod, Boris Knyazev, Guy Wolf et al.· 0 citations
Text content is the dominant factor in annotation decisions, far outweighing annotator demographics, and that content-focused SHAP explanations are more effective than demographic persona prompting for guiding LLM annotations, showing that explainability methods can improve both the reliability and the transparency of NLP systems.
An engineering-oriented, end-to-end roadmap that structures the full lifecycle of clinical language model systems—from model design and domain adaptation to optimization and real-world evaluation is introduced.
The emergence of large language models offers unprecedented opportunities to transform clinical workflows. However, reliance on proprietary closed-source models poses significant risks to data privacy, institutional autonomy, and reasoning transparency. This study investigates the optimization of open-weight architectures, specifically the Llama-3.1-8B and Qwen2.5-14B-Instruct families, to function as specialized diagnostic tools rather than general-purpose conversational agents. Utilizing the MedQA USMLE-style benchmark, we evaluate the synergistic impact of context window scaling and multi-stage supervised fine-tuning within a localized Retrieval-Augmented Generation framework. Our findings identify a critical scaling threshold at 512 tokens, beyond which diagnostic accuracy stabilizes while preserving the computational efficiency required for on-site clinical deployment. Furthermore, we document a pronounced Brevity Shift, in which multi-stage supervised fine-tuning reduced model verbosity by over 99.7%. The models transitioned from long-form reasoning with an average of 465 tokens to single-token outputs for Qwen and near-single-token outputs for Llama. This shift effectively mitigated verbosity compensation, defined as the tendency of models to mask uncertainty through excessive text, while simultaneously improving diagnostic performance, which peaked at 70.0% accuracy for the Qwen architecture on the benchmark task. Additionally, our analysis reveals that systematic fine-tuning substantially reduces option-preference bias, as measured by Total Variation Distance, leading to more objective and consistent model outputs. Conducted under the INFOSTRATEG Strategic Program (PARROT AI) and funded by the National Centre for Research and Development (NCBR), Poland, this work establishes a robust technical framework for developing trustworthy, high-efficiency medical AI systems capable of operating entirely within hospital-controlled infrastructure.
L. Pawlik, Stanisław Deniziak· Scientific Reports· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
