The work identifies five distinct stages of the documentation review process: self review, technical review, editorial review, play testing, and post-publication feedback, and draws on practitioners with distinct expertise to address quality across content, presentation, and user experience.
Abstract
Organizations assign documentation work to technical writers, yet the knowledge required to produce it is distributed across developers, managers, and other practitioners. Prior work has established quality criteria for judging"good"documentation, but it has not examined how practitioners bring that expertise to improve documentation quality or the challenges they face in doing so. Through semi-structured interviews with experienced technical writers ($n=31$) from different organizations, our work reveals the individual and collaborative effort required to maintain documentation quality. We identify five distinct stages of the documentation review process: self review, technical review, editorial review, play testing, and post-publication feedback. Each stage draws on practitioners with distinct expertise to address quality across content, presentation, and user experience. Our findings surface organizational and technical challenges writers face in recruiting expert reviewers, navigating development timelines, and contending with tools not specifically designed for documentation workflows. Our work positions documentation review as a crucial yet understudied site of collaborative work and opens new research and design directions for process improvement and tool support.
LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense.. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model--language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation--utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9--95.2\%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.
Albérick Euraste Djiré, Iyiola E. Olatunji, Melissa Tessa et al.· 1 citation
MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.
James C. Davis, Kelechi G. Kalu, Huiyun Peng et al.· 1 citation
An audit-and-placebo protocol is proposed that separates verifier artifacts, interaction scaffolding, and grounded feedback credit in evaluations of self-evolving test generators in evaluations of self-evolving test generators.
Yunhao Liang, Chengguang Gan, Ruixuan Ying et al.· 0 citations
It is found that functional correctness alone is insufficient to assess the operational reliability of LLM-generated software before deployment in continuously running environments, and aging trends can also emerge in manually developed implementations.
Cesar Santos, Michele Vitagliano, Roberto Natella et al.· 0 citations
This prototype MRG image translocation software was helpful to 69% of patients with binocular diplopia, but limited by large angle strabismus because of the limited instrument field of view.
Edsel B Ing, Kevin Sha, Sarosh Dandoti et al.· Journal of neuro-ophthalmolo...· 0 citations
A diagnostic support system based on a unified web platform that classifies patients according to the risks of developing three diseases based on regularly collected clinical or audio data using classical supervised learning algorithms is presented.
Vedamurthy D R, Dr. Anup Ritti, A. Bibi et al.· International Journal for Re...· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduAug 17, 2026
A USAF cadet and a Lincoln Laboratory researcher found AI chatbots can help nontechnical service members produce viable software applications for their unique problems.
