Skip to content

HandsOnAid: Building and Evaluating an Offline First-Aid Assistant for Emergency Response Scenarios

Nov 2026 · 0 citations · 49 references
Computer Science

TL;DR

HandsOnAid, a 4-bit Gemma 2B model fine-tuned with QLoRA on FirstAidQA dataset, a corpus of 5,550 first-aid question-answer pairs, finds limited coverage of several details of ANZCOR guidelines in the training corpus, which suggests that missing information in the training data could be a cause of the low scores.

Abstract

Emergencies can occur where there is no phone signal and no ambulance nearby. A small language model on a phone can be helpful in such situations, but its advice needs to be safe and accurate. We built HandsOnAid, a 4-bit Gemma 2B model fine-tuned with QLoRA on FirstAidQA dataset, a corpus of 5,550 first-aid question-answer pairs. We compared seven configurations, including the base, 4-bit and 8-bit fine-tuned models, and versions with a length floor, safety gate, or retrieval. We used a 41-pair question-answer bank with medically reviewed references that followed ANZCOR guidelines, and a strict 0-5 score rubric, with twelve safety overrides. Four frontier LLM judges (GPT-5.6 Sol, Claude Opus 5, DeepSeek V4 Pro, GLM-5.3) blindly scored every answer, with 45 hidden controls used to check judge reliability. Fine-tuning improved the mean score by +1.08 over the base model to 2.34. The best configuration scored 2.51 out of 5 which was the fine-tuned 4-bit model with retrieval. A medical professional also conducted a blind evaluation of 40 answers across the base, the 4-bit fine-tuned, and the retrieval configurations, and the scores were similar to the LLM judges'average scores. A keyword audit found limited coverage of several details of ANZCOR guidelines in the training corpus, that are required by the rubric, which suggests that missing information in the training data could be a cause of the low scores. On a smartphone device (Pixel 7 Pro) with six CPU threads and no network connection, the 4-bit fine-tuned model produced its first displayable text in 6.86 seconds and a complete answer in 14.88 seconds on average over ten questions.

View source

Similar papers

#artificial intelligence Review Sep 2026

From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization

A 202-scenario benchmark with Reference Decisions under a seven-class taxonomy is introduced and Decision Alignment and safety-specific error metrics are evaluated, finding structured LLM decisions are insufficient as a standalone safety mechanism and deployment requires an independent enforcement layer.

Diba Afroze, Xing-Li Zhang, Ya-Zhou Tu et al. · 0 citations
Preprint Oct 2026

What Does a Safety Label Measure? Auditing Situational-Safety Benchmarks for Multimodal LLMs

Situational-safety benchmarks label each scene SAFE or UNSAFE and score a multimodal large language model (MLLM) against that label, usually by judging its free-form answer, and these scores guide deployment. We ask what such a label measures. A label score can come from the scene, the request or the benchmark's own fi...

Wayner Barrios, Kenneth Barrios · 0 citations

Efficient LLM Distillation for Bangladesh Legal Context: A Smartphone-Compatible Retrieval-Augmented Generation Model

Legal information in Bangladesh is inaccessible to most citizens. Statutory text is English-only, trained lawyers are concentrated in urban centres, and cloud-dependent AI fails where mobile connectivity is unreliable, a setting in which hallucinated legal text causes direct harm. The system addresses statutory interpr...

M. Kamal, Mahadi Hasan Fahim, Talha Ridwan et al. · 0 citations
Conference Oct 2026

Evaluating Edge and Server-Offloaded Retrieval-Augmented Generation for a Rehabilitation Voice Assistant on Raspberry Pi

Voice-based assistants offer a promising interface for rehabilitation patients with limited digital literacy, but reliable domain-specific responses require grounding in relevant knowledge rather than relying solely on language model parametric memory. Retrieval-augmented generation (RAG) provides this grounding, yet d...

Aleksandr Bosler · 0 citations
#human-computer interacti... Preprint Oct 2026

A Stratified Multi-Rater Evaluation of LLM-Based Virtual Standardized Patients with a Deployed Data-Generation Platform

The Objective Structured Clinical Examination (OSCE) is widely used to assess clinical communication skills in medical education, requiring trainees to take histories and reason clinically with standardized patients (SPs). Large language model (LLM)-based virtual standardized patients (VSPs), typically built by persona...

Yong-Jin Yi, Jin-Ho Park, Sejoong Kim · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.