A User-Centered Approach to Customizing, Validating, and Evaluating a Large Language Model for Empowering Healthcare Providers with Up-to-Date Information on Adolescent and Child Health

Abstract

Healthcare professionals (HCPs) in Kenya face critical challenges in accessing reliable, context-specific information on adolescent and child health, which hampers clinical decision-making and compromises patient outcomes. Large Language Models (LLMs) offer a promising solution by delivering context-aware, localized knowledge tailored to the needs of Kenya’s healthcare system. This study aimed to (1) assess current knowledge and ethical perspectives of HCPs regarding LLMs, (2) iteratively refine and evaluate a web-based LLM prototype, (3) examine its real-world usability and clinical relevance, and (4) develop a proposed regulatory compliance and quality assurance framework. Employing a mixed-methods design that included surveys, interviews, and co-design workshops that engaged stakeholders across Nairobi, Kajiado, and Migori counties. Participants included medical doctors, nurses, midwives, clinical officers, and community health promoters involved in adolescent and child health. Additionally, policymakers, ICT specialists, facility administrators, and quality assurance officers contributed to the proposed framework development. A total of 678 participants were projected to be enrolled: 240 HCPs for key interviews, 360 for co-design, and 360 for usability and utility testing (with overlap), plus 78 stakeholders for regulatory discussions. Purposive sampling ensured diverse representation across professional roles and facility levels. Conducted over 12 months, with data collection from May to July 2025, the study employed both quantitative and qualitative analyses in profiling participant demographics, assessing associations between awareness, ethics, and usability, and identifying thematic patterns around implementation barriers and user feedback. A Stakeholder workshop guided framework review and refinement. This research strategy recognizes that ethical, usable AI implementation demands more than technical precision. By centering HCPs, supporting meaningful policy dialogue, and addressing regulatory challenges, the study aimed to promote equitable adoption, sustainable impact, and ethical integration of LLMs into Kenya’s healthcare ecosystem.