HEALTHASSIST AI: A MULTIMODAL LARGE LANGUAGE MODEL FRAMEWORK FOR INTELLIGENT HEALTHCARE ASSISTANCE

Abstract

Aim/Background: The HealthAssist architecture is a complex MLLM which seeks to provide a dependable and reliable healthcare assistance through the use of artificial intelligence, particularly in underserved areas. It uses the Mistral Large 3 MoE model of 675 billion parameters while using only 41 billion at any point during execution. The healthassist system has an independent 2.5 billion parameter vision encoder used in the interpretation of medical imagery, alongside a large context of 256,000 tokens. Methods: This system incorporates seven different clinical services that include: medication consultation, disease identification, lab results interpretation, prescription decryption, symptom-based diagnosis, emergency services, and recommending hospitals locally. The architecture is made to ensure security through the use of SHA-256 authentication and compliance with the FDA SaMD and EU AI Act framework. Results: The results showed that the system attained an overall accuracy of 90.54% (CI=89.21-91.87), 94.64% semantic similarity, 86.54% METEOR, 88.12% ROUGE-L, and 92.89% anti-hallucination rate, performing better than other existing state-of-the-art baseline models on three different benchmark databases. Conclusion: The secure authentication and regulatory compliance make HealthAssist a scalable and implementable system for real-world clinical deployment, with demonstrated clinical acceptability across expert panel evaluation.