A user centered evaluation framework for mobile health applications using Fuzzy AHP and TOPSIS: evidence from expert reviews and user experience data

Abstract

Background The rapid growth of digital mental health applications has created a need for comprehensive evaluation approaches that simultaneously consider clinical effectiveness, user experience, and ethical requirements. Existing assessment methods often focus on limited aspects of application quality and may not adequately capture uncertainty in expert judgments. Objective This study aimed to develop and apply a hybrid Fuzzy Analytic Hierarchy Process (Fuzzy AHP) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) framework for the user-centered evaluation and ranking of digital mental health applications. Methods A publicly available Mobile Health Apps Evaluation Dataset comprising 62 applications and 274 evaluation records was analyzed. Four main criteria and twelve subcriteria were established through expert consultation, including usability, engagement, clinical relevance, and ethics and privacy. An expert panel consisting of 18 specialists from public health, healthcare, digital health, health informatics, and user experience fields participated in the weighting process. Fuzzy AHP was employed to determine criteria weights under uncertainty, while TOPSIS was used to rank the selected applications. Sensitivity analysis was conducted to evaluate the robustness of the ranking outcomes. Results Clinical relevance received the highest importance weight (0.352), followed by ethics and privacy (0.284), usability (0.211), and engagement (0.153). Among the subcriteria, clinical expert involvement (0.132), privacy policy availability (0.118), and guideline compliance (0.115) were identified as the most influential factors. The TOPSIS analysis ranked A1, A2, and A3 as the highest-performing applications, with closeness coefficients of 0.842, 0.816, and 0.791, respectively. Sensitivity analysis demonstrated strong ranking stability across alternative weighting scenarios, with Spearman correlation coefficients exceeding 0.90. Conclusion The proposed hybrid Fuzzy AHP–TOPSIS framework provides a robust and transparent approach for evaluating digital mental health applications. The findings highlight the importance of clinical credibility, privacy protection, and evidence-based design in determining application quality. The framework offers practical guidance for developers, healthcare professionals, and policymakers seeking to identify and promote high-quality digital mental health technologies.