How Applied Researchers Engage with Methodological Tutorials: A Three-step Test Interview Study of an R Tutorial for Trial-Based Economic Evaluation.
TL;DR
How applied researchers engaged with the R tutorial and identified factors influencing its usability and perceived value may inform future tutorial design and the development and validation of more structured guidance, such as a checklist for tutorial development.
Abstract
Objectives
Methodological tutorials support the application of statistical methods in health technology assessment. Using an R tutorial for trial-based economic evaluation as a case study, we explored how applied researchers engaged with the tutorial and identified factors influencing its usability and perceived value.
Methods
We conducted a three-step test interview study with applied researchers without experience with the tutorial (n = 17) and semi-structured interviews with applied researchers who had previously used it in their own work (n = 3). Participants worked through the tutorial whilst thinking aloud, followed by focussed and reflective interviews. Qualitative data were analysed using reflexive thematic analysis.
Results
Three themes were generated: (1) 'getting in and getting it to run', describing technical barriers; (2) 'understanding and making sense of what tutorials teach', reflecting challenges in applying methods and interpreting outputs; and (3) 'adapting and applying tutorials in practice', highlighting difficulties modifying code for real-world datasets. Barriers stemmed not only from methodological complexity but also from software setup, unclear terminology, limited coding skills and lack of support. Despite these, participants described tutorials in general as efficient learning tools when clearly structured, transparent and adaptable.
Conclusions
We derived practical considerations that may help improve the accessibility, clarity and adaptability of future methodological tutorials. Methodological tutorials can support the learning and application of statistical methods, but their effective use depends on technical accessibility, user capability and access to support. These findings may inform future tutorial design and the development and validation of more structured guidance, such as a checklist for tutorial development.
