Knowledge Documentation Framework for AI Initiatives: Development and Validation Across Organizational AI Contexts

Abstract

Despite the growing organizational reliance on artificial intelligence (AI) systems, knowledge documentation (KD) practices in AI initiatives remain largely ad hoc, unstandardized, and disconnected from project lifecycle management. This study addressed this gap by proposing and validating the Knowledge Documentation Framework for AI Initiatives (KDF-AI), a twelve-component, five-phase, maturity-tiered framework synthesized through a literature review of peer-reviewed studies on KD practices in organizations implementing AI. Using Design Science Research (DSR) as the methodological paradigm, the study completed two iterative cycles: a literature-based synthesis that produced KDF-AI v1 and an expert evaluation cycle that resulted in the refined KDF-AI v2. Expert content validation was conducted with three domain validators representing academic, governance, and AI practitioner perspectives using a mixed-method approach that combined quantitative Content Validity Index (CVI) assessment with deductive thematic analysis of semi-structured interviews. The results showed that 57 of 66 items (86.4%) achieved universal inter-rater agreement, producing S-CVI/UA = 0.864 and S-CVI/Ave = 0.955, both exceeding the recommended threshold of 0.80. The nine items that did not meet the threshold consistently reflected issues of clarity rather than relevance, indicating strong conceptual acceptance of the framework while highlighting the need for more operationally specific articulation in several Advanced-tier components. Nine targeted revisions resulted in the development of KDF-AI v2. The study contributed: (1) a validated lifecycle-integrated KD framework for AI initiatives; (2) a taxonomy of ten systematically identified gaps in current AI KD practices; and (3) a methodological demonstration of mixed-method CVI validation for framework development in information systems research.