Repositioning Human Agency: An Analysis of Governance Capabilities in Responsible Human-GenAI Collaboration

Abstract

This study examines the gaps that still exist between technology adoption, the responsible use of AI, and information system governance, particularly in the use of Generative AI. To address these issues, this study proposes a multi-layered Human Oversight Architecture grounded in the Human-Centered AI (HCAI) approach. Within this framework, four capabilities — Class-Level Risk Governance, Prompt Literacy, Epistemic Vigilance, and Human Agency — are positioned as complementary elements in fostering the intent for responsible collaboration between humans and AI. The PLS-SEM method was applied using survey data from 492 elementary school teachers in Indonesia. The analysis results indicate that the willingness to collaborate responsibly with AI is most strongly influenced by the presence of clear boundaries and governance mechanisms, as well as practical skills in using AI. The two factors with the strongest influence were Risk Governance (β = 0.368) and Prompt Literacy (β = 0.265), followed by Epistemic Caution (β = 0.186) and Human Agency (β = 0.112). Meanwhile, a top-down approach to Responsible Collaboration Intent did not show a significant direct effect (β = 0.074, p = 0.129), indicating that abstract rules without operational governance are not yet sufficient. Human Agency, which serves as a normative anchor, ensures that operational capabilities remain under meaningful human control. This study offers a practical framework for strengthening accountability in AI-based decision-support systems.