A survey that can be used to assess students' AI knowledge and foresee its usefulness as a diagnostic that goes beyond understanding students' attitudes and perceptions of AI and GenAI use and tests multiple aspects of students' knowledge and conceptual understanding to enable the development of targeted instruction.
Abstract
In this research-to-practice paper we present a survey that can be used to assess students'AI knowledge. As the use of artificial intelligence (AI), including generative artificial intelligence (GenAI), has proliferated, so has the need to educate students about the topic. A range of AI literacy frameworks have been proposed, outlining the essential knowledge that students should have. Alongside, different ways of assessing AI knowledge have been developed. As yet, there is a lack of assessment instruments capable of evaluating multiple forms of student knowledge, including technical concepts, practical applications, and ethical concerns about AI use. In this article, we present a study implementing a comprehensive instrument to assess AI knowledge. The instrument combines measures from multiple scales to capture a range of literacy features and actual knowledge. We implemented the instrument in a higher education setting to assess its viability and usefulness and found that the instrument exhibited useful diagnostic capabilities and was able to identify common misconceptions among students. Although students performed well overall, there was a significant misunderstanding of how AI, especially GenAI systems, work. It also identified a lack of higher-level knowledge. The instrument is publicly available for use by others. We foresee its usefulness as a diagnostic that goes beyond understanding students'attitudes and perceptions of AI and GenAI use and tests multiple aspects of students'knowledge and conceptual understanding. This can enable the development of targeted instruction.
The widespread use of Generative Artificial Intelligence tools among students across different educational levels is reshaping students’ attitude and education experience. They use it in various fields, including idea generation, writing assistance, and content creation. This study investigates the impact of the Modern Digital Skills course offered by the University of Jordan on students' digital literacy in Generative Artificial Intelligence. The study will employ a quantitative approach by conducting pre- and post-course surveys to measure changes in students' knowledge, skills, and attitudes across approximately 1,000 undergraduates from different majors at the university. Key variables include students' understanding of Generative Artificial Intelligence, their confidence in using Generative Artificial Intelligence tools, and their ability to critically evaluate AI-generated content and its ethical implications. The findings will determine whether the course significantly enhances students' knowledge, proficiency, critical thinking, and ethical awareness, while also exploring challenges and opportunities in integrating such a course into the academic curriculum. Ultimately, this study aims to provide insights for curriculum committees and decision-makers at universities in Jordan and the Middle East, emphasizing the importance of designing educational programs that foster essential Artificial Intelligence competencies and prepare students for a professional landscape increasingly shaped by Artificial Intelligence technologies.
Esra Alzaghoul, A. Assaf, Tahani Al-Khatib et al.· International Journal of Mod...· 0 citations
Artificial intelligence (AI) has become increasingly prominent in higher education, offering new possibilities for improving teaching practices and student learning experiences. Despite its growing adoption, limited evidence exists regarding how students perceive the benefits, challenges, and ethical implications associated with AI-supported educational tools, particularly in Sri Lanka. This study explored the perceptions of technical education students toward the use of AI applications in higher education. A mixed-methods approach was adopted, involving 200 students from the University of Moratuwa, German Tech, and NAITA. Data were collected through a structured questionnaire consisting of seven Likert-scale items and semi structured interviews with four open-ended questions. Quantitative data were analyzed using descriptive statistics and Chi-square tests, whereas qualitative responses were examined through thematic analysis. The results indicated that students generally viewed AI applications favorably and acknowledged their usefulness in activities such as research, information retrieval, problem-solving, and individualized learning support. Among the various AI tools, ChatGPT was identified as the most widely used application. Participants also demonstrated positive expectations regarding the future integration of AI into educational settings. Nevertheless, concerns relating to data privacy, algorithmic fairness, academic integrity, excessive dependence on AI-generated outputs, and system transparency were frequently reported. Interview findings further revealed students‟ interest in AI systems capable of providing personalized guidance, improved accessibility, enhanced support for ethical academic practices, and clearer policies governing data usage. Overall, the findings suggest that AI technologies have considerable potential to enrich higher education; however, their successful implementation requires careful attention to ethical, technological, and pedagogical considerations to ensure that educational benefits are achieved while minimizing associated risks.
H. U. C. P. Hewawasam, D. Saparamadu· Sabaragamuwa University Jour...· 0 citations
This paper presents the development and initial validation of an instrument to measure self-efficacy while using GenAI to learn programming, and finds strong support for the validity of the existing Steinhorst instrument in a new context, specifically an introductory programming course that fully integrates GenAI.
J. Prather, Lauren E. Margulieux, Yekaterina Kharitonova et al.· International Computing Educ...· 0 citations
It is proposed that mathematical literacy in the AI era should extend beyond traditional competencies to include the ability to critically evaluate AI-generated outputs, identify algorithmic limitations, and use AI responsibly a construct this review tentatively terms Mathematical AI Literacy.
Andi Mangaraja, D. Hasibuan, Ramadhan Herianto et al.· Mathline : Jurnal Matematika...· 0 citations
As an emerging technology, Generative Artificial Intelligence (GAI) tools are currently subject to constant re-inventions, and they may be employed in widely different ways by users. This study sought to develop understanding pertaining to the adoption of GAI tools on the basis of a cross-sectional survey conducted within a specific institutional context. It explored the following research question: “How do staff and students in an academic setting perceive their own responses to the specific emerging technology of Large Language Models?”.
The study found that appropriate uses of GAI tools in the research setting were not straightforwardly obvious for participants. Variation was identified in both the use of GAI tools and in expertise on the part of staff, so that it would have been difficult for many staff participants to appreciate educationally appropriate uses of GAI tools and the extent to which students were manifesting those uses. While student respondents were more positive than staff about likely gains to be had from making use of these tools, and considered that facilitating conditions to support their use of GAI tools were in place, the main use that came consistently into view for students was to use GAI tools to improve language-usage and for text production.
The findings suggest that more is needed than to frame the challenges posed by GAI tools for Higher Education across the world as a technical challenge for learning design or for training that addresses the AI literacy of students. The affordances of GAI tools play a key in determining their effective use. These affordances need to be imagined more adequately by both students and staff if this technology is to serve educational purposes effectively, with this imagination on the part of staff also needing to account for understanding of the ways that students themselves imagine the affordances of GAI tools.
P. Kahn, Mark Carrigan, Ignacio Wyman et al.· Frontiers in Education· 0 citations
Different disciplinary differences emerged: students in language-focused disciplines exhibited cautious attitudes regarding AI’s influence on authorship and creativity, whereas those in technical sciences generally regarded AI as a neutral assistant with minimal concern for ethical or cognitive implications.
A. Kereibayeva, Fariza Abdraimova, T. Toktarova et al.· AI@DTESI· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
